diff --git a/2021/2021-04-20_IT101-DM/slides/code_versioning.md b/2021/2021-04-20_IT101-DM/slides/code_versioning.md new file mode 100644 index 0000000000000000000000000000000000000000..eccd4a189300591de6be79ffbee91fcc987c631c --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/code_versioning.md @@ -0,0 +1,33 @@ +# Code versioning +<div style="position:absolute; width:40%"> + +**git** + + * Current standard for code versioning + * Maintain versions of your code as it develops + * Local system, which does not require an online repository + * Repositories allow distributed development + +<img align="middle" height="300px" src="slides/img/Git-logo.png"> +</div> + +<div class="fragment" style="position:absolute; left:50%; width:40%""> + +**git@lcsb** +* Recommended, supported repository +* Allows tracking of issues +* Ready for continous integration - code checked on commits to the repository. +* [https://git-r3lab.uni.lu](https://git-r3lab.uni.lu) + + + **Use at LCSB** + + * All analyses code should be in a repository + * Minimally at submission of a manuscript + * Better daily + * Even better "analyses chunkwise" +</div> + +<div style="position:absolute; width:45%; left:50%; top:28em; text-align:right"> +<a href="https://howto.lcsb.uni.lu/?policies:LCSB-POL-BIC-07" style="color:grey; font-size:0.8em;">LCSB-POL-BIC-07 Source Code Management Policy</a> +</div> diff --git a/2021/2021-04-20_IT101-DM/slides/data-housekeeping.md b/2021/2021-04-20_IT101-DM/slides/data-housekeeping.md new file mode 100644 index 0000000000000000000000000000000000000000..8cb4d014ea2b7732e1aa2b6913379a74021cf527 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/data-housekeeping.md @@ -0,0 +1,136 @@ +# Data housekeeping +## File names +<div style="display:flex; position:static; width:100%"> +<div class="fragment" data-fragment-index="0" style="position:static; width:30%"> + +### General pricinples + * Machine readable + * Human readable + * Plays well with default ordering +</div> +<div class="fragment" data-fragment-index="1" style="position:absolute; left:33%; width:30%"> + +### Separators + * No spaces + * Underscore to separate + * Hyphen to combine + +</div> +<div class="fragment" data-fragment-index="2" style="position:absolute; left:66%; width:30%"> + +### Date format follows **ISO 8601**<br> + + 2018-12-03<br> + 2018-12-06_1700 + +</div> +</div> + + +<div class="fragment" data-fragment-index="3" style="width:100%; position:static"> +<div style="position:absolute;width:55%"> +<b>Bad</b> names + +```bash + PhD-project-Jan19 alldata_final.foo + Finacial detailes BIocore 19/11/12.xls + ATACseq1Londonmapped.bam + Hlad.jez.M-L-průtoky JÃObj.z OhÅ™e-od 10-2011.xlsx +``` +</div> +<div style="position:relative;width:55%; bottom:20%; left:50%"> +<b>Good</b> names + +```bash +Iris-setosa_samples_1927-05-12.csv +PI102_Mouse12_EEG_2018-11-03_1245.tsv +Bioinfiniti_FullProposal_2018-11-15_1655.do +``` +</div> +</div> +<br> +<br> +<div class="fragment" data-fragment-index="3" style="width:100%;"> +From Jenny Bryan by CC-BY +(https://speakerdeck.com/jennybc/how-to-name-files) +</div> + + + +# Data housekeeping +## File organization +* Have folder organization conventions for your **group** + * Per Paper + * Per Study/Project + * Per Collaborator +* Keep <b>readme files</b> for data + * Title + * Date of Creation/Receipt + * Instrument or software specific information + * People involved + * Relations between multiple files/folders + +* Separate files you are actively working from the old ones +* Orient newcomers to the group's conventions + + + +# Data housekeeping +<div style="position:absolute"> + +## When working + * Clarify and separate source and intermediate data + * Keep data copies to a **minimum** + * Cleanup post-analysis + * Cleanup copies created for presentations or for sharing +</div> +<div style="position:relative;left:50%; width:40%"> +<img src="slides/img/cleaning-table.jpg" height="450px"> +</div> + + + +# Data housekeeping +## End of project + * handover data to a new responsible when leaving + * data should be kept as a single copy on server-side storage + * no copies on desktops and external devices + * non-proprietary formats + * minimal metadata + * sensitive data (e.g. whole genome) **must** be encrypted + <br/> + <br/> + * If not specified otherwise, data must be kept for **10 years** following project end for reproducibility purposes +<aside class="notes"> +Note: sometimes it is hard to find/understand dataset 10 days old +</aside> + +## In doubt on data archival? +Contact R<sup>3</sup> for support on archival of datasets using tickets: + * https://service.uni.lu/sp + * Home > Catalog > LCSB > Biocore: Application services > Request for: Support + +<div style="position:absolute; width:45%; left:50%; top:28em; text-align:right"> +<a href=" https://howto.lcsb.uni.lu/?policies:LCSB-POL-BIC-03" style="color:grey; font-size:0.8em;">Research Data Retention and Archival Policy</a> +</div> + + + +# Data housekeeping - Summary +## Server is your friend! + * Allows a consistent backup policy for your datasets + * Keeps number of copies to minimum + * Specification of clear access rights + * High accessibility + * Data are discoverable + * Server can't be stolen + +## General guidelines + * Use institutional media for storage of **all** data + * Research data (particularly sensitive data) should be in a single source location + * Enable encryption for data stored on movable media + * Clarify and separate source and intermediate data + * Disable write access to relevant source data (read-only) + * Backup research data! + * Download Anti-virus software + * Generate checksums diff --git a/2021/2021-04-20_IT101-DM/slides/data-introduction.md b/2021/2021-04-20_IT101-DM/slides/data-introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..c269d80286a4bb458eec2d5640664eeb1e718713 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/data-introduction.md @@ -0,0 +1,86 @@ +# Data and metadata +<div style="display:grid;grid-gap:100px;grid-template-columns: 40% 40%"> +<div > + +## Data + * "*information in digital form that can be transmitted or processed*" + <p align="right">-- Merriam-Webster dictionary</p> + * "*information in an electronic form that can be stored and processed by a computer*" + <p align="right">--Cambridge dictionary</p> + +</div> +<div> + +## Metadata + * data describing other data + * information that is given to describe or help you use other information + * metadata are data + * can be processed and analyzed +</div> +</div> + +<div class="fragment"> + +## Metadata examples: +<div style="position:absolute"> + <ul> + <li> LabBook </li> + <li> author/owner of the data</li> + <li> origin of the data + <li> data type + </ul> +</div> +<div style="position:absolute;left:25%"> + <ul> + <li> description of content </li> + <li> modification date </li> + <li> description of modification </li> + <li> location </li> + </ul> +</div> +<div style="position:relative;left:50%;top:0.7em"> + <ul> + <li> calibration readings</li> + <li> software/firmware version</li> + <li> data purpose</li> + <li> means of creation</li> + </ul> +</div> +</div> + +<div class="fragment"> +<br> +</center> +<center style="color:red">!Insufficient metadata make the data useless!</center> +</div> +<aside class="notes"> +Sometimes metadata collection takes more time than data collection +</aside> + + + +# LCSB research data +three categories: + * **Primary data** + * scientific data + * measurements, images, observations, notes, surveys, ... + * models, software codes, libraries, ... + * metadata directly describing the data + * data dictionaries + * format, version, coverage descriptions, ... + + * **Research record** + * description of the research process, including experiment + * experiment set-up + * followed protocols + * ... + + * **Project accompanying documentation** + * ethical approvals, information on the consent + * collaboration agreements + * intellectual property ownership + * other relevant documentation + +<div style="position:absolute; width:45%; left:50%; top:28em; text-align:right;"> +<a href="https://howto.lcsb.uni.lu/internal/policies/LCSB-POL-BIC-03/" style="color:grey; font-size:0.8em;">LCSB-POL-BIC-03 Research Data Retention and Archival Policy</a> +</div> \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/data_flow.md b/2021/2021-04-20_IT101-DM/slides/data_flow.md new file mode 100644 index 0000000000000000000000000000000000000000..396c1ec70825f3695e46344ac94dc20617c71c2f --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/data_flow.md @@ -0,0 +1,106 @@ +# Typical flow of data + +<div style="display:grid;grid-gap:10px;grid-template-columns: 30% 20% 30%; + grid-auto-flow:column;grid-template-rows: repeat(4,auto);position:relative;left:8%"> + +<div class="content-box fragment" data-fragment-index="1"> + <div class="box-title red">Source data</div> + <div class="content"> + + * Experimental results + * Large data sets + * Manually collected data + * External + + </div> +</div> + +<div class="content-box fragment" data-fragment-index="2"> + <div class="box-title yellow">Intermediate</div> + <div class="content"> + + * Derived data + * Tidy data + * Curated sets + + </div> +</div> + +<div class="content-box fragment" data-fragment-index="3"> + <div class="box-title blue">Analyses</div> + <div class="content"> + + * Exploratory + * Model building + * Hypothesis testing + + </div> +</div> + +<div class="content-box fragment" data-fragment-index="4"> + <div class="box-title green">Dissemination</div> + <div class="content"> + + * Manuscript, report, presentation, ... + + </div> +</div> +<center> +<img src="slides/img/data-flow_sources.png" height=60%> +</center> +<center> +<img src="slides/img/data-flow_transformation.png" height=60%> +</center> +<center> +<img src="slides/img/data-flow_chart.png" height=60%> +</center> +<center> +<img src="slides/img/data-flow_paper.png" height=60%> +</center> + +<div class="content-box fragment" data-fragment-index="5"> +<div class="box-title red">Preserve</div> + <div class="content"> + + * Version data sets + * Backup + * Protect + + </div> +</div> + +<div class="content-box fragment" data-fragment-index="6"> + <div class="box-title yellow">Reproduce</div> + <div class="content"> + + * Automate your builds + * Use workflow tools (e.g. Snakemake) + + </div> +</div> + + +<div class="content-box fragment" data-fragment-index="7"> +<div class="box-title blue">Trace</div> + <div class="content"> + + * Multiple iterations. + * Code versioning (Git) + + </div> +</div> + +<div class="content-box fragment" data-fragment-index="8"> +<div class="box-title green">Track</div> + <div class="content"> + + * Through multiple versions + + </div> +</div> + +</div> +<aside class="notes"> +flow of the data is downstream (mostly), but you are going back and forth +applies to all data (financial report, lab safety assessment) +</aside> diff --git a/2021/2021-04-20_IT101-DM/slides/fair-principles.md b/2021/2021-04-20_IT101-DM/slides/fair-principles.md new file mode 100644 index 0000000000000000000000000000000000000000..be70f11a6f0b941a0e7f7b86b990a1c098096282 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/fair-principles.md @@ -0,0 +1,12 @@ +# FAIR (meta)data principles + * dates back to 2014 + * well accepted by scientific community + * necessity in data driven science + * officially embraced by EU and G20 + * required by funding agencies and journal publishers + +<center> +<img src="slides/img/fair-principles.png" height="400px"> +</center> +<br> +<br> diff --git a/2021/2021-04-20_IT101-DM/slides/howtos.md b/2021/2021-04-20_IT101-DM/slides/howtos.md new file mode 100644 index 0000000000000000000000000000000000000000..34998e54e7604a3ee50213dfe9650785c9ced8ed --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/howtos.md @@ -0,0 +1,6 @@ +# LCSB How-Tos +<br> +https://howto.lcsb.uni.lu/ +<center> + <img src="slides/img/howtocard.png" width="50%"> +</center> diff --git a/2021/2021-04-20_IT101-DM/slides/img/3pillars-full.png b/2021/2021-04-20_IT101-DM/slides/img/3pillars-full.png new file mode 120000 index 0000000000000000000000000000000000000000..12dff9269e9392fd5e8fe1987512e31255b817db --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/3pillars-full.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/3pillars-full.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/DinoSequentialSmaller.gif b/2021/2021-04-20_IT101-DM/slides/img/DinoSequentialSmaller.gif new file mode 120000 index 0000000000000000000000000000000000000000..b93395506b04518889f7af6dadd740ac95abe158 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/DinoSequentialSmaller.gif @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/DinoSequentialSmaller.gif \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/Git-logo.png b/2021/2021-04-20_IT101-DM/slides/img/Git-logo.png new file mode 120000 index 0000000000000000000000000000000000000000..0b9b18c57e58e19956b8bf4a9f87a1bc0035e527 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/Git-logo.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/Git-logo.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_backed-up.png b/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_backed-up.png new file mode 120000 index 0000000000000000000000000000000000000000..a21575d0f8cc6bdc6c16c5f887951f9559e8ef69 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_backed-up.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/LCSB_storages_backed-up.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_backup.png b/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_backup.png new file mode 120000 index 0000000000000000000000000000000000000000..94cfd695982584f7e0f5f7220596efc504836cf1 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_backup.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/LCSB_storages_backup.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_full.png b/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_full.png new file mode 120000 index 0000000000000000000000000000000000000000..0f0df3127847f58ccfc388193b187a006f746d1d --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_full.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/LCSB_storages_full.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_personal-crossed.png b/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_personal-crossed.png new file mode 120000 index 0000000000000000000000000000000000000000..a731b8d3fa99d5ff09bae84b1cb221b76a12fb21 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/LCSB_storages_personal-crossed.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/LCSB_storages_personal-crossed.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/alexey_kolodkin.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/alexey_kolodkin.png new file mode 120000 index 0000000000000000000000000000000000000000..2640a53bdb8ad6f19b71c41f36329925047bcd28 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/alexey_kolodkin.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/alexey_kolodkin.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/christophe_trefois.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/christophe_trefois.png new file mode 120000 index 0000000000000000000000000000000000000000..4ffa6cf033923f9fa06bf42af59227e36f616df3 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/christophe_trefois.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/christophe_trefois.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/karim_chaouch.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/karim_chaouch.png new file mode 100644 index 0000000000000000000000000000000000000000..2cee6cd10e847533de46a0a7a11a11c2dc6a32c8 Binary files /dev/null and b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/karim_chaouch.png differ diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/laurent_heirendt.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/laurent_heirendt.png new file mode 120000 index 0000000000000000000000000000000000000000..dc2cb33d19e972d5df9c1baa8db3f5099f2f625d --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/laurent_heirendt.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/laurent_heirendt.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/maharshi_vyas.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/maharshi_vyas.png new file mode 120000 index 0000000000000000000000000000000000000000..efd014a29e3a94a790569e3f74695e9e7a26a80a --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/maharshi_vyas.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/maharshi_vyas.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/nene_barry.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/nene_barry.png new file mode 100644 index 0000000000000000000000000000000000000000..10b5a918f57ba78a377e1276987f059137c0f24f Binary files /dev/null and b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/nene_barry.png differ diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/noua_toukourou.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/noua_toukourou.png new file mode 120000 index 0000000000000000000000000000000000000000..e1670ac385b6cd898c0be9e4f14bf3e289b59332 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/noua_toukourou.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/noua_toukourou.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/pinar_alper.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/pinar_alper.png new file mode 120000 index 0000000000000000000000000000000000000000..c55e60712e946f34ecc5138f4940b4a1311ee0a3 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/pinar_alper.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/pinar_alper.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/reinhard_schneider.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/reinhard_schneider.png new file mode 120000 index 0000000000000000000000000000000000000000..25af93e660cf70eed5c4b2b0ca7298476089a068 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/reinhard_schneider.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/reinhard_schneider.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/sarah_peter.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/sarah_peter.png new file mode 120000 index 0000000000000000000000000000000000000000..ad689955c4e6dd2b9d4305d8209415b206306438 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/sarah_peter.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/sarah_peter.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/valentine_groues.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/valentine_groues.png new file mode 100644 index 0000000000000000000000000000000000000000..07adbe182897c374b531df3311593a1d03fff68b Binary files /dev/null and b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/valentine_groues.png differ diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/vilem_ded.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/vilem_ded.png new file mode 120000 index 0000000000000000000000000000000000000000..fb56a09d1b7f0bfce3c35d84159f6a8b2f0434a6 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/vilem_ded.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/vilem_ded.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/yohan_yarosz.png b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/yohan_yarosz.png new file mode 120000 index 0000000000000000000000000000000000000000..fb619f735038d3313465727660309181f12e00b8 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/R3_profile_pictures/yohan_yarosz.png @@ -0,0 +1 @@ +../../../../../2020/2020-06-09_IT101-DM/slides/img/R3_profile_pictures/yohan_yarosz.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/banned_exchange_channels.png b/2021/2021-04-20_IT101-DM/slides/img/banned_exchange_channels.png new file mode 120000 index 0000000000000000000000000000000000000000..cabe9a708f22c3aa73d794dc8c8db628a8f7413c --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/banned_exchange_channels.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/banned_exchange_channels.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/checksum.png b/2021/2021-04-20_IT101-DM/slides/img/checksum.png new file mode 120000 index 0000000000000000000000000000000000000000..4db19c34e78d1a3359283d4c7ab417dd706c111c --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/checksum.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/checksum.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/cleaning-table.jpg b/2021/2021-04-20_IT101-DM/slides/img/cleaning-table.jpg new file mode 120000 index 0000000000000000000000000000000000000000..7c6edfee0d05597fe67e94271797da6e7f7603d9 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/cleaning-table.jpg @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/cleaning-table.jpg \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/code-example.png b/2021/2021-04-20_IT101-DM/slides/img/code-example.png new file mode 120000 index 0000000000000000000000000000000000000000..ae90b43c8b827b4738396fcc776da6f2b18ef05a --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/code-example.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/code-example.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/data-flow_chart.png b/2021/2021-04-20_IT101-DM/slides/img/data-flow_chart.png new file mode 120000 index 0000000000000000000000000000000000000000..f2197339938b7a01d0e41b4fded1f871f690afc0 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/data-flow_chart.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/data-flow_chart.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/data-flow_paper.png b/2021/2021-04-20_IT101-DM/slides/img/data-flow_paper.png new file mode 120000 index 0000000000000000000000000000000000000000..dd68ecf6381ae6a932e1dde5c56aec55c1783873 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/data-flow_paper.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/data-flow_paper.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/data-flow_sources.png b/2021/2021-04-20_IT101-DM/slides/img/data-flow_sources.png new file mode 120000 index 0000000000000000000000000000000000000000..3f5b7183630e7e1459e813dacfef65ff89103f1d --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/data-flow_sources.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/data-flow_sources.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/data-flow_transformation.png b/2021/2021-04-20_IT101-DM/slides/img/data-flow_transformation.png new file mode 120000 index 0000000000000000000000000000000000000000..80a6566ac70c8ae390a79617b8e27172ee2461dc --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/data-flow_transformation.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/data-flow_transformation.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/excel_analyses-sheet.jpeg b/2021/2021-04-20_IT101-DM/slides/img/excel_analyses-sheet.jpeg new file mode 120000 index 0000000000000000000000000000000000000000..b4772fb84ab26d3474fce0e5005c374040cfdab3 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/excel_analyses-sheet.jpeg @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/excel_analyses-sheet.jpeg \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/excel_data-sheet.png b/2021/2021-04-20_IT101-DM/slides/img/excel_data-sheet.png new file mode 120000 index 0000000000000000000000000000000000000000..d3cc7213505f9d2b515feabee38e5baff0fdc240 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/excel_data-sheet.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/excel_data-sheet.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/fair-principles.png b/2021/2021-04-20_IT101-DM/slides/img/fair-principles.png new file mode 120000 index 0000000000000000000000000000000000000000..dc08b40550920efe75d2c6c90a9aef12f911b4bf --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/fair-principles.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/fair-principles.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/favicon.ico b/2021/2021-04-20_IT101-DM/slides/img/favicon.ico new file mode 120000 index 0000000000000000000000000000000000000000..9d22edefbb4dcf55b9b46dc3c3e868e4ef0232dd --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/favicon.ico @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/favicon.ico \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/howtocard.png b/2021/2021-04-20_IT101-DM/slides/img/howtocard.png new file mode 100644 index 0000000000000000000000000000000000000000..02732890b427096df005547b0f1b0af7ec2558c6 Binary files /dev/null and b/2021/2021-04-20_IT101-DM/slides/img/howtocard.png differ diff --git a/2021/2021-04-20_IT101-DM/slides/img/plot-data.png b/2021/2021-04-20_IT101-DM/slides/img/plot-data.png new file mode 120000 index 0000000000000000000000000000000000000000..1c85224dd2ec7415d8022a1caff7acecfe3d3752 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/plot-data.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/plot-data.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/r3-training-logo.png b/2021/2021-04-20_IT101-DM/slides/img/r3-training-logo.png new file mode 120000 index 0000000000000000000000000000000000000000..a294348c13e792f62efd588aff1fd26f00326573 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/r3-training-logo.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/r3-training-logo.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/red-cross.png b/2021/2021-04-20_IT101-DM/slides/img/red-cross.png new file mode 120000 index 0000000000000000000000000000000000000000..2bd34a326fbbbbc4a8740725a8a18ac1124cbc67 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/red-cross.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/red-cross.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/reproducibility_nature.png b/2021/2021-04-20_IT101-DM/slides/img/reproducibility_nature.png new file mode 120000 index 0000000000000000000000000000000000000000..fd870ff209c9e0d02382f028efd456e190439540 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/reproducibility_nature.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/reproducibility_nature.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/rudi_balling.jpg b/2021/2021-04-20_IT101-DM/slides/img/rudi_balling.jpg new file mode 120000 index 0000000000000000000000000000000000000000..ee9c8f3bf5e31b19d84f3042ac89a28b9226aa6d --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/rudi_balling.jpg @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/rudi_balling.jpg \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/scripts b/2021/2021-04-20_IT101-DM/slides/img/scripts new file mode 120000 index 0000000000000000000000000000000000000000..e4928330211eccc2cf6a879dc3c353b91870281f --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/scripts @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/scripts \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/undraw_secure_server_s9u8.png b/2021/2021-04-20_IT101-DM/slides/img/undraw_secure_server_s9u8.png new file mode 120000 index 0000000000000000000000000000000000000000..2c613b42b06370c8fa870819ed804b591f085941 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/undraw_secure_server_s9u8.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/undraw_secure_server_s9u8.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/img/wordcloud.png b/2021/2021-04-20_IT101-DM/slides/img/wordcloud.png new file mode 120000 index 0000000000000000000000000000000000000000..09c65d628005e8e1a0f9f32d80b2fe91e8c77df9 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/img/wordcloud.png @@ -0,0 +1 @@ +../../../../2020/2020-06-09_IT101-DM/slides/img/wordcloud.png \ No newline at end of file diff --git a/2021/2021-04-20_IT101-DM/slides/index.md b/2021/2021-04-20_IT101-DM/slides/index.md new file mode 100644 index 0000000000000000000000000000000000000000..987e37711c5525d92252eaf51cdba8c03f89466c --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/index.md @@ -0,0 +1,20 @@ +# IT101 - Working with computers +<br>IT101 - Working with computers<br> +## April 20th, 2021 + +<div style="top: 6em; left: 0%; position: absolute;"> + <img src="theme/img/lcsb_bg.png"> +</div> + +<div style="top: 5em; left: 60%; position: absolute;"> + <img src="slides/img/r3-training-logo.png" height="200px"> + <br><br><br><br> + <h3></h3> + <br><br><br> + <h4> + Vilem Ded<br> + Data Steward<br> + vilem.ded@uni.lu<br> + <i>Luxembourg Centre for Systems Biomedicine</i> + </h4> +</div> diff --git a/2021/2021-04-20_IT101-DM/slides/ingestion.md b/2021/2021-04-20_IT101-DM/slides/ingestion.md new file mode 100644 index 0000000000000000000000000000000000000000..7d168db8c9e03e261196b436ccae1b460f49fc4e --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/ingestion.md @@ -0,0 +1,79 @@ +# Data housekeeping +## Available data storage +<div class='fragment' style="position:absolute"> +<img src="slides/img/LCSB_storages_full.png" height="750px"> +</div> + +<div class='fragment' style="position:absolute"> +<img src="slides/img/LCSB_storages_personal-crossed.png" height="750px"> +</div> + +<div style="position:absolute; width:45%; left:50%; top:28em; text-align:right"> +<a href=" https://howto.lcsb.uni.lu/?policies:LCSB-POL-BIC-02" style="color:grey; font-size:0.8em;">Data Storage and Backup Policy</a> +</div> + + + +# Data ingestion/transfer +## Receiving and sending data + +<img height="400px" style="position:relative;left:10%" src="slides/img/banned_exchange_channels.png"><br> +<div style="position:absolute; left:10%;width:30%"> + +## E-mail is not for data transfer + +* Avoid transfer of any data by e-mail +* E-mail is a poor repository +* Data can be read on passage + +</div> +<div class="fragment" style="left:50%; width:30%; position:absolute"> + +## Exchanging data +* Share on Atlas server +* OwnCloud share (LCSB - BioCore) +* DropIt service (SIU) +* LFT (IBM Aspera) share for sensitive data +</div> +</div> + +<div style="position:absolute; width:45%; left:50%; top:28em; text-align:right"> +<a href=" https://howto.lcsb.uni.lu/?policies:LCSB-POL-BIC-05" style="color:grey; font-size:0.8em;">Research Human Data Sharing Policy</a> +</div> + + + +# Data ingestion/transfer +Data can be corrupted: + * (non-)malicious modification + * faulty file transfer + * disk corruption + +<div class="fragment"> + +### Solution + + * disable write access to the source data + * generate checksums! + +<div style="position:absolute;left:40%"> +<img src="slides/img/checksum.png" width="500px"> +</div> +</div> + +<div class="fragment" style="position:relative; left:0%"> + + +## When to generate checksums? +* before data transfer + - new dataset from collaborator + - upload to remote repository + +* long term storage + - master version of dataset + - snapshot of data for publication +</div> + +<div style="position:absolute; width:45%; left:50%; top:28em; text-align:right"> +<a href=" https://howto.lcsb.uni.lu/?policies:LCSB-POL-BIC-02" style="color:grey; font-size:0.8em;">Data Storage and Backup Policy</a> +</div> diff --git a/2021/2021-04-20_IT101-DM/slides/introduction.md b/2021/2021-04-20_IT101-DM/slides/introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..95e30815a03c5be0f1dd409701ec0edc15d2aa73 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/introduction.md @@ -0,0 +1,26 @@ +# Introduction +<div class="fragment" style="position:absolute"> +<img height="450px" src="slides/img/wordcloud.png"><br> + +## Learning objectives + + * How to manage your data + * How to look and analyze your data + * Solving issues with computers + * Reproduciblity in the research data life cycle + +</div> +<div class="fragment" style="position:relative;left:50%; width:40%"> +<div > +<center> +<img height="405px" src="slides/img/rudi_balling.jpg"><br> +Prof. Dr. Rudi Balling, director +</center> +</div> + +## Pertains to practically all people at LCSB + * Scientists + * PhD candidates + * Technicians + * Administrators +</div> diff --git a/2021/2021-04-20_IT101-DM/slides/list.json b/2021/2021-04-20_IT101-DM/slides/list.json new file mode 100644 index 0000000000000000000000000000000000000000..5dbc2360d7b5b8d68580868e6431ce26e8746eb1 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/list.json @@ -0,0 +1,17 @@ +[ + {"filename": "index.md"}, + {"filename": "introduction.md"}, + {"filename": "data-introduction.md"}, + {"filename": "data_flow.md"}, + {"filename": "ingestion.md"}, + {"filename": "storage_setup.md"}, + {"filename": "data-housekeeping.md"}, + {"filename": "howtos.md"}, + {"filename": "reproducibility.md"}, + {"filename": "code_versioning.md"}, + {"filename": "visualization.md"}, + {"filename": "problem_solving.md"}, + {"filename": "fair-principles.md"}, + {"filename": "r3_group.md"}, + {"filename": "thanks.md"} +] diff --git a/2021/2021-04-20_IT101-DM/slides/overview.md b/2021/2021-04-20_IT101-DM/slides/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..6f9143c7c64fa539eb44c3ea5ff9eec076701879 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/overview.md @@ -0,0 +1,25 @@ +## Overview + +0. Introduction - learning objectives + targeted audience +1. Data workflow +1. Ingestion: + * receiving/sending/sharing data + * file naming + * checksums + * backup + 1. making data tidy + * what is table + * + 1. Learning to code workflows and analyses - excel files, coding + 1. Code versioning and reproducibility + 1. Visualization + * see the data + 1. problem solving + * guide + * rubberducking + * google for help + * oracle + 1. R3 team + 1. Acknowledgment + + 1. data minimization diff --git a/2021/2021-04-20_IT101-DM/slides/problem_solving.md b/2021/2021-04-20_IT101-DM/slides/problem_solving.md new file mode 100644 index 0000000000000000000000000000000000000000..904240dd79feb49dddb92bb2865fb35460ff1b71 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/problem_solving.md @@ -0,0 +1,11 @@ +# Problem solving +A guide for solving computing issues + +1. Express the problem + * Write down what you want to achieve +2. Search for help + * Read **FAQs**, **help pages** and the **official documentation** well before turning to Google + * Use stack exchange, forums and related resources (carefully) +3. Ask an expert + * You have to submit the problem in writing + * Make the question interesting diff --git a/2021/2021-04-20_IT101-DM/slides/r3_group.md b/2021/2021-04-20_IT101-DM/slides/r3_group.md new file mode 100644 index 0000000000000000000000000000000000000000..919aa4839ccb71bb9570fdfece3a0041ee1bfd29 --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/r3_group.md @@ -0,0 +1,140 @@ +# Responsible and Reproducible Research (R<sup>3</sup>) +## What is R<sup>3</sup>? + + +A multi-facetted change management +process built on 3 pillars: + +- R3 pathfinder + +- R3 school + +- R3 accelerator + +Common link module: R3 clinic + +<div style="top: -1em; left: 50%; position: absolute;"> + <img src="slides/img/3pillars-full.png"> +</div> + +<br> +<br> +<br> +<br> + +<aside class="notes"> +Pathfinder - policies, finding optimal data management changes<br> +School - courses, howtos, trainnings<br> +Accelerator - advanced teams and their boost/support, CI/CD setup<br> +Clinic - hands-on, meetings in groups, code review + suggestions<br> +</aside> + +## R<sup>3</sup> Training + * LCSB's Monthly Data Management and Data Protection training + * ELIXIR Luxembourg's trainings <br> + https://elixir-luxembourg.org/training + * R<sup>3</sup> school Git basics - every 4 months + <aside class="notes"> + Direct newcommers to this monthly training + </aside> + + + +# Responsible and Reproducible Research (R<sup>3</sup>) +<section data-transition="none" data-background-image="slides/img/r3-training-logo.png" data-background-size="1000px" data-background-opacity="0.1"> +</section> +<div style="display:block;text-align:center;position:relative;" > +<div class="profile-container"> + + * Reinhard Schneider + * <img src="slides/img/R3_profile_pictures/reinhard_schneider.png"> + * Head of Bioinformatics Core + +</div> +<div class="profile-container"> + + * Pinar Alper + * <img src="slides/img/R3_profile_pictures/pinar_alper.png"> + * Datasteward + +</div> +<div class="profile-container"> + + * Yohan Yarosz</li> + * <img src="slides/img/R3_profile_pictures/yohan_yarosz.png"> + * Development + +</div> +<div class="profile-container"> + + * Laurent Heirendt</li> + * <img src="slides/img/R3_profile_pictures/laurent_heirendt.png"> + * Git, CI + +</div> +<div class="profile-container"> + + * Sarah Peter</li> + * <img src="slides/img/R3_profile_pictures/sarah_peter.png"> + * Infrastructure + +</div> +<div class="profile-container"> + + * Valentine Groues</li> + * <img src="slides/img/R3_profile_pictures/valentine_groues.png"> + * Development + +</div> +<div class="profile-container"> + + * Vilem Ded</li> + * <img src="slides/img/R3_profile_pictures/vilem_ded.png"> + * Datasteward + +</div> +<div class="profile-container"> + + * Noua Toukourou</li> + * <img src="slides/img/R3_profile_pictures/noua_toukourou.png"> + * Infrastructure + +</div> +<div class="profile-container"> + + * Alexey Kolodkin</li> + * <img src="slides/img/R3_profile_pictures/alexey_kolodkin.png"> + * Datasteward + +</div> +<div class="profile-container"> + + * Maharshi Vyas</li> + * <img src="slides/img/R3_profile_pictures/maharshi_vyas.png"> + * Infrastructure + +</div> + +<div class="profile-container"> + + * Nene Barry</li> + * <img src="slides/img/R3_profile_pictures/nene_barry.png"> + * Datasteward + +</div> + +<div class="profile-container"> + + * Karim Chaouch</li> + * <img src="slides/img/R3_profile_pictures/karim_chaouch.png"> + * Development + +</div> +<div class="profile-container"> + + * Christophe Trefois + * <img src="slides/img/R3_profile_pictures/christophe_trefois.png"> + * R<sup>3</sup> team lead + +</div> +</div> diff --git a/2021/2021-04-20_IT101-DM/slides/reproducibility.md b/2021/2021-04-20_IT101-DM/slides/reproducibility.md new file mode 100644 index 0000000000000000000000000000000000000000..a73310ceaf1031a2e79e0f55083319e52680f5cf --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/reproducibility.md @@ -0,0 +1,96 @@ +# Reproducibility +* ensures credibility +* key requirement for follow-up and collaborative studies + +<div style="position:absolute"> +<img src="slides/img/reproducibility_nature.png" height="650px"> +</div> + +<div class="fragment" style="position:relative;left:50%"> + +## Why is our workflow not reproducible? + + Lack of provenance: + * Input data downloaded from “some website†+ * Copy & paste operations + * Manual text entry + * Analysis not coded + * Intermediate and final data not generated by deterministic processes +</div> + + + +# Reproducibility +## Learning to code workflows and analyses +<div style="display:inline-grid;grid-gap: 40px;grid-template-columns: auto auto;position:relative;left:12%"> +<div class="fragment"> +<div class="content-box"> +<div class="box-title red">Spreadsheets alone</div> +<div class="content"> + + * Is great for looking at data. + * Data entry is fast. + * Analysis flow is hidden and not in focus. + +</div> +</div> +<div style="text-align:center"> +<img src="slides/img/excel_data-sheet.png" height="280px"> +</div> +</div> + +<div class="fragment"> +<div class="content-box"> +<div class="box-title">Coding</div> +<div class="content"> + + * Is great for controlling analysis + * Data is hidden. + * Flow is visible. +</div> +</div> +<img src="slides/img/code-example.png" height="280px"> +</div> +</div> + +<div class="content-box fragment" style="left:15%;width:60%;position:relative"> +<div class="box-title green">Develop data science skills</div> +<div class="content"> + + * Develop good data management and analysis habits. + * Start coding your analysis within spreadsheets. + * Make yourself familiar with a statistics environment such as R, Python or Matlab + * No need to learn a high level programming language such as C++ or Java. + +</div> +</div> + +</div> + + + +# Table +<div style="position:absolute"> +"Tabular format of data" + +### Header + + * one line! + * **good** names of columns + +### Rows + * represent observations/entities + +### Columns + * represent property of the observations + * one data type +</div> +<div style="left:50%; position:relative; top:-2em"> +<img src="slides/img/excel_data-sheet.png" width="700px"> +<div class="fragment" data-fragment-index="3" style="position:absolute"> +<img src="slides/img/excel_analyses-sheet.jpeg" width="700px"><br> +</div> +<div class="fragment" data-fragment-index="4" style="position:relative"> +<img src="slides/img/red-cross.png" width="700px"><br> +</div> +</div> diff --git a/2021/2021-04-20_IT101-DM/slides/storage_setup.md b/2021/2021-04-20_IT101-DM/slides/storage_setup.md new file mode 100644 index 0000000000000000000000000000000000000000..c4f5e9070cec562fe4ecf14589be1a07c99c647f --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/storage_setup.md @@ -0,0 +1,54 @@ +# Storage set-up + +* Download Anti-virus software +* Regularly update your SW/OS +* Encrypt movable media + +### Passwords + +* Strong passwords +* Password manager +* Safe password exchange channels +* Expiration time on password share + +### Backup + * take care of your own backups! + * don't work on your backup copy! + * minimum is <b>3-2-1 backup rule</b> + + +<div style="position:absolute;right:10%;top:10%"> +<img src="slides/img/undraw_secure_server_s9u8.png" height="750px"> +</div> +<div style="position:absolute; width:45%; left:50%; top:28em; text-align:right"> +<a href=" https://howto.lcsb.uni.lu/?policies:LCSB-POL-BIC-02" style="color:grey; font-size:0.8em;">Data Storage and Backup Policy</a> +</div> + + + +# Storage set-up +## Backup - Central IT/LCSB +<div style="position:relative"> +<img src="slides/img/LCSB_storages_backed-up.png" height="750px"> +</div> +<div style="position:absolute;left:65%;top:60%"> + +Server administrators take care of: +* server backups +* LCSB OwnCloud backups +* group/application server backups (not always) + +</div> + + + +# Storage set-up +## Backup - personal research data +<div style="position:relative"> +<img src="slides/img/LCSB_storages_backup.png" height="750px"> +</div> +<div style="position:absolute;left:55%;top:70%"> + +<font color="red">One version should reside on Atlas!</font> + +</div> diff --git a/2021/2021-04-20_IT101-DM/slides/thanks.md b/2021/2021-04-20_IT101-DM/slides/thanks.md new file mode 100644 index 0000000000000000000000000000000000000000..7852f522c4d40df3ff9c9346e8b79dffc1178b3a --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/thanks.md @@ -0,0 +1,42 @@ +# Thank you.<sup> </sup> +<center><img src="slides/img/r3-training-logo.png" height="200px"></center> +<br> +<br> +<br> +<br> +<center> +Contact us if you need help: + +<a href="mailto:lcsb-r3@uni.lu">lcsb-r3@uni.lu</a> +</center> +<div style="position:absolute"> +Links: + +HowTo Cards / Policies: https://howto.lcsb.uni.lu/ + +Course Slides: https://courses.lcsb.uni.lu/ + +Internal Presentations: https://presentations.lcsb.uni.lu/ + +LCSB GitLab: https://git-r3lab.uni.lu/ + +HPC: https://hpc.uni.lu/ + +Service Portal: https://service.uni.lu/sp + +LCSB intranet: https://intranet.uni.lux +</div> +<div style="position:relative;top:1.5em;left:55%;width:45%"> +Avalable SW and tools: +<div style="margin-left: 20px;"> +SIU managed: + +  - Service Portal > All Catalogs > IT > Softwares +</div> +<div style="margin-left: 20px;"> +LCSB managed: + +  - Service Portal > Knowledge > FAQ - Corporate Software\ +  - LCSB intranet > Science tab > Tools +</div> +</div> diff --git a/2021/2021-04-20_IT101-DM/slides/visualization.md b/2021/2021-04-20_IT101-DM/slides/visualization.md new file mode 100644 index 0000000000000000000000000000000000000000..78847e73a297f77e3753623d7e546dcf499d4e2f --- /dev/null +++ b/2021/2021-04-20_IT101-DM/slides/visualization.md @@ -0,0 +1,21 @@ +# Visualization +<center> + +**Plot your data!** +<figure> + <img src="slides/img/DinoSequentialSmaller.gif" height="500px"> + <blockquote>"never trust summary statistics alone; always visualize your data"</blockquote> + <figcaption>--Alberto Cairo</figcaption> +</figure> +</center> + + + +# Visualization +<center> + +**Plot your data!** +<figure> + <img src="slides/img/plot-data.png" height="800px"> +</figure> +</center> diff --git a/2021/2021-06-08_IT101-DM/slides/code_versioning.md b/2021/2021-06-08_IT101-DM/slides/code_versioning.md new file mode 120000 index 0000000000000000000000000000000000000000..b5c7e06fec7c70db4c46438f42df5aada0c4e3b6 --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/code_versioning.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/code_versioning.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/data-housekeeping.md b/2021/2021-06-08_IT101-DM/slides/data-housekeeping.md new file mode 120000 index 0000000000000000000000000000000000000000..6146c0087b6fb77fe25b41811ba800db5a2f6f45 --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/data-housekeeping.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/data-housekeeping.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/data-introduction.md b/2021/2021-06-08_IT101-DM/slides/data-introduction.md new file mode 120000 index 0000000000000000000000000000000000000000..a38b5e87a9dad6508aabfeb86be6ee262cfe2dfc --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/data-introduction.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/data-introduction.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/data_flow.md b/2021/2021-06-08_IT101-DM/slides/data_flow.md new file mode 120000 index 0000000000000000000000000000000000000000..b7a2b3166efee639efba7cbf54e7f69b9d69f53f --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/data_flow.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/data_flow.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/fair-principles.md b/2021/2021-06-08_IT101-DM/slides/fair-principles.md new file mode 120000 index 0000000000000000000000000000000000000000..cb5bc476d4cec8f2137074d0b3f71da0653d415a --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/fair-principles.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/fair-principles.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/howtos.md b/2021/2021-06-08_IT101-DM/slides/howtos.md new file mode 120000 index 0000000000000000000000000000000000000000..4ff818a5d3cb5e82adddc255ff18c2cf4fc89ae5 --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/howtos.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/howtos.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/img b/2021/2021-06-08_IT101-DM/slides/img new file mode 120000 index 0000000000000000000000000000000000000000..ad68bb48c4730adc2a31dfcacfca1ae5bebc8311 --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/img @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/img \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/index.md b/2021/2021-06-08_IT101-DM/slides/index.md new file mode 100644 index 0000000000000000000000000000000000000000..e8d2e30cab782477b3e2ecd19baa5085758fc09c --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/index.md @@ -0,0 +1,20 @@ +# IT101 - Working with computers +<br>IT101 - Working with computers<br> +## June 8th, 2021 + +<div style="top: 6em; left: 0%; position: absolute;"> + <img src="theme/img/lcsb_bg.png"> +</div> + +<div style="top: 5em; left: 60%; position: absolute;"> + <img src="slides/img/r3-training-logo.png" height="200px"> + <br><br><br><br> + <h3></h3> + <br><br><br> + <h4> + Vilem Ded<br> + Data Steward<br> + vilem.ded@uni.lu<br> + <i>Luxembourg Centre for Systems Biomedicine</i> + </h4> +</div> diff --git a/2021/2021-06-08_IT101-DM/slides/ingestion.md b/2021/2021-06-08_IT101-DM/slides/ingestion.md new file mode 120000 index 0000000000000000000000000000000000000000..57dfd695c79678991180d8e7cbd8a899098989ec --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/ingestion.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/ingestion.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/introduction.md b/2021/2021-06-08_IT101-DM/slides/introduction.md new file mode 120000 index 0000000000000000000000000000000000000000..f4837a3cd8c480b8e28ff81a28ba9d69228b8524 --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/introduction.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/introduction.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/list.json b/2021/2021-06-08_IT101-DM/slides/list.json new file mode 120000 index 0000000000000000000000000000000000000000..df07566dc8dc33d5b4f759a23c0abbac0d2ea3f8 --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/list.json @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/list.json \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/overview.md b/2021/2021-06-08_IT101-DM/slides/overview.md new file mode 120000 index 0000000000000000000000000000000000000000..50f8000fb02fa74861ecd6e1f32132f206276317 --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/overview.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/overview.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/problem_solving.md b/2021/2021-06-08_IT101-DM/slides/problem_solving.md new file mode 100644 index 0000000000000000000000000000000000000000..f34e0a2bc914e35010f492936be5a9506c8ecf27 --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/problem_solving.md @@ -0,0 +1,11 @@ +# Problem solving +A guide for solving computing issues + +1. Express the problem + * Write down what you want to achieve +2. Search for help + * Read **FAQs**, **help pages** and the **official documentation** well before turning to Google + * Use stack exchange, forums and related resources (carefully) +3. Ask an expert + * Submit the problem in writing + * Make the question interesting diff --git a/2021/2021-06-08_IT101-DM/slides/r3_group.md b/2021/2021-06-08_IT101-DM/slides/r3_group.md new file mode 100644 index 0000000000000000000000000000000000000000..9005ade96c9eaf2a31864ab2dfd0f465fc16c8ba --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/r3_group.md @@ -0,0 +1,140 @@ +# Responsible and Reproducible Research (R<sup>3</sup>) +## What is R<sup>3</sup>? + + +A multi-facetted change management +process built on 3 pillars: + +- R3 pathfinder + +- R3 school + +- R3 accelerator + +Common link module: R3 clinic + +<div style="top: -1em; left: 50%; position: absolute;"> + <img src="slides/img/3pillars-full.png"> +</div> + +<br> +<br> + +<aside class="notes"> +Pathfinder - policies, finding optimal data management changes<br> +School - courses, howtos, trainnings<br> +Accelerator - advanced teams and their boost/support, CI/CD setup<br> +Clinic - hands-on, meetings in groups, code review + suggestions<br> +</aside> + +## R<sup>3</sup> Training + * LCSB's Monthly Data Management and Data Protection training + * ELIXIR Luxembourg's trainings <br> + https://elixir-luxembourg.org/training + * Reproducible analysis workflows with Snakemake and Conda (2021-07-14) + * Best practices in research data management and stewardship (2021-06-14) + * R<sup>3</sup> school Git basics - every 4 months + <aside class="notes"> + Direct newcommers to this monthly training + </aside> + + + +# Responsible and Reproducible Research (R<sup>3</sup>) +<section data-transition="none" data-background-image="slides/img/r3-training-logo.png" data-background-size="1000px" data-background-opacity="0.1"> +</section> +<div style="display:block;text-align:center;position:relative;" > +<div class="profile-container"> + + * Reinhard Schneider + * <img src="slides/img/R3_profile_pictures/reinhard_schneider.png"> + * Head of Bioinformatics Core + +</div> +<div class="profile-container"> + + * Pinar Alper + * <img src="slides/img/R3_profile_pictures/pinar_alper.png"> + * Datasteward + +</div> +<div class="profile-container"> + + * Yohan Yarosz</li> + * <img src="slides/img/R3_profile_pictures/yohan_yarosz.png"> + * Development + +</div> +<div class="profile-container"> + + * Laurent Heirendt</li> + * <img src="slides/img/R3_profile_pictures/laurent_heirendt.png"> + * Git, CI + +</div> +<div class="profile-container"> + + * Sarah Peter</li> + * <img src="slides/img/R3_profile_pictures/sarah_peter.png"> + * Infrastructure + +</div> +<div class="profile-container"> + + * Valentine Groues</li> + * <img src="slides/img/R3_profile_pictures/valentine_groues.png"> + * Development + +</div> +<div class="profile-container"> + + * Vilem Ded</li> + * <img src="slides/img/R3_profile_pictures/vilem_ded.png"> + * Datasteward + +</div> +<div class="profile-container"> + + * Noua Toukourou</li> + * <img src="slides/img/R3_profile_pictures/noua_toukourou.png"> + * Infrastructure + +</div> +<div class="profile-container"> + + * Alexey Kolodkin</li> + * <img src="slides/img/R3_profile_pictures/alexey_kolodkin.png"> + * Datasteward + +</div> +<div class="profile-container"> + + * Maharshi Vyas</li> + * <img src="slides/img/R3_profile_pictures/maharshi_vyas.png"> + * Infrastructure + +</div> + +<div class="profile-container"> + + * Nene Barry</li> + * <img src="slides/img/R3_profile_pictures/nene_barry.png"> + * Datasteward + +</div> + +<div class="profile-container"> + + * Karim Chaouch</li> + * <img src="slides/img/R3_profile_pictures/karim_chaouch.png"> + * Development + +</div> +<div class="profile-container"> + + * Christophe Trefois + * <img src="slides/img/R3_profile_pictures/christophe_trefois.png"> + * R<sup>3</sup> team lead + +</div> +</div> diff --git a/2021/2021-06-08_IT101-DM/slides/reproducibility.md b/2021/2021-06-08_IT101-DM/slides/reproducibility.md new file mode 120000 index 0000000000000000000000000000000000000000..8f6e4d1f386bc7a0ca5a6939191cc7a5b7b50f89 --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/reproducibility.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/reproducibility.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/storage_setup.md b/2021/2021-06-08_IT101-DM/slides/storage_setup.md new file mode 120000 index 0000000000000000000000000000000000000000..9fe2d27f633b272789cb2dd57b3d963895e2643d --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/storage_setup.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/storage_setup.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/thanks.md b/2021/2021-06-08_IT101-DM/slides/thanks.md new file mode 120000 index 0000000000000000000000000000000000000000..85dc36b89cdbfa538b1619e7b429499db2196fac --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/thanks.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/thanks.md \ No newline at end of file diff --git a/2021/2021-06-08_IT101-DM/slides/visualization.md b/2021/2021-06-08_IT101-DM/slides/visualization.md new file mode 120000 index 0000000000000000000000000000000000000000..d946af686d2d755178f317b134b6f102f7d52e0a --- /dev/null +++ b/2021/2021-06-08_IT101-DM/slides/visualization.md @@ -0,0 +1 @@ +../../2021-04-20_IT101-DM/slides/visualization.md \ No newline at end of file