diff --git a/.ci/generator b/.ci/generator index d73c076d4d529517a1457fc4a7a09d8906b6a13c..53a922a13c4e8916c97599b5242493a47ea0a2e2 160000 --- a/.ci/generator +++ b/.ci/generator @@ -1 +1 @@ -Subproject commit d73c076d4d529517a1457fc4a7a09d8906b6a13c +Subproject commit 53a922a13c4e8916c97599b5242493a47ea0a2e2 diff --git a/cards.md b/cards.md index 36659543c0d49db911699ba455005a8a0bbddcea..f812acce2c1e823cd0d8f5ea57e56e90da2cb00a 100644 --- a/cards.md +++ b/cards.md @@ -67,11 +67,13 @@ order: -1 <div class="index-box noborderbox" id="exchange-channels-card"> <h3>Exchange channels</h3> <ul> - <li><a href="{{ 'external/exchange-channels/lft' | relative_url }}">LCSB file transfer (LFT) Quick Guide</a></li> - <li><a href="{{ 'external/exchange-channels/calendar' | relative_url }}">Sharing calendar in Microsoft Exchange</a></li> <li><a href="{{ 'external/exchange-channels/asperaweb' | relative_url }}">AsperaWEB Quick Guide</a></li> - <li><a href="{{ 'external/exchange-channels/owncloud' | relative_url }}">Owncloud</a></li> + <li><a href="{{ 'external/exchange-channels/atlas-hpc' | relative_url }}">Data transfer between Atlas and UL HPC Clusters</a></li> + <li><a href="{{ 'external/exchange-channels/calendar' | relative_url }}">Sharing calendar in Microsoft Exchange</a></li> <li><a href="{{ 'external/exchange-channels/cryptomator' | relative_url }}">Cryptomator</a></li> + <li><a href="{{ 'external/exchange-channels/lft' | relative_url }}">LCSB file transfer (LFT) Quick Guide</a></li> + <li><a href="{{ 'external/exchange-channels/owncloud' | relative_url }}">Owncloud</a></li> + </ul> </div> <div class="index-box noborderbox" id="general-card"> @@ -122,6 +124,13 @@ order: -1 </ul> </div> + <div class="index-box noborderbox" id="lab-good-practice-card"> + <h3>Lab: Good Practice</h3> + <ul> + <li><a href="{{ 'external/lab-good-practice/mycoplasma' | relative_url }}">Mycoplasma contamination check</a></li> + + </ul> + </div> <div class="index-box noborderbox" id="lab-hsa-card"> <h3>Lab: Health & Safety, Access</h3> <ul> @@ -134,7 +143,7 @@ order: -1 <li><a href="{{ 'external/lab-hsa/shipment' | relative_url }}">Shipment of biological or chemical samples with carrier</a></li> <li><a href="{{ 'external/lab-hsa/spill' | relative_url }}">Spill in a laboratory</a></li> <li><a href="{{ 'external/lab-hsa/spill-bsc' | relative_url }}">Spill in a Biosafety Cabinet</a></li> - <li><a href="{{ 'external/lab-hsa/waste' | relative_url }}">Chemical and Biological Waste Management</a></li> + <li><a href="{{ 'external/lab-hsa/waste' | relative_url }}">Waste Management: Chemical and Biological waste</a></li> </ul> </div> diff --git a/external/exchange-channels/atlas-hpc/atlas-hpc.md b/external/exchange-channels/atlas-hpc/atlas-hpc.md new file mode 100644 index 0000000000000000000000000000000000000000..a2945058077e5486e1af706d514860fbbc916353 --- /dev/null +++ b/external/exchange-channels/atlas-hpc/atlas-hpc.md @@ -0,0 +1,42 @@ +--- +layout: page +permalink: /external/integrity/transfer/atlas-hpc/ +shortcut: integrity:transfer:atlas-hpc +redirect_from: + - /cards/integrity/transfer:atlas-hpc + - /external/integrity/transfer:atlas-hpc + - /integrity/transfer/atlas-hpc + - /external/external/integrity/transfer/atlas-hpc/ +--- + +# Data transfer between Atlas and UL HPC Clusters +A recommended storage pattern is to have the master copy of data on Atlas (project folder) and only store data on the UL HPC Clusters temporarily for the required practical duration of computational analysis. The derived data and results should hereafter be transferred back to Atlas. This How-to Card describes the different methods to transfer data between Atlas and the UL HPC Clusters. The three recommended methods to transfer data are: + +1. [Via laptop with ```scp``` or ```rsync```](#1. Via laptop using scp or rsync) +2. [Via dedicated Virtual Machine (VM)](#2. Via dedicated Virtual Machine (VM) using rsync) +3. [Via Large File Transfer (LFT)](#3. Via Large File Transfer (LFT)) + +Please refer to the dedicated knowledge bases to see how to [connect to UL HPC Clusters](https://hpc-docs.uni.lu/connect/access/) and to [mount Atlas](https://service.uni.lu/sp?id=kb_article_view&sysparm_article=KB0010233). + +<img src="img/data-transfer-flow.png"> + +## 1. Via laptop using `scp` or `rsync` +When using the UL laptop to transfer data between UL HPC Clusters and Atlas, you must [mount Atlas via smb on laptop](https://service.uni.lu/sp?id=kb_article_view&sysparm_article=KB0010233) before using `scp` and `rsync` for the transfer. While both commands ensure a secure transfer of data between the UL HPC Clusters and Atlas, `rsync` may be much faster for handling large amounts of small files (which are transferred very quickly in batches), and for selective incremental updates of large datasets (it is capable to automatically transferring only the changed files, thus saving time). +* `scp`: transfers all files and directories. +* `rsync`: transfers only the files which differ between the source and the destination. + +Please visit the [UL HPC documentation](https://hpc-docs.uni.lu/data/transfer/#data-transfer-tofromwithin-ul-hpc-clusters) to see how to use `rsync` and `scp`. + +## 2. Via dedicated Virtual Machine (VM) using rsync +Data can be transferred via a dedicated VM, which can be requested via [ServiceNow](https://service.uni.lu/sp?id=sc_cat_item&table=sc_cat_item&sys_id=49956812db3fa010ca53454039961978). +Instead of transferring data between Atlas and UL HPC Clusters through the laptop as described above, the transfer will go through the dedicated VM. Once connected to the VM and mounted to Atlas, the ```rsync``` command can be used in the same way as described in the [UL HPC documentation](https://hpc-docs.uni.lu/data/transfer/#data-transfer-tofromwithin-ul-hpc-clusters). This method is recommended for **recurring transfers of very large datasets** that benefit from high-speed network connection between the VM and the HPC. + + +**Note**: For larger transfers between Atlas and UL HPC Clusters, you may want to run the operations in background using `screen` or `tmux`. These prevent interruption of data transfer in cases when your `ssh` connection gets interrupted. + +## 3. Via Large File Transfer (LFT) +An alternative solution is to use LFT for transferring data between Atlas and UL HPC Clusters. This method can reliably transfer large data volumes (typically several terabytes). However, LFT can only be used if the data is already on LFT (e.g., received from external collaborators). In this case, you can make a copy of the data and directly download it to the UL HPC Clusters for computational analysis. Note that a **master copy of the data must still be manually uploaded to Atlas** for internal archival. + +Please refer to the [dedicated How-to Card on LFT]({{'/?exchange-channels:lft' | relative_url }}) for detailed information. + +**Note**: In cases when the analysis data are not already received via LFT, we strongly recommend to use one of the other (simpler) methods instead. diff --git a/external/exchange-channels/atlas-hpc/img/data-transfer-flow.png b/external/exchange-channels/atlas-hpc/img/data-transfer-flow.png new file mode 100644 index 0000000000000000000000000000000000000000..72da75ad15c6c19c48e32296a67d90ce5adc648f Binary files /dev/null and b/external/exchange-channels/atlas-hpc/img/data-transfer-flow.png differ diff --git a/external/lab-equipment/cold-traps/cold-traps.md b/external/lab-equipment/cold-traps/cold-traps.md index 46271b9422d5754d29339e1ca507eb29438baaae..859da9bea5154b9c39f1abc555f698f46ced2991 100644 --- a/external/lab-equipment/cold-traps/cold-traps.md +++ b/external/lab-equipment/cold-traps/cold-traps.md @@ -1,6 +1,6 @@ --- layout: page -permalink: /external/lab-equipment/cold-traps +permalink: /external/lab-equipment/cold-traps/ shortcut: lab:cold-traps redirect_from: - /cards/lab-equipment:maintenance_of_cold_traps @@ -89,4 +89,4 @@ When necessary, the acid cartridge (LIMS ID: 10608) should be exchanged * Remove the cartridge and replace it with the new one * Close back the container * Place the old cartridge in the chemical hood for evaporation in a retention container -* 24 hours later, trash it in a blue bin UN3291 \ No newline at end of file +* 24 hours later, trash it in a blue bin UN3291 diff --git a/external/lab-good-practice/mycoplasma/mycoplasma.md b/external/lab-good-practice/mycoplasma/mycoplasma.md index 4927bf8f3770abf69b3f4a4c07c77f89d364d278..0f146d60834345ad42ad9e7e4605a12e83a79cf8 100644 --- a/external/lab-good-practice/mycoplasma/mycoplasma.md +++ b/external/lab-good-practice/mycoplasma/mycoplasma.md @@ -11,7 +11,7 @@ redirect_from: - /external/cards/lab:mycoplasma --- -# Mycoplasam contamination check +# Mycoplasma contamination check Mycoplasma is a genus of bacteria that belongs to the class Mollicutes. What sets mycoplasmas apart from many other bacteria is their lack of a cell wall. The absence of a rigid cell wall makes them resistant to certain antibiotics that target cell wall synthesis. Meaning that even if you have antibiotics in your cell culture media, you can have a mycoplasma contamination.