Skip to content

GitLab

Commit 0a14705f authored by Pinar Alper's avatar Pinar Alper
Browse files

Merge branch 'issue-19' into develop 

# Conflicts:
#	external/exchange-channels/R3labCARD-exchange-channels.md
parents 1b8f47ca 645a0127
Pipeline #11547 passed with stage
in 1 minute and 11 seconds
Hide whitespace changes
Inline Side-by-side
external/exchange-channels/R3labCARD-exchange-channels.md
View file @ 0a14705f
......@@ -3,65 +3,13 @@ layout: page
permalink: /external/exchange-channels/
---
# Exchanging Research Data with Collaborators
{:.no_toc}
LCSB provides two channels for the exchange of data with research collaborators:
* **LCSB ownCloud**: This is a private cloud storage service for the use of LCSB staff and collaborators. It is suitable for exchanging small-sized data files (up to 1-2 gigabyte). All communication with the LCSB ownCloud server is SSL encrypted.
* **LCSB Download/Upload Manager (DUMA)**: This is an IBM Aspera deployment. It is suitable for large-sized data files (several gigabytes or terabytes). Aspera has a built in end-to-end encryption feature, meaning your data will not only be encrypted during transmission, it will stay encrypted at the other end until an authorised party picks it up.
## Using LCSB ownCloud
A LUMS account is needed in order to use LCSB ownCloud. LUMS accounts for staff are normally created within the first few days of starting work at the LCSB. In addition to staff, researchers at Partner Institutes may be given LUMS accounts. If you have inquiries/requests on LUMS accounts, please create a ticket on [ServiceNow](https://service.uni.lu/) using catalog item **Home/Catalog/LCSB/BioCore**.
Similar to other cloud storage systems, ownCloud is accessible both via a browser and also via a client application. On the web, LCSB's ownCloud is at [https://owncloud.lcsb.uni.lu/](https://owncloud.lcsb.uni.lu/) <br/> ![Alt](owcld_1.png "Title")
You can download the ownCloud client suitable for your staff computer [here](https://owncloud.lcsb.uni.lu/). User documentation on ownCloud tools and portal can be found [here](https://doc.owncloud.com/server/index.html).
LCSB staff should observe the following guidance when using ownCloud:
* Limit folder shares to only the personnel that needs to access data.
* When sharing via Links, always set a **password** and an **expiration date** for the link. <br/> ![Alt](owcld_2.png "Title")
* When sharing via Links, **please do not send the link passwords to collaborators in email**. Instead, use secure password sharing tools listed [here](../../internal/passwords/).
## Using LCSB DUMA
The use of DUMA is mediated by the LCSB Data Stewardship team.
DUMA can be accessed via a web browser or via a command-line utility. In order to use DUMA, create a ticket on [ServiceNow](https://service.uni.lu/) using the catalog item **Home/Catalog/LCSB/BioCore**.
* If you want to make data available for download, then please specify in your ticket the current source location of your data (e.g. HPC, staff computer).
* If collaborators will be uploading data, then please specify in your ticket the target storage location you want your data to arrive (e.g. HPC, staff computer).
* In your ticket also specify whether you will use DUMA via the web or command-line interface.
* Depending on your choice of interface you will receive:
- a **token** (for command line)
- an **access link and a token** (for web)
When sharing DUMA access tokens with collaborators, use password link generator tools listed [here](../../internal/passwords/).
In the following we describe instructions on how to use DUMA web interface. The instructions for using the command-line client can be found [here](https://asperaweb.lcsb.uni.lu/help/-Sc1j1kj_zkkFAVjwKgMwbGMa7_iyiv6Ob7GwKV_)
1. you will receive an **access link** and a **token** from the Data Steward. Access link is a download/upload area that can be reached by you and/or collaborator. You will be prompted for the token to reach the access link (see below). ![Alt](duma_1.png "Title")
2. When you visit a DUMA access link for the first time, you will be prompted to install **IBM Aspera Connect** client.
* click **Download** or **Download latest version** links (see below). ![Alt](duma_2.png "Title")
* Your browser will download the client installer. <br/> ![Alt](duma_3.png "Title")
* Run the installer till you receive the completion message. <br/> ![Alt](duma_4.png "Title")
* Once **IBM Aspera Connect** client is installed, refresh the **access link**. You will be prompted whether you want to open **IBM Aspera Launcher**, click **Open**. <br/> ![Alt](duma_5.png "Title")
3. The **access link** page will display a **File Browser** section. Depending on the settings per access link, users can create or delete folders in the File Browser and upload or download data.<br/> ![Alt](duma_6.png "Title")
4. Clicking **Upload** or **Download** will launch the **IBM Aspera Connect** client on your computer. You first will be asked whether you allow the client to connect to aspera.lcsb.uni.lu. Choose **Allow**.<br/> ![Alt](duma_7.png "Title")
5. At any time you can launch **IBM Aspera Connect** to display the status of uploads to or downloads from your computer. <br/> ![Alt](duma_8.png "Title")
LCSB provides two channels for the exchange of data with research collaborators **ownCloud** and **Data Upload Manager (DUMA)**. This labcard provides information on the use of these exchange channels.
* TOC
{:toc}
{% include_relative owncloud.md %}
{% include_relative duma.md %}
external/exchange-channels/duma.md 0 → 100644
View file @ 0a14705f
# Data Upload Manager - DUMA Quick Guide
{{TOC}}
## Overview
DUMA is an IBM Aspera deployment at the LCSB-University of Luxembourg. DUMA supports end-to-end encrypted data transfer and can handle high data volumes e.g. several tera bytes.
## Obtaining a DUMA access link
You need an **access link** to use DUMA. An access link is a temporary, password-protected space, much like a drop box, on LCSB's DUMA server. In order to obtain an access link, you should contact your research collaborator at the LCSB-University of Luxembourg. Once created, you will receive your **access link** and associated **password** (link) by e-mail.
**IMPORTANT NOTE: Whenever the LCSB shares a password for a DUMA endpoint (collaborator), the password be reachable via a link, which will expire. Therefore you should make a record of your password once you view it.**
An access link can be reached via standard web browsers. Data can be transferred to/from an access link in two ways:
* Through the web browser by visiting the link, which is our recommended way of data transfer, described in this [section of the guide](#DUMA_WEB)
* Through the use of a command line tool. If your data sits in an environment, where you can not launch a web browser, then you may use use a command line client tool to reach an access link. This process is described in this [section of this guide](#DUMA_CLI).
The use of DUMA is mediated by LCSB's data stewards. If you require assistance in using DUMA, you should send an email to [LCSB admin team](lcsb-sysadmins@uni.lu).
<a name="DUMA_WEB"></a>
## Accessing DUMA via Web Interface
In the following steps we provide instructions on how to use DUMA web interface.
1. Once you receive your **access link** and **password** from the LCSB, visit the link using a standard web browser. Firefox 66.x (or higher) recommended, but others should also work. You will be prompted for your password (see below).<br/>
![Alt](duma_1.png "Title")
2. When you visit a DUMA access link for the very first time, you will be prompted to install **IBM Aspera Connect** client.
* click **Download** or **Download latest version** links (see below). <br/>![Alt](duma_2.png "Title")
* Your browser will download the client installer.
![Alt](duma_3.png "Title")
* Run the installer till you receive the completion message.
![Alt](duma_4.png "Title")
* Once **IBM Aspera Connect** client is installed, refresh the **access link**. You will be prompted whether you want to open **IBM Aspera Launcher**, click **Open**.
![Alt](duma_5.png "Title")
3. The **access link** page will display a **File Browser** section. Depending on the settings per access link, users can create or delete folders in the File Browser and upload or download data.<br/>
![Alt](duma_6.png "Title")
4. Clicking **Upload** or **Download** will launch the **IBM Aspera Connect** client on your computer. You first will be asked whether you allow the client to connect to aspera.lcsb.uni.lu. Choose **Allow**. <br/>
![Alt](duma_7.png "Title")
5. At any time you can launch **IBM Aspera Connect** to display the status of uploads to or downloads from your computer. <br/>
![Alt](duma_8.png "Title")
6. All data are encrypted on server side and they stay encrypted also upon download. For decryption, you have to navigate into your **IBM Aspera Connect** window and click "**Unlock encrypted files**". <br/>
![Alt](duma_9.png "Title") <br/>
You will be prompted for encryption passphrase which you should already have received from the repository administrator. <br/>
![Alt](duma_10.png "Title") <br/>
Encrypted files are by default kept on your disc after decryption. If you want to change this behaviour, navigate to Options->Settings and check "Delete encrypted files when finished" box.
<a name="DUMA_CLI"></a>
## Accessing DUMA via Command-Line Tool
In the following steps we describe instructions on how to use DUMA web interface.
1. **Install the aspera command line client.** The _ascp_ tool, which can be found [here](https://downloads.asperasoft.com/en/downloads/62) as instructed.
2. **Obtain tokens.** You will need **tokens** while using _ascp_. You will need a **download token** for download and a separate **upload token** for upload. Your tokens can be viewed by clicking the **Help** icon on your access link (see below).<br/>
![Alt](duma_help_link.png "Title") <br/><br/><br/>
![Alt](duma_cli_tokens.png "Title")
3. **Obtain SSH key.** You will also need an **SSH private key** to authenticate. This is the Aspera key that comes with the Aspera Connect installation named `asperaweb_id_dsa.openssh`. Below are locations where the ssh_key can be found:
* On MAC
* Local installation of Aspera connect:
* `/Users/username/Applications/Aspera Connect.app/Contents/Resources/asperaweb_id_dsa.openssh`
* System wide installation of Connect:
* `/Applications/Aspera Connect.app/Contents/Resources/asperaweb_id_dsa.openssh`
* On WINDOWS
* Local installation of Aspera connect:
* `C:\\Users\username\AppData\Local\Programs\Aspera\Aspera Connect\etc\asperaweb_id_dsa.openssh`
* System wide installation of Connect:
* `C:\\Program Files (x86)\Aspera\Aspera Connect\etc\asperaweb_id_dsa.openssh`
* On LINUX
* Local installation of Aspera connect:
* `/home/username/.aspera/connect/etc/asperaweb_id_dsa.openssh`
* System wide installation of Connect:
* `/opt/aspera/etc/asperaweb_id_dsa.openssh`
4. **Set necessary environment variables**. `KEY` and `TOKEN` environment variables should be set as follows. If you do not have any of the tokens use `None` as the value.
* `KEY=<path to the ssh key>`
* `DTOKEN=<<<<YOUR DOWNLOAD TOKEN>>>>>>`
* `UTOKEN=<<<<YOUR UPLOAD TOKEN>>>>>>`
5. **Run the relevant _ascp_ commands**.
* Upload:
* To upload all *png* files to the root of the share:
* `ascp -d -i $KEY -P 33001 -W $UTOKEN *.png app_duma_public_read@aspera.lcsb.uni.lu:/`
* To upload *document.txt* to a subfolder:
* `ascp -d -i $KEY -P 33001 -W $UTOKEN document.txt app_duma_public_read@aspera.lcsb.uni.lu:/subfolder`
* Download:
* To download *subfolder* and all it's content to the current directory:
* `ascp -d -i $KEY -P 33001 -W $DTOKEN app_duma_public_read@aspera.lcsb.uni.lu:subfolder`
You can find more information also on the [Aspera website](https://downloads.asperasoft.com/en/documentation/62).
external/exchange-channels/owncloud.md 0 → 100644
View file @ 0a14705f
# Owncloud Quickguide
**LCSB ownCloud** is a private cloud storage service for the use of LCSB staff and collaborators. It is suitable for exchanging small-sized files (up to 1-2 gigabyte). All communication with the LCSB ownCloud server is SSL encrypted.
## Obtaining a LUMS account
A LUMS (LCSB User Management System) account is needed in order to use LCSB ownCloud. LUMS accounts for LCSB staff are normally created within the first few days of starting work at the LCSB. In addition to staff, researchers at Partner Institutes may be given LUMS accounts. If you want to create a LUMS account, or require support, you should contact [LCSB admin team](lcsb-sysadmins@uni.lu).
## Using LCSB ownCloud
Similar to other cloud storage systems, ownCloud is accessible both via a browser and also via a client application. On the web, LCSB's ownCloud is at [https://owncloud.lcsb.uni.lu/](https://owncloud.lcsb.uni.lu/)
![LCSB ownCloud web login](owcld_1.png)
You can download the ownCloud client suitable for your staff computer [here](https://owncloud.org/download/). User documentation on ownCloud tools and portal can be found [here](https://doc.owncloud.com/server/index.html).
When sharing research data, one should observe the following guidance using Owncloud:
* Limit folder shares to only the personnel that needs to access data.
* When sharing via Links, always set a **password** and an **expiration date** for the link.
![](owcld_2.png)
* When sharing folders via Links, **the link passwords should not be sent plain in email**. Instead, password links should be used. If you need to generate link passwords can use the [Private bin](https://privatebin.lcsb.uni.lu) service hosted by the LCSB.
**IMPORTANT NOTE: Whenever the LCSB shares an ownCloud folder with you (collaborator), the share password will be reachable via link, which will expire. Therefore you should make a record of your password once you view it.**
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment