Changes for page Getting Started with HDC

Last modified by Dennis Segebarth on 2025/02/24 16:29
From version 7.1
edited by Dennis Segebarth
on 2025/02/24 16:27
Change comment: There is no comment for this version
To version 3.4
edited by Dennis Segebarth
on 2024/12/18 11:33
Change comment: clear view right for XWiki.XWikiGuest

Summary

Details

Page properties
Content
... ... @@ -6,7 +6,7 @@
6 6  
7 7  == Creating a User Account ==
8 8  
9 -All active EBRAINS users can use their EBRAINS credentials to create a user account in the HealthDataCloud central node platform. If you don’t already have a valid EBRAINS account, please visit the [[EBRAINS registration page>>url:https://ebrains.eu/register/]] to obtain one.
9 +All active EBRAINS users can use their EBRAINS credentials to create a user account on HealthDataCloud. The basic user account allows you to browse the Portal and see the list of publicly visible Projects. Accessing an HDC Project and using the Project’s data storage and computing resources requires invitation from a Project Administrator. If you’re a Principal Investigator who is interested in setting up a Project on HDC, please refer to the section //Obtaining your own HDC Project// later in this article//.//
10 10  
11 11  To create an HDC user account:
12 12  
... ... @@ -13,105 +13,17 @@
13 13  * Visit the HDC Portal at [[https:~~/~~/hdc.humanbrainproject.eu/>>url:https://hdc.humanbrainproject.eu/]] and click the **Login** button.
14 14  * (((
15 15  Click the **EBRAINS **button and enter your EBRAINS username and password.
16 +
17 +* If you don’t already have a valid EBRAINS account, please visit the [[EBRAINS registration page>>url:https://ebrains.eu/register/]] to obtain one, then return to the [[HealthDataCloud Portal>>url:https://hdc.humanbrainproject.eu]] and log in with your EBRAINS username and password.
16 16  )))
17 17  * (((
18 18  Review and familiarize yourself with the **Terms of Use**, then click **Accept **to finalize your registration.
19 19  
20 -* Note, the **Accept **button isn’t activated until you scroll down to the end of the document. You may export the Terms of Use as a PDF file or review them at any time from both the [[HDC website>>url:https://hdc.humanbrainproject.eu]] or at the bottom left corner (“Terms of Use”) of each page after logging into the HDC.
22 +* Note, the **Accept **button isn’t activated until you scroll down to the end of the document. You may export the Terms of Use as a pdf file or review them at any time from both the [[HDC website>>url:https://hdc.humanbrainproject.eu]] or at the bottom left corner (“Terms of Use”) of each page after logging into the HDC.
21 21  )))
22 22  
23 23  Your personal information is collected to provide a user account and access to HDC services in accordance with the [[HDC Privacy Policy>>url:https://object.hdc.humanbrainproject.eu/public-resources/HDC-Privacy-Policy.pdf]].
24 24  
25 -== Discovering the HDC - a Quick Start Guide ==
26 -
27 -With the release of Pilot-HDC v1.3, you can now discover all features of the HealthDataCloud central node platform on a self-serve basis. Here, we prepared a quick start guide that you can follow along, or explore the platform on your own - it´s now easier than ever!
28 -
29 -(% class="box warningmessage" %)
30 -(((
31 -Important: Please note that the HDC central node platform has not yet undergone a GDPR audit and you are, therefore, not allowed to upload any sensitive data to the platform.
32 -)))
33 -
34 -=== Join the HealthDataCloud Test Project ===
35 -
36 -Before being able to test the platforms features, you have to join one of its Projects. A **Project** is an isolated, access-controlled data management unit in HDC that holds all of the data and computational resources relating to a research project. Projects are owned by a **Project Administrator**,** **a role analogous to an academic Principal Investigator. Usually, joining a Project requires the corresponding Project Admin to invite you to join. However, for testing and discovering of the HDC features, we created the **HealthDataCloud Test Project**, which you can join yourself at any moment by the click of a button. To do so:
37 -
38 -1. Go to the **Projects** page using the the top-level navigation bar:
39 -\\[[image:Projects_navigation_bar.png||height="80" width="839"]]
40 -
41 -1. Click on the **Join Starting Project** button:
42 -[[image:Join_starting_project.png||height="291" width="827"]]
43 -
44 -1. A pop-out window will open and confirm that you successfully joined the HealthDataCloud Test Project with the role of a Project Collaborator (see [[this section>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Managing%20HDC%20Projects/#HProjectRoles]] of our User Guide for more information about the different roles and associated permissions) and remind you again to **not upload any sensitive information or data **to this Project:
45 -\\[[image:Screenshot 2024-11-25 104224.png||height="294" width="458"]]
46 -
47 -You can now proceed to the landing page of the HealthDataCloud Test Project, either by clicking on the **Go to Project** button in the pop-out, or via the **Projects** page using the navigation bar, where the HealthDataCloud Test Project will now be listed:
48 -
49 -[[image:HDC_test_project_on_Projects_page.png||height="283" width="869"]]
50 -
51 -
52 -The landing page of the HealthDataCloud Test Project will look like this (please refer to [[this section>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/HDC%20Portal%20Navigation/#HProjectsInterface]] of our User Guide article to learn more about the Projects Interface):
53 -
54 -[[image:Landing_page_HDC_test_project.png||height="430" width="872"]]
55 -
56 -
57 -(% class="box infomessage" %)
58 -(((
59 -Access permissions within the HDC are tied to your EBRAINS account and setting of new permissions may sometimes take a while to be updated. This can sometimes cause an "Access denied" error, when you try to access the HealthDataCloud Test Project after completing the steps above. To enforce an update of your permissions in your EBRAINS Account, simply logout and log back in to the HDC central node platform. The associated permissions of your EBRAINS account are now refreshed and you will be able to access the HealthDataCloud Test Project.
60 -)))
61 -
62 -=== Learn about Zone Restrictions ===
63 -
64 -As a Research Data Management Platform, the central features of the HDC evolve around data flows, and how compliance with data privacy regulations can be ensured by default and by design. In this quick start guide, we´ll help you gain a general understanding of the Platforms design and access restrictions that apply in it´s different zones. Once you complete this guide, however, we highly recommend you have a look at our in-depth articles about the respective topics (for instance, see [[Working with Project Files in the Portal>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Project%20Files%20in%20the%20Portal/]]), which introduce many additional features we won´t touch upon here.
65 -
66 -Any data that is uploaded to the platform first lands in the //Green Room// zone of the respective Project. The //Green Room// is a dedicated and highly isolated storage area within each Project, where only the Project Administrator(s) and the user who uploaded the data have access to it. This allows the Project Administrator(s) to verify that the data are compliant with the respective Project’s data management plan before being copied to the //Core //zone of the Project. This is important, as data in the //Core// automatically becomes accessible to all members who have at least the Project Collaborator role (see [[this section>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Managing%20HDC%20Projects/#HProjectRoles]] of our User Guide for more information about the different roles and associated permissions). Data that was transferred into the //Core// zone of a Project can then also be organized into Datasets. In brief, Datasets are collections of related Project files, folders, and detailed metadata. The Dataset feature allows users to organize Project files, enrich with annotations, validate data structures, and publish controlled versions for sharing (see [[this article>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Datasets/]] of our User Guide for more information about Datasets).
67 -
68 -The HDC also offers Workspace Tools, i.e. direct integration of computational tools like JupyterHub that allow you to view, process, analyze, or annotate and describe your data. In general, Workspace Tools can be deployed in both //Green Room //and //Core// zones, depending on the Projects needs. In the HealthDataCloud Test Project, however, all Workspace Tools (the virtual programming environment JupyterHub, the business intelligence tool Superset, the Virtual Machine interface Guacamole, and the documentation tool XWiki) are deployed in the //Core// zone.
69 -
70 -=== Upload Data ===
71 -
72 -Now that you have a high-level understanding of the different zones within the HDC, let´s start by uploading some test data to the platform via the it´s web interface, i.e. the Portal. Please be reminded again, however, to abide by the Platform Terms of Use and any Project-specific restrictions when using the Portal, and - specifcally - that **you are not allowed to upload any sensitive data to the HealthDataCloud Test Project**. To upload data:
73 -
74 -1. Go to the **File Explorer// //**of the HealthDataCloud Test Project:
75 -\\[[image:File_explorer.png||height="407" width="827"]]
76 -
77 -1. You automatically land in your home directory within the //Green Room// zone, which is also indicated in the UI. Feel free to create a new folder and, once you selected the destination folder to which you´d like to upload, please click on **Upload:**
78 -\\[[image:Upload.png||height="319" width="830"]]
79 -
80 -1. A pop-out window will open and prompt you to specify the file(s) or folder(s) you´d like to upload. Once you selected data to upload, click on **Upload** to start the upload, or click on **Cancel** to abort. Once more: **Please remember that you are not allowed to upload sensitive data to the HealthDataCloud Test Project.**
81 -\\[[image:1740389127404-790.png]]
82 -
83 -1. You can check the progress of the upload by clicking on the **File Status Icon**. Please refer to [[this article>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Project%20Files%20in%20the%20Portal/]] in our User Guide for many more features that are available when uploading data to the HDC, including the [[//Resumable Upload//>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Project%20Files%20in%20the%20Portal/#HResumingFailedUploads]] feature that allows you to continue previous uploads that got interrupted, e.g. due to unstable network connections.
84 -
85 -Congratulations - you now know about the most important and basic functionality of the HDC. As a next step, you´d usually want to copy the newly uploaded data from the //Green Room// to the //Core// of the Project, in order to work on them collaboratively with your team. If you are not a Project Administrator, though, you do not have the permissions to directly copy data between zones. Instead, you will have to open a //Copy to Core Request//, which needs to be approved by a Project Administrator - giving them the chance to validate that the data complies with the Projects Data Management plan (see [[this section>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Project%20Files%20in%20the%20Portal/#HRequestingFileCopyfromtheGreenRoomtotheCore]] in our User Guide for a more detailed description of this process). In our Quick Start Guide, we´ll instead continue with working on some data that already resides in the //Core// of the Project and can be accessed collaboratively.
86 -
87 -=== Working with Data in the //Core// ===
88 -
89 -As outlined in above section //Learn about Zone Restrictions//, data in the //Core// zone of a Project is shared with all Project members that have at least the //Project Collaborator// role to foster collaboration. You can browse all accessible data by using again the **File Explorer**, click on **Core**, and then navigate to the folder of interest. In this Quick Start Guide, we will now introduce how you can leverage Workspace Tools like JupyterHub to process, analyze, and visualize your data. For this,
90 -
91 -1. Click on the **JupyterHub **icon to launch your own server within the HealthDataCloud Test Project:
92 -\\[[image:JHub.png||height="399" width="827"]]
93 -
94 -1. If prompted, select to "Sign in with Keycloak" which allows you to continue using single sign-on (SSO). Next, make sure to select the //Datascience Environment// which will launch your instance of JupyterHub with all the required packages preinstalled. After clicking on start, you can see the status of your server launch indicated by a progress bar:
95 -\\[[image:DataScience.png||height="197" width="836"]]
96 -
97 -1. Once your server has launched, you can use JupyterHub as you´re used to in order to create Jupyter Notebooks or write scripts. For this Quick Start Guide, we´ll continue to use the **Terminal** in order to leverage the platform´s command line interface (CLI), //pilotcli// to move data between your Project and the Workspace Tools:
98 -\\[[image:StartTerminal.png||height="584" width="831"]]
99 -
100 -1. Now you´ll authenticate your session with the //pilotcli// by executing the command {{code language="none"}}pilotcli user login{{/code}} to initiate the login process. This will output a link:
101 -\\[[image:pilotcli_login.png||height="428" width="827"]]
102 -
103 -1. Please use a new browser tab or window to access the link from step 4. Then grant the requested permission to complete your login process:
104 -\\[[image:grant_permissions.png||height="329" width="448"]]
105 -
106 -1. Back in JupyterHub, the Terminal will confirm your successful login, which allows you to perform the final step of this section, which is to copy a demo Juypter Notebook that guides you through the entire remaining process by executing the following command in the JupyterHub Terminal: {{code language="none"}}pilotcli file sync hdctestproject/dsegebarth/tabula_muris_facs/tabula_muris_demo.ipynb .{{/code}}
107 -\\[[image:download_demo_ipynb.png||height="440" width="830"]]
108 -
109 -1. This will download a demo Jupyter Notebook with rich annotations and explanations, guiding you through the process of how a typical analysis workflow within HDC using Jupyter Notebooks could look like. Please feel free to follow along and execute the cells of the notebook, which will download the remaining data as needed.
110 -\\[[image:Demo_NB.png||height="312" width="830"]]
111 -
112 -
113 -There are obviously many more features that the HDC has to offer, which are explained in great detail in other sections of this User Guide. We hope this Quick Start Guide allowed you to gain a good understanding of the core features and functionalities of the HDC and how they can support your collaborative day to day work. For more information on how to request the creation of your own HDC Project that allows you to on-board your team and specify the computational resources needed, please have a look at the following sections.
114 -
115 115  == Obtaining your own HDC Project ==
116 116  
117 117  A **Project** is an isolated, access-controlled data management unit in HDC that holds all of the data and computational resources relating to a research project. Projects are owned by a **Project Administrator**,** **a role analogous to an academic Principal Investigator. If you’re interested in starting up your own HDC Project, please contact the HDC team (see //Contact Us//). Access to HDC is governed by the [[HDC Access Policy>>url:https://object.hdc.humanbrainproject.eu/public-resources/HDC-Access-Policy.pdf]].
DataScience.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -279.5 KB
Content
Demo_NB.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -1.1 MB
Content
HDC_test_project_on_Projects_page.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -328.0 KB
Content
JHub.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -664.8 KB
Content
Join_starting_project.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -257.7 KB
Content
Landing_page_HDC_test_project.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -1010.6 KB
Content
Projects_navigation_bar.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -98.4 KB
Content
Screenshot 2024-11-25 104224.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -68.4 KB
Content
StartTerminal.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -682.6 KB
Content
Welcome_to_HDC_test_project.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -68.4 KB
Content
download_demo_ipynb.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -843.4 KB
Content
grant_permissions.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -213.5 KB
Content
pilotcli_login.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -351.5 KB
Content