Wiki source code of Getting Started with HDC

Last modified by Dennis Segebarth on 2025/02/24 16:29
Show last authors
1 {{box cssClass="floatinginfobox" title="Table of Contents"}}
2 {{toc/}}
3 {{/box}}
4
5 = Accessing HealthDataCloud (HDC) =
6
7 == Creating a User Account ==
8
9 All active EBRAINS users can use their EBRAINS credentials to create a user account in the HealthDataCloud central node platform. If you don’t already have a valid EBRAINS account, please visit the [[EBRAINS registration page>>url:https://ebrains.eu/register/]] to obtain one.
10
11 To create an HDC user account:
12
13 * Visit the HDC Portal at [[https:~~/~~/hdc.humanbrainproject.eu/>>url:https://hdc.humanbrainproject.eu/]] and click the **Login** button.
14 * (((
15 Click the **EBRAINS **button and enter your EBRAINS username and password.
16 )))
17 * (((
18 Review and familiarize yourself with the **Terms of Use**, then click **Accept **to finalize your registration.
19
20 * Note, the **Accept **button isn’t activated until you scroll down to the end of the document. You may export the Terms of Use as a PDF file or review them at any time from both the [[HDC website>>url:https://hdc.humanbrainproject.eu]] or at the bottom left corner (“Terms of Use”) of each page after logging into the HDC.
21 )))
22
23 Your personal information is collected to provide a user account and access to HDC services in accordance with the [[HDC Privacy Policy>>url:https://object.hdc.humanbrainproject.eu/public-resources/HDC-Privacy-Policy.pdf]].
24
25 == Discovering the HDC - a Quick Start Guide ==
26
27 With the release of Pilot-HDC v1.3, you can now discover all features of the HealthDataCloud central node platform on a self-serve basis. Here, we prepared a quick start guide that you can follow along, or explore the platform on your own - it´s now easier than ever!
28
29 (% class="box warningmessage" %)
30 (((
31 Important: Please note that the HDC central node platform has not yet undergone a GDPR audit and you are, therefore, not allowed to upload any sensitive data to the platform.
32 )))
33
34 === Join the HealthDataCloud Test Project ===
35
36 Before being able to test the platforms features, you have to join one of its Projects. A **Project** is an isolated, access-controlled data management unit in HDC that holds all of the data and computational resources relating to a research project. Projects are owned by a **Project Administrator**,** **a role analogous to an academic Principal Investigator. Usually, joining a Project requires the corresponding Project Admin to invite you to join. However, for testing and discovering of the HDC features, we created the **HealthDataCloud Test Project**, which you can join yourself at any moment by the click of a button. To do so:
37
38 1. Go to the **Projects** page using the the top-level navigation bar:
39 \\[[image:Projects_navigation_bar.png||height="80" width="839"]]
40
41 1. Click on the **Join Starting Project** button:
42 [[image:Join_starting_project.png||height="291" width="827"]]
43
44 1. A pop-out window will open and confirm that you successfully joined the HealthDataCloud Test Project with the role of a Project Collaborator (see [[this section>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Managing%20HDC%20Projects/#HProjectRoles]] of our User Guide for more information about the different roles and associated permissions) and remind you again to **not upload any sensitive information or data **to this Project:
45 \\[[image:Screenshot 2024-11-25 104224.png||height="294" width="458"]]
46
47 You can now proceed to the landing page of the HealthDataCloud Test Project, either by clicking on the **Go to Project** button in the pop-out, or via the **Projects** page using the navigation bar, where the HealthDataCloud Test Project will now be listed:
48
49 [[image:HDC_test_project_on_Projects_page.png||height="283" width="869"]]
50
51
52 The landing page of the HealthDataCloud Test Project will look like this (please refer to [[this section>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/HDC%20Portal%20Navigation/#HProjectsInterface]] of our User Guide article to learn more about the Projects Interface):
53
54 [[image:Landing_page_HDC_test_project.png||height="430" width="872"]]
55
56
57 (% class="box infomessage" %)
58 (((
59 Access permissions within the HDC are tied to your EBRAINS account and setting of new permissions may sometimes take a while to be updated. This can sometimes cause an "Access denied" error, when you try to access the HealthDataCloud Test Project after completing the steps above. To enforce an update of your permissions in your EBRAINS Account, simply logout and log back in to the HDC central node platform. The associated permissions of your EBRAINS account are now refreshed and you will be able to access the HealthDataCloud Test Project.
60 )))
61
62 === Learn about Zone Restrictions ===
63
64 As a Research Data Management Platform, the central features of the HDC evolve around data flows, and how compliance with data privacy regulations can be ensured by default and by design. In this quick start guide, we´ll help you gain a general understanding of the Platforms design and access restrictions that apply in it´s different zones. Once you complete this guide, however, we highly recommend you have a look at our in-depth articles about the respective topics (for instance, see [[Working with Project Files in the Portal>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Project%20Files%20in%20the%20Portal/]]), which introduce many additional features we won´t touch upon here.
65
66 Any data that is uploaded to the platform first lands in the //Green Room// zone of the respective Project. The //Green Room// is a dedicated and highly isolated storage area within each Project, where only the Project Administrator(s) and the user who uploaded the data have access to it. This allows the Project Administrator(s) to verify that the data are compliant with the respective Project’s data management plan before being copied to the //Core //zone of the Project. This is important, as data in the //Core// automatically becomes accessible to all members who have at least the Project Collaborator role (see [[this section>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Managing%20HDC%20Projects/#HProjectRoles]] of our User Guide for more information about the different roles and associated permissions). Data that was transferred into the //Core// zone of a Project can then also be organized into Datasets. In brief, Datasets are collections of related Project files, folders, and detailed metadata. The Dataset feature allows users to organize Project files, enrich with annotations, validate data structures, and publish controlled versions for sharing (see [[this article>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Datasets/]] of our User Guide for more information about Datasets).
67
68 The HDC also offers Workspace Tools, i.e. direct integration of computational tools like JupyterHub that allow you to view, process, analyze, or annotate and describe your data. In general, Workspace Tools can be deployed in both //Green Room //and //Core// zones, depending on the Projects needs. In the HealthDataCloud Test Project, however, all Workspace Tools (the virtual programming environment JupyterHub, the business intelligence tool Superset, the Virtual Machine interface Guacamole, and the documentation tool XWiki) are deployed in the //Core// zone.
69
70 === Upload Data ===
71
72 Now that you have a high-level understanding of the different zones within the HDC, let´s start by uploading some test data to the platform via the it´s web interface, i.e. the Portal. Please be reminded again, however, to abide by the Platform Terms of Use and any Project-specific restrictions when using the Portal, and - specifcally - that **you are not allowed to upload any sensitive data to the HealthDataCloud Test Project**. To upload data:
73
74 1. Go to the **File Explorer// //**of the HealthDataCloud Test Project:
75 \\[[image:File_explorer.png||height="410" width="832"]]
76
77 1. You automatically land in your home directory within the //Green Room// zone, which is also indicated in the UI. Feel free to create a new folder and, once you selected the destination folder to which you´d like to upload, please click on **Upload:**
78 \\[[image:Upload.png||height="318" width="827"]]
79
80 1. A pop-out window will open and prompt you to specify the file(s) or folder(s) you´d like to upload. Once you selected data to upload, click on **Upload** to start the upload, or click on **Cancel** to abort. Once more: **Please remember that you are not allowed to upload sensitive data to the HealthDataCloud Test Project.**
81 \\[[image:1740414584737-871.png||height="218" width="381"]]
82
83 1. You can check the progress of the upload by clicking on the **File Status Icon**. Please refer to [[this article>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Project%20Files%20in%20the%20Portal/]] in our User Guide for many more features that are available when uploading data to the HDC, including the [[//Resumable Upload//>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Project%20Files%20in%20the%20Portal/#HResumingFailedUploads]] feature that allows you to continue previous uploads that got interrupted, e.g. due to unstable network connections.
84
85 Congratulations - you now know about the most important and basic functionality of the HDC. As a next step, you´d usually want to copy the newly uploaded data from the //Green Room// to the //Core// of the Project, in order to work on them collaboratively with your team. If you are not a Project Administrator, though, you do not have the permissions to directly copy data between zones. Instead, you will have to open a //Copy to Core Request//, which needs to be approved by a Project Administrator - giving them the chance to validate that the data complies with the Projects Data Management plan (see [[this section>>https://xwiki.hdc.humanbrainproject.eu/bin/view/userguide/Working%20with%20HDC%20Project%20Files%20in%20the%20Portal/#HRequestingFileCopyfromtheGreenRoomtotheCore]] in our User Guide for a more detailed description of this process). In our Quick Start Guide, we´ll instead continue with working on some data that already resides in the //Core// of the Project and can be accessed collaboratively.
86
87 === Working with Data in the //Core// ===
88
89 As outlined in above section //Learn about Zone Restrictions//, data in the //Core// zone of a Project is shared with all Project members that have at least the //Project Collaborator// role to foster collaboration. You can browse all accessible data by using again the **File Explorer**, click on **Core**, and then navigate to the folder of interest. In this Quick Start Guide, we will now introduce how you can leverage Workspace Tools like JupyterHub to process, analyze, and visualize your data. For this,
90
91 1. Click on the **JupyterHub **icon to launch your own server within the HealthDataCloud Test Project:
92 \\[[image:JHub.png||height="399" width="827"]]
93
94 1. If prompted, select to "Sign in with Keycloak" which allows you to continue using single sign-on (SSO). Next, make sure to select the //Datascience Environment// which will launch your instance of JupyterHub with all the required packages preinstalled. After clicking on start, you can see the status of your server launch indicated by a progress bar:
95 \\[[image:DataScience.png||height="197" width="836"]]
96
97 1. Once your server has launched, you can use JupyterHub as you´re used to in order to create Jupyter Notebooks or write scripts. For this Quick Start Guide, we´ll continue to use the **Terminal** in order to leverage the platform´s command line interface (CLI), //pilotcli// to move data between your Project and the Workspace Tools:
98 \\[[image:StartTerminal.png||height="584" width="831"]]
99
100 1. Now you´ll authenticate your session with the //pilotcli// by executing the command {{code language="none"}}pilotcli user login{{/code}} to initiate the login process. This will output a link:
101 \\[[image:pilotcli_login.png||height="428" width="827"]]
102
103 1. Please use a new browser tab or window to access the link from step 4. Then grant the requested permission to complete your login process:
104 \\[[image:grant_permissions.png||height="329" width="448"]]
105
106 1. Back in JupyterHub, the Terminal will confirm your successful login, which allows you to perform the final step of this section, which is to copy a demo Juypter Notebook that guides you through the entire remaining process by executing the following command in the JupyterHub Terminal: {{code language="none"}}pilotcli file sync hdctestproject/dsegebarth/tabula_muris_facs/tabula_muris_demo.ipynb .{{/code}}
107 \\[[image:download_demo_ipynb.png||height="440" width="830"]]
108
109 1. This will download a demo Jupyter Notebook with rich annotations and explanations, guiding you through the process of how a typical analysis workflow within HDC using Jupyter Notebooks could look like. Please feel free to follow along and execute the cells of the notebook, which will download the remaining data as needed.
110 \\[[image:Demo_NB.png||height="312" width="830"]]
111
112 There are obviously many more features that the HDC has to offer, which are explained in great detail in other sections of this User Guide. We hope this Quick Start Guide allowed you to gain a good understanding of the core features and functionalities of the HDC and how they can support your collaborative day to day work. For more information on how to request the creation of your own HDC Project that allows you to on-board your team and specify the computational resources needed, please have a look at the following sections.
113
114 == Obtaining your own HDC Project ==
115
116 A **Project** is an isolated, access-controlled data management unit in HDC that holds all of the data and computational resources relating to a research project. Projects are owned by a **Project Administrator**,** **a role analogous to an academic Principal Investigator. If you’re interested in starting up your own HDC Project, please contact the HDC team (see //Contact Us//). Access to HDC is governed by the [[HDC Access Policy>>url:https://object.hdc.humanbrainproject.eu/public-resources/HDC-Access-Policy.pdf]].
117
118 == Joining a Project ==
119
120 If a Project Administrator has invited you to join an HDC Project and you already have an HDC user account, you’ll receive an email informing you of your new Project membership and role. You’ll be able to access the Project the next time you log in.
121
122 If you received an invitation to join an HDC Project but don’t already have a user account, you can complete your account registration following the instructions in the invitation email (see //Creating a User Account//) and then log in to the Portal and access your Project.
123
124 If you’re a Project Administrator, the Platform Administrator will invite you to your new Project. Consult the article //Managing HDC Projects// for more information.
125
126 == Changing your Password or Forgot Password ==
127
128 Your HDC password is your EBRAINS password.
129
130 * To change your EBRAINS password, sign into [[EBRAINS>>url:https://www.ebrains.eu]] and click Manage Account in the upper right corner.
131 * If you forgot your EBRAINS password, click Forgot Password from the EBRAINS Login window and follow the prompts to create a new password.
132
133 == Logging into the HDC Portal ==
134
135 * (((
136 Open the [[HDC Portal>>url:https://hdc.humanbrainproject.eu/]] in a supported web browser.
137
138 * Popular browsers like Google Chrome, Mozilla Firefox, Microsoft Edge, and Apple Safari are supported. Some older browser versions as well as other browsers like Internet Explorer may not be supported.
139 )))
140 * Click the **Login **button
141 * (((
142 Click the **EBRAINS **button and enter your EBRAINS username and password.
143
144 * If you don’t already have a valid EBRAINS account, please visit the [[EBRAINS registration page>>url:https://ebrains.eu/register/]] to obtain one, then return to the [[HDC Portal>>url:https://hdc.humanbrainproject.eu]] and log in with your EBRAINS username and password.
145 )))
146
147 == Viewing your User Profile ==
148
149 Click your username in the top right corner of any page and click **Account **to open your account profile page. The account page displays your user profile information, Project membership, and recent activities relating to your account.
150
151 == Logging out of the HDC Portal ==
152
153 It’s recommended to always log out of the Portal whenever you finish using the platform or step away from your workstation.
154
155 * To log out of your session manually, click your username in the top right of the page and select **Logout**.
156 * Idle sessions are logged out for added security. After a period of inactivity, a warning reminds you that your session is about to expire. If no further action is performed and no active processes are running, your session will be logged out automatically.
157
158 = How to get help =
159
160 == Consult the User Guide ==
161
162 This guide provides information on how to use the features of the HDC platform.
163
164 == Submit a Support Inquiry ==
165
166 * Users logged into the HDC portal can view the Support panel by clicking **Support **from the top right of any page. Here, you will find Frequently Asked Questions (FAQs) and, after scrolling to the bottom of the support panel, a form to submit a Support inquiry.
167 * Anyone can submit an inquiry to the [[EBRAINS support team>>url:https://www.ebrains.eu/contact]].
168
169 == Watch for Upcoming Maintenance updates ==
170
171 The support team announces planned maintenance with notifications displayed after logging into the portal. If you see an Upcoming Maintenance notice, take note of future downtime periods and plan your platform use to minimize the impact on your work.
172
173 == Contact us ==
174
175 * Log in and submit a Support inquiry from the Portal Support section.
176 * Submit a support inquiry to the [[EBRAINS Support Team>>url:https://www.ebrains.eu/contact]].
177
178 = HDC Portal organization =
179
180 The Portal consists of the following areas:
181
182 * Main Menu (common to all pages)
183 * Dashboard
184 * Projects Landing Page
185 * Projects Interface
186 * Datasets
187 * Platform Management (for Platform Administrators only)
188
189 Detailed information on these components is offered in dedicated sections of the User Guide.
190
191 HDC also offers a full-featured Command Line Interface, pilotcli, a binary executable program that provides advanced users with convenient tools for performing file actions and platform-related tasks programmatically. For more information on the Command Line Interface, see //Working with HDC Project Files in the Command Line Interface//.
192
193
194 ----
195
196 Copyright © 2023-2024 [[Indoc Systems>>url:https://www.indocsystems.com]].
197
198 HealthDataCloud is powered by Pilot technology, a product of [[Indoc Systems>>url:https://www.indocsystems.com]].