Changes for page Using JupyterHub in HDC

Last modified by Dennis Segebarth on 2026/02/26 12:16
From version 21.1
edited by Dennis Segebarth
on 2026/02/26 12:16
Change comment: There is no comment for this version
To version 1.6
edited by Susan Evans
on 2023/07/11 14:00
Change comment: (Autosaved)

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.dsegebarth
1 +XWiki.sgevans
Content
... ... @@ -1,17 +10,5 @@
1 -{{box cssClass="floatinginfobox" title="Table of Contents"}}
2 -{{toc depth="2"/}}
3 -{{/box}}
4 -
5 -{{warning}}
6 -**Disclaimer:** This article may contain screenshots from other Pilot-based deployments. Differences only apply to variations in branding and color schemes, not the functionalities covered in this article, though.
7 -{{/warning}}
8 -
9 -
10 10  JupyterHub is an open-source, multi-user version of Jupyter Notebook for performing analysis of Project files in the Core. More information can be found in the application documentation [[https:~~/~~/jupyter.org/>>https://jupyter.org/]].
11 11  
12 -[[image:1723798240534-407.png||height="189" width="291"]]
13 -
14 -
15 15  = How it Works =
16 16  
17 17  JupyterHub allows Project members to create or import Jupyter Notebooks into the Project Workspace environment, retrieve Project files from the Core, perform computational workflows on the data, and write the outputs back to the Core where they can be accessed by other Project members. JupyterHub spins up a new JupyterLab instance for each Project member.
... ... @@ -29,13 +29,9 @@
29 29  
30 30  JupyterHub is configured at the time of Project Setup. If you launch JupyterHub and receive a notice that it hasn’t been deployed for your project, please contact your Platform Administrator.
31 31  
32 -{{info}}
33 -If you access JupyterHub of the HealthDataCloud Test Project, please be aware that the resources are limited for each user to: 2 GB of persistent storage volume, 4 GB memory, and a single CPU. These limitations can easily be adjusted for new Projects.
34 -{{/info}}
35 -
36 36  = Launching JupyterHub =
37 37  
38 -[[image:1723798257792-201.png||height="121" width="349"]]
22 +[[image:HDC Project Workspace tool navigation Jupyterhub v1.0.0 2023-05-25.png||height="10%" width="30%"]]
39 39  
40 40  1. Launch your Project and click the **JupyterHub icon** in the left menu bar.
41 41  1. Click **Sign in with Keycloak** to initiate your session. JupyterHub automatically authenticates with your existing username and password and launches your session - no additional sign-in is required.
... ... @@ -50,7 +50,7 @@
50 50  1. In the Launcher, click the **Python 3 Notebook **icon, or click **File > New > Notebook**.
51 51  1. Create your Notebook.
52 52  
53 -[[image:1723798278604-114.png||height="376" width="865"]]
37 +[[image:Project Workspace Jupyter Create Python Notebook v2.1.6 2023-02-07.png||height="22%" width="50%"]]
54 54  
55 55  = Launching the Terminal =
56 56  
... ... @@ -59,7 +59,7 @@
59 59  1. In the Launcher, click the **Terminal **icon, or click **File > New > Terminal**.
60 60  1. The terminal window opens.
61 61  
62 -[[image:1723798293872-992.png||height="162" width="863"]]
46 +[[image:Project Workspace Jupyter Launch Terminal v2.1.6 2023-02-07.png||height="9%" width="50%"]]
63 63  
64 64  Ubuntu is used to host Jupyter Notebook. Use the command cat /etc/os-release to determine to current version of Ubuntu:
65 65  
... ... @@ -196,12 +196,12 @@
196 196  
197 197  Afterwards, the environment will be listed when you open the Launcher to open a new Jupyter Notebook:
198 198  
199 -[[image:1723798325144-485.png||height="436" width="867"]]
183 +[[image:Project Workspace Jupyter view new Kernel 2023-07-11.png||height="25%" width="50%"]]
200 200  
201 201  
202 202  and also from each opened Notebook, e.g., via **Kernel > Change Kernel…** :
203 203  
204 -[[image:1723798338447-469.png||height="317" width="247"]]
188 +[[image:Project Workspace Jupyter Kernel change Kernel dropdown 2023-07-11.png||height="64%" width="50%"]]
205 205  
206 206  = Installing New Python Packages =
207 207  
... ... @@ -291,9 +291,9 @@
291 291  * Open the login window and enter your HDC username and password (i.e. your EBRAINS account credentials).
292 292  * Grant access by clicking **Yes**.
293 293  
294 -[[image:1723798355215-434.png||height="352" width="379"]]
278 +[[image:Pilotcli Jupyter user login Grant Access window v2.4.0 2023-05-25.png||height="46%" width="50%"]]
295 295  
296 -[[image:1723798365454-527.png||height="123" width="376"]]
280 +[[image:Pilotcli Jupyter user login Device Login Successful v2.4.0 2023-05-25.png||height="16%" width="50%"]]
297 297  
298 298  * After successful confirmation, return to the terminal in your JupyterHub browser tab.
299 299  
... ... @@ -356,9 +356,9 @@
356 356  Reminder: Please follow Linux conventions for file management. If your filename contains spaces, wrap it in single or double quotes.
357 357  
358 358  * //Filename~:// “Chemical Tracking Data.csv”
359 -* //Source~:// Project “Indoc Test Project”, “Core” storage zone, folder “collaborator4” {{code}}indoctestproject/collaborator4/Chemical Tracking Data.csv -z core{{/code}}
360 -* //Destination: //user's Home directory in the Guacamole or JupyterHub VM {{code}}.{{/code}}
361 -* //Command group/option: //{{code}}file sync{{/code}}
343 +* //Source~:// Project “Indoc Test Project”, “Core” storage zone, folder “collaborator4” indoctestproject/collaborator4/Chemical Tracking Data.csv -z core
344 +* //Destination: //user's Home directory in the Guacamole or JupyterHub VM .
345 +* //Command group/option: //file sync
362 362  
363 363  {{code language="none"}}
364 364  collaborator4@jupyter-collaborator4:~$ pilotcli file sync indoctestproject/collaborator4/'Chemical Tracking Data.csv' . -z core
... ... @@ -367,60 +367,10 @@
367 367  File has been downloaded successfully and saved to: ./Chemical Tracking Data.csv
368 368  {{/code}}
369 369  
370 -To confirm successful download, type {{code}}ls{{/code}} and verify the file "Chemical Tracking Data.csv" is stored in the Home folder.
371 371  
372 -{{code language="none"}}
373 -collaborator4@jupyter-collaborator4:~$ ls
374 -'Chemical Tracking Data.csv' pilotcli
375 -{{/code}}
376 376  
377 -The file “Chemical Tracking Data.csv” can be viewed in the JupyterHub graphical user interface:
378 378  
379 -[[image:1723798383409-873.png||height="267" width="874"]]
380 380  
358 +To confirm successful download, type ls and verify the file {{code}}Chemical Tracking Data.csv{{/code}} is stored in the Home folder.
381 381  
382 -== Uploading Project Data from JupyterHub using the Pilot Command Line Interface ==
383 -
384 -After analyzing Project data inside the JupyterHub, you can upload the generated outputs back into the Project via the Pilot Command Line Interface.
385 -
386 -=== Example ===
387 -
388 -* //Filename//: Chemical Tracking Data rev.csv
389 -* //Source~:// user's Home directory in JupyterHub {{code}}.{{/code}}
390 -* //Destination//: Project “Indoc Test Project”, folder “collaborator4”, “Core” storage zone,
391 -{{code}}indoctestproject/collaborator4{{/code}} {{code}}-z core{{/code}}
392 -* //Command group/option~:// {{code}}file upload{{/code}}
393 -* //User message// (for upload back to the Core): “my workbench output, no additional sensitive data"
394 -* //Command~:// {{code}}pilotcli file upload ./'Chemical Tracking Data rev.csv' -p{{/code}} {{code}}indoctestproject/collaborator4 -z core -m "my workbench output, no additional sensitive data"{{/code}}
395 -
396 -When uploading data to the Core, you are reminded that you are bypassing the usual Green Room upload workflow. To confirm, type {{code}}y{{/code}} at the prompt, or {{code}}N{{/code}} to cancel.
397 -
398 -{{code language="none"}}
399 -collaborator4@jupyter-collaborator4:~$ pilotcli file upload ./'Chemical Tracking Data rev.csv' -p indoctestproject/collaborator4 -z core -m "my workbench output, no additional sensitive data"
400 -You are about to transfer data directly to the PILOT Core! In accordance with the PILOT Terms of Use, please confirm that you have made your best efforts to
401 -pseudonymize or anonymize the data and that you have the legal authority to transfer and make this data available for dissemination and use within the PILOT .If you
402 -need to process the data to remove sensitive identifiers, please cancel this transfer and upload the data to the Green Room to perform these actions.
403 -To cancel this transfer, enter [n/No]
404 -To confirm and proceed with the data transfer, enter [y/Yes]
405 - [y/N]: y
406 -Starting upload of: ./Chemical Tracking Data rev.csv
407 -Pre-upload complete.
408 -Uploading Chemical Tracking Data rev.csv: |██████████████████████████████ 100% 00:00
409 -Upload Time: 2.92s for 1 files
410 -All uploading jobs have finished.
411 -{{/code}}
412 -
413 -After completing the upload, you can confirm the new file “Chemical Tracking Data rev.csv" exists in the correct directory using the pilotcli file list command and/or in the Portal File Explorer.
414 -
415 -{{code language="none"}}
416 -collaborator4@jupyter-collaborator4:~$ pilotcli file list indoctestproject/collaborator4 -z core
417 -Chemical Tracking Data rev.csv Chemical Tracking Data.csv
418 -{{/code}}
419 -
420 -[[image:1723798397694-530.png||height="217" width="863"]]
421 -
422 -----
423 -
424 -Copyright © 2023-2026 [[Indoc Systems>>url:https://www.indocsystems.com]].
425 -
426 -HealthDataCloud is powered by Pilot technology, a product of [[Indoc Systems>>url:https://www.indocsystems.com]].
360 +[[image:Jupyter downloaded file in Home folder v2.4.11 2023-05-25 1850.png||height="15%" width="50%"]]
1723798240534-407.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -18.0 KB
Content
1723798257792-201.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -42.2 KB
Content
1723798278604-114.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -216.4 KB
Content
1723798293872-992.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -113.5 KB
Content
1723798325144-485.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -423.2 KB
Content
1723798338447-469.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -27.1 KB
Content
1723798355215-434.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -14.1 KB
Content
1723798365454-527.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -10.0 KB
Content
1723798383409-873.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -30.1 KB
Content
1723798397694-530.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.dsegebarth
Size
... ... @@ -1,1 +1,0 @@
1 -166.8 KB
Content
HDC Project Workspace tool navigation Jupyterhub v1.0.0 2023-05-25.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.sgevans
Size
... ... @@ -1,0 +1,1 @@
1 +40.6 KB
Content
Jupyter downloaded file in Home folder v2.4.11 2023-05-25 1850.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.sgevans
Size
... ... @@ -1,0 +1,1 @@
1 +35.0 KB
Content
Pilotcli Jupyter user login Device Login Successful v2.4.0 2023-05-25.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.sgevans
Size
... ... @@ -1,0 +1,1 @@
1 +10.3 KB
Content
Pilotcli Jupyter user login Grant Access window v2.4.0 2023-05-25.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.sgevans
Size
... ... @@ -1,0 +1,1 @@
1 +13.9 KB
Content
Project Workspace Jupyter Create Python Notebook v2.1.6 2023-02-07.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.sgevans
Size
... ... @@ -1,0 +1,1 @@
1 +793.7 KB
Content
Project Workspace Jupyter Kernel change Kernel dropdown 2023-07-11.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.sgevans
Size
... ... @@ -1,0 +1,1 @@
1 +38.9 KB
Content
Project Workspace Jupyter Launch Terminal v2.1.6 2023-02-07.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.sgevans
Size
... ... @@ -1,0 +1,1 @@
1 +346.3 KB
Content
Project Workspace Jupyter view new Kernel 2023-07-11.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.sgevans
Size
... ... @@ -1,0 +1,1 @@
1 +490.0 KB
Content