Changes for page Using JupyterHub in HDC
Last modified by Dennis Segebarth on 2024/10/02 18:14
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 1 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -291,26 +291,14 @@ 291 291 292 292 When using the Pilot Command Line Interface in JupyterHub and the following actions are possible on the derivative files generated in JupyterHub: 293 293 294 -{table} 295 -**File Operation** | **Permitted in the ** 296 -**Green Room** 297 -Word 1 | Word 2 298 -{table} 299 - 300 300 |=(% colspan="1" rowspan="1" %)((( 301 301 **File Operation** 302 - 303 - 304 304 )))|=(% colspan="1" rowspan="1" %)((( 305 305 **Permitted in the ** 306 -**Green Room** 307 - 308 - 298 +**Green Room** 309 309 )))|=(% colspan="1" rowspan="1" %)((( 310 310 **Permitted in the ** 311 -**Core** 312 - 313 - 301 +**Core** 314 314 ))) 315 315 |(% colspan="1" rowspan="1" %)File upload 316 316 (upload derivative output files from JupyterHub to the Green Room or Core storage)|(% colspan="1" rowspan="1" %)((( ... ... @@ -325,6 +325,85 @@ 325 325 Yes 326 326 ))) 327 327 316 +== Downloading Project Data to JupyterHub using the Pilot Command Line Interface == 328 328 318 +After logging into the Pilot Command Line Interface, you can download data from the Project Core into the JupyterHub environment to start your data analyses. 329 329 330 - 320 +File related commands are grouped in the {{code}}file{{/code}} category. To view the full list of commands in this category, type {{code}}pilotcli file --help{{/code}}. To download project data, use the file sync command. To view the full list of commands in this category, type {{code}}pilotcli file sync --help{{/code}}. 321 + 322 + 323 +{{code language="none"}} 324 +collaborator4@jupyter-collaborator4:~$ pilotcli file sync --help 325 +Usage: pilotcli file sync [OPTIONS] [PATHS]... OUTPUT_PATH 326 + 327 + Download files/folders from a given Project/folder/file in core zone. 328 + 329 +Options: 330 + -z, --zone TEXT Target Zone (i.e., core/greenroom) 331 + --zip Download files as a zip. 332 + -i, --geid Enable downloading by geid. 333 + --help Show this message and exit. 334 +{{/code}} 335 + 336 +=== Example === 337 + 338 +Downloading a file from the Core to your Home Directory: 339 + 340 +Reminder: Please follow Linux conventions for file management. If your filename contains spaces, wrap it in single or double quotes. 341 + 342 +* //Filename~:// “Chemical Tracking Data.csv” 343 +* //Source~:// Project “Indoc Test Project”, “Core” storage zone, folder “collaborator4” {{code}}indoctestproject/collaborator4/Chemical Tracking Data.csv -z core{{/code}} 344 +* //Destination: //user's Home directory in the Guacamole or JupyterHub VM {{code}}.{{/code}} 345 +* //Command group/option: //{{code}}file sync{{/code}} 346 + 347 +{{code language="none"}} 348 +collaborator4@jupyter-collaborator4:~$ pilotcli file sync indoctestproject/collaborator4/'Chemical Tracking Data.csv' . -z core 349 +start downloading... 350 +Downloading Chemical Tracking Data.csv |██████████████████████████████ 100% 00:00 351 +File has been downloaded successfully and saved to: ./Chemical Tracking Data.csv 352 +{{/code}} 353 + 354 +To confirm successful download, type {{code}}ls{{/code}} and verify the file "Chemical Tracking Data.csv" is stored in the Home folder. 355 + 356 +{{code language="none"}} 357 +collaborator4@jupyter-collaborator4:~$ ls 358 +'Chemical Tracking Data.csv' pilotcli 359 +{{/code}} 360 + 361 +The file “Chemical Tracking Data.csv” can be viewed in the JupyterHub graphical user interface: 362 + 363 +[[image:Jupyter downloaded file in Home folder v2.4.11 2023-05-25 1850.png||height="15%" width="50%"]] 364 + 365 + 366 +== Uploading Project Data from JupyterHub using the Pilot Command Line Interface == 367 + 368 +After analyzing Project data inside the JupyterHub, you can upload the generated outputs back into the Project via the Pilot Command Line Interface. 369 + 370 +=== Example === 371 + 372 +* //Filename//: Chemical Tracking Data rev.csv 373 +* //Source~:// user's Home directory in JupyterHub {{code}}.{{/code}} 374 +* //Destination//: Project “Indoc Test Project”, folder “collaborator4”, “Core” storage zone, 375 +{{code}}indoctestproject/collaborator4{{/code}} {{code}}-z core{{/code}} 376 +* //Command group/option~:// {{code}}file upload{{code}} 377 +* //User message// (for upload back to the Core): “my workbench output, no additional sensitive data" 378 +* //Command~:// pilotcli file upload ./'Chemical Tracking Data rev.csv' -p indoctestproject/collaborator4 -z core -m "my workbench output, no additional sensitive data" 379 + 380 +When uploading data to the Core, you are reminded that you are bypassing the usual Green Room upload workflow. To confirm, type y at the prompt, or N to cancel. 381 + 382 +{{code language="none"}} 383 +collaborator4@jupyter-collaborator4:~$ pilotcli file upload ./'Chemical Tracking Data rev.csv' -p indoctestproject/collaborator4 -z core -m "my workbench output, no additional sensitive data" 384 +You are about to transfer data directly to the PILOT Core! In accordance with the PILOT Terms of Use, please confirm that you have made your best efforts to 385 +pseudonymize or anonymize the data and that you have the legal authority to transfer and make this data available for dissemination and use within the PILOT .If you 386 +need to process the data to remove sensitive identifiers, please cancel this transfer and upload the data to the Green Room to perform these actions. 387 +To cancel this transfer, enter [n/No] 388 +To confirm and proceed with the data transfer, enter [y/Yes] 389 + [y/N]: y 390 +Starting upload of: ./Chemical Tracking Data rev.csv 391 +Pre-upload complete. 392 +Uploading Chemical Tracking Data rev.csv: |██████████████████████████████ 100% 00:00 393 +Upload Time: 2.92s for 1 files 394 +All uploading jobs have finished. 395 +{{/code}} 396 + 397 +
- Jupyter downloaded file in Home folder v2.4.11 2023-05-25 1850.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.sgevans - Size
-
... ... @@ -1,0 +1,1 @@ 1 +35.0 KB - Content