Changes for page Working with HDC Project Files in the Command Line Interface
Last modified by Dennis Segebarth on 2024/10/02 18:38
Summary
-
Page properties (1 modified, 0 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -252,16 +252,344 @@ 252 252 253 253 * **Options**: -p ~-~-project-code 254 254 * **Usage**: List all attributes from a specified Project. 255 +* **Prerequisite**: 256 + 257 +* User login is required, otherwise an error is returned: ‘The current login session is invalid. Please login to continue.’ 258 +* Requires the Project code, which can be retrieved via {{code}}pilotcli project list{{/code}}. 259 + 260 +{{code language="none"}} 261 +$ pilotcli file attribute-list --help 262 +Usage: pilotcli file attribute-list [OPTIONS] 263 + 264 + List attribute templates of a given Project. 265 + 266 +Options: 267 + -p, --project-code TEXT Project Code 268 + --help Show this message and exit. 269 +{{/code}} 270 + 271 +=== Examples === 272 + 273 +List attributes from a given Project by providing the Project Code at the prompt. In this example, “cli” is the corresponding Project Code: 274 + 275 +{{code language="none"}} 276 +$ pilotcli file attribute-list 277 +ProjectCode: cli 278 + 279 +Manifest1 280 +-------------------------------------------------------------------------- 281 +| Attribute Name | Type | Value | Optional | 282 +-------------------------------------------------------------------------- 283 +| attr1 | multiple_choice | a1,a2,a3,a4,a5 | False | 284 +-------------------------------------------------------------------------- 285 +| attr2 | text | None | False | 286 +-------------------------------------------------------------------------- 287 +| attr3 | multiple_choice | t1,t2,t3,t4,t5 | True | 288 +-------------------------------------------------------------------------- 289 + 290 +Manifest2 291 +-------------------------------------------------------------------------- 292 +| Attribute Name | Type | Value | Optional | 293 +-------------------------------------------------------------------------- 294 +| attr1 | multiple_choice | a1,a2,a3,a4 | True | 295 +-------------------------------------------------------------------------- 296 +| attr2 | text | None | False | 297 +-------------------------------------------------------------------------- 298 +| attr3 | multiple_choice | t1,t2,t3 | True | 299 +-------------------------------------------------------------------------- 300 +All Attributes fetched successfully. 301 +{{/code}} 302 + 303 +List attributes from given a Project and provide the Project Code “cli” as an option in the same line {{code}}-p cli{{/code}}: 304 + 305 +{{code language="none"}} 306 +$ pilotcli file attribute-list -p cli 307 + 308 +Manifest1 309 +-------------------------------------------------------------------------- 310 +| Attribute Name | Type | Value | Optional | 311 +-------------------------------------------------------------------------- 312 +| attr1 | multiple_choice | a1,a2,a3,a4,a5 | False | 313 +-------------------------------------------------------------------------- 314 +| attr2 | text | None | False | 315 +-------------------------------------------------------------------------- 316 +| attr3 | multiple_choice | t1,t2,t3,t4,t5 | True | 317 +-------------------------------------------------------------------------- 318 + 319 +Manifest2 320 +-------------------------------------------------------------------------- 321 +| Attribute Name | Type | Value | Optional | 322 +-------------------------------------------------------------------------- 323 +| attr1 | multiple_choice | a1,a2,a3,a4 | True | 324 +-------------------------------------------------------------------------- 325 +| attr2 | text | None | False | 326 +-------------------------------------------------------------------------- 327 +| attr3 | multiple_choice | t1,t2,t3 | True | 328 +-------------------------------------------------------------------------- 329 +All Attributes fetched successfully. 330 +{{/code}} 331 + 332 +== Command: **attribute-export** == 333 + 334 +* **Options**: -p ~-~-project-code, -n ~-~-attribute-name 335 +* **Usage**: Export an attribute template as JSON-file from a given Project 255 255 * ((( 256 256 **Prerequisite**: 257 257 258 -* User login is required, otherwise an error is returned: ‘The current login session is invalid. Please login to continue.’ 259 -* Requires the Project code, which can be retrieved via `pilotcli project list. 339 +* User login is required, otherwise an error is returned: "The current login session is invalid. Please login to continue." 340 +* Project must be accessible to the user, otherwise an error is returned: "Project Code not found in list. Please verify and try again." 341 +* Requires the Project code, which can be retrieved via pilotcli project list. 342 +* Requires the Attribute template name. If the attribute template name does not exist or is spelled incorrectly, an error is returned: "Attribute 'sample-template' not found in Project. Please verify and try again." 343 +* pilotcli must have the permission to write a file in the directory, otherwise an error is returned. 260 260 ))) 261 261 346 +{{code language="none"}} 347 +$ pilotcli file attribute-export --help 348 +Usage: pilotcli file attribute-export [OPTIONS] 262 262 350 + Export attribute template from a given Project. 263 263 352 +Options: 353 + -p, --project-code TEXT Project Code 354 + -n, --attribute-name TEXT Attribute Template Name 355 + --help Show this message and exit. 356 +{{/code}} 264 264 358 +=== Examples === 359 + 360 +Export an attribute template from a given Project. The user is prompted to provide the Project Code (“cli”) and the Attribute Template Name (“Manifest1”): 361 + 362 +{{code language="none"}} 363 +$ pilotcli file attribute-export 364 +ProjectCode: cli 365 +AttributeName: Manifest1 366 + 367 +Manifest1 368 +-------------------------------------------------------------------------- 369 +| Attribute Name | Type | Value | Optional | 370 +-------------------------------------------------------------------------- 371 +| attr1 | multiple_choice | a1,a2,a3,a4,a5 | False | 372 +-------------------------------------------------------------------------- 373 +| attr2 | text | None | False | 374 +-------------------------------------------------------------------------- 375 +| attr3 | multiple_choice | t1,t2,t3,t4,t5 | True | 376 +-------------------------------------------------------------------------- 377 +Template saved successfully: cli_Manifest1_template.json 378 +Attribute definition saved successfully: cli_Manifest1_definition.json 379 +{{/code}} 380 + 381 +Export an attribute template from a given Project using options to provide both the Project Code {{code}}(-p cli){{/code}} and the Attribute Template Name {{code}}(-n Manifest1){{/code}}, “cli” and “Manifest1” in this case, respectively: 382 + 383 +{{code language="none"}} 384 +$ pilotcli file attribute-export -p cli -n Manifest1 385 + 386 +Manifest1 387 +-------------------------------------------------------------------------- 388 +| Attribute Name | Type | Value | Optional | 389 +-------------------------------------------------------------------------- 390 +| attr1 | multiple_choice | a1,a2,a3,a4,a5 | False | 391 +-------------------------------------------------------------------------- 392 +| attr2 | text | None | False | 393 +-------------------------------------------------------------------------- 394 +| attr3 | multiple_choice | t1,t2,t3,t4,t5 | True | 395 +-------------------------------------------------------------------------- 396 +Template saved successfully: cli_Manifest1_template.json 397 +Attribute definition saved successfully: cli_Manifest1_definition.json 398 +{{/code}} 399 + 400 +== Command: **list** == 401 + 402 +* **Options**: -z ~-~-zone 403 +* **Usage**: List files and folders in the specified directory within the Platform. Folders will be displayed in blue and files will be displayed in white (or black, depending on the background of your terminal). If no zone (i.e. Green Room or Core) is specified, Green Room will be used as default. 404 +* ((( 405 +**Prerequisite**: 406 + 407 +* User login is required, otherwise an error is returned: "The current login session is invalid. Please login to continue." 408 +* Project must be accessible to the user, otherwise an error is returned: "Project Code not found in list. Please verify and try again." 409 +* Requires the Project code, which can be retrieved via pilotcli project list. 410 +))) 411 + 412 +{{code language="none"}} 413 +$ pilotcli file list --help 414 +Usage: pilotcli file list [OPTIONS] PATHS 415 + 416 + List files and folders inside a given Project/folder. 417 + 418 +Options: 419 + -z, --zone TEXT Target Zone (i.e., core/greenroom) [default: 420 + greenroom] [default: greenroom] 421 + 422 + --page INTEGER The page to be listed [default: 0] 423 + --page-size INTEGER number of objects per page [default: 10] 424 + -d, --detached whether run in detached mode 425 + --help Show this message and exit. 426 +{{/code}} 427 + 428 +=== Examples === 429 + 430 +List folders in the directory “admin” in the Project with the Project Code “indoctestproject”. Since no zone was specified, the corresponding directory in the Green Room will be queried: 431 + 432 +{{code language="none"}} 433 +$ pilotcli file list indoctestproject/admin 434 +test_folder1 test_folder2 file1 file2 file3 435 +{{/code}} 436 + 437 +To list the corresponding elements in the “indoctestproject/admin” directory in the Projects' Core, the zone needs to be specified {{code}}(-z core){{/code}}: 438 + 439 +{{code language="none"}} 440 +$ pilotcli file list indoctestproject/admin -z core 441 +folder1 folder2 test_folder.zip file.txt sample.txt large-file.json 442 +{{/code}} 443 + 444 +== Command: **upload** == 445 + 446 +* **Options**: -p ~-~-project, -a ~-~-attribute, -t ~-~-tag, -z ~-~-zone, -m ~-~-upload-message, -s ~-~-source-file, ~-~-zip, -td ~-~-thread, -o ~-~-output path 447 +* **Usage**: Upload files/folders to the given project, with optional tags and attributes, to the target folder. 448 +* ((( 449 +**Prerequisite**: 450 + 451 +* User login is required, otherwise an error is returned: "The current login session is invalid. Please login to continue." 452 +* Project must be accessible to the user, otherwise an error is returned: "Project Code not found in list. Please verify and try again." 453 +* When uploading file to the Project Core, the user must have access to the Core (Project Contributors cannot upload files/folders to the Project Core). 454 +))) 455 + 456 +{{code language="none"}} 457 +$ pilotcli file upload --help 458 +Usage: pilotcli file upload [OPTIONS] [PATHS]... 459 + 460 + Upload files/folders to a given Project path. 461 + 462 +Options: 463 + -p, --project-path TEXT Project folder path starting from Project code. 464 + (i.e., indoctestproject/user/folder) [required] 465 + 466 + -a, --attribute TEXT File Attribute Template used for annotating files 467 + during upload. 468 + 469 + -t, --tag TEXT Add a tag to the file. This option could be used 470 + multiple times for adding multiple tags. 471 + 472 + -z, --zone TEXT Target Zone (i.e., core/greenroom) [default: 473 + greenroom] [default: greenroom] 474 + 475 + -m, --upload-message TEXT The message used to comment on the purpose of 476 + uploading your processed file [default: ] 477 + 478 + -s, --source-file TEXT The Project path of the source file of your 479 + processed files. 480 + 481 + --zip Upload folder as a compressed zip file. 482 + -td, --thread INTEGER The number of thread for upload a file [default: 483 + 1] 484 + 485 + -o, --output-path TEXT The output path for the manifest file of 486 + resumable upload [default: ./manifest.json] 487 + 488 + --help Show this message and exit. 489 +{{/code}} 490 + 491 +=== Examples === 492 + 493 +Upload a file (or folder) to a specified directory of a Project’s Green Room, in this case the file located at {{code}}./sample_folder/sample_file1{{/code}} will be uploaded to the folder “admin” in the Project with the Project Code “cli” {{code}}(-p cli/admin){{/code}}: 494 + 495 +{{code language="none"}} 496 +$ pilotcli file upload ./sample_folder/sample_file1 -p cli/admin 497 +Starting upload of: ./sample_folder/sample_file1 498 +Pre-upload complete. 499 +uploader: admin container_id: 18061 total_size: 1048576 total_chunks: 1 uniq_identifier: e958247c-ce05-48b2-9038-78b774836256-1627498771 resumable_relative_path admin/ 500 + 501 +Upload job is finalizing, please wait... 502 +Upload job complete. 503 +All files uploaded successfully. 504 +{{/code}} 505 + 506 +In case the specified destination directory does not exist yet, you will be prompted to confirm its creation by entering “y”, or to abort the command by entering “N”: 507 + 508 +{{code language="none"}} 509 +Target folder does not exist. Would you like to create a new folder? [y/N]: y 510 +{{/code}} 511 + 512 +In order to upload a file (or folder) to a destination directory in a Project's Core, core must be specified as corresponding zone {{code}}(-z core){{/code}} and you will be required to provide a message to that should describe the files` content and why you are uploading it directly into the Core {{code}}(-m "sample message"){{/code}}. Moreover, you will be reminded about the Terms of Use of the Platform and additional considerations regarding the direct upload of data into the Core zone, and are prompted to confirm your intention to proceed with the upload by entering “y” or “Yes”: 513 + 514 +{{code language="none"}} 515 +$ pilotcli file upload ./sample_folder/sample_file2 -p cli/admin -z core -m "sample message" 516 +You are about to transfer data directly to the Core! In accordance with the Terms of Use, please confirm that you 517 +have made your best efforts to pseudonymize or anonymize the data and that you have the legal authority to transfer and make this 518 +data available for dissemination and use within the Platform.To review the Terms of Use, please the Portal. If you 519 +need to process the data to remove sensitive identifiers, please cancel this transfer and upload the data to the Green Room to 520 +perform these actions. 521 +To cancel this transfer, enter [n/No] 522 +To confirm and proceed with the data transfer, enter [y/Yes] 523 + [y/N]: y 524 +Starting upload of: ./sample_folder/sample_file2 525 +Pre-upload complete. 526 +uploader: admin container_id: 18061 total_size: 1048576 total_chunks: 1 uniq_identifier: fa807a0f-ceb0-4fff-9c9e-563bdcbb309f-1627498851 resumable_relative_path admin/ 527 + 528 +Upload job is finalizing, please wait... 529 +Upload job complete. 530 +All files uploaded successfully. 531 +{{/code}} 532 + 533 +Upload a folder to Project Green Room 534 + 535 +{{code language="none"}} 536 +$ pilotcli file upload ./sample_folder/example_folder -p cli/admin 537 +Starting upload of: ./sample_folder/example_folder 538 +Current version does not support folder tagging, any selected tags will be ignored 539 +Pre-upload complete. 540 +uploader: admin container_id: 18061 total_size: 1048576 total_chunks: 1 uniq_identifier: 4f0eee2e-70f4-4b03-9e72-040544ec7d3a-1627498907 resumable_relative_path admin/example_folder/ 541 + 542 +Starting finalization... 543 +uploader: admin container_id: 18061 total_size: 1048576 total_chunks: 1 uniq_identifier: 41ec70e2-0859-4aba-9009-06e700c7c79e-1627498907 resumable_relative_path admin/example_folder/ 544 + 545 +Starting finalization... 546 +uploader: admin container_id: 18061 total_size: 1048576 total_chunks: 1 uniq_identifier: 28454c51-ec9a-4dc5-ab3c-0f7e80e3050d-1627498907 resumable_relative_path admin/example_folder/ 547 + 548 +Upload job is finalizing, please wait... 549 +Upload job complete. 550 +All files uploaded successfully. 551 +{{/code}} 552 + 553 +Upload file to existing folder in Green Room 554 + 555 +* Specify a target folder to upload files by inputting folder name after the Project code separated by the '/'. 556 + 557 +{{code language="none"}} 558 +$ pilotcli file upload ./sample_folder/sample_file5 -p cli/admin/example_folder 559 +Starting upload of: ./sample_folder/sample_file5 560 +Pre-upload complete. 561 +uploader: admin container_id: 18061 total_size: 1048576 total_chunks: 1 uniq_identifier: 7ef58005-d408-4ad9-a741-fa938f58e67f-1627498977 resumable_relative_path admin/example_folder/ 562 + 563 +Upload job is finalizing, please wait... 564 +Upload job complete. 565 +All files uploaded successfully. 566 +{{/code}} 567 + 568 +Upload folder as zip to a specified directory of a Project's Green Room by adding the ~-~-zip option. 569 + 570 +* The specified folder will then be compressed as a zip file and subsequently be uploaded to the specified destination. Once the upload is completed, the compressed zip file will be removed from the directory. Note: pilotcli must have the permission to create and delete zip files in the corresponding directory of your local machine: 571 + 572 +{{code language="none"}} 573 +$ pilotcli file upload ./sample_folder/example_folder -p cli/admin --zip 574 +Starting upload of: ./sample_folder/example_folder 575 +Started zipping files. 576 +Pre-upload complete. 577 +uploader: admin container_id: 18061 total_size: 3595 total_chunks: 1 uniq_identifier: 61a729ce-cdd6-4a87-82b5-3f8ea63e6c4a-1627499012 resumable_relative_path admin/ 578 + 579 +Upload job is finalizing, please wait... 580 +Upload job complete. 581 +All files uploaded successfully. 582 +{{/code}} 583 + 584 +Upload a file with tags and/or attributes. 585 + 586 +* When entering the upload command, the user can add a maximum of ten tags to the file. For example, in order to add “sample_tag1” and “sample_tag2” to your file, simply add {{code}}-t sample_tag1{{/code}} {{code}}-t sample_tag2{{/code}} to the command (see example below using {{code}}tag1{{/code}} and {{code}}tag2{{/code}} as tags). 587 +* In addition, you can also use attribute templates for a more enriched annotation of your files. Please note, that the corresponding attribute template file needs to be edited & saved prior to uploading. To get the blank template, see the **attribute-export** instructions above for more details. To attach a attribute JSON-file to your file upon uploading, use -a cli_Manifest1_template.json, where “cli_Manifest1_template.json" points to the location of the attributes JSON-file on your local machine. 588 +* Upon uploading, pilotcli will also validate the content of the attached attribute JSON-file. Whenever a template is invalid, including missing non-optional fields, non-existing choice in multiple choice attribute, exceeding length text field and so on, pilotcli will raise error to user and user must update the attribute template and upload again: 589 + 590 + 591 + 592 + 265 265 ---- 266 266 267 267 - ... ... @@ -271,4 +271,5 @@ 271 271 272 272 Copyright © 2023 [[Indoc Research>>url:https://www.indocresearch.org/]]. 273 273 274 -HealthDataCloud is powered by Pilot technology, a product of [[Indoc Research>>url:https://www.indocresearch.org/]]. 602 +HealthDataCloud is powered by Pilot technology, a product of [[Indoc Research>>url:https://www.indocresearch.org/]].~{~{/code}} 603 +~)~)~)