Changes for page Working with HDC Datasets

Last modified by Dennis Segebarth on 2024/10/02 18:25
From version 18.1
edited by Dennis Segebarth
on 2024/08/16 08:58
Change comment: Deleted image "Datasets Metadata Manage Custom Schema Template v2.1.5 2023-01-25.png"
To version 23.1
edited by Dennis Segebarth
on 2024/10/02 18:25
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -2,7 +2,7 @@
2 2  {{toc depth="2"/}}
3 3  {{/box}}
4 4  
5 -Datasets extend Projects with comprehensive data management functionalities. Importantly, each Project can have multiple Datasets associated with it, but each Dataset can only be populated with data from a single Project.
5 +Datasets extend Projects with comprehensive data management functionalities. Importantly, each Project can have multiple Datasets associated with it, but each Dataset can only be populated with data from a single Project. Each Dataset can be viewed, accessed, and interacted with by the user who created it, as well as all users with the Project Administrator role in the Project the Dataset is associated with.
6 6  
7 7  In brief, Datasets are collections of related Project files, folders, and detailed metadata. The Dataset feature allows users to organize Project files, enrich with annotations, validate data structures, and publish controlled versions for sharing.
8 8  
... ... @@ -20,11 +20,11 @@
20 20  
21 21  = Creating a Dataset =
22 22  
23 -New Datasets are empty containers ready for you to add data files and/or folders from your Project Core. Your Dataset is private - only you can view and work with it until you download and share it with others.
23 +New Datasets are empty containers ready for you to add data files and/or folders from your Project Core. Each Dataset can be viewed, accessed, and interacted with by the user who created it, as well as all users with the Project Administrator role in the Project the Dataset is associated with.
24 24  
25 25  Datasets are created from the Main Menu. To create a new dataset,
26 26  
27 -[[image:Main Menu Datasets NonAdmin v2.26 2023-04-12.png||height="5%" width="50%"]]
27 +[[image:1723802288344-928.png||height="92" width="859"]]
28 28  
29 29  
30 30  1. Click **Datasets** from the Main Menu to open the Datasets landing page. Datasets you previously created are displayed in this page in the “My Datasets” tab, and you can continue working on them as described in the following sections of this article, e.g. see //Adding Files to a Dataset//.
... ... @@ -48,11 +48,11 @@
48 48  
49 49  = Adding Files to a Dataset =
50 50  
51 -As mentioned above, each Dataset is associated to a single Project. Therefore, files can be added to a Dataset only from the Core of the respective Project. This will create a copy of the file so the original file in the Core is unaffected by subsequent manipulations inside the Dataset. The Dataset must have already been created before files can be added (see //Creating a Dataset//).
51 +As mentioned above, each Dataset is associated to a single Project. Therefore, files can be added to a Dataset only from the Core of the respective Project. This will create a copy of the file so the original file in the Core is unaffected by subsequent manipulations inside the Dataset. The Dataset must have already been created before files can be added (see //Creating a Dataset//). Once this association between the Dataset and a Project has been established, the Dataset automatically becomes visible and accessible for all members of the Project that have the Project Administrator role.
52 52  
53 53  To add files to a Dataset,
54 54  
55 -[[image:Datasets Add to Datasets v2.1.5 2023-01-24.png||height="11%" width="50%"]]
55 +[[image:1723802315115-691.png||height="191" width="870"]]
56 56  
57 57  1. Navigate your Project’s File Explorer and click **Core**.
58 58  1. Select one or more files and/or folders by clicking the checkbox beside each one, then click **Add to Datasets** in the File Explorer menu.
... ... @@ -70,7 +70,7 @@
70 70  
71 71  Use the **Explorer** tab to organize and validate your Dataset.
72 72  
73 -[[image:Datasets Explorer v2.1.5 2023-01-24.png||height="10%" width="50%"]]
73 +[[image:1723802327251-439.png||height="182" width="897"]]
74 74  
75 75  1. Open the Dataset from the Datasets landing page, then select the **Explorer **tab.
76 76  1. Use **+ New Folder** or **Move to **to rearrange files and/or folders.
... ... @@ -89,13 +89,13 @@
89 89  
90 90  === BIDS Validation ===
91 91  
92 -[[image:BIDS_Logo.png||height="9%" width="20%"]]
92 +[[image:1723802339509-521.png||height="83" width="192"]]
93 93  
94 94  The **Validate BIDS** tool allows you to check your Dataset for conformation with the [[Brain Imaging Data Structure (BIDS)>>url:https://bids.neuroimaging.io/specification.html]] standard. The tool uses the open-source [[bids-validator 1.8.4 Python Package>>url:https://pypi.org/project/bids-validator/#quickstart]].
95 95  
96 96  Before running the Validate BIDS tool, ensure that the Dataset Type is set to BIDS (defined at Dataset creation or modified in the **Metadata **tab under Schemas > Essential).
97 97  
98 -[[image:Datasets Validate BIDS v2.1.5 2023-01-24.png||height="13%" width="50%"]]
98 +[[image:1723802351811-730.png||height="230" width="878"]]
99 99  
100 100  1. Open the Dataset from the Datasets landing page and select the **Explorer **tab.
101 101  1. Click **Validate BIDS**. If the Validate BIDS button isn’t visible, make sure your Dataset Type is set to BIDS (see above).
... ... @@ -125,7 +125,7 @@
125 125  
126 126  Use the **Metadata** tab to annotate your Dataset.
127 127  
128 -[[image:Datasets Metadata v2.1.5 2023-01-24.png||height="10%" width="50%"]]
128 +[[image:1723802365719-520.png||height="176" width="876"]]
129 129  
130 130  These steps describe how to annotate your dataset using the Default Schema. If you have created a Custom Schema (see the section titled //Creating a Custom Schema// in this article), you can follow the same steps described here, selecting your Custom Schema name instead of Default Schema in Step 2.
131 131  
... ... @@ -134,7 +134,7 @@
134 134  1. (((
135 135  In the **Schemas **section on the right panel, click the **Select schema to complete **dropdown menu and select one of the Default schema templates or a Custom Schema created by you (see the section //Creating a Custom Schema //in this article).
136 136  
137 -[[image:Datasets Metadata Default Schema v2.1.5 2023-01-25.png||height="38%" width="50%"]]
137 +[[image:1723802376639-303.png||height="246" width="326"]]
138 138  
139 139  Default schema categories include the following:
140 140  
... ... @@ -170,7 +170,7 @@
170 170  
171 171  To create a Custom Schema,
172 172  
173 -[[image:Datasets Metadata Create Custom Schema v2.1.5 2023-01-25.png||height="24%" width="50%"]]
173 +[[image:1723802393964-978.png||height="237" width="485"]]
174 174  
175 175  1. Open the Dataset from the Datasets landing page and select the **Metadata **tab. Under **Existing Schema** (left panel), select **Default Schemas**.
176 176  1. In the **Schemas **panel (right side), click the **Select schema to complete **dropdown menu and select **+ Create Custom Schema** to open the Custom Schema Template creation window.
... ... @@ -199,7 +199,7 @@
199 199  
200 200  To manage a Custom Schema template that contains saved annotations,
201 201  
202 -[[image:Datasets Metadata Manage Custom Schema Template v2.1.5 2023-01-25.png||height="17%" width="50%"]]
202 +[[image:1723802410617-147.png||height="163" width="481"]]
203 203  
204 204  1. Navigate to **Existing Schema**// //(left panel of the **Metadata **tab) and select the **Default Schema** category.
205 205  1. Click the Custom schema name, then click the **eye icon** to open the Schemas view (right side). The existing entries are displayed.
... ... @@ -215,7 +215,7 @@
215 215  
216 216  == Annotating with OpenMINDS ==
217 217  
218 -[[image:openMINDS_logo.png||height="10%" width="20%"]]
218 +[[image:1723802431881-142.png||height="111" width="222"]]
219 219  
220 220  The **open** **M**etadata **I**nitiative for **N**euroscience **D**ata **S**tructures ([[openMINDS>>url:https://github.com/HumanBrainProject/openMINDS]])// //is an open-source, community-driven research infrastructure initiative powered by EBRAINS and the Human Brain Project. The openMINDS schema gathers a set of metadata models that can be used for describing heterogeneous neuroscience data. The data can originate from human, animal or simulated studies, computational models, and software tools, as well as metadata or data models.
221 221  
... ... @@ -223,7 +223,7 @@
223 223  
224 224  Community tools are offered to help generate openMINDS JSON schemas, including a [[wizard>>url:https://wizard.kg-ppd.ebrains.eu/]] and a [[python package>>url:https://pypi.org/project/openMINDS/]]. For information or support, please contact the [[openMINDS team>>url:https://github.com/HumanBrainProject/openMINDS]].
225 225  
226 -[[image:Datasets Metadata openMINDS Schemas v2.1.5 2023-01-25.png||height="18%" width="50%"]]
226 +[[image:1723802444445-662.png||height="320" width="864"]]
227 227  
228 228  To upload an openMINDS JSON schema file,
229 229  
... ... @@ -244,7 +244,7 @@
244 244  
245 245  To view Activity, open the Dataset from the Datasets landing page and select the **Activity **tab. Use the date filters to narrow the range of activities displayed.
246 246  
247 -[[image:Datasets Activity v2.1.5 2023-01-24.png||height="10%" width="50%"]]
247 +[[image:1723802458591-860.png||height="174" width="863"]]
248 248  
249 249  
250 250  == Anytime Dataset Download ==
... ... @@ -259,7 +259,7 @@
259 259  
260 260  To release a Dataset version,
261 261  
262 -[[image:Datasets Release new version v2.1.5 2023-01-26.png||height="10%" width="50%"]]
262 +[[image:1723802471096-883.png||height="169" width="864"]]
263 263  
264 264  
265 265  1. Open the Dataset from the **Datasets **landing page and ensure you have made all the desired updates to the files, folder structure and metadata.
... ... @@ -273,7 +273,7 @@
273 273  
274 274  To view all released Dataset versions, click **Versions **in the Dataset information bar located next to the Dataset title. This opens the **Versions **sidebar where you can browse and download the Versions you have created.
275 275  
276 -[[image:Datasets Versions v2.1.5 2023-01-26.png||height="24%" width="50%"]]
276 +[[image:1723802481440-617.png||height="234" width="488"]]
277 277  
278 278  ----
279 279  
... ... @@ -282,5 +282,4 @@
282 282  HealthDataCloud is powered by Pilot technology, a product of [[Indoc Systems>>url:https://www.indocsystems.com]].
283 283  
284 284  
285 -
286 286  
Datasets Metadata v2.1.5 2023-01-24.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.sgevans
Size
... ... @@ -1,1 +1,0 @@
1 -324.1 KB
Content
Datasets Release new version v2.1.5 2023-01-26.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.sgevans
Size
... ... @@ -1,1 +1,0 @@
1 -325.5 KB
Content
Datasets Validate BIDS v2.1.5 2023-01-24.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.sgevans
Size
... ... @@ -1,1 +1,0 @@
1 -120.8 KB
Content
1723802288344-928.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +4.1 KB
Content
1723802310399-642.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +4.1 KB
Content
1723802315115-691.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +101.2 KB
Content
1723802327251-439.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +115.1 KB
Content
1723802339509-521.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +121.0 KB
Content
1723802351811-730.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +93.2 KB
Content
1723802365719-520.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +112.8 KB
Content
1723802376639-303.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +415.8 KB
Content
1723802393964-978.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +244.9 KB
Content
1723802410617-147.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +210.6 KB
Content
1723802431881-142.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +49.0 KB
Content
1723802444445-662.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +291.0 KB
Content
1723802458591-860.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +108.8 KB
Content
1723802471096-883.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +115.2 KB
Content
1723802481440-617.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.dsegebarth
Size
... ... @@ -1,0 +1,1 @@
1 +319.9 KB
Content