Wiki source code of Changelog

Version 2.1 by Dennis Segebarth on 2026/02/25 09:03
Show last authors
1 {{box cssClass="floatinginfobox" title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
4
5 Pilot-HDC is the central-node research data management platform software ecosystem for the eBRAIN-Health / EBRAINS HealthDataCloud central node platform. It provides a secure, project-based environment for managing, annotating, and sharing research data, integrates with EBRAINS core services (including federated identity management and the EBRAINS Knowledge Graph), and combines data management capabilities with co-located workspace and compute resources. This changelog documents the functional enhancements, fixes, and operational improvements delivered across Pilot-HDC releases.
6
7 = Open-source availability =
8
9 Pilot-HDC is developed by Indoc Research Europe gGmbH as an open-source software ecosystem and publicly accessible [[here>>https://github.com/orgs/PilotDataPlatform/repositories?q=pilot-hdc]]. The source code of the Pilot-HDC services, as well as the related Infrastructure-as-Code (IaC) components used for deployment and operation, is released under an OSI-compatible license (AGPL-3.0).
10
11 = Version History =
12
13 == Pilot-HDC v1.0 (29.09.2023): ==
14
15 === New Features: ===
16
17 * Integration with EBRAINS Knowledge Graph (KG): Users can now directly interact with the EBRAINS KG through the Platform user interface (UI):
18 ** Create dedicated KG Spaces for Datasets via the Dataset UI.
19 ** Upload openMINDS schema instances directly to KG Spaces through the Dataset UI.
20 ** Access permissions for KG Spaces are automatically mapped to the permissions of the corresponding HDC Project members.
21 ** Changes to Project membership are automatically propagated to reflect updated permissions in associated KG Spaces.
22 * The Platform Support contact form is now integrated with the EBRAINS Support System for seamless user assistance.
23 * Virtual Machines (VMs) are now available as Workspace Tools for Projects, enhancing computational flexibility.
24
25 === Improvements: ===
26
27 * Release of the full User Guide to provide comprehensive support for platform users.
28 * Upgraded the Workspace Tool XWiki to enable access to the "WYSIWYG" editor mode for easier content creation.
29 * Integrated monitoring of non-sensitive platform metrics via the centralized EBRAINS monitoring system for improved operational oversight.
30 * The latest version of pilotcli, the Platform’s Command Line Interface (CLI), is now pre-installed and readily available in all Workspace VMs and JupyterHub environments.
31 * Refined several user notifications to include direct links to relevant User Guide articles for enhanced usability.
32
33 === Bug Fixes: ===
34
35 * Fixed an issue where the "Last time logged in" was not displayed correctly on both the user Account page and the Platform Administration table.
36 * Resolved an issue with platform-wide notification banners that displayed scheduled maintenance times incorrectly due to UTC-to-local timezone conversion errors.
37 * Fixed a bug where the BIDS Validator in the Dataset interface failed to display results if validation was unsuccessful.
38 * Addressed inconsistencies in file and folder structures of Datasets upon download.
39
40 ----
41
42 == Pilot-HDC v1.1 (15.05.2024): ==
43
44 === New Feature: ===
45
46 * Management of Workspace VM-specific credentials: Users can now manage their Workspace VM-specific credentials fully self-served. This includes creation, reset, and change of their Workspace VM password
47
48 === Improvements: ===
49
50 * Removing a user from a Project automatically disables their access to all Workspace VMs of that Project
51 * Failed upload events are now sorted from most recent to oldest for resumable upload
52 * Failed upload events are now deleted after 24 hours
53
54 === Bug Fixes: ===
55
56 * Fixed the bug that caused failure of direct Project invitation for new EBRAINS users
57 * Fixed the bug that caused failure of updating users permissions in EBRAINS Collabs for Datasets with enabled Knowledge Graph integration service in cases with too many Datasets per Project
58 * Fixed the bug that prevented Project Admins from completing their own Guacamole access request
59
60 ----
61
62 == Pilot-HDC v1.2 (30.09.2024): ==
63
64 === New Features: ===
65
66 * Enhancements to integration with EBRAINS Knowledge Graph (KG) via the HDC Dataset user interface:
67 ** Users can now delete openMINDS metadata instances in both HDC Dataset and/or in the associated KG space
68 ** Users can now download openMINDS metadata instances into HDC Datasets from KG by providing the KG UUID
69 ** Users can now push the latest version of an openMINDS metadata instance from HDC Datasets to the associated KG Space
70 ** Users can now pull the latest version of an openMINDS metadata instance from KG to the HDC Dataset
71 ** User interactions with EBRAINS KG integration are now logged in the Activity tab of HDC Datasets
72 * Access to Datasets by Project Administrators: Project Administrators can now view, access, and interact with all Datasets associated with their Project, even if it was created by a different user
73
74 === Improvements: ===
75
76 * Project Administrators can now filter lists of accessible Datasets to see only those they created, or to include those created by members of their Project, both via the platform user interface and via the command line interface
77 * Enhanced performance and user experience through reduced build times for the platform frontend and improved error handling
78 * Increased overall security by updating several components to address vulnerabilities. This includes upgrades of some Workspace Tools:
79 ** JupyterHub was upgrade to version 4.0.2
80 ** JupyterLab was upgraded to version 4.0.7
81 ** XWiki was upgraded to version 16.6.0
82 * Improved response time and incident management capabilities through upgraded CI/CD pipelines and enhanced alerting based on platform system metrics monitoring
83 * Started enforcing resource limits of Workspace Tools to ensure reliable platform operation
84
85 === Bug Fixes: ===
86
87 * Fixed the bug that prevented deletion of files from Metadata tab of Datasets
88 * Fixed the bug that prevented correct display of the Terms of Use upon first login
89
90 ----
91
92 == Pilot-HDC v1.3 (22.11.2024): ==
93
94 === New Features: ===
95
96 * All registered EBRAINS users can now join the HealthDataCloud Test Project on a self-serve basis, allowing them to explore the HealthDataCloud central node platform features.
97 * Association between a Dataset and a Project is now being made upon Dataset creation.
98 * Users can now filter lists of accessible Datasets by Project they are associated to.
99
100 === Improvements: ===
101
102 * When adding data to a Dataset from the core zone of a Project, only those Datasets that are associated with the corresponding Project are shown, not all Datasets across Projects the user has access to.
103 * Introduced enhanced logging capabilities by enabling "Audit" log levels acroess all services for improved traceability and accountability.
104 * Refined error messages in the Command Line Interface (pilotcli) to provide more detailed and user-friendly guidance during platform interactions.
105 * Increased platform security through component updates to address vulnerabilities.
106
107 === Bug Fixes: ===
108
109 * Resolved inconsistent references to openMINDS schema instances across the platform.
110 * Fixed the issue that prevented seamless switching between zones in the Project Search interface.
111 * Added additional health checks to ensure service availability following partial or complete cluster restarts.
112
113 ----
114
115 == Pilot-HDC v1.4 (08.04.2025): ==
116
117 === New Features: ===
118
119 * Dataset Sharing between Projects within the Platform:
120 ** Project Administrators can now share specific versions of a Dataset with other Projects by sending a sharing request directly through the user interface.
121 ** Project Administrators of the recipient Project can review and process these requests via a newly added Requests tab in their Project view.
122 ** All interactions related to Dataset sharing are logged in the Activity tab of the Dataset UI. Sharing requests and their processing status are persistently visible in the Requests tab of the recipient Project.
123 * New UI elements now allow users to bulk-synchronize all openMINDS metadata between a Dataset within the Platform and the EBRAINS Knowledge Graph in a single action.
124 * The use of identity management and role-based access control (RBAC) was expanded to further strengthen security and ensure consistent access policies across all components of the system.
125
126 === Improvements: ===
127
128 * Enhanced and refined metrics collection and log retention policies across key platform components to improve system observability, ensure log availability, and support proactive issue detection.
129 * Improved the responsiveness and efficiency of the Platform Portal while reducing resource consumption, contributing to a smoother user experience.
130 * Simplified the deployment and maintenance processes across core Platform services by consolidating and standardizing configuration management, reducing operational complexity and improving consistency.
131
132 === Bug Fixes: ===
133
134 * Rolled out the latest version of pilotcli (v2.2.7) across all Workspace Tools.
135
136 ----
137
138 == Pilot-HDC v1.5 (25.09.2025): ==
139
140 === New Features: ===
141
142 * Download origin indicator: To strengthen security and monitoring for Project members, each "download" action now shows whether data was retrieved internally (within the Platform, e.g., to Workspace Tools such as JupyterHub or a Workspace VM) or externally.
143 ** Applies to downloads initiated via both the Platform UI and the pilotcli.
144 ** Logged in the Activity streams of both Projects and Datasets.
145 ** Note: Only the "internal" vs "external" status is displayed; no IP addresses are revealed.
146 * Automated end-to-end testing: A comprehensive test suite now runs every 8 hours, covering major Platform features and triggering tight alerting mechanisms if errors occur.
147
148 === Improvements: ===
149
150 * Dataset sharing: Shared Dataset content is no longer delivered as a .zip archive. Instead, the full folder structure is recreated in the recipient Project Green Room.
151 * Deployment and configuration: Refactored automated pipelines to reduce stack complexity and dependency surface.
152 * Image registry hardening: Expanded the private image registry coverage for platform-critical service images to reduce reliance on external sources.
153 * Monitoring: Extended non-sensitive performance metrics collection to Workspace VM services.
154
155 === Bug Fixes: ===
156
157 * Fixed a rendering issue affecting certain XWiki plugins.
158
159 ----
160
161 == Pilot-HDC v1.6 (16.02.2026): ==
162
163 === New Features: ===
164
165 * File & Folder deletion and restoration:
166 ** Project members can now delete and restore files and folders directly via the Portal UI.
167 ** From the Project zones Green Room and Core, Project members can move files and folders to the Trash Bin and restore them to their original location when needed. Visibility and access permissions in the Trash Bin are governed by the Platform’s RBAC policies.
168 ** Only Project Administrators can mark items in the Trash Bin for permanent deletion. Marked items are automatically deleted after a grace period of 30 days.
169 ** Via the User Account page, Project Administrators can review an overview of items marked for permanent deletion, including when they were marked. Until the grace period expires, items can be restored (back to the Trash Bin and from there to the Project zone) or permanently deleted immediately.
170 ** **Note:** For forensic and compliance purposes, general retention policies apply to basic metadata (e.g., unique file/folder IDs) and audit logs after permanent deletion.
171
172 === Improvements: ===
173
174 * Search UI enhancements: Refined and streamlined the Search UI and aligned it with the new file/folder deletion workflows.
175 * Interoperability preparations (Pilot-HDC central node & Pilot-HDC-lite satellite nodes): Implemented foundational changes to support interoperable deployments across central and satellite nodes, including configurable Portal behavior and more flexible deployment configuration.
176 * Security: Implemented additional hardening measures for platform isolation and access control, including improved safeguards for CI/CD job execution.
177 * Governance: Expanded audit logging coverage across Platform services to improve traceability and accountability.
178
179 === Bug Fixes: ===
180
181 * Fixed an issue affecting resumable upload when large numbers of files were temporarily stored for recovery.
182 * Fixed an SSL certificate issue affecting the platform ingress/proxy configuration.
183
184 ----
185
186 = Acknowledgements =
187
188 The development of the HealthDataCloud open source software was supported by the EBRAINS research infrastructure, funded from the European Union's Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3) and H2020 Research and Innovation Action Grant Interactive Computing E-Infrastructure for the Human Brain Project ICEI 800858.
189
190 This project has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101058516. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or other granting authorities. Neither the European Union nor other granting authorities can be held responsible for them.
191
192 [[image:https://camo.githubusercontent.com/e89d9272e7323d4da28995429506909aa85ea704b4082474f4f35fbccdaa3e24/68747470733a2f2f6864632e68756d616e627261696e70726f6a6563742e65752f696d672f4844432d45552d61636b6e6f776c656467656d656e742e706e67||height="84" width="880"]]
193
194 ----
195
196 Copyright © 2023-2026 [[Indoc Systems>>url:https://www.indocsystems.com]].
197
198 HealthDataCloud is powered by Pilot technology, a product of [[Indoc Systems>>url:https://www.indocsystems.com]].