Purpose
The purpose of this release note document is to provide the major features in the CDH 4.4 release of Clinical Data Hub (CDH).
Acronyms & Definitions
Term | Definition |
CDH | Clinical Data Hub |
PDA | Patient Data Analytics |
PDL | Patient Data Lake |
ODL | Operation Data Lake |
CDM | Common Data Model |
USDM | Unified Study Data Model |
EDC | Electronic Data Capture |
DQ | Data Quality |
SDTM | Study Data Tabulation Model |
AWS | Amazon Web Services |
KPI | Key Performance Indicator |
DEL | Data Element Library |
Scope
The scope of this document is to provide the List of Features that went into this release and to list out the known issues. This document also ensures that all the listed features for this release are validated as per SAAMA’s validation process.
Release Notes
As part of the CDH 4.4 release, the features listed below are implemented on AWS:
New! Incremental File Ingestion across Global & Study Admin
The CDH application has the ability to load large and frequently updated datasets in an efficient way. This is achieved through the use of incremental loading, which only loads new data based on a specific column identifier (last refresh date or sequence ID) in the dataset.
This feature is especially beneficial for managing third-party (patient) data or operational data.
Enhancement! Increased performance in handling huge volumes of data
The PDL and ODL jobs will facilitate the direct reading and writing of large incoming data files in Parquet, Avro or ORC formats into the CDH database, thereby obviating the need for conversion to CSV format. The objective of this approach is to achieve good performance and to decrease the probability of job execution failures.
Enhancement! Handling multiple Parquet file load
The data in multiple Parquet files is loaded in discrete portions, which are subsequently persisted in a temporary table. These individual segments are then amalgamated and persisted in the main table.
Enhancement! Increased performance of Audit table
The Audit History table has been overhauled to enable the loading and handling of large volumes of audit records from source, while mitigating performance overload concerns during data load job executions.
New! SCALE adapter code fix
The SCALE adapter EDC Protocol First has been applied with code fix to resolve data loss issues in the Client environment. A new endpoint, namely, GetTransactionStatus, has also been implemented to get required information such as TransactionXmlRemaining, TotalTransactionCount and so on.
The details of the requirements are available in Requirement Specification Document (Reference Document ID: CDH-RSD-001, v12.0)
Known Bugs/De-Scoped Functionalities/Limitation
Below is the list of Deferred bugs from this release:
ID # | JIRA Description | Impact & Justification | Product Release No. |
PDA-8986 | The dataset summary is missed in the logs when a custom job pull is executed. | There is no impact on CDH v4.4 due to this defect. This defect is deferred and will be taken up as an enhancement, for future release, that will be added to Job execution log summary output to capture the datasets that were not processed and to obtain clear information related with Custom Job failure. | CDH v4.4 |
PDA-9009 | ODL extract job fails for big data files | There is no impact on CDH v4.4 due to this defect. We need to consider the redesign of the extract process to handle large files more efficiently. Deferring this defect to be considered for a future release. | CDH v4.4 |
PDA-8977 | The Metadata Approval status shows as ‘In Progress’ for a long time. | There is no such impact on CDH v4.4 due to this defect. However, this is deferred and will be addressed in the future release as we need a permanent solution to the Metadata Approval problem. For now, as a workaround we have created a standard operating procedure to temporarily prevent this issue from occurring. | CDH v4.4 |
PDA-9203 | Rave-ODM job fails due to incorrect date format in one of the patient forms. | This is not a defect on the system adapter end. It’s the source API that is providing incorrect/informal date value. This will be analysed to find & accommodate the best approach at code level. There is no impact on CDH v4.4 due to this defect. Hence deferring this defect to a future release. | CDH v4.4 |
PDA-9204 | The Patient API in Data Browser app is taking one minute to render 9000000 (9 million) records. | There is no impact on CDH v4.4 due to this defect. This is not a defect. It will be taken up as an enhancement for a future release. As per the Engg. team’s observations & comments and as per the PM team, the application is displaying the count of records on the UI, and this is resulting in a prolonged page refresh activity since the volume of records is large. Discussion is required on how best we can display the count of records and not delay the page refresh activity. | CDH v4.4 |
PDA-9192 | If the study name has a dot (.) in it, then the user does not have access to the Study Registration module. | There is no impact on CDH v4.4 due to this defect. We have not found any occurrence of this defect in the PROD environment since none of our customers have a dot (.) in their study name. Deferring this issue to a future release. | CDH v4.4 |
PDA-9194 | Concurrent execution of GSDM job for a set of studies | There is no impact on CDH v4.4 due to this defect. Due to an out-of-memory error, it is not possible to execute parallel studies. To enable parallel execution, the current system needs to be augmented to a great degree. We have noted this issue with current configuration and will defer this defect and will take it up as an enhancement in a future release. | CDH v4.4 |
PDA-9219 | ODL Job for 98GB file got killed after 6 hrs. | There is no impact on CDH v4.4 due to this defect. The recommended course of action is to retest by extending the default timeout values and adjusting the job configuration parameters. We have noted this issue with current configuration and will defer this defect and will take it up as an enhancement in a future release. | CDH v4.4 |
Below is the list of Deferred bugs from Past releases:
ID # | JIRA Description | Impact & Justification | Product Release No. |
PDA-8733 | An Inform Scale Job that is in execution is marked as Failed when the LSAC server is down. | There is no major impact. The user is required to re-trigger the job when the server is up. As the workaround is mentioned and the functionality was designed as per the planned release scope of LSAC CDH v4.3, this issue will be addressed in the future release. | CDH v4.3 |
PDA-8740 | For a scale job, the Trigger button is enabled (to trigger/start a job) even when the current api pdl job is running.
| There is no impact on functionality. This feature will add protection on multiple job submissions. The job fails if multiple jobs are submitted and hence there is no data loss. | CDH v4.3 |
PDA-8764 | In the Data Browser (Metadata Section), the data length of most columns in cdhops table and of some columns in cdhclinical tables is stated as Null. | There is no impact on the functionality. Instead of seeing a blank field there is a Null value visible in the UI. This issue will be addressed in the future release. | CDH v4.3 |
PDA-8798 | The Audit Trail issues as identified in the CDH application. | This is not actually a defect. This will be addressed as an enhancement of the Audit Trail feature and will be taken up in the future release. | CDH v4.3 |
PDA-8127 | Display adapter description is not present for protocol first. | There is no impact on current release. There is no description available for individual fields in the database and for this reason it's not showing any help info for protocol first. Once the description is added for Protocol First it would be visible. This is a minor defect, with minimal impact, hence deferred to future release. Discussion is agreed & approved by Product Owner. | CDH v4.0 |
PDA-8142 | Role access details not visible in Data Browser for all accounts. | There is no impact on current release. CDH shows the user’s role for all applications thus if it is not visible in the data browser then user can check the role in the Global/Study admin applications. It is not impacting any functionality in the system hence deferred to future release. Discussion is agreed & approved by Product Owner. | CDH v4.0 |
PDA-8265 | CDH-val: CDM extract job status is failed for huge dataset file. | There is no impact on current release. This issue is observed in the LSAC Dev environment due to lower cluster configuration. Nodes within the cluster are 8 core, 32 GB RAM of size hence the job fails to execute for a massive table. Production cluster is big enough to run such a massive load hence failure won’t occur. Hence this defect is deferred to future release. Discussion is agreed & approved by Product Owner. | CDH v4.0 |
PDA-8205 | Switch account is not working on LSAC Console. | There is no impact on current release. Switch account feature is given in all CDH apps; hence if it is not working in LSAC console then users can navigate to other CDH apps, namely, Global Admin, Study Admin or Data Access and then they can switch account. In future release (SCH 5.0), there is no need to access the LSAC console and its access would be restricted. Hence this defect is deferred to future release. Discussion is agreed & approved by Product Owner. | CDH v4.0 |
PDA-8385 | DQ Recon Job not working, not able to see data on DQ. | There is no impact on current release. This will be a fix in the next release due to its impact on timeline and will be shipped as a code fix. | CDH v4.0.3 |
Bug Fixes
Below is the list of Resolved Issues from this release:
ID # | JIRA Description | Resolution Status |
PDA-8996 | The form data is not reflecting in the cdhops_subject_form table after running the EDC P1 job. This is causing huge data loss. | Resolved. |
PDA-8995 | On executing the EDC P1 job, deleted AE records are still visible in the RAW_PDL and CQS layer. | Resolved. |
PDA-8418 | When DQ is applied, the ODL job takes more time to complete and fails due to default partition and batch sizes. | Resolved. |
PDA-8687 | The DEL approval file is not executed successfully in all new/old CDH environments. | Resolved. |
PDA-8857 | Dataset records are missing in the file that is downloaded from the ODL/PDL extract. | Resolved. |
PDA-8858 | The download functionality is not working on datasets with more than 10K records. | Resolved. |
PDA-8726 | There is no notification of a job failure for Custom_KPI and Medical Review jobs. | Resolved. |
References
Sr. No. | Document Name | Document ID |
1 | Project Change Request and Impact Analysis – CDH 4.4 | CR-CDH-012 |
2 | Quality Plan - CDH 4.4 | CDH-4.4-QP-001 |
3 | CDH - Requirement Specification Document | CDH-RSD-001 |
4 | CDH 4.4 - Quality Summary Report | CDH-4.4-QSR-001 |
Technical Support
Customers can submit issues via our support portal (support.saama.com) or calling the support line (866) 619-9995 during normal business hours. For any other query customers can contact us on the email address: support@saama.com
Annexure
Not Applicable.