The scope of this document is to provide the List of Features that went into this release and to list out the known issues. This document also ensures that all the listed features for this release are validated as per SAAMA’s validation process.
As part of the CDH 4.4 release, the features listed below are implemented on AWS:
New! Incremental File Ingestion across Global & Study Admin
The CDH application has the ability to load large and frequently updated datasets in an efficient way. This is achieved through the use of incremental loading, which only loads new data based on a specific column identifier (last refresh date or sequence ID) in the dataset.
This feature is especially beneficial for managing third-party (patient) data or operational data.
Enhancement! Increased performance in handling huge volumes of data
The PDL and ODL jobs will facilitate the direct reading and writing of large incoming data files in Parquet, Avro or ORC formats into the CDH database, thereby obviating the need for conversion to CSV format. The objective of this approach is to achieve good performance and to decrease the probability of job execution failures.
Enhancement! Handling multiple Parquet file load
The data in multiple Parquet files is loaded in discrete portions, which are subsequently persisted in a temporary table. These individual segments are then amalgamated and persisted in the main table.
Enhancement! Increased performance of Audit table
The Audit History table has been overhauled to enable the loading and handling of large volumes of audit records from source, while mitigating performance overload concerns during data load job executions.
New! SCALE adapter code fix
The SCALE adapter EDC Protocol First has been applied with code fix to resolve data loss issues in the Client environment. A new endpoint, namely, GetTransactionStatus, has also been implemented to get required information such as TransactionXmlRemaining, TotalTransactionCount and so on.
The details of the requirements are available in Requirement Specification Document (Reference Document ID: CDH-RSD-001, v12.0)
Known Bugs/De-Scoped Functionalities/Limitation
Below is the list of Deferred bugs from this release:
Impact & Justification
Product Release No.
The dataset summary is missed in the logs when a custom job pull is executed.
There is no impact on CDH v4.4 due to this defect. This defect is deferred and will be taken up as an enhancement, for future release, that will be added to Job execution log summary output to capture the datasets that were not processed and to obtain clear information related with Custom Job failure.
ODL extract job fails for big data files
There is no impact on CDH v4.4 due to this defect. We need to consider the redesign of the extract process to handle large files more efficiently. Deferring this defect to be considered for a future release.
The Metadata Approval status shows as ‘In Progress’ for a long time.
There is no such impact on CDH v4.4 due to this defect. However, this is deferred and will be addressed in the future release as we need a permanent solution to the Metadata Approval problem.
For now, as a workaround we have created a standard operating procedure to temporarily prevent this issue from occurring.
Rave-ODM job fails due to incorrect date format in one of the patient forms.
This is not a defect on the system adapter end. It’s the source API that is providing incorrect/informal date value. This will be analysed to find & accommodate the best approach at code level. There is no impact on CDH v4.4 due to this defect. Hence deferring this defect to a future release.
The Patient API in Data Browser app is taking one minute to render 9000000 (9 million) records.
There is no impact on CDH v4.4 due to this defect.
This is not a defect. It will be taken up as an enhancement for a future release. As per the Engg. team’s observations & comments and as per the PM team, the application is displaying the count of records on the UI, and this is resulting in a prolonged page refresh activity since the volume of records is large. Discussion is required on how best we can display the count of records and not delay the page refresh activity.
If the study name has a dot (.) in it, then the user does not have access to the Study Registration module.
There is no impact on CDH v4.4 due to this defect. We have not found any occurrence of this defect in the PROD environment since none of our customers have a dot (.) in their study name. Deferring this issue to a future release.
Concurrent execution of GSDM job for a set of studies
There is no impact on CDH v4.4 due to this defect. Due to an out-of-memory error, it is not possible to execute parallel studies. To enable parallel execution, the current system needs to be augmented to a great degree. We have noted this issue with current configuration and will defer this defect and will take it up as an enhancement in a future release.
ODL Job for 98GB file got killed after 6 hrs.
There is no impact on CDH v4.4 due to this defect. The recommended course of action is to retest by extending the default timeout values and adjusting the job configuration parameters. We have noted this issue with current configuration and will defer this defect and will take it up as an enhancement in a future release.
Below is the list of Deferred bugs from Past releases:
Impact & Justification
Product Release No.
An Inform Scale Job that is in execution is marked as Failed when the LSAC server is down.
There is no major impact. The user is required to re-trigger the job when the server is up. As the workaround is mentioned and the functionality was designed as per the planned release scope of LSAC CDH v4.3, this issue will be addressed in the future release.
For a scale job, the Trigger button is enabled (to
trigger/start a job) even when the current api pdl job is running.
There is no impact on functionality. This feature will add protection on multiple job submissions. The job fails if multiple jobs are submitted and hence there is no data loss.
In the Data Browser (Metadata Section), the data length of most columns in cdhops table and of some columns in cdhclinical tables is stated as Null.
There is no impact on the functionality. Instead of seeing a blank field there is a Null value visible in the UI. This issue will be addressed in the future release.
The Audit Trail issues as identified in the CDH application.
This is not actually a defect. This will be addressed as an enhancement of the Audit Trail feature and will be taken up in the future release.
Display adapter description is not present for protocol first.
There is no impact on current release. There is no description available for individual fields in the database and for this reason it's not showing any help info for protocol first. Once the description is added for Protocol First it would be visible. This is a minor defect, with minimal impact, hence deferred to future release. Discussion is agreed & approved by Product Owner.
Role access details not visible in Data Browser for all accounts.
There is no impact on current release. CDH shows the user’s role for all applications thus if it is not visible in the data browser then user can check the role in the Global/Study admin applications. It is not impacting any functionality in the system hence deferred to future release. Discussion is agreed & approved by Product Owner.
CDH-val: CDM extract job status is failed for huge dataset file.
There is no impact on current release. This issue is observed in the LSAC Dev environment due to lower cluster configuration. Nodes within the cluster are 8 core, 32 GB RAM of size hence the job fails to execute for a massive table. Production cluster is big enough to run such a massive load hence failure won’t occur. Hence this defect is deferred to future release. Discussion is agreed & approved by Product Owner.
Switch account is not working on LSAC Console.
There is no impact on current release. Switch account feature is given in all CDH apps; hence if it is not working in LSAC console then users can navigate to other CDH apps, namely, Global Admin, Study Admin or Data Access and then they can switch account. In future release (SCH 5.0), there is no need to access the LSAC console and its access would be restricted. Hence this defect is deferred to future release. Discussion is agreed & approved by Product Owner.
DQ Recon Job not working, not able to see data on DQ.
There is no impact on current release. This will be a fix in the next release due to its impact on timeline and will be shipped as a code fix.