Real World Testing Real World Testing Results 2022

menu close

Real World Testing Results 2022 (RWT)

General Information

Developer Name CloudCraft Software
Product Name CloudCraft Software
Version Number 9.0
Certified Health IT Edition 2015 Cures Update
Product List (CHPL) ID 15.04.04.3071.Clou.09.01.1.221227
Real World Testing Public URL https://cloudcraftsoftware.com/realworldtesting.html

Changes to Original Plan

Summary of Change Reason Impact
Test data instead of production data was used in testing (b)(1) Transitions of Care. Providers are still largely dependent on the fax machine for transferring records. There was only one instance of a production user performing a TOC electronically. While the single electronic TOC was demonstrably successful, we chose to perform several more TOCs using synthetic data in order to fully capture the system’s functionality.

We will also be implementing strategic policies and training plans to encourage more providers to adopt the electronic method, as it is faster, more secure, and more accurate.
CCDAs to ACOs versus qrda files The ACO specifications are not ingesting qrda files, only CCDAs Since the ACOs are monitoring care gaps and assisting our customer in closing those gaps through daily huddle sheets, the customers are not using these features in CloudCraft in 2022.
Pre-emptive CQMs for providers to pre plan for patients on the calendar in order to close care gaps. There is a latency for ACOs to ingest data and provide daily huddle sheets and analytics. CloudCraft, in 2023, shall highlight patient CQMs real time during the visit. This will remind doctors to cover these CQMs at the point of care, and auto-order procedures that close these gaps.
Serve as an audit against the ACOs data. Provide analytics that compare with the ACO analytics for auditing purposes Sending data to the ACOs for ingesting is fairly a black box. We know what we sent, but the analytic output needs to be audited.
Verified CQMs in Cypress rather than with live data. See above for current implementation with ACOs. Test completed with test data rather than production data.
Verified Health Care Surveys against NIST tool. The functionality has not yet been adopted widely in production. Results reflect test data rather than production data.
Altered the test procedure for Health Care Surveys from what is presented on the RWT plan. The planned steps were based on a previous understanding of f7; the actual testing was in line with the current ONC standard. The steps completed during testing do not match what was on the RWT plan.
Verified eCR files are created via trigger codes and are compliant to standard. Electronic case reporting is not currently being used by trading partners. Test completed with test data rather than production data.

Summary of Testing Methods and Key Findings

Accurate and secure transmission of patient health information is paramount to ensuring interoperability between different health IT solutions. We chose to demonstrate this capability in CloudCraft by performing transitions-of-care for a number of patients directly from internal provider to external provider, by bulk exporting a batch of C-CDA files containing patient data, transmitting public health data to appropriate registries, and accessing patient information via FHIR API. In each scenario, we confirmed that the data transmitted and/or exported was accurate, up to date, and in conformance to ONC standards.

Though our trading partners are not currently transmitting Electronic Case Reporting functionality to public health registries, we have ensured that the functionality is in place in preparation for future adoption. There is a patient check out event which automatically checks against a table of “trigger codes” composed of diagnoses, test results, and lab orders, and if the trigger event results in a positive match of any of these codes, an eCR document is automatically created. This document has been verified as compliant with the latest standards. Additionally, future plans include utilizing the eCR Now FHIR® application to deliver data to the CDC.

Several patients were successfully accessed via the app MyLinks utilizing the Dynamic FHIR API. All patient information was thoroughly inspected to ensure the data shown in MyLinks was identical to the original data in CloudCraft. No discrepancies or missing data were found.

The below criteria were tested:

§ 170.315(b)(1) Transitions of care
§ 170.315(b)(2) Clinical information reconciliation and incorporation
§ 170.315(b)(6) Data export
§ 170.315(c)(1) - Clinical quality measures (CQMs) — record and export
§ 170.315(c)(2) - Clinical quality measures (CQMs) — import and calculate
§ 170.315(c)(3) - Clinical quality measures (CQMs) — report
§ 170.315(f)(1) Transmission to immunization registries
§ 170.315(f)(5) Transmission to public health agencies — electronic case reporting
§ 170.315(f)(7) Transmission to public health agencies — health care surveys
§ 170.315(g)(7) Application access— patient selection
§ 170.315(g)(8) Application access— data category request
§ 170.315(g)(9) Application access— all data request
§ 170.315(h)(1) Direct Project

Care setting(s) that were tested for real world interoperability

Ambulatory

Voluntary updates to the standards and implementation specifications approved through SVAP (if applicable)

N/A

Metrics and Outcomes

Measurement/Metric Associated Criteria Outcomes Relied Upon Software
80% or more of outbound TOC's successfully received by HISP (b)(1) Transition of Care

(h)(1) Direct Project: from the Electronic Exchange Category		 75% successfully received (172/256) ConnectEHR
75% or more of trading partner's TOC C-CDAs are successfully incorporated by SUT. (b)(1) Transition of Care

(b)(2) Clinical information reconciliation and incorporation

(h)(1) Direct Project: from the Electronic Exchange Category 		93% successfully incorporated (574/616) ConnectEHR
Work towards 10% of patients where a referral is initiated complete the referral loop using DIRECT. (b)(1) Transition of Care

(h)(1) Direct Project: from the Electronic Exchange Category 		100% completed referral loop (7/7) ConnectEHR
100% of Exports ran timely, or a user friendly error will be provided. (b)(6) - Data export 100% of exports ran at the configured time ConnectEHR
C-CDA count matches actual patient count for requested selection criteria. (b)(6) - Data export All patients were requested for export, and all C-CDAs were generated ConnectEHR
100% matching data elements in CQMsolution vs EHR. This will be confirmed by visual validation of the following data:
  • Demographics
  • Problems/Diagnosis
  • Medications
  • Allergies
  • Vitals
  • Results
  • Procedures
 (c)(1) - Clinical quality measures (CQMs) — record and export All data for all test patients matched what was received in CQMsolution. CQMsolution
100% discrepancy will be identified thru Quarterly Reconciliation of measures calculation between ACO calculations and SUT calculations (c)(2) - Clinical quality measures (CQMs) — import and calculate

(c)(3) - Clinical quality measures (CQMs) — report 		Calculation of measures in CQMsolution matched the Cypress calculation when the test patient QRDA files were uploaded to Cypress. CQMsolution
100% correct immunization history records successfully received in EHR confirmed by visual validation. (f)(1) Transmission to immunization registries 100% completed ConnectEHR
Successful Transmission to Public Health Registry will be reviewed for ACK & NAK to ensure 100% successful transmission. (f)(1) Transmission to immunization registries 100% completed ConnectEHR
100% correct immunization records successfully posted to registry confirmed by visual validation. (f)(1) Transmission to immunization registries 100% completed ConnectEHR
100% of eCR messages successfully received and processed by public health agency based on either:
a) Logging into agency web site and validating, or
b) Using a report provided by agency 		(f)(5) Transmission to public health agencies — electronic case reporting 100% of eCR messages triggered appropriately and are valid according to current standards.

Metric adjusted due to pending production adoption by trading partners. 		ConnectEHR
Survey document is created according to HL7 guidelines and passes the ONC test tool. (f)(7) Transmission to public health agencies — health care surveys Created and passed. NA
Patient is able to retrieve FHIR API data from PHR app for 100% of encounters. (g)(7) Application access— patient selection

(g)(8) Application access— data category request

(g)(9) Application access— all data request 		100% data retrieved. ConnectEHR
Dynamic FHIR
MyLinks
In 100% of encounters from Step #1, PHR data matches data from EHR. This will be confirmed by visual validation of CCDS. (g)(7) Application access— patient selection

(g)(8) Application access— data category request

(g)(9) Application access— all data request		 All information matched. ConnectEHR
Dynamic FHIR
MyLinks

Key Milestones

Key Milestone Care Setting Date/Timeframe
  • Confirm Trading Partner
  • Confirm ability to send and receive clinical documents
  • Confirm with TP that production data will be used, whether in an actual live environment or a copy of a live environment
 Ambulatory May 2022
  • SUT links to the correct Direct addresses and initiates sending of Clinical Document.
  • C-CDA Care Referral or Referral Note is triggered to send via Direct Protocol
  • Referral Dept. checks Direct Status screen (under Direct Outgoing menu choice) to ensure Clinical Document was successfully transmitted.
Ambulatory June 2022
Recipient uses scorecard to grade C-CDA Ambulatory July 2022
  • Tester uses Document Center to locate Clinical Document.
  • Care provider reviews the Direct Status screen (under Direct Outgoing menu choice).
  • Recipient validates that Social History section of C-CDA is flagged as restricted
Ambulatory August 2022
  • The care provider reviews the record, and the patient's problems, medications, and medication allergies are reviewed and optionally merged into the system under test with no duplicates.
  • The data from the TP becomes part of the patients permanent record.
  • The main dashboard is updated to accurately reflect for the provider all 3rd party visits to be reconciled
Ambulatory September 2022
  • Date and time ranges can be configurable via the UI
  • Targeted Practices can be configurable via the UI
  • Patients exported can be configurable via the UI
Ambulatory July 2022
Use the Edge Test Tool to check validity of output file(s) Ambulatory July 2022
  • Has a state immunization registry that is enabled for bi-directional send/receive of immunization data.
  • Already has a functional bi-directional immunization interface or would like to implement one to their registry.
 Ambulatory July 2022
Validate that immunization interface is functioning as expected Ambulatory July 2022
Verify immunization data was received in registry. Ambulatory July 2022
Verify immunization data was received in EHR. Ambulatory July 2022
eCR messages were successfully received and processed by public health agency. Ambulatory July 2022
Confirmed functioning eCR interface to public health agency Ambulatory July 2022
  • Confirm Trading Partner
  • Confirm ability to calculate and report eCQMs
  • Confirm with TP that production data will be used, whether in an actual live environment or a copy of a live environment
Ambulatory August 2022
The file should upload and be accepted by the environment without error. Ambulatory August 2022
All populations of all measures should match. Ambulatory August 2022
  • Partner with PHR or identify existing PHR that can receive patient clinical data as described in this RWT plan.
  • Ensure that PHR has functionality to access the Dynamic FHIR API, as described here.
  • Partner with EHR that is integrated with the Dynamic FHIR API and Patient Portal modules of ConnectEHR.
Ambulatory August 2022
Encounter is created and visually confirmed Ambulatory August 2022
  • Dynamic FHIR API has transformed C-CDA into FHIR resources.
  • PHR app consumes FHIR resources to populate EHR data
Ambulatory August 2022
Visually validate Assessment, Plan of Treatment and Health Concerns narrative text Ambulatory August 2022
  • The identified testing location is able to select a subset of survey questions that are relevant to its activities.
  • Location confirms availability of survey option in patient portal.
Ambulatory July 2022
Responses are converted to HL7 CDA documents for transmission. Ambulatory July 2022