Problem (general statement)
There are increasing numbers of outages in the HL7 feed, causing issues with the accuracy of data on EMAP. @stefpiatek has a log of recent outages over the last couple of months.
For example, hospital visits appear to be ongoing in EMAP Star, when Clarity records them as finished.
This creates problems identifying the patients currently in the hospital at a point in time, as a query against star.hospital_visit picks up erroneously unfinished visits.
Problem this causes for the patient flow predictor
We can't produce a census of current ED/SDEC or current inpatients, using real-time data from EMAP. Our census is too large, because we include the unfinished visits. This is very serious problem, meaning that our application can't deliver on its goal of providing real-time insights.
Certain workarounds are possible but highly undesirable, for example:
- checking clarity for discharge datetimes and other transfer events, but this negates the value of EMAP's real-time status
- imposing some kind of cut-off on how long we consider a visit to be an ongoing visit; this might be tenable for ED (eg only consider visits of less than 24 hours) but very problematic for inpatient visits
Possible solution
Switch to hoovering all historical data from caboodle and clarity, and then only process the HL7 feed from the start of a run.
According to @stefpiatek , this is not trivial because it would need a work up of SQL scripts for all data which we don't already parse in hoover, which is most of the HL7 feed (except for flowsheets)
Problem (general statement)
There are increasing numbers of outages in the HL7 feed, causing issues with the accuracy of data on EMAP. @stefpiatek has a log of recent outages over the last couple of months.
For example, hospital visits appear to be ongoing in EMAP Star, when Clarity records them as finished.
This creates problems identifying the patients currently in the hospital at a point in time, as a query against
star.hospital_visitpicks up erroneously unfinished visits.Problem this causes for the patient flow predictor
We can't produce a census of current ED/SDEC or current inpatients, using real-time data from EMAP. Our census is too large, because we include the unfinished visits. This is very serious problem, meaning that our application can't deliver on its goal of providing real-time insights.
Certain workarounds are possible but highly undesirable, for example:
Possible solution
Switch to hoovering all historical data from caboodle and clarity, and then only process the HL7 feed from the start of a run.
According to @stefpiatek , this is not trivial because it would need a work up of SQL scripts for all data which we don't already parse in hoover, which is most of the HL7 feed (except for flowsheets)