add transpiled versions of mimic-iii concepts

Author: alistairewj

mimic-iii/concepts/README.md

@@ -28,11 +28,11 @@ for dialect in postgres duckdb; do
     --destination_dialect "${dialect}" \
     --derived_schema mimiciii_derived \
     --schema_map mimiciii_clinical=mimiciii,mimiciii_notes=mimiciii \
-    --exclude cookbook other-languages functions pivot/pivoted_oasis.sql
+    --exclude cookbook functions
 done
 ```
 
-Source tables are referenced via the `mimiciii` schema (the schema created by the [buildmimic](../buildmimic) scripts), and the derived concepts are created in the `mimiciii_derived` schema. `pivot/pivoted_oasis.sql` is excluded from transpilation as it is not currently runnable on any engine.
+Source tables are referenced via the `mimiciii` schema (the schema created by the [buildmimic](../buildmimic) scripts), and the derived concepts are created in the `mimiciii_derived` schema.
 
 Continuous integration rebuilds both dialects against the [MIMIC-III demo](https://physionet.org/content/mimiciii-demo/1.4/) on every approved pull request and verifies that the PostgreSQL and DuckDB results are identical.
 

mimic-iii/concepts_duckdb/README.md

@@ -0,0 +1,25 @@
+# DuckDB concepts (MIMIC-III)
+
+This folder contains scripts to generate useful abstractions of raw MIMIC-III data ("concepts") in [DuckDB](https://duckdb.org/).
+Almost all of these scripts were generated automatically from the BigQuery SQL dialect using the [sqlglot](https://github.com/tobymao/sqlglot) package.
+If you would like to contribute a correction, do not make it here. Instead, make your correction in the [concepts folder](/mimic-iii/concepts/) using the BigQuery SQL syntax, and regenerate this folder as described in that folder's [README](/mimic-iii/concepts/README.md).
+
+One file is hand-written for DuckDB and has no BigQuery source: [diagnosis/ccs_multi_dx.sql](diagnosis/ccs_multi_dx.sql) loads the ICD-9 to CCS mapping from [diagnosis/ccs_multi_dx.csv.gz](diagnosis/ccs_multi_dx.csv.gz).
+
+## Using these concepts
+
+The scripts assume the MIMIC-III data have been loaded into a `mimiciii` schema of the DuckDB database, and create the concept tables in the `mimiciii_derived` schema. For example, to load the data with the standard DDL:
+
+```sh
+duckdb mimic3.db -c "CREATE SCHEMA mimiciii; CREATE SCHEMA mimiciii_derived;"
+# create the tables in the mimiciii schema and COPY each table's CSV into it,
+# e.g. using mimic-iii/buildmimic/duckdb/duckdb_add_tables.sql after USE mimiciii;
+```
+
+The `duckdb.sql` file calls all the concepts in the correct order and outputs them to the `mimiciii_derived` schema.
+You should connect to your DuckDB database file and run this file from this folder (the paths are relative):
+
+```sh
+cd mimic-iii/concepts_duckdb
+duckdb /path/to/mimic3.db -c ".read duckdb.sql"
+```

mimic-iii/concepts_duckdb/code_status.sql

@@ -0,0 +1,62 @@
+-- THIS SCRIPT IS AUTOMATICALLY GENERATED. DO NOT EDIT IT DIRECTLY.
+DROP TABLE IF EXISTS mimiciii_derived.code_status; CREATE TABLE mimiciii_derived.code_status AS
+WITH t1 AS (
+  SELECT
+    icustay_id,
+    charttime,
+    value,
+    ROW_NUMBER() OVER (PARTITION BY icustay_id ORDER BY charttime NULLS FIRST) AS rnfirst,
+    ROW_NUMBER() OVER (PARTITION BY icustay_id ORDER BY charttime DESC) AS rnlast,
+    CASE WHEN value IN ('Full Code', 'Full code') THEN 1 ELSE 0 END AS fullcode,
+    CASE WHEN value IN ('Comfort Measures', 'Comfort measures only') THEN 1 ELSE 0 END AS cmo,
+    CASE WHEN value = 'CPR Not Indicate' THEN 1 ELSE 0 END AS dncpr,
+    CASE
+      WHEN value IN ('Do Not Intubate', 'DNI (do not intubate)', 'DNR / DNI')
+      THEN 1
+      ELSE 0
+    END AS dni,
+    CASE
+      WHEN value IN ('Do Not Resuscita', 'DNR (do not resuscitate)', 'DNR / DNI')
+      THEN 1
+      ELSE 0
+    END AS dnr
+  FROM mimiciii.chartevents
+  WHERE
+    itemid IN (128, 223758)
+    AND NOT value IS NULL
+    AND value <> 'Other/Remarks'
+    AND (
+      error IS NULL OR error = 0
+    )
+)
+SELECT
+  ie.subject_id,
+  ie.hadm_id,
+  ie.icustay_id,
+  MAX(CASE WHEN rnfirst = 1 THEN t1.fullcode ELSE NULL END) AS fullcode_first,
+  MAX(CASE WHEN rnfirst = 1 THEN t1.cmo ELSE NULL END) AS cmo_first,
+  MAX(CASE WHEN rnfirst = 1 THEN t1.dnr ELSE NULL END) AS dnr_first,
+  MAX(CASE WHEN rnfirst = 1 THEN t1.dni ELSE NULL END) AS dni_first,
+  MAX(CASE WHEN rnfirst = 1 THEN t1.dncpr ELSE NULL END) AS dncpr_first,
+  MAX(CASE WHEN rnlast = 1 THEN t1.fullcode ELSE NULL END) AS fullcode_last,
+  MAX(CASE WHEN rnlast = 1 THEN t1.cmo ELSE NULL END) AS cmo_last,
+  MAX(CASE WHEN rnlast = 1 THEN t1.dnr ELSE NULL END) AS dnr_last,
+  MAX(CASE WHEN rnlast = 1 THEN t1.dni ELSE NULL END) AS dni_last,
+  MAX(CASE WHEN rnlast = 1 THEN t1.dncpr ELSE NULL END) AS DNCPR_last,
+  MAX(t1.fullcode) AS fullcode,
+  MAX(t1.cmo) AS cmo,
+  MAX(t1.dnr) AS dnr,
+  MAX(t1.dni) AS dni,
+  MAX(t1.dncpr) AS dncpr,
+  MIN(CASE WHEN t1.dnr = 1 THEN t1.charttime ELSE NULL END) AS dnr_first_charttime,
+  MIN(CASE WHEN t1.dni = 1 THEN t1.charttime ELSE NULL END) AS dni_first_charttime,
+  MIN(CASE WHEN t1.dncpr = 1 THEN t1.charttime ELSE NULL END) AS dncpr_first_charttime,
+  MIN(CASE WHEN t1.cmo = 1 THEN t1.charttime ELSE NULL END) AS timecmo_chart
+FROM mimiciii.icustays AS ie
+LEFT JOIN t1
+  ON ie.icustay_id = t1.icustay_id
+GROUP BY
+  ie.subject_id,
+  ie.hadm_id,
+  ie.icustay_id,
+  ie.intime