Skip to content

README.md

Latest commit

 

History

History
85 lines (57 loc) · 3.56 KB

README.md

File metadata and controls

85 lines (57 loc) · 3.56 KB
Raw

MIMIC III Data Generation

Data Loading

MIMIC III Clinical Database is a large database of anonymized data of more than forty thousand patients. The data provided at the source, should first be downloaded, and built. The building process is provided by the official Github package. We utilize PostgreSQL version of the built database for quick and convenient data querying.

Data Formatting

Having built the MIMIC III on a PostgreSQL DBMS, the data needs to be preprocessed. This process comprises three essential stages:

  • Database parsing
  • Preprocessing
  • Data cleanup

Database Parsing

The parse_db.py script connects to the database to build essential materialized views, which could further be quieried for quick data extraction.

Usage

python parse_db.py \
    --host localhost \
    --db mimic \
    --user postgres \
    --password postgres \
    --out_path ./formatted

Arguments

  • --host <HOST>: The host of PostgreSQL DBMS containing build MIMIC III database. The default value is set to localhost.
  • --db <DATABASE>: The name of the database of PostgreSQL DBMS, containing build MIMIC III database. The default value is set to mimic.
  • --user <USER>: The user owning the PostgreSQL database, containing build MIMIC III database. The default value is set to postgres.
  • --password <PASSWORD>: The password of the user owning the PostgreSQL database, containing build MIMIC III database. The default value is set to postgres.
  • --out_path <OUT_PATH>: The target directory for intermediate files, produced by the script. The default value is set to ./formatted.

Output

Results will be saved under the path, specified in OUT_PATH. The list of files in the directory is expected to consist of:

  • AKI_KIDIGO_7D_SQL_CREATININE_DBSOURCE.csv
  • labstay_DBSOURCE.csv
  • chart_vitals_stay_DBSOURCE.csv
  • comorbidities_DBSOURCE.csv

Preprocessing

The preprocess.py script combines the data from the previous step with the source data files to build the pre-final version of the dataset.

Usage

python preprocess.py \
    --formatted_path ./formatted \
    --data_path ./data

Arguments

  • --formatted_path <FORMATTED_PATH>: The path, where the files, produced by parse_db.py are stored. The default value is set to ./formatted.
  • --data_path <DATA_PATH>: The path to gzipped MIMIC III data. The default value is set to ./data.

Output

Results will be saved under the path, specified in FORMATTED_PATH. The list of files in the directory is expected to be extended by INFO_DATASET_7days_creatinine.csv.

Data Cleanup

The cleanup.py script cleans up the data from the previous step and prepares it for the analysis.

Usage

python cleanup.py \
    --data_path ./formatted/INFO_DATASET_7days_creatinine.csv \
    --formatted_path ./formatted/data.csv

Arguments

  • --data_path <DATA_PATH>: The path, where the file, produced by preprocess.py (INFO_DATASET_7days_creatinine.csv) is stored. The default value is set to ./formatted/INFO_DATASET_7days_creatinine.csv.
  • --formatted_path <FORMATTED_PATH>: The target path to store the final version of the data, ready for analysis. The default value is set to ./formatted/data.csv.

Output

Results will be saved under the path, specified in FORMATTED_PATH. The list of files in the directory is expected to be extended by data.csv.