diff --git a/statvar_imports/eurostat/live_births_total_by_month/README.md b/statvar_imports/eurostat/live_births_total_by_month/README.md new file mode 100644 index 0000000000..7482f29e80 --- /dev/null +++ b/statvar_imports/eurostat/live_births_total_by_month/README.md @@ -0,0 +1,87 @@ +# Eurostat Live Births Total By Month Import + +## Overview +This dataset contains monthly live births data at national and regional levels, sourced from Eurostat. The data tracks the total number of live births per month across various European countries and regions. + +**type of place:** Country, NUTS Regions (Level 0-3) +**years:** Historical data to present (1960-2025) +**place_resolution:** Resolved to DCIDs (e.g., dcid:country/FRA, dcid:nuts/AT113) + +## Data Source +**Source URL:** +https://ec.europa.eu/eurostat/databrowser/view/DEMO_FMONTH__custom_270818/default/table?lang=en + +**Provenance Description:** +The data is provided by Eurostat, the statistical office of the European Union. It is part of the "Demography and migration" database, specifically the "Live births by month" (DEMO_FMONTH) dataset. + +## Refresh Type +Automatic Refresh + +The refresh is automated using the provided `run.sh` script, which handles both data download and processing. + +## How To Run Import +To execute the complete import process (download and processing), run: +```bash +./run.sh +``` + +### Script Details: +- **Download**: Uses `curl` to fetch the latest SDMX-CSV data from Eurostat's dissemination API. +- **Processing**: Uses `stat_var_processor.py` to map raw data to Data Commons StatVarObservations using the PV map and metadata configuration. + +## Processing Instructions +To process the Eurostat Live Births data and generate statistical variables, use the following commands from your current import data directory: + +Download input file + +```bash +mkdir -p input_files +curl -L --retry 3 "https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/DEMO_FMONTH/?format=SDMX-CSV&compressed=false" -o input_files/live_births_total_by_month_data_input.csv +``` + +For Test Data Run + +```bash +python3 ../../../tools/statvar_importer/stat_var_processor.py \ + "--input_data=./test_data/live_births_total_by_month_data_input.csv" \ + "--pv_map=./live_births_total_by_month_pvmap.csv" \ + "--output_path=./test_data/live_births_total_by_month_output" \ + "--config_file=./live_births_total_by_month_metadata.csv" \ + "--existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf" +``` + +For Main data processing run + +```bash +python3 ../../../tools/statvar_importer/stat_var_processor.py \ + "--input_data=./input_files/*.csv" \ + "--pv_map=./live_births_total_by_month_pvmap.csv" \ + "--config_file=./live_births_total_by_month_metadata.csv" \ + "--generate_statvar_name=True" \ + "--skip_constant_csv_columns=False" \ + "--output_columns=observationDate,observationAbout,variableMeasured,value,observationPeriod,unit" \ + "--output_path=output_files/live_births_total_by_month_output" \ + "--places_resolved_csv=./places_resolved.csv" \ + "--existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf" +``` + +## Key Files +- `run.sh`: Main execution script for download and processing. +- `live_births_total_by_month_pvmap.csv`: Property-Value mapping for StatVar definitions and dimensions. +- `live_births_total_by_month_metadata.csv`: Configuration parameters for the processor. +- `places_resolved.csv`: Mapping of place codes to Data Commons DCIDs. +- `output_files/live_births_total_by_month_output.csv`: Processed statistical observations. +- `output_files/live_births_total_by_month_output.tmcf`: Template MCF mapping the CSV columns to Data Commons schema. + +## Validation +To validate the generated data, use the Data Commons import tool (lint mode): +```bash +java -jar datacommons-import-tool.jar lint output_files/live_births_total_by_month_output.csv output_files/live_births_total_by_month_output.tmcf +``` +The resulting reports (`report.json`, `summary_report.html`) in `dc_generated/` provide detailed insights into data quality and validation status. + +## Testing +Testing is performed using the `test_data` directory: +- Raw Input: `test_data/live_births_total_by_month_data_input.csv` +- Expected Output: `test_data/live_births_total_by_month_output.csv` +- Expected TMCF: `test_data/live_births_total_by_month_output.tmcf` diff --git a/statvar_imports/eurostat/live_births_total_by_month/live_births_total_by_month_metadata.csv b/statvar_imports/eurostat/live_births_total_by_month/live_births_total_by_month_metadata.csv new file mode 100644 index 0000000000..a5e0ad3df3 --- /dev/null +++ b/statvar_imports/eurostat/live_births_total_by_month/live_births_total_by_month_metadata.csv @@ -0,0 +1,3 @@ +parameter,value +header_rows,1 + diff --git a/statvar_imports/eurostat/live_births_total_by_month/live_births_total_by_month_pvmap.csv b/statvar_imports/eurostat/live_births_total_by_month/live_births_total_by_month_pvmap.csv new file mode 100644 index 0000000000..1463b616bc --- /dev/null +++ b/statvar_imports/eurostat/live_births_total_by_month/live_births_total_by_month_pvmap.csv @@ -0,0 +1,49 @@ +key,property1,value1,property2,value2,property3,value3,property4,value4 + +# Global Dataflow properties (Standard Data Commons Demographic Modeling) +"DATAFLOW:ESTAT:DEMO_FMONTH(1.0)",populationType,dcs:BirthEvent,measuredProperty,dcs:count,statType,dcs:measuredValue,, + +# --- AUTOMATED CLEANUP: Ignored Columns --- +LAST UPDATE,#ignore,"",,,,,, +freq,#ignore,"",,,,,, +CONF_STATUS,#ignore,"",,,,,, + +# Month mapping & observationPeriod variable passing (Grounded on standard ISO-8601 durations) +month:M01,Month,01,observationPeriod,P1M,,,, +month:M02,Month,02,observationPeriod,P1M,,,, +month:M03,Month,03,observationPeriod,P1M,,,, +month:M04,Month,04,observationPeriod,P1M,,,, +month:M05,Month,05,observationPeriod,P1M,,,, +month:M06,Month,06,observationPeriod,P1M,,,, +month:M07,Month,07,observationPeriod,P1M,,,, +month:M08,Month,08,observationPeriod,P1M,,,, +month:M09,Month,09,observationPeriod,P1M,,,, +month:M10,Month,10,observationPeriod,P1M,,,, +month:M11,Month,11,observationPeriod,P1M,,,, +month:M12,Month,12,observationPeriod,P1M,,,, +month:TOTAL,Month,"",observationPeriod,P1Y,,,, +month:UNK,#ignore,"",,,,,, + +# Unit dimension (dcs:Person is standard for demographic counts) +unit:NR,unit,dcs:Person,,,,,, + +# Time column capture (Stores year for the strict date checker) +TIME_PERIOD,Year,{Data},,,,,, + +geo,observationAbout,{Data},,,,,, + +# Measure and Date Resolution +OBS_VALUE,#Eval,"observationDate='{Year}-{Month}' if '{Month}' else '{Year}'",value,{Number},,,, + +# Safely Ignoring Transient and Composite Status Flags to Prevent Ingestion Warnings +OBS_FLAG:b,#ignore,"",,,,,, +OBS_FLAG:e,#ignore,"",,,,,, +OBS_FLAG:p,#ignore,"",,,,,, +OBS_FLAG:f,#ignore,"",,,,,, +OBS_FLAG:n,#ignore,"",,,,,, +OBS_FLAG:u,#ignore,"",,,,,, +OBS_FLAG:be,#ignore,"",,,,,, +OBS_FLAG:bep,#ignore,"",,,,,, + +# Missing values +OBS_FLAG:M,MissingValue,"",,,,,, diff --git a/statvar_imports/eurostat/live_births_total_by_month/manifest.json b/statvar_imports/eurostat/live_births_total_by_month/manifest.json new file mode 100644 index 0000000000..19f65f9c53 --- /dev/null +++ b/statvar_imports/eurostat/live_births_total_by_month/manifest.json @@ -0,0 +1,34 @@ +{ + "import_specifications": [ + { + "import_name": "EuroStat_Live_Births_Total_By_Month", + "curator_emails": [ + "support@datacommons.org" + ], + "provenance_url": "https://ec.europa.eu/eurostat/databrowser/view/DEMO_FMONTH__custom_270818/default/table?lang=en", + "provenance_description": "Annual tracking of the number of live births by month across reporting European countries", + "scripts": [ + "run.sh" + ], + "import_inputs": [ + { + "template_mcf": "output_files/live_births_total_by_month_output.tmcf", + "cleaned_csv": "output_files/live_births_total_by_month_output.csv" + } + ], + "source_files": [ + "input_files/*.csv" + ], + "cron_schedule": "5 1 1,15 * *", + "resource_limits": {"cpu": 4, "memory": 8, "disk":100}, + "config_override": { + "invoke_import_validation": true, + "invoke_import_tool": true, + "invoke_differ_tool": true, + "skip_input_upload": false, + "skip_gcs_upload": false, + "cleanup_gcs_volume_mount": false + } + } + ] +} diff --git a/statvar_imports/eurostat/live_births_total_by_month/places_resolved.csv b/statvar_imports/eurostat/live_births_total_by_month/places_resolved.csv new file mode 100644 index 0000000000..acc65e505e --- /dev/null +++ b/statvar_imports/eurostat/live_births_total_by_month/places_resolved.csv @@ -0,0 +1,265 @@ +"place_name","dcid","placeId","wikidataId","name","alternateName","typeOf","containedInPlace" +"AD","dcid:country/AND","","","","","","" +"AE","dcid:country/ARE","","","","","","" +"AF","dcid:country/AFG","","","","","","" +"AG","dcid:country/ATG","","","","","","" +"AI","dcid:country/AIA","","","","","","" +"AL","dcid:country/ALB","","","","","","" +"AM","dcid:country/ARM","","","","","","" +"AN","dcid:country/ANT","","","","","","" +"AO","dcid:country/AGO","","","","","","" +"AQ","dcid:country/ATA","","","","","","" +"AR","dcid:country/ARG","","","","","","" +"AS","dcid:country/ASM","","","","","","" +"AT","dcid:country/AUT","","","","","","" +"AU","dcid:country/AUS","","","","","","" +"AW","dcid:country/ABW","","","","","","" +"AZ","dcid:country/AZE","","","","","","" +"BA","dcid:country/BIH","","","","","","" +"BB","dcid:country/BRB","","","","","","" +"BD","dcid:country/BGD","","","","","","" +"BE","dcid:country/BEL","","","","","","" +"BF","dcid:country/BFA","","","","","","" +"BG","dcid:country/BGR","","","","","","" +"BH","dcid:country/BHR","","","","","","" +"BI","dcid:country/BDI","","","","","","" +"BJ","dcid:country/BEN","","","","","","" +"BL","dcid:country/BLM","","","","","","" +"BM","dcid:country/BMU","","","","","","" +"BN","dcid:country/BRN","","","","","","" +"BO","dcid:country/BOL","","","","","","" +"BR","dcid:country/BRA","","","","","","" +"BS","dcid:country/BHS","","","","","","" +"BT","dcid:country/BTN","","","","","","" +"BV","dcid:country/BVT","","","","","","" +"BW","dcid:country/BWA","","","","","","" +"BY","dcid:country/BLR","","","","","","" +"BZ","dcid:country/BLZ","","","","","","" +"CA","dcid:country/CAN","","","","","","" +"CC","dcid:country/CCK","","","","","","" +"CD","dcid:country/COD","","","","","","" +"CF","dcid:country/CAF","","","","","","" +"CG","dcid:country/COG","","","","","","" +"CH","dcid:country/CHE","","","","","","" +"CI","dcid:country/CIV","","","","","","" +"CK","dcid:country/COK","","","","","","" +"CL","dcid:country/CHL","","","","","","" +"CM","dcid:country/CMR","","","","","","" +"CN","dcid:country/CHN","","","","","","" +"CO","dcid:country/COL","","","","","","" +"CR","dcid:country/CRI","","","","","","" +"CU","dcid:country/CUB","","","","","","" +"CV","dcid:country/CPV","","","","","","" +"CX","dcid:country/CXR","","","","","","" +"CY","dcid:country/CYP","","","","","","" +"CZ","dcid:country/CZE","","","","","","" +"DE","dcid:country/DEU","","","","","","" +"DJ","dcid:country/DJI","","","","","","" +"DK","dcid:country/DNK","","","","","","" +"DM","dcid:country/DMA","","","","","","" +"DO","dcid:country/DOM","","","","","","" +"DZ","dcid:country/DZA","","","","","","" +"EC","dcid:country/ECU","","","","","","" +"EE","dcid:country/EST","","","","","","" +"EG","dcid:country/EGY","","","","","","" +"EH","dcid:country/ESH","","","","","","" +"EL","dcid:country/GRC","","","","","","" +"ER","dcid:country/ERI","","","","","","" +"ES","dcid:country/ESP","","","","","","" +"ET","dcid:country/ETH","","","","","","" +"FI","dcid:country/FIN","","","","","","" +"FJ","dcid:country/FJI","","","","","","" +"FK","dcid:country/FLK","","","","","","" +"FM","dcid:country/FSM","","","","","","" +"FO","dcid:country/FRO","","","","","","" +"FR","dcid:country/FRA","","","","","","" +"FX","dcid:country/FXX","","","","","","" +"GA","dcid:country/GAB","","","","","","" +"UK","dcid:country/GBR","","","","","","" +"GD","dcid:country/GRD","","","","","","" +"GE","dcid:country/GEO","","","","","","" +"GF","dcid:country/GUF","","","","","","" +"GG","dcid:country/GGY","","","","","","" +"GH","dcid:country/GHA","","","","","","" +"GI","dcid:country/GIB","","","","","","" +"GL","dcid:country/GRL","","","","","","" +"GM","dcid:country/GMB","","","","","","" +"GN","dcid:country/GIN","","","","","","" +"GQ","dcid:country/GNQ","","","","","","" +"GS","dcid:country/SGS","","","","","","" +"GT","dcid:country/GTM","","","","","","" +"GU","dcid:country/GUM","","","","","","" +"GW","dcid:country/GNB","","","","","","" +"GY","dcid:country/GUY","","","","","","" +"HK","dcid:country/HKG","","","","","","" +"HM","dcid:country/HMD","","","","","","" +"HN","dcid:country/HND","","","","","","" +"HR","dcid:country/HRV","","","","","","" +"HT","dcid:country/HTI","","","","","","" +"HU","dcid:country/HUN","","","","","","" +"ID","dcid:country/IDN","","","","","","" +"IE","dcid:country/IRL","","","","","","" +"IL","dcid:country/ISR","","","","","","" +"IM","dcid:country/IMN","","","","","","" +"IN","dcid:country/IND","","","","","","" +"IO","dcid:country/IOT","","","","","","" +"IQ","dcid:country/IRQ","","","","","","" +"IR","dcid:country/IRN","","","","","","" +"IS","dcid:country/ISL","","","","","","" +"IT","dcid:country/ITA","","","","","","" +"JE","dcid:country/JEY","","","","","","" +"JM","dcid:country/JAM","","","","","","" +"JO","dcid:country/JOR","","","","","","" +"JP","dcid:country/JPN","","","","","","" +"KE","dcid:country/KEN","","","","","","" +"KG","dcid:country/KGZ","","","","","","" +"KH","dcid:country/KHM","","","","","","" +"KI","dcid:country/KIR","","","","","","" +"KM","dcid:country/COM","","","","","","" +"KN","dcid:country/KNA","","","","","","" +"KP","dcid:country/PRK","","","","","","" +"KR","dcid:country/KOR","","","","","","" +"KW","dcid:country/KWT","","","","","","" +"KY","dcid:country/CYM","","","","","","" +"KZ","dcid:country/KAZ","","","","","","" +"LA","dcid:country/LAO","","","","","","" +"LB","dcid:country/LBN","","","","","","" +"LC","dcid:country/LCA","","","","","","" +"LI","dcid:country/LIE","","","","","","" +"LK","dcid:country/LKA","","","","","","" +"LR","dcid:country/LBR","","","","","","" +"LS","dcid:country/LSO","","","","","","" +"LT","dcid:country/LTU","","","","","","" +"LU","dcid:country/LUX","","","","","","" +"LV","dcid:country/LVA","","","","","","" +"LY","dcid:country/LBY","","","","","","" +"MA","dcid:country/MAR","","","","","","" +"MC","dcid:country/MCO","","","","","","" +"MD","dcid:country/MDA","","","","","","" +"ME","dcid:country/MNE","","","","","","" +"MF","dcid:country/MAF","","","","","","" +"MG","dcid:country/MDG","","","","","","" +"MH","dcid:country/MHL","","","","","","" +"MK","dcid:country/MKD","","","","","","" +"ML","dcid:country/MLI","","","","","","" +"MM","dcid:country/MMR","","","","","","" +"MN","dcid:country/MNG","","","","","","" +"MO","dcid:country/MAC","","","","","","" +"MP","dcid:country/MNP","","","","","","" +"MQ","dcid:country/MTQ","","","","","","" +"MR","dcid:country/MRT","","","","","","" +"MS","dcid:country/MSR","","","","","","" +"MT","dcid:country/MLT","","","","","","" +"MU","dcid:country/MUS","","","","","","" +"MV","dcid:country/MDV","","","","","","" +"MW","dcid:country/MWI","","","","","","" +"MX","dcid:country/MEX","","","","","","" +"MY","dcid:country/MYS","","","","","","" +"MZ","dcid:country/MOZ","","","","","","" +"NA","dcid:country/NAM","","","","","","" +"NC","dcid:country/NCL","","","","","","" +"NE","dcid:country/NER","","","","","","" +"NF","dcid:country/NFK","","","","","","" +"NG","dcid:country/NGA","","","","","","" +"NI","dcid:country/NIC","","","","","","" +"NL","dcid:country/NLD","","","","","","" +"NO","dcid:country/NOR","","","","","","" +"NP","dcid:country/NPL","","","","","","" +"NR","dcid:country/NRU","","","","","","" +"NU","dcid:country/NIU","","","","","","" +"NZ","dcid:country/NZL","","","","","","" +"OM","dcid:country/OMN","","","","","","" +"PA","dcid:country/PAN","","","","","","" +"PE","dcid:country/PER","","","","","","" +"PF","dcid:country/PYF","","","","","","" +"PG","dcid:country/PNG","","","","","","" +"PH","dcid:country/PHL","","","","","","" +"PK","dcid:country/PAK","","","","","","" +"PL","dcid:country/POL","","","","","","" +"PM","dcid:country/SPM","","","","","","" +"PN","dcid:country/PCN","","","","","","" +"PR","dcid:country/PRI","","","","","","" +"PS","dcid:country/PSE","","","","","","" +"PT","dcid:country/PRT","","","","","","" +"PW","dcid:country/PLW","","","","","","" +"PY","dcid:country/PRY","","","","","","" +"QA","dcid:country/QAT","","","","","","" +"RE","dcid:country/REU","","","","","","" +"RO","dcid:country/ROU","","","","","","" +"RS","dcid:country/SRB","","","","","","" +"RU","dcid:country/RUS","","","","","","" +"RW","dcid:country/RWA","","","","","","" +"SA","dcid:country/SAU","","","","","","" +"SB","dcid:country/SLB","","","","","","" +"SC","dcid:country/SYC","","","","","","" +"SD","dcid:country/SDN","","","","","","" +"SE","dcid:country/SWE","","","","","","" +"SG","dcid:country/SGP","","","","","","" +"SH","dcid:country/SHN","","","","","","" +"SI","dcid:country/SVN","","","","","","" +"SJ","dcid:country/SJM","","","","","","" +"SK","dcid:country/SVK","","","","","","" +"SL","dcid:country/SLE","","","","","","" +"SM","dcid:country/SMR","","","","","","" +"SN","dcid:country/SEN","","","","","","" +"SO","dcid:country/SOM","","","","","","" +"SR","dcid:country/SUR","","","","","","" +"ST","dcid:country/STP","","","","","","" +"SV","dcid:country/SLV","","","","","","" +"SX","dcid:country/SXM","","","","","","" +"SY","dcid:country/SYR","","","","","","" +"SZ","dcid:country/SWZ","","","","","","" +"TC","dcid:country/TCA","","","","","","" +"TD","dcid:country/TCD","","","","","","" +"TF","dcid:country/ATF","","","","","","" +"TG","dcid:country/TGO","","","","","","" +"TH","dcid:country/THA","","","","","","" +"TJ","dcid:country/TJK","","","","","","" +"TK","dcid:country/TKL","","","","","","" +"TL","dcid:country/TLS","","","","","","" +"TM","dcid:country/TKM","","","","","","" +"TN","dcid:country/TUN","","","","","","" +"TO","dcid:country/TON","","","","","","" +"TR","dcid:country/TUR","","","","","","" +"TT","dcid:country/TTO","","","","","","" +"TV","dcid:country/TUV","","","","","","" +"TW","dcid:country/TWN","","","","","","" +"TZ","dcid:country/TZA","","","","","","" +"UA","dcid:country/UKR","","","","","","" +"UG","dcid:country/UGA","","","","","","" +"UM","dcid:country/UMI","","","","","","" +"US","dcid:country/USA","","","","","","" +"UY","dcid:country/URY","","","","","","" +"UZ","dcid:country/UZB","","","","","","" +"VA","dcid:country/VAT","","","","","","" +"VC","dcid:country/VCT","","","","","","" +"VE","dcid:country/VEN","","","","","","" +"VG","dcid:country/VGB","","","","","","" +"VI","dcid:country/VIR","","","","","","" +"VN","dcid:country/VNM","","","","","","" +"VU","dcid:country/VUT","","","","","","" +"WF","dcid:country/WLF","","","","","","" +"WS","dcid:country/WSM","","","","","","" +"XK","dcid:country/XKS","","","","","","" +"YE","dcid:country/YEM","","","","","","" +"YT","dcid:country/MYT","","","","","","" +"ZA","dcid:country/ZAF","","","","","","" +"ZM","dcid:country/ZMB","","","","","","" +"ZW","dcid:country/ZWE","","","","","","" +"DE_TOT","dcid:nuts/DE_TOT","","","","","","" +"EA","dcid:nuts/EA","","","","","","" +"EA12","dcid:nuts/EA12","","","","","","" +"EA18","dcid:nuts/EA18","","","","","","" +"EA19","dcid:nuts/EA19","","","","","","" +"EEA","dcid:nuts/EEA","","","","","","" +"EEA30_2007","dcid:nuts/EEA30_2007","","","","","","" +"EEA31","dcid:nuts/EEA31","","","","","","" +"EFTA","dcid:nuts/EFTA","","","","","","" +"EU","dcid:nuts/EU","","","","","","" +"EU27","dcid:nuts/EU27","","","","","","" +"EU27_2007","dcid:nuts/EU27_2007","","","","","","" +"EU27_2020","dcid:nuts/EU27_2020","","","","","","" +"EA20","dcid:nuts/EA20","","","","","","" +"EA21","dcid:nuts/EA21","","","","","","" +"EU28","dcid:nuts/EU28","","","","","","" +"EEA28","dcid:nuts/EEA28","","","","","","" \ No newline at end of file diff --git a/statvar_imports/eurostat/live_births_total_by_month/run.sh b/statvar_imports/eurostat/live_births_total_by_month/run.sh new file mode 100644 index 0000000000..2e6e28ea3d --- /dev/null +++ b/statvar_imports/eurostat/live_births_total_by_month/run.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +# Step 1: Data Download +mkdir -p input_files +mkdir -p output_files +curl -L --retry 3 "https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/DEMO_FMONTH/?format=SDMX-CSV&compressed=false" -o ./input_files/live_births_total_by_month_data_input.csv + +# Step 2: Data Processing + +python3 ../../../tools/statvar_importer/stat_var_processor.py \ + "--input_data=./input_files/*.csv" \ + "--pv_map=./live_births_total_by_month_pvmap.csv" \ + "--config_file=./live_births_total_by_month_metadata.csv" \ + "--generate_statvar_name=True" \ + "--skip_constant_csv_columns=False" \ + "--output_columns=observationDate,observationAbout,variableMeasured,value,observationPeriod,unit" \ + "--output_path=output_files/live_births_total_by_month_output" \ + "--places_resolved_csv=./places_resolved.csv" \ + "--existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf" + \ No newline at end of file diff --git a/statvar_imports/eurostat/live_births_total_by_month/test_data/live_births_total_by_month_data_input.csv b/statvar_imports/eurostat/live_births_total_by_month/test_data/live_births_total_by_month_data_input.csv new file mode 100644 index 0000000000..e348119eea --- /dev/null +++ b/statvar_imports/eurostat/live_births_total_by_month/test_data/live_births_total_by_month_data_input.csv @@ -0,0 +1,100 @@ +DATAFLOW,LAST UPDATE,freq,unit,month,geo,TIME_PERIOD,OBS_VALUE,OBS_FLAG,CONF_STATUS +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AD,2005,69,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AD,2006,52,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AD,2007,78,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AD,2008,67,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AD,2009,69,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AD,2010,69,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AD,2011,68,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AD,2012,59,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AD,2018,40,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,1997,4743,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,1998,4743,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2004,3588,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2005,3092,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2006,3191,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2007,2893,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2013,2839,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2014,2837,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2015,2742,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2016,2517,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2017,2423,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2018,2468,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2019,2393,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2020,2316,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2021,2066,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2022,2277,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2023,1991,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AL,2024,1985,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2005,3319,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2006,3354,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2007,3362,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2008,3389,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2009,3715,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2014,3597,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2015,3444,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2016,4009,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2017,3081,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2018,3031,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2019,3020,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2020,2751,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2022,2890,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AM,2023,3275,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1960,10666,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1961,11372,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1962,11433,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1963,11805,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1964,11343,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1965,11378,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1966,10723,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1967,10993,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1968,11197,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1969,10987,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1970,9759,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1971,9507,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1972,9187,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1973,8750,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1974,8545,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1975,8343,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1976,8047,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1977,7259,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1978,7157,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1979,7554,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1980,7481,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1981,7890,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1982,8142,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1983,7825,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1984,7424,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1985,7546,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1986,7468,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1987,7279,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1988,7442,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1989,7470,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1990,7612,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1991,7925,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1992,7932,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1993,8266,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1994,7798,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1995,7603,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1996,7436,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1997,7318,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1998,6860,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,1999,6697,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2000,6587,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2001,6611,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2002,6594,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2003,6570,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2004,6780,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2005,6482,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2006,6526,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2007,6504,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2008,6739,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2009,6475,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2010,6414,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2011,6651,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2012,6389,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2013,6629,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2014,6666,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2015,6985,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2016,6980,, +ESTAT:DEMO_FMONTH(1.0),20/04/26 23:00:00,A,NR,M01,AT,2017,7040,, diff --git a/statvar_imports/eurostat/live_births_total_by_month/test_data/live_births_total_by_month_output.csv b/statvar_imports/eurostat/live_births_total_by_month/test_data/live_births_total_by_month_output.csv new file mode 100644 index 0000000000..aaec54d7a4 --- /dev/null +++ b/statvar_imports/eurostat/live_births_total_by_month/test_data/live_births_total_by_month_output.csv @@ -0,0 +1,100 @@ +observationDate,observationAbout,variableMeasured,value,observationPeriod,unit +2005-01,dcid:country/AND,dcid:Count_BirthEvent,69,P1M,dcs:Person +2006-01,dcid:country/AND,dcid:Count_BirthEvent,52,P1M,dcs:Person +2007-01,dcid:country/AND,dcid:Count_BirthEvent,78,P1M,dcs:Person +2008-01,dcid:country/AND,dcid:Count_BirthEvent,67,P1M,dcs:Person +2009-01,dcid:country/AND,dcid:Count_BirthEvent,69,P1M,dcs:Person +2010-01,dcid:country/AND,dcid:Count_BirthEvent,69,P1M,dcs:Person +2011-01,dcid:country/AND,dcid:Count_BirthEvent,68,P1M,dcs:Person +2012-01,dcid:country/AND,dcid:Count_BirthEvent,59,P1M,dcs:Person +2018-01,dcid:country/AND,dcid:Count_BirthEvent,40,P1M,dcs:Person +1997-01,dcid:country/ALB,dcid:Count_BirthEvent,4743,P1M,dcs:Person +1998-01,dcid:country/ALB,dcid:Count_BirthEvent,4743,P1M,dcs:Person +2004-01,dcid:country/ALB,dcid:Count_BirthEvent,3588,P1M,dcs:Person +2005-01,dcid:country/ALB,dcid:Count_BirthEvent,3092,P1M,dcs:Person +2006-01,dcid:country/ALB,dcid:Count_BirthEvent,3191,P1M,dcs:Person +2007-01,dcid:country/ALB,dcid:Count_BirthEvent,2893,P1M,dcs:Person +2013-01,dcid:country/ALB,dcid:Count_BirthEvent,2839,P1M,dcs:Person +2014-01,dcid:country/ALB,dcid:Count_BirthEvent,2837,P1M,dcs:Person +2015-01,dcid:country/ALB,dcid:Count_BirthEvent,2742,P1M,dcs:Person +2016-01,dcid:country/ALB,dcid:Count_BirthEvent,2517,P1M,dcs:Person +2017-01,dcid:country/ALB,dcid:Count_BirthEvent,2423,P1M,dcs:Person +2018-01,dcid:country/ALB,dcid:Count_BirthEvent,2468,P1M,dcs:Person +2019-01,dcid:country/ALB,dcid:Count_BirthEvent,2393,P1M,dcs:Person +2020-01,dcid:country/ALB,dcid:Count_BirthEvent,2316,P1M,dcs:Person +2021-01,dcid:country/ALB,dcid:Count_BirthEvent,2066,P1M,dcs:Person +2022-01,dcid:country/ALB,dcid:Count_BirthEvent,2277,P1M,dcs:Person +2023-01,dcid:country/ALB,dcid:Count_BirthEvent,1991,P1M,dcs:Person +2024-01,dcid:country/ALB,dcid:Count_BirthEvent,1985,P1M,dcs:Person +2005-01,dcid:country/ARM,dcid:Count_BirthEvent,3319,P1M,dcs:Person +2006-01,dcid:country/ARM,dcid:Count_BirthEvent,3354,P1M,dcs:Person +2007-01,dcid:country/ARM,dcid:Count_BirthEvent,3362,P1M,dcs:Person +2008-01,dcid:country/ARM,dcid:Count_BirthEvent,3389,P1M,dcs:Person +2009-01,dcid:country/ARM,dcid:Count_BirthEvent,3715,P1M,dcs:Person +2014-01,dcid:country/ARM,dcid:Count_BirthEvent,3597,P1M,dcs:Person +2015-01,dcid:country/ARM,dcid:Count_BirthEvent,3444,P1M,dcs:Person +2016-01,dcid:country/ARM,dcid:Count_BirthEvent,4009,P1M,dcs:Person +2017-01,dcid:country/ARM,dcid:Count_BirthEvent,3081,P1M,dcs:Person +2018-01,dcid:country/ARM,dcid:Count_BirthEvent,3031,P1M,dcs:Person +2019-01,dcid:country/ARM,dcid:Count_BirthEvent,3020,P1M,dcs:Person +2020-01,dcid:country/ARM,dcid:Count_BirthEvent,2751,P1M,dcs:Person +2022-01,dcid:country/ARM,dcid:Count_BirthEvent,2890,P1M,dcs:Person +2023-01,dcid:country/ARM,dcid:Count_BirthEvent,3275,P1M,dcs:Person +1960-01,dcid:country/AUT,dcid:Count_BirthEvent,10666,P1M,dcs:Person +1961-01,dcid:country/AUT,dcid:Count_BirthEvent,11372,P1M,dcs:Person +1962-01,dcid:country/AUT,dcid:Count_BirthEvent,11433,P1M,dcs:Person +1963-01,dcid:country/AUT,dcid:Count_BirthEvent,11805,P1M,dcs:Person +1964-01,dcid:country/AUT,dcid:Count_BirthEvent,11343,P1M,dcs:Person +1965-01,dcid:country/AUT,dcid:Count_BirthEvent,11378,P1M,dcs:Person +1966-01,dcid:country/AUT,dcid:Count_BirthEvent,10723,P1M,dcs:Person +1967-01,dcid:country/AUT,dcid:Count_BirthEvent,10993,P1M,dcs:Person +1968-01,dcid:country/AUT,dcid:Count_BirthEvent,11197,P1M,dcs:Person +1969-01,dcid:country/AUT,dcid:Count_BirthEvent,10987,P1M,dcs:Person +1970-01,dcid:country/AUT,dcid:Count_BirthEvent,9759,P1M,dcs:Person +1971-01,dcid:country/AUT,dcid:Count_BirthEvent,9507,P1M,dcs:Person +1972-01,dcid:country/AUT,dcid:Count_BirthEvent,9187,P1M,dcs:Person +1973-01,dcid:country/AUT,dcid:Count_BirthEvent,8750,P1M,dcs:Person +1974-01,dcid:country/AUT,dcid:Count_BirthEvent,8545,P1M,dcs:Person +1975-01,dcid:country/AUT,dcid:Count_BirthEvent,8343,P1M,dcs:Person +1976-01,dcid:country/AUT,dcid:Count_BirthEvent,8047,P1M,dcs:Person +1977-01,dcid:country/AUT,dcid:Count_BirthEvent,7259,P1M,dcs:Person +1978-01,dcid:country/AUT,dcid:Count_BirthEvent,7157,P1M,dcs:Person +1979-01,dcid:country/AUT,dcid:Count_BirthEvent,7554,P1M,dcs:Person +1980-01,dcid:country/AUT,dcid:Count_BirthEvent,7481,P1M,dcs:Person +1981-01,dcid:country/AUT,dcid:Count_BirthEvent,7890,P1M,dcs:Person +1982-01,dcid:country/AUT,dcid:Count_BirthEvent,8142,P1M,dcs:Person +1983-01,dcid:country/AUT,dcid:Count_BirthEvent,7825,P1M,dcs:Person +1984-01,dcid:country/AUT,dcid:Count_BirthEvent,7424,P1M,dcs:Person +1985-01,dcid:country/AUT,dcid:Count_BirthEvent,7546,P1M,dcs:Person +1986-01,dcid:country/AUT,dcid:Count_BirthEvent,7468,P1M,dcs:Person +1987-01,dcid:country/AUT,dcid:Count_BirthEvent,7279,P1M,dcs:Person +1988-01,dcid:country/AUT,dcid:Count_BirthEvent,7442,P1M,dcs:Person +1989-01,dcid:country/AUT,dcid:Count_BirthEvent,7470,P1M,dcs:Person +1990-01,dcid:country/AUT,dcid:Count_BirthEvent,7612,P1M,dcs:Person +1991-01,dcid:country/AUT,dcid:Count_BirthEvent,7925,P1M,dcs:Person +1992-01,dcid:country/AUT,dcid:Count_BirthEvent,7932,P1M,dcs:Person +1993-01,dcid:country/AUT,dcid:Count_BirthEvent,8266,P1M,dcs:Person +1994-01,dcid:country/AUT,dcid:Count_BirthEvent,7798,P1M,dcs:Person +1995-01,dcid:country/AUT,dcid:Count_BirthEvent,7603,P1M,dcs:Person +1996-01,dcid:country/AUT,dcid:Count_BirthEvent,7436,P1M,dcs:Person +1997-01,dcid:country/AUT,dcid:Count_BirthEvent,7318,P1M,dcs:Person +1998-01,dcid:country/AUT,dcid:Count_BirthEvent,6860,P1M,dcs:Person +1999-01,dcid:country/AUT,dcid:Count_BirthEvent,6697,P1M,dcs:Person +2000-01,dcid:country/AUT,dcid:Count_BirthEvent,6587,P1M,dcs:Person +2001-01,dcid:country/AUT,dcid:Count_BirthEvent,6611,P1M,dcs:Person +2002-01,dcid:country/AUT,dcid:Count_BirthEvent,6594,P1M,dcs:Person +2003-01,dcid:country/AUT,dcid:Count_BirthEvent,6570,P1M,dcs:Person +2004-01,dcid:country/AUT,dcid:Count_BirthEvent,6780,P1M,dcs:Person +2005-01,dcid:country/AUT,dcid:Count_BirthEvent,6482,P1M,dcs:Person +2006-01,dcid:country/AUT,dcid:Count_BirthEvent,6526,P1M,dcs:Person +2007-01,dcid:country/AUT,dcid:Count_BirthEvent,6504,P1M,dcs:Person +2008-01,dcid:country/AUT,dcid:Count_BirthEvent,6739,P1M,dcs:Person +2009-01,dcid:country/AUT,dcid:Count_BirthEvent,6475,P1M,dcs:Person +2010-01,dcid:country/AUT,dcid:Count_BirthEvent,6414,P1M,dcs:Person +2011-01,dcid:country/AUT,dcid:Count_BirthEvent,6651,P1M,dcs:Person +2012-01,dcid:country/AUT,dcid:Count_BirthEvent,6389,P1M,dcs:Person +2013-01,dcid:country/AUT,dcid:Count_BirthEvent,6629,P1M,dcs:Person +2014-01,dcid:country/AUT,dcid:Count_BirthEvent,6666,P1M,dcs:Person +2015-01,dcid:country/AUT,dcid:Count_BirthEvent,6985,P1M,dcs:Person +2016-01,dcid:country/AUT,dcid:Count_BirthEvent,6980,P1M,dcs:Person +2017-01,dcid:country/AUT,dcid:Count_BirthEvent,7040,P1M,dcs:Person diff --git a/statvar_imports/eurostat/live_births_total_by_month/test_data/live_births_total_by_month_output.tmcf b/statvar_imports/eurostat/live_births_total_by_month/test_data/live_births_total_by_month_output.tmcf new file mode 100644 index 0000000000..39a28d1ba4 --- /dev/null +++ b/statvar_imports/eurostat/live_births_total_by_month/test_data/live_births_total_by_month_output.tmcf @@ -0,0 +1,8 @@ +Node: E:output->E0 +observationDate: C:output->observationDate +observationAbout: C:output->observationAbout +variableMeasured: C:output->variableMeasured +value: C:output->value +observationPeriod: C:output->observationPeriod +unit: C:output->unit +typeOf: dcs:StatVarObservation diff --git a/statvar_imports/eurostat/marriage_indicators/README.md b/statvar_imports/eurostat/marriage_indicators/README.md new file mode 100644 index 0000000000..d83a2d2d86 --- /dev/null +++ b/statvar_imports/eurostat/marriage_indicators/README.md @@ -0,0 +1,87 @@ +# Eurostat Marriage Indicators Import + +## Overview +This dataset contains annual marriage indicators at the national level, sourced from Eurostat. The data tracks various metrics including total marriages, crude marriage rates, mean age at first marriage, and total first marriage rates across various European countries. + +**type of place:** Country +**years:** Historical data to present (1960-2024) +**place_resolution:** Resolved to DCIDs (e.g., dcid:country/ARM, dcid:country/EST) + +## Data Source +**Source URL:** +https://ec.europa.eu/eurostat/databrowser/view/DEMO_NIND/default/table?lang=en + +**Provenance Description:** +The data is provided by Eurostat, the statistical office of the European Union. It is part of the "Demography and migration" database, specifically the "Marriage indicators" (DEMO_NIND) dataset. + +## Refresh Type +Automatic Refresh + +The refresh is automated using the provided `run.sh` script, which handles both data download and processing. + +## How To Run Import +To execute the complete import process (download and processing), run: +```bash +./run.sh +``` + +### Script Details: +- **Download**: Uses `curl` to fetch the latest SDMX-CSV data from Eurostat's dissemination API. +- **Processing**: Uses `stat_var_processor.py` to map raw data to Data Commons StatVarObservations using the PV map and metadata configuration. + +## Processing Instructions +To process the Eurostat Marriage Indicators data and generate statistical variables, use the following commands from your current import data directory: + +Download input file + +```bash +mkdir -p input_files +curl -L --retry 3 "https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/DEMO_NIND/?format=SDMX-CSV&compressed=false" -o ./input_files/marriage_indicators_data_input.csv +``` + +For Test Data Run + +```bash +python3 ../../../tools/statvar_importer/stat_var_processor.py \ + "--input_data=./test_data/marriage_indicators_data_input.csv" \ + "--pv_map=./marriage_indicators_pvmap.csv" \ + "--output_path=./test_data/marriage_indicators_output" \ + "--config_file=./marriage_indicators_metadata.csv" \ + "--existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf" +``` + +For Main data processing run + +```bash +python3 ../../../tools/statvar_importer/stat_var_processor.py \ + "--input_data=./input_files/*.csv" \ + "--pv_map=./marriage_indicators_pvmap.csv" \ + "--config_file=./marriage_indicators_metadata.csv" \ + "--generate_statvar_name=True" \ + "--skip_constant_csv_columns=False" \ + "--output_columns=observationDate,observationAbout,variableMeasured,value,observationPeriod,unit,scalingFactor" \ + "--output_path=output_files/marriage_indicators_output" \ + "--places_resolved_csv=./places_resolved.csv" \ + "--existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf" +``` + +## Key Files +- `run.sh`: Main execution script for download and processing. +- `marriage_indicators_pvmap.csv`: Property-Value mapping for StatVar definitions and dimensions. +- `marriage_indicators_metadata.csv`: Configuration parameters for the processor. +- `places_resolved.csv`: Mapping of place codes to Data Commons DCIDs. +- `output_files/marriage_indicators_output.csv`: Processed statistical observations. +- `output_files/marriage_indicators_output.tmcf`: Template MCF mapping the CSV columns to Data Commons schema. + + +## Validation +To validate the generated data, use the Data Commons import tool (lint mode). Note that you must include the StatVar MCF files to resolve schema references: +```bash +java -jar datacommons-import-tool.jar lint output_files/marriage_indicators_output.csv output_files/marriage_indicators_output.tmcf +``` +The resulting reports (`report.json`, `summary_report.html`) in `dc_generated/` provide detailed insights into data quality and validation status. + +## Testing +Testing is performed using the `test_data` directory: +- Expected Output: `test_data/marriage_indicators_output.csv` +- Expected TMCF: `test_data/marriage_indicators_output.tmcf` diff --git a/statvar_imports/eurostat/marriage_indicators/manifest.json b/statvar_imports/eurostat/marriage_indicators/manifest.json new file mode 100644 index 0000000000..e2ab9c8ac0 --- /dev/null +++ b/statvar_imports/eurostat/marriage_indicators/manifest.json @@ -0,0 +1,34 @@ +{ + "import_specifications": [ + { + "import_name": "EuroStat_Marriage_Indicators", + "curator_emails": [ + "support@datacommons.org" + ], + "provenance_url": "https://ec.europa.eu/eurostat/databrowser/view/DEMO_NIND/default/table?lang=en", + "provenance_description": "Annual tracking of marriage indicators (counts, crude rates, mean ages, and total first marriage rates) across reporting European countries", + "scripts": [ + "run.sh" + ], + "import_inputs": [ + { + "template_mcf": "output_files/marriage_indicators_output.tmcf", + "cleaned_csv": "output_files/marriage_indicators_output.csv" + } + ], + "source_files": [ + "input_files/*.csv" + ], + "cron_schedule": "5 2 1,15 * *", + "resource_limits": {"cpu": 4, "memory": 8, "disk":100}, + "config_override": { + "invoke_import_validation": true, + "invoke_import_tool": true, + "invoke_differ_tool": true, + "skip_input_upload": false, + "skip_gcs_upload": false, + "cleanup_gcs_volume_mount": false + } + } + ] +} diff --git a/statvar_imports/eurostat/marriage_indicators/marriage_indicators_metadata.csv b/statvar_imports/eurostat/marriage_indicators/marriage_indicators_metadata.csv new file mode 100644 index 0000000000..8c68e1ecd6 --- /dev/null +++ b/statvar_imports/eurostat/marriage_indicators/marriage_indicators_metadata.csv @@ -0,0 +1,3 @@ +parameter,value +header_rows,1 +place_type,Country diff --git a/statvar_imports/eurostat/marriage_indicators/marriage_indicators_pvmap.csv b/statvar_imports/eurostat/marriage_indicators/marriage_indicators_pvmap.csv new file mode 100644 index 0000000000..f59b53dfd9 --- /dev/null +++ b/statvar_imports/eurostat/marriage_indicators/marriage_indicators_pvmap.csv @@ -0,0 +1,52 @@ +key,property1,value1,property2,value2,property3,value3,property4,value4,property5,value5,property6,value6 + + +# --- AUTOMATED CLEANUP: Ignored Columns --- +DATAFLOW,#ignore,"",,,,,,,,,, +LAST UPDATE,#ignore,"",,,,,,,,,, +CONF_STATUS,#ignore,"",,,,,,,,,, + +# Frequency dimension (P1Y for Annual series) +freq:A,observationPeriod,P1Y,,,,,,,,, + +# --- Geopolitical Entity Translation (Bypasses offline API call failures) --- +geo,observationAbout,{Data},,,,,,, +EU27_2007,#ignore,"",,,,,,,,,, +EU28,#ignore,"",,,,,,,,,, +EU27_2020,#ignore,"",,,,,,, + +# --- Mapped Marriage Indicators (Strict Person-Based Tracking) --- +# Mean age at first marriage (Mapped to Person population with FirstMarriage status) +indic_de:FAGEMAR1,populationType,dcs:Person,measuredProperty,dcs:age,statType,dcs:meanValue,gender,dcs:Female,maritalStatus,dcs:FirstMarriage,unit,dcs:Year +indic_de:MAGEMAR1,populationType,dcs:Person,measuredProperty,dcs:age,statType,dcs:meanValue,gender,dcs:Male,maritalStatus,dcs:FirstMarriage,unit,dcs:Year + +# First Marriage Rates (TFMR) modeled under Person population +indic_de:FMAR1CUM,populationType,dcs:Person,measuredProperty,dcs:count,statType,dcs:measuredValue,gender,dcs:Female,maritalStatus,dcs:FirstMarriage,measurementDenominator,dcs:Count_Person_Female +indic_de:MMAR1CUM,populationType,dcs:Person,measuredProperty,dcs:count,statType,dcs:measuredValue,gender,dcs:Male,maritalStatus,dcs:FirstMarriage,measurementDenominator,dcs:Count_Person_Male + +# First Marriage Proportions (NeverMarried as fraction of EverMarried females/males) +indic_de:FMAR1PC,populationType,dcs:Person,measuredProperty,dcs:count,statType,dcs:measuredValue,gender,dcs:Female,maritalStatus,dcs:NeverMarried,measurementDenominator,dcs:Count_Person_Female_EverMarried +indic_de:MMAR1PC,populationType,dcs:Person,measuredProperty,dcs:count,statType,dcs:measuredValue,gender,dcs:Male,maritalStatus,dcs:NeverMarried,measurementDenominator,dcs:Count_Person_Male_EverMarried + +# Absolute Counts (Coded as MarriedInThePast12Months) +indic_de:MARRIAGE,populationType,dcs:Person,measuredProperty,dcs:count,statType,dcs:measuredValue,maritalStatus,dcs:MarriedInThePast12Months,,, + +# Crude Marriage Rate (Count of persons married in the past 12 months as a fraction of total population) +indic_de:GNUPRT,populationType,dcs:Person,measuredProperty,dcs:count,statType,dcs:measuredValue,maritalStatus,dcs:MarriedInThePast12Months,measurementDenominator,dcs:Count_Person,scalingFactor,1000 + +# Catch-All Fallback: Cleanly ignore any remaining unmapped indicator codes +indic_de,#ignore,"",,,,,,,,,, + +# --- Time, Value, and Flags --- +TIME_PERIOD,observationDate,{Data},,,,,,,,,, +OBS_VALUE,value,{Number},,,,,,,,,, + +# Safely Ignoring Transient and Composite Status Flags to Prevent Ingestion Warnings +OBS_FLAG,#ignore,"",,,,,,,,,, +OBS_FLAG:b,#ignore,"",,,,,,,,,, +OBS_FLAG:p,#ignore,"",,,,,,,,,, +OBS_FLAG:e,#ignore,"",,,,,,, +OBS_FLAG:ep,#ignore,"",,,,,,,,,, +OBS_FLAG:i,#ignore,"",,,,,,,,,, +OBS_FLAG:M,MissingValue,"",,,,,,,,,, +OBS_FLAG:NA,MissingValue,"",,,,,,,,,, \ No newline at end of file diff --git a/statvar_imports/eurostat/marriage_indicators/places_resolved.csv b/statvar_imports/eurostat/marriage_indicators/places_resolved.csv new file mode 100644 index 0000000000..acc65e505e --- /dev/null +++ b/statvar_imports/eurostat/marriage_indicators/places_resolved.csv @@ -0,0 +1,265 @@ +"place_name","dcid","placeId","wikidataId","name","alternateName","typeOf","containedInPlace" +"AD","dcid:country/AND","","","","","","" +"AE","dcid:country/ARE","","","","","","" +"AF","dcid:country/AFG","","","","","","" +"AG","dcid:country/ATG","","","","","","" +"AI","dcid:country/AIA","","","","","","" +"AL","dcid:country/ALB","","","","","","" +"AM","dcid:country/ARM","","","","","","" +"AN","dcid:country/ANT","","","","","","" +"AO","dcid:country/AGO","","","","","","" +"AQ","dcid:country/ATA","","","","","","" +"AR","dcid:country/ARG","","","","","","" +"AS","dcid:country/ASM","","","","","","" +"AT","dcid:country/AUT","","","","","","" +"AU","dcid:country/AUS","","","","","","" +"AW","dcid:country/ABW","","","","","","" +"AZ","dcid:country/AZE","","","","","","" +"BA","dcid:country/BIH","","","","","","" +"BB","dcid:country/BRB","","","","","","" +"BD","dcid:country/BGD","","","","","","" +"BE","dcid:country/BEL","","","","","","" +"BF","dcid:country/BFA","","","","","","" +"BG","dcid:country/BGR","","","","","","" +"BH","dcid:country/BHR","","","","","","" +"BI","dcid:country/BDI","","","","","","" +"BJ","dcid:country/BEN","","","","","","" +"BL","dcid:country/BLM","","","","","","" +"BM","dcid:country/BMU","","","","","","" +"BN","dcid:country/BRN","","","","","","" +"BO","dcid:country/BOL","","","","","","" +"BR","dcid:country/BRA","","","","","","" +"BS","dcid:country/BHS","","","","","","" +"BT","dcid:country/BTN","","","","","","" +"BV","dcid:country/BVT","","","","","","" +"BW","dcid:country/BWA","","","","","","" +"BY","dcid:country/BLR","","","","","","" +"BZ","dcid:country/BLZ","","","","","","" +"CA","dcid:country/CAN","","","","","","" +"CC","dcid:country/CCK","","","","","","" +"CD","dcid:country/COD","","","","","","" +"CF","dcid:country/CAF","","","","","","" +"CG","dcid:country/COG","","","","","","" +"CH","dcid:country/CHE","","","","","","" +"CI","dcid:country/CIV","","","","","","" +"CK","dcid:country/COK","","","","","","" +"CL","dcid:country/CHL","","","","","","" +"CM","dcid:country/CMR","","","","","","" +"CN","dcid:country/CHN","","","","","","" +"CO","dcid:country/COL","","","","","","" +"CR","dcid:country/CRI","","","","","","" +"CU","dcid:country/CUB","","","","","","" +"CV","dcid:country/CPV","","","","","","" +"CX","dcid:country/CXR","","","","","","" +"CY","dcid:country/CYP","","","","","","" +"CZ","dcid:country/CZE","","","","","","" +"DE","dcid:country/DEU","","","","","","" +"DJ","dcid:country/DJI","","","","","","" +"DK","dcid:country/DNK","","","","","","" +"DM","dcid:country/DMA","","","","","","" +"DO","dcid:country/DOM","","","","","","" +"DZ","dcid:country/DZA","","","","","","" +"EC","dcid:country/ECU","","","","","","" +"EE","dcid:country/EST","","","","","","" +"EG","dcid:country/EGY","","","","","","" +"EH","dcid:country/ESH","","","","","","" +"EL","dcid:country/GRC","","","","","","" +"ER","dcid:country/ERI","","","","","","" +"ES","dcid:country/ESP","","","","","","" +"ET","dcid:country/ETH","","","","","","" +"FI","dcid:country/FIN","","","","","","" +"FJ","dcid:country/FJI","","","","","","" +"FK","dcid:country/FLK","","","","","","" +"FM","dcid:country/FSM","","","","","","" +"FO","dcid:country/FRO","","","","","","" +"FR","dcid:country/FRA","","","","","","" +"FX","dcid:country/FXX","","","","","","" +"GA","dcid:country/GAB","","","","","","" +"UK","dcid:country/GBR","","","","","","" +"GD","dcid:country/GRD","","","","","","" +"GE","dcid:country/GEO","","","","","","" +"GF","dcid:country/GUF","","","","","","" +"GG","dcid:country/GGY","","","","","","" +"GH","dcid:country/GHA","","","","","","" +"GI","dcid:country/GIB","","","","","","" +"GL","dcid:country/GRL","","","","","","" +"GM","dcid:country/GMB","","","","","","" +"GN","dcid:country/GIN","","","","","","" +"GQ","dcid:country/GNQ","","","","","","" +"GS","dcid:country/SGS","","","","","","" +"GT","dcid:country/GTM","","","","","","" +"GU","dcid:country/GUM","","","","","","" +"GW","dcid:country/GNB","","","","","","" +"GY","dcid:country/GUY","","","","","","" +"HK","dcid:country/HKG","","","","","","" +"HM","dcid:country/HMD","","","","","","" +"HN","dcid:country/HND","","","","","","" +"HR","dcid:country/HRV","","","","","","" +"HT","dcid:country/HTI","","","","","","" +"HU","dcid:country/HUN","","","","","","" +"ID","dcid:country/IDN","","","","","","" +"IE","dcid:country/IRL","","","","","","" +"IL","dcid:country/ISR","","","","","","" +"IM","dcid:country/IMN","","","","","","" +"IN","dcid:country/IND","","","","","","" +"IO","dcid:country/IOT","","","","","","" +"IQ","dcid:country/IRQ","","","","","","" +"IR","dcid:country/IRN","","","","","","" +"IS","dcid:country/ISL","","","","","","" +"IT","dcid:country/ITA","","","","","","" +"JE","dcid:country/JEY","","","","","","" +"JM","dcid:country/JAM","","","","","","" +"JO","dcid:country/JOR","","","","","","" +"JP","dcid:country/JPN","","","","","","" +"KE","dcid:country/KEN","","","","","","" +"KG","dcid:country/KGZ","","","","","","" +"KH","dcid:country/KHM","","","","","","" +"KI","dcid:country/KIR","","","","","","" +"KM","dcid:country/COM","","","","","","" +"KN","dcid:country/KNA","","","","","","" +"KP","dcid:country/PRK","","","","","","" +"KR","dcid:country/KOR","","","","","","" +"KW","dcid:country/KWT","","","","","","" +"KY","dcid:country/CYM","","","","","","" +"KZ","dcid:country/KAZ","","","","","","" +"LA","dcid:country/LAO","","","","","","" +"LB","dcid:country/LBN","","","","","","" +"LC","dcid:country/LCA","","","","","","" +"LI","dcid:country/LIE","","","","","","" +"LK","dcid:country/LKA","","","","","","" +"LR","dcid:country/LBR","","","","","","" +"LS","dcid:country/LSO","","","","","","" +"LT","dcid:country/LTU","","","","","","" +"LU","dcid:country/LUX","","","","","","" +"LV","dcid:country/LVA","","","","","","" +"LY","dcid:country/LBY","","","","","","" +"MA","dcid:country/MAR","","","","","","" +"MC","dcid:country/MCO","","","","","","" +"MD","dcid:country/MDA","","","","","","" +"ME","dcid:country/MNE","","","","","","" +"MF","dcid:country/MAF","","","","","","" +"MG","dcid:country/MDG","","","","","","" +"MH","dcid:country/MHL","","","","","","" +"MK","dcid:country/MKD","","","","","","" +"ML","dcid:country/MLI","","","","","","" +"MM","dcid:country/MMR","","","","","","" +"MN","dcid:country/MNG","","","","","","" +"MO","dcid:country/MAC","","","","","","" +"MP","dcid:country/MNP","","","","","","" +"MQ","dcid:country/MTQ","","","","","","" +"MR","dcid:country/MRT","","","","","","" +"MS","dcid:country/MSR","","","","","","" +"MT","dcid:country/MLT","","","","","","" +"MU","dcid:country/MUS","","","","","","" +"MV","dcid:country/MDV","","","","","","" +"MW","dcid:country/MWI","","","","","","" +"MX","dcid:country/MEX","","","","","","" +"MY","dcid:country/MYS","","","","","","" +"MZ","dcid:country/MOZ","","","","","","" +"NA","dcid:country/NAM","","","","","","" +"NC","dcid:country/NCL","","","","","","" +"NE","dcid:country/NER","","","","","","" +"NF","dcid:country/NFK","","","","","","" +"NG","dcid:country/NGA","","","","","","" +"NI","dcid:country/NIC","","","","","","" +"NL","dcid:country/NLD","","","","","","" +"NO","dcid:country/NOR","","","","","","" +"NP","dcid:country/NPL","","","","","","" +"NR","dcid:country/NRU","","","","","","" +"NU","dcid:country/NIU","","","","","","" +"NZ","dcid:country/NZL","","","","","","" +"OM","dcid:country/OMN","","","","","","" +"PA","dcid:country/PAN","","","","","","" +"PE","dcid:country/PER","","","","","","" +"PF","dcid:country/PYF","","","","","","" +"PG","dcid:country/PNG","","","","","","" +"PH","dcid:country/PHL","","","","","","" +"PK","dcid:country/PAK","","","","","","" +"PL","dcid:country/POL","","","","","","" +"PM","dcid:country/SPM","","","","","","" +"PN","dcid:country/PCN","","","","","","" +"PR","dcid:country/PRI","","","","","","" +"PS","dcid:country/PSE","","","","","","" +"PT","dcid:country/PRT","","","","","","" +"PW","dcid:country/PLW","","","","","","" +"PY","dcid:country/PRY","","","","","","" +"QA","dcid:country/QAT","","","","","","" +"RE","dcid:country/REU","","","","","","" +"RO","dcid:country/ROU","","","","","","" +"RS","dcid:country/SRB","","","","","","" +"RU","dcid:country/RUS","","","","","","" +"RW","dcid:country/RWA","","","","","","" +"SA","dcid:country/SAU","","","","","","" +"SB","dcid:country/SLB","","","","","","" +"SC","dcid:country/SYC","","","","","","" +"SD","dcid:country/SDN","","","","","","" +"SE","dcid:country/SWE","","","","","","" +"SG","dcid:country/SGP","","","","","","" +"SH","dcid:country/SHN","","","","","","" +"SI","dcid:country/SVN","","","","","","" +"SJ","dcid:country/SJM","","","","","","" +"SK","dcid:country/SVK","","","","","","" +"SL","dcid:country/SLE","","","","","","" +"SM","dcid:country/SMR","","","","","","" +"SN","dcid:country/SEN","","","","","","" +"SO","dcid:country/SOM","","","","","","" +"SR","dcid:country/SUR","","","","","","" +"ST","dcid:country/STP","","","","","","" +"SV","dcid:country/SLV","","","","","","" +"SX","dcid:country/SXM","","","","","","" +"SY","dcid:country/SYR","","","","","","" +"SZ","dcid:country/SWZ","","","","","","" +"TC","dcid:country/TCA","","","","","","" +"TD","dcid:country/TCD","","","","","","" +"TF","dcid:country/ATF","","","","","","" +"TG","dcid:country/TGO","","","","","","" +"TH","dcid:country/THA","","","","","","" +"TJ","dcid:country/TJK","","","","","","" +"TK","dcid:country/TKL","","","","","","" +"TL","dcid:country/TLS","","","","","","" +"TM","dcid:country/TKM","","","","","","" +"TN","dcid:country/TUN","","","","","","" +"TO","dcid:country/TON","","","","","","" +"TR","dcid:country/TUR","","","","","","" +"TT","dcid:country/TTO","","","","","","" +"TV","dcid:country/TUV","","","","","","" +"TW","dcid:country/TWN","","","","","","" +"TZ","dcid:country/TZA","","","","","","" +"UA","dcid:country/UKR","","","","","","" +"UG","dcid:country/UGA","","","","","","" +"UM","dcid:country/UMI","","","","","","" +"US","dcid:country/USA","","","","","","" +"UY","dcid:country/URY","","","","","","" +"UZ","dcid:country/UZB","","","","","","" +"VA","dcid:country/VAT","","","","","","" +"VC","dcid:country/VCT","","","","","","" +"VE","dcid:country/VEN","","","","","","" +"VG","dcid:country/VGB","","","","","","" +"VI","dcid:country/VIR","","","","","","" +"VN","dcid:country/VNM","","","","","","" +"VU","dcid:country/VUT","","","","","","" +"WF","dcid:country/WLF","","","","","","" +"WS","dcid:country/WSM","","","","","","" +"XK","dcid:country/XKS","","","","","","" +"YE","dcid:country/YEM","","","","","","" +"YT","dcid:country/MYT","","","","","","" +"ZA","dcid:country/ZAF","","","","","","" +"ZM","dcid:country/ZMB","","","","","","" +"ZW","dcid:country/ZWE","","","","","","" +"DE_TOT","dcid:nuts/DE_TOT","","","","","","" +"EA","dcid:nuts/EA","","","","","","" +"EA12","dcid:nuts/EA12","","","","","","" +"EA18","dcid:nuts/EA18","","","","","","" +"EA19","dcid:nuts/EA19","","","","","","" +"EEA","dcid:nuts/EEA","","","","","","" +"EEA30_2007","dcid:nuts/EEA30_2007","","","","","","" +"EEA31","dcid:nuts/EEA31","","","","","","" +"EFTA","dcid:nuts/EFTA","","","","","","" +"EU","dcid:nuts/EU","","","","","","" +"EU27","dcid:nuts/EU27","","","","","","" +"EU27_2007","dcid:nuts/EU27_2007","","","","","","" +"EU27_2020","dcid:nuts/EU27_2020","","","","","","" +"EA20","dcid:nuts/EA20","","","","","","" +"EA21","dcid:nuts/EA21","","","","","","" +"EU28","dcid:nuts/EU28","","","","","","" +"EEA28","dcid:nuts/EEA28","","","","","","" \ No newline at end of file diff --git a/statvar_imports/eurostat/marriage_indicators/run.sh b/statvar_imports/eurostat/marriage_indicators/run.sh new file mode 100644 index 0000000000..9caa754e32 --- /dev/null +++ b/statvar_imports/eurostat/marriage_indicators/run.sh @@ -0,0 +1,19 @@ +#!/bin/bash + +# Step 1: Data Download +mkdir -p input_files +mkdir -p output_files +curl -L --retry 3 "https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/DEMO_NIND/?format=SDMX-CSV&compressed=false" -o ./input_files/marriage_indicators_data_input.csv + +# Step 2: Data Processing +python3 ../../../tools/statvar_importer/stat_var_processor.py \ + "--input_data=./input_files/*.csv" \ + "--pv_map=./marriage_indicators_pvmap.csv" \ + "--config_file=./marriage_indicators_metadata.csv" \ + "--generate_statvar_name=True" \ + "--skip_constant_csv_columns=False" \ + "--output_columns=observationDate,observationAbout,variableMeasured,value,observationPeriod,unit,scalingFactor" \ + "--output_path=output_files/marriage_indicators_output" \ + "--places_resolved_csv=./places_resolved.csv" \ + "--existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf" + \ No newline at end of file diff --git a/statvar_imports/eurostat/marriage_indicators/test_data/marriage_indicators_data_input.csv b/statvar_imports/eurostat/marriage_indicators/test_data/marriage_indicators_data_input.csv new file mode 100644 index 0000000000..c18243f7c2 --- /dev/null +++ b/statvar_imports/eurostat/marriage_indicators/test_data/marriage_indicators_data_input.csv @@ -0,0 +1,100 @@ +DATAFLOW,LAST UPDATE,freq,indic_de,geo,TIME_PERIOD,OBS_VALUE,OBS_FLAG,CONF_STATUS +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AM,2006,25.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AM,2007,25.8,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AM,2008,25.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AM,2009,25.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AM,2015,26.8,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AM,2017,26.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AM,2018,26.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1990,25.2,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1991,25.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1992,25.6,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1993,25.8,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1994,26.1,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1995,26.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1996,26.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1997,26.8,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1998,26.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,1999,27.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2000,27.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2001,27.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2002,27.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2003,27.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2004,28.2,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2005,28.6,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2006,29.1,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2007,29.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2008,29.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2009,29.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2011,30.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2012,30.6,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2013,30.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2014,30.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2015,31.2,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2016,31.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2017,31.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2018,31.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2019,32.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2023,32.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AT,2024,32.8,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AZ,2006,24.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AZ,2007,24.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AZ,2008,24.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AZ,2009,24.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AZ,2014,24.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AZ,2017,23.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AZ,2018,23.8,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,AZ,2019,24.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1990,24.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1991,24.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1992,24.8,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1993,25.1,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1994,25.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1995,25.6,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1996,25.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1997,25.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1998,26.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,1999,26.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2000,26.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2001,26.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2002,27.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2003,27.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2004,27.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2005,28.1,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2006,28.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2007,28.6,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2008,28.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2009,29.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2010,29.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2011,,b, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2017,31.2,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2020,32.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2021,32.2,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2022,32.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BE,2023,32.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1990,21.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1991,21.5,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1992,21.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1993,22.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1994,22.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1995,22.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1996,23.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1997,23.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1998,23.4,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,1999,23.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2000,24.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2001,24.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2002,24.6,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2003,25.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2004,25.3,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2005,25.6,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2006,25.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2007,25.8,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2008,25.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2009,26.2,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2010,26.6,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2011,26.6,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2013,26.7,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2014,26.9,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2015,27.0,, +ESTAT:DEMO_NIND(1.0),30/04/26 23:00:00,A,FAGEMAR1,BG,2016,27.1,, diff --git a/statvar_imports/eurostat/marriage_indicators/test_data/marriage_indicators_output.csv b/statvar_imports/eurostat/marriage_indicators/test_data/marriage_indicators_output.csv new file mode 100644 index 0000000000..44cd9f2054 --- /dev/null +++ b/statvar_imports/eurostat/marriage_indicators/test_data/marriage_indicators_output.csv @@ -0,0 +1,100 @@ +observationDate,observationAbout,variableMeasured,value,observationPeriod,unit,scalingFactor +2006,dcid:country/ARM,dcid:Mean_Age_Person_Female_FirstMarriage,25.4,P1Y,dcs:Year, +2007,dcid:country/ARM,dcid:Mean_Age_Person_Female_FirstMarriage,25.8,P1Y,dcs:Year, +2008,dcid:country/ARM,dcid:Mean_Age_Person_Female_FirstMarriage,25.7,P1Y,dcs:Year, +2009,dcid:country/ARM,dcid:Mean_Age_Person_Female_FirstMarriage,25.5,P1Y,dcs:Year, +2015,dcid:country/ARM,dcid:Mean_Age_Person_Female_FirstMarriage,26.8,P1Y,dcs:Year, +2017,dcid:country/ARM,dcid:Mean_Age_Person_Female_FirstMarriage,26.7,P1Y,dcs:Year, +2018,dcid:country/ARM,dcid:Mean_Age_Person_Female_FirstMarriage,26.7,P1Y,dcs:Year, +1990,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,25.2,P1Y,dcs:Year, +1991,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,25.4,P1Y,dcs:Year, +1992,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,25.6,P1Y,dcs:Year, +1993,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,25.8,P1Y,dcs:Year, +1994,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,26.1,P1Y,dcs:Year, +1995,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,26.3,P1Y,dcs:Year, +1996,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,26.5,P1Y,dcs:Year, +1997,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,26.8,P1Y,dcs:Year, +1998,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,26.9,P1Y,dcs:Year, +1999,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,27.3,P1Y,dcs:Year, +2000,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,27.4,P1Y,dcs:Year, +2001,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,27.5,P1Y,dcs:Year, +2002,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,27.7,P1Y,dcs:Year, +2003,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,27.9,P1Y,dcs:Year, +2004,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,28.2,P1Y,dcs:Year, +2005,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,28.6,P1Y,dcs:Year, +2006,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,29.1,P1Y,dcs:Year, +2007,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,29.3,P1Y,dcs:Year, +2008,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,29.5,P1Y,dcs:Year, +2009,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,29.9,P1Y,dcs:Year, +2011,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,30.3,P1Y,dcs:Year, +2012,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,30.6,P1Y,dcs:Year, +2013,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,30.7,P1Y,dcs:Year, +2014,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,30.9,P1Y,dcs:Year, +2015,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,31.2,P1Y,dcs:Year, +2016,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,31.3,P1Y,dcs:Year, +2017,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,31.5,P1Y,dcs:Year, +2018,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,31.7,P1Y,dcs:Year, +2019,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,32,P1Y,dcs:Year, +2023,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,32.7,P1Y,dcs:Year, +2024,dcid:country/AUT,dcid:Mean_Age_Person_Female_FirstMarriage,32.8,P1Y,dcs:Year, +2006,dcid:country/AZE,dcid:Mean_Age_Person_Female_FirstMarriage,24.5,P1Y,dcs:Year, +2007,dcid:country/AZE,dcid:Mean_Age_Person_Female_FirstMarriage,24.5,P1Y,dcs:Year, +2008,dcid:country/AZE,dcid:Mean_Age_Person_Female_FirstMarriage,24.3,P1Y,dcs:Year, +2009,dcid:country/AZE,dcid:Mean_Age_Person_Female_FirstMarriage,24.3,P1Y,dcs:Year, +2014,dcid:country/AZE,dcid:Mean_Age_Person_Female_FirstMarriage,24.4,P1Y,dcs:Year, +2017,dcid:country/AZE,dcid:Mean_Age_Person_Female_FirstMarriage,23.9,P1Y,dcs:Year, +2018,dcid:country/AZE,dcid:Mean_Age_Person_Female_FirstMarriage,23.8,P1Y,dcs:Year, +2019,dcid:country/AZE,dcid:Mean_Age_Person_Female_FirstMarriage,24,P1Y,dcs:Year, +1990,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,24.4,P1Y,dcs:Year, +1991,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,24,P1Y,dcs:Year, +1992,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,24.8,P1Y,dcs:Year, +1993,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,25.1,P1Y,dcs:Year, +1994,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,25.4,P1Y,dcs:Year, +1995,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,25.6,P1Y,dcs:Year, +1996,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,25.7,P1Y,dcs:Year, +1997,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,25.9,P1Y,dcs:Year, +1998,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,26,P1Y,dcs:Year, +1999,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,26.3,P1Y,dcs:Year, +2000,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,26.9,P1Y,dcs:Year, +2001,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,26.7,P1Y,dcs:Year, +2002,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,27,P1Y,dcs:Year, +2003,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,27.5,P1Y,dcs:Year, +2004,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,27.9,P1Y,dcs:Year, +2005,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,28.1,P1Y,dcs:Year, +2006,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,28.4,P1Y,dcs:Year, +2007,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,28.6,P1Y,dcs:Year, +2008,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,28.9,P1Y,dcs:Year, +2009,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,29.3,P1Y,dcs:Year, +2010,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,29.4,P1Y,dcs:Year, +2017,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,31.2,P1Y,dcs:Year, +2020,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,32,P1Y,dcs:Year, +2021,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,32.2,P1Y,dcs:Year, +2022,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,32.7,P1Y,dcs:Year, +2023,dcid:country/BEL,dcid:Mean_Age_Person_Female_FirstMarriage,32.9,P1Y,dcs:Year, +1990,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,21.5,P1Y,dcs:Year, +1991,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,21.5,P1Y,dcs:Year, +1992,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,21.7,P1Y,dcs:Year, +1993,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,22,P1Y,dcs:Year, +1994,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,22.4,P1Y,dcs:Year, +1995,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,22.7,P1Y,dcs:Year, +1996,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,23,P1Y,dcs:Year, +1997,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,23.3,P1Y,dcs:Year, +1998,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,23.4,P1Y,dcs:Year, +1999,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,23.7,P1Y,dcs:Year, +2000,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,24.7,P1Y,dcs:Year, +2001,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,24.7,P1Y,dcs:Year, +2002,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,24.6,P1Y,dcs:Year, +2003,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,25,P1Y,dcs:Year, +2004,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,25.3,P1Y,dcs:Year, +2005,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,25.6,P1Y,dcs:Year, +2006,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,25.7,P1Y,dcs:Year, +2007,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,25.8,P1Y,dcs:Year, +2008,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,25.9,P1Y,dcs:Year, +2009,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,26.2,P1Y,dcs:Year, +2010,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,26.6,P1Y,dcs:Year, +2011,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,26.6,P1Y,dcs:Year, +2013,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,26.7,P1Y,dcs:Year, +2014,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,26.9,P1Y,dcs:Year, +2015,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,27,P1Y,dcs:Year, +2016,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,27.1,P1Y,dcs:Year, +2017,dcid:country/BGR,dcid:Mean_Age_Person_Female_FirstMarriage,27.3,P1Y,dcs:Year diff --git a/statvar_imports/eurostat/marriage_indicators/test_data/marriage_indicators_output.tmcf b/statvar_imports/eurostat/marriage_indicators/test_data/marriage_indicators_output.tmcf new file mode 100644 index 0000000000..75cb4c9d12 --- /dev/null +++ b/statvar_imports/eurostat/marriage_indicators/test_data/marriage_indicators_output.tmcf @@ -0,0 +1,9 @@ +Node: E:output->E0 +observationDate: C:output->observationDate +observationAbout: C:output->observationAbout +variableMeasured: C:output->variableMeasured +value: C:output->value +observationPeriod: C:output->observationPeriod +unit: C:output->unit +scalingFactor: C:output->scalingFactor +typeOf: dcs:StatVarObservation