Skip to content

Commit e73500f

Browse files
mmckyclaude
mmcky
and
claude
authored
P2 pilot: add the pandas_panel trio (realwage, countries, employ) with manifests (#17)
* P2 pilot: add the pandas_panel trio (realwage, countries, employ) with manifests Second pilot (PLAN Phase 8, P2) — the cross-series shared-static case: three files consumed by BOTH lecture-python-programming and lecture-python.myst via their own-repo copies today, migrated here to a single canonical copy each. - realwage.csv — OECD real minimum wages (CC BY 4.0); constructed, builder unrecovered (inherited). - employ.csv — Eurostat employment (reuse w/ attribution, Decision 2011/833/EU); constructed, builder unrecovered (inherited). - countries.csv — WorldData.info country table; verbatim. Source is all-rights-reserved (restricted) — cached as an inherited exposure (served by the lectures for years), recorded honestly and logged for licence review in QuantEcon/workspace-lectures#20. AGENTS.md gains a one-line exception for this already-served case; it must be resolved before data.quantecon.org goes public. Integrity: each file is byte-identical across the two consuming repos (compared 2026-07-17) and its manifest sha256 matches the committed bytes, so the repoints cannot change lecture output. Upstream provenance is `unverifiable` for all three (inherited, undated, no builder) — recorded honestly per the P1 pattern. Repoints in the two lecture repos follow as separate PRs. Part of QuantEcon/meta#338; retires the last of the pandas_panel legacy-repo references. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * P2: correct countries.csv 'Dialing prefix' dtype to string Copilot review (data-lectures#17): the column holds non-integer values for territories with hyphenated prefixes (1-684, 599-9, 44-1624, 1-284, 1-670, 1-721, 1-649), so int64 was inaccurate and would break a schema-enforcing validator. pandas reads it as object/string. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 2ee7ca1 commit e73500f

7 files changed

Lines changed: 17027 additions & 0 deletions

File tree

‎AGENTS.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,8 @@ Because this repo is a **stability cache, not a content-distribution host** (see
7676

7777
For the public data sources most snapshots come from (World Bank, FRED, Eurostat, …) the answer is a **known yes, recorded once per source** — permissive terms plus attribution. Record what the source states and move on; don't re-litigate it per snapshot. Treat the manifest's `redistribution` field as a **cheap binary gate** (`permitted` / `restricted`): a fast `permitted` for public statistics agencies, `restricted` blocking only the genuinely restricted source before it goes public — e.g. FRED re-serves third-party series that may not be redistributed, and anything under non-commercial or no-redistribution terms must not be cached here, since attribution alone does not cure those. Capture licence detail richly when the source provides it; where it is genuinely unavailable, record the gap rather than blocking the file.
7878

79+
One exception: a `restricted` file the lectures have already served publicly may be cached here if it is marked `redistribution: restricted` with a `note` and logged for licence review in the migration-licensing tracker ([workspace-lectures#20](https://github.com/QuantEcon/workspace-lectures/issues/20)) — resolve it (permission, an open replacement, or removal) before `data.quantecon.org` is promoted as a public open-data host.
80+
7981
## Cross-repo hygiene
8082

8183
- Changes here often pair with PRs in lecture repos and issues in `QuantEcon/meta`. In commit messages and PR bodies, **never place a GitHub closing keyword (`fixes`, `closes`, `resolves`, …) immediately before a cross-repo reference** like `QuantEcon/meta#336` — GitHub will auto-close the referenced issue when the commit lands on the default branch. Write "See QuantEcon/meta#336" or "Part of QuantEcon/meta#336".

‎lectures/countries.csv‎

Lines changed: 249 additions & 0 deletions
Large diffs are not rendered by default.

‎lectures/countries.csv.yml‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Manifest for countries.csv — pandas_panel pilot (PLAN Phase 8, P2).
2+
# Shared static consumed by TWO series (programming + python.myst); see consumers.
3+
#
4+
# LICENCE NOTE: this source is `restricted` (no redistribution grant). It is
5+
# cached here as an inherited exposure — the lectures have served it publicly
6+
# for years — recorded honestly and logged for licence review in
7+
# QuantEcon/workspace-lectures#20. It must be resolved (permission confirmed,
8+
# replaced with an open equivalent, or removed) before data.quantecon.org is
9+
# promoted as a public open-data host.
10+
11+
filename: countries.csv
12+
title: WorldData.info country reference table
13+
description: >
14+
Per-country reference attributes (continent, capital, population, area,
15+
currency, vital statistics, …), semicolon-separated. The pandas panel-data
16+
lecture uses only two columns — "Country (en)" and "Continent" — to attach a
17+
continent to each country in a merge.
18+
19+
# Verbatim: the WorldData.info download as distributed — original column names
20+
# and semicolon delimiter preserved, no re-serialisation.
21+
class: verbatim
22+
23+
source:
24+
name: WorldData.info — country data downloads
25+
url: https://www.worlddata.info/downloads/
26+
citation: >
27+
WorldData.info, country reference data (https://www.worlddata.info/downloads/).
28+
29+
license:
30+
name: Proprietary — © WorldData.info, all rights reserved
31+
url: https://www.worlddata.info/downloads/
32+
# "Data without guarantee © WorldData.info, all rights reserved" — no
33+
# redistribution grant stated. Attribution alone does not cure a
34+
# no-redistribution term, so this is `restricted`, not `permitted`.
35+
redistribution: restricted
36+
verified: 2026-07-17
37+
note: >
38+
Inherited exposure (served by the lectures for years); cached here pending
39+
licence review — see QuantEcon/workspace-lectures#20. Not a blessed
40+
exception: resolve before promoting data.quantecon.org publicly.
41+
42+
retrieved: null # inherited from the lecture repos; original
43+
# WorldData.info retrieval date not recorded
44+
maintainer: QuantEcon
45+
46+
integrity:
47+
sha256: b334ddac9bf8b1788c6b9abf626959e2ff98a7f40a9dcdaa22e226d84595c3be
48+
upstream:
49+
status: unverifiable
50+
date: 2026-07-17
51+
against: null
52+
note: >
53+
WorldData.info updates this table over time (population, vital stats), so
54+
a current re-fetch will not byte-match this retrieval. Byte-identical
55+
across both consuming repos (compared 2026-07-17).
56+
57+
schema:
58+
format: csv
59+
delimiter: ";" # read with pd.read_csv(url, sep=';')
60+
columns:
61+
- {name: Country (en), dtype: string, description: country name in English (used by the lecture)}
62+
- {name: Country (de), dtype: string, description: country name in German}
63+
- {name: Country (local), dtype: string, description: local-language country name}
64+
- {name: Country code, dtype: string, description: ISO 3166 alpha-2 code}
65+
- {name: Continent, dtype: string, description: continent (used by the lecture)}
66+
- {name: Capital, dtype: string, description: capital city}
67+
- {name: Population, dtype: int64, description: population count}
68+
- {name: Area, dtype: int64, description: land area (km²)}
69+
- {name: Coastline, dtype: int64, description: coastline length (km)}
70+
- {name: Government form, dtype: string, description: form of government}
71+
- {name: Currency, dtype: string, description: currency name}
72+
- {name: Currency code, dtype: string, description: ISO 4217 currency code}
73+
- {name: Dialing prefix, dtype: string, description: international dialing prefix (e.g. "1-684" for territories)}
74+
- {name: Birthrate, dtype: float64, description: births per 1000}
75+
- {name: Deathrate, dtype: float64, description: deaths per 1000}
76+
- {name: Life expectancy, dtype: float64, description: life expectancy (years)}
77+
- {name: Url, dtype: string, description: source page on laenderdaten.info}
78+
row_count_floor: 248 # frozen historical extract -- exact row count
79+
# date_range omitted: country-attribute table, no time dimension.
80+
known_nulls:
81+
Capital: 248 # Capital column is empty throughout this export
82+
Country (local): 3
83+
Currency: 1
84+
Currency code: 1
85+
Dialing prefix: 5
86+
87+
consumers:
88+
- repo: QuantEcon/lecture-python-programming
89+
file: lectures/pandas_panel.md
90+
- repo: QuantEcon/lecture-python.myst
91+
file: lectures/pandas_panel.md
92+
93+
# verbatim — no builder
94+
builder_status: not-applicable

0 commit comments

Comments
 (0)