Skip to content

Commit ae7aee6

Browse files
AtlasNexusTechAtlasNexusTech
committed
feat: publish public extractor sample
1 parent 71a2bde commit ae7aee6

18 files changed

Lines changed: 400 additions & 0 deletions

‎extracteur-donnees-publiques/exemple/README.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Exemple public — extracteur de données publiques
2+
3+
Cet exemple démontre la **structure et la reproductibilité** du livrable annoncé. Il utilise uniquement dix pages HTML locales et des données synthétiques (`SYN-001` à `SYN-010`). Les URL en `.invalid` ne désignent aucun site réel.
4+
5+
## Contenu
6+
7+
- `source-pages/` : 10 pages HTML synthétiques homogènes ;
8+
- `extracteur_demo.py` : script Python 3 sans dépendance externe ;
9+
- `exemple-extraction.csv` : 10 lignes × 8 champs ;
10+
- `exemple-extraction.json` : les mêmes 10 enregistrements structurés ;
11+
- `journal-execution.txt` : compte rendu d’exécution ;
12+
- `exemple-extracteur-donnees-publiques.zip` : archive complète.
13+
14+
## Exécution
15+
16+
```bash
17+
python3 extracteur_demo.py
18+
```
19+
20+
Sortie attendue :
21+
22+
```text
23+
OK: 10 pages, 8 champs, 0 erreur
24+
```
25+
26+
## Limites de démonstration
27+
28+
Cet exemple local ne valide ni la légalité, ni les droits, ni les conditions d’utilisation, ni `robots.txt`, ni la stabilité d’une source réelle. Une source réelle doit rester publique, autorisée, sans connexion, CAPTCHA, contrôle d’accès ou données sensibles. Aucun fichier confidentiel n’est nécessaire pour la qualification initiale.

‎extracteur-donnees-publiques/exemple/exemple-extracteur-donnees-publiques.zip‎

6.97 KB
Binary file not shown.

‎extracteur-donnees-publiques/exemple/exemple-extraction.csv‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
record_id,title,category,city,price_eur,availability,updated_at,source_url
2+
SYN-001,Lampe de bureau,Bureau,Lille,42.00,disponible,2026-07-28,https://exemple.invalid/catalogue/syn-001/
3+
SYN-002,Support écran,Bureau,Rennes,59.00,disponible,2026-07-29,https://exemple.invalid/catalogue/syn-002/
4+
SYN-003,Clavier compact,Informatique,Nantes,74.50,rupture,2026-07-29,https://exemple.invalid/catalogue/syn-003/
5+
SYN-004,Sacoche ordinateur,Mobilité,Lyon,38.90,disponible,2026-07-30,https://exemple.invalid/catalogue/syn-004/
6+
SYN-005,Hub USB-C,Informatique,Paris,49.90,disponible,2026-07-30,https://exemple.invalid/catalogue/syn-005/
7+
SYN-006,Carnet relié,Papeterie,Bordeaux,18.00,disponible,2026-07-30,https://exemple.invalid/catalogue/syn-006/
8+
SYN-007,Webcam HD,Informatique,Toulouse,69.00,précommande,2026-07-31,https://exemple.invalid/catalogue/syn-007/
9+
SYN-008,Gourde inox,Mobilité,Grenoble,24.90,disponible,2026-07-31,https://exemple.invalid/catalogue/syn-008/
10+
SYN-009,Repose-poignets,Bureau,Strasbourg,21.50,disponible,2026-07-31,https://exemple.invalid/catalogue/syn-009/
11+
SYN-010,Casque antibruit,Bureau,Marseille,84.00,rupture,2026-08-01,https://exemple.invalid/catalogue/syn-010/

‎extracteur-donnees-publiques/exemple/exemple-extraction.json‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
[
2+
{
3+
"record_id": "SYN-001",
4+
"title": "Lampe de bureau",
5+
"category": "Bureau",
6+
"city": "Lille",
7+
"price_eur": "42.00",
8+
"availability": "disponible",
9+
"updated_at": "2026-07-28",
10+
"source_url": "https://exemple.invalid/catalogue/syn-001/"
11+
},
12+
{
13+
"record_id": "SYN-002",
14+
"title": "Support écran",
15+
"category": "Bureau",
16+
"city": "Rennes",
17+
"price_eur": "59.00",
18+
"availability": "disponible",
19+
"updated_at": "2026-07-29",
20+
"source_url": "https://exemple.invalid/catalogue/syn-002/"
21+
},
22+
{
23+
"record_id": "SYN-003",
24+
"title": "Clavier compact",
25+
"category": "Informatique",
26+
"city": "Nantes",
27+
"price_eur": "74.50",
28+
"availability": "rupture",
29+
"updated_at": "2026-07-29",
30+
"source_url": "https://exemple.invalid/catalogue/syn-003/"
31+
},
32+
{
33+
"record_id": "SYN-004",
34+
"title": "Sacoche ordinateur",
35+
"category": "Mobilité",
36+
"city": "Lyon",
37+
"price_eur": "38.90",
38+
"availability": "disponible",
39+
"updated_at": "2026-07-30",
40+
"source_url": "https://exemple.invalid/catalogue/syn-004/"
41+
},
42+
{
43+
"record_id": "SYN-005",
44+
"title": "Hub USB-C",
45+
"category": "Informatique",
46+
"city": "Paris",
47+
"price_eur": "49.90",
48+
"availability": "disponible",
49+
"updated_at": "2026-07-30",
50+
"source_url": "https://exemple.invalid/catalogue/syn-005/"
51+
},
52+
{
53+
"record_id": "SYN-006",
54+
"title": "Carnet relié",
55+
"category": "Papeterie",
56+
"city": "Bordeaux",
57+
"price_eur": "18.00",
58+
"availability": "disponible",
59+
"updated_at": "2026-07-30",
60+
"source_url": "https://exemple.invalid/catalogue/syn-006/"
61+
},
62+
{
63+
"record_id": "SYN-007",
64+
"title": "Webcam HD",
65+
"category": "Informatique",
66+
"city": "Toulouse",
67+
"price_eur": "69.00",
68+
"availability": "précommande",
69+
"updated_at": "2026-07-31",
70+
"source_url": "https://exemple.invalid/catalogue/syn-007/"
71+
},
72+
{
73+
"record_id": "SYN-008",
74+
"title": "Gourde inox",
75+
"category": "Mobilité",
76+
"city": "Grenoble",
77+
"price_eur": "24.90",
78+
"availability": "disponible",
79+
"updated_at": "2026-07-31",
80+
"source_url": "https://exemple.invalid/catalogue/syn-008/"
81+
},
82+
{
83+
"record_id": "SYN-009",
84+
"title": "Repose-poignets",
85+
"category": "Bureau",
86+
"city": "Strasbourg",
87+
"price_eur": "21.50",
88+
"availability": "disponible",
89+
"updated_at": "2026-07-31",
90+
"source_url": "https://exemple.invalid/catalogue/syn-009/"
91+
},
92+
{
93+
"record_id": "SYN-010",
94+
"title": "Casque antibruit",
95+
"category": "Bureau",
96+
"city": "Marseille",
97+
"price_eur": "84.00",
98+
"availability": "rupture",
99+
"updated_at": "2026-08-01",
100+
"source_url": "https://exemple.invalid/catalogue/syn-010/"
101+
}
102+
]

‎extracteur-donnees-publiques/exemple/extracteur_demo.py‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
#!/usr/bin/env python3
2+
"""Extrait 8 champs depuis les 10 pages HTML synthétiques incluses."""
3+
from __future__ import annotations
4+
5+
import csv
6+
import json
7+
import re
8+
from pathlib import Path
9+
10+
ROOT = Path(__file__).resolve().parent
11+
SOURCE = ROOT / "source-pages"
12+
FIELDS = ["record_id", "title", "category", "city", "price_eur", "availability", "updated_at", "source_url"]
13+
14+
15+
def capture(pattern: str, html: str, field: str) -> str:
16+
match = re.search(pattern, html, flags=re.I | re.S)
17+
if not match:
18+
raise ValueError(f"champ absent: {field}")
19+
return re.sub(r"<[^>]+>", "", match.group(1)).strip()
20+
21+
22+
def extract(path: Path) -> dict[str, str]:
23+
html = path.read_text(encoding="utf-8")
24+
return {
25+
"record_id": capture(r'data-record-id="([^"]+)"', html, "record_id"),
26+
"title": capture(r"<h1>(.*?)</h1>", html, "title"),
27+
"category": capture(r'class="category">(.*?)</p>', html, "category"),
28+
"city": capture(r'class="city">(.*?)</p>', html, "city"),
29+
"price_eur": capture(r'class="price" value="([^"]+)"', html, "price_eur"),
30+
"availability": capture(r'class="availability">(.*?)</p>', html, "availability"),
31+
"updated_at": capture(r'<time datetime="([^"]+)"', html, "updated_at"),
32+
"source_url": f"https://exemple.invalid/catalogue/{path.stem}/",
33+
}
34+
35+
36+
def main() -> None:
37+
records = [extract(path) for path in sorted(SOURCE.glob("*.html"))]
38+
if len(records) != 10:
39+
raise SystemExit(f"10 pages attendues, {len(records)} trouvées")
40+
with (ROOT / "exemple-extraction.csv").open("w", encoding="utf-8", newline="") as handle:
41+
writer = csv.DictWriter(handle, fieldnames=FIELDS, lineterminator="\n")
42+
writer.writeheader()
43+
writer.writerows(records)
44+
(ROOT / "exemple-extraction.json").write_text(json.dumps(records, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
45+
(ROOT / "journal-execution.txt").write_text(
46+
"STATUT=SUCCES\nPAGES_LUES=10\nLIGNES_PRODUITES=10\nERREURS=0\nDONNEES=SYNTHETIQUES\n",
47+
encoding="utf-8",
48+
)
49+
print("OK: 10 pages, 8 champs, 0 erreur")
50+
51+
52+
if __name__ == "__main__":
53+
main()

‎extracteur-donnees-publiques/exemple/generate_sample.py‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
#!/usr/bin/env python3
2+
"""Génère puis extrait un mini-catalogue 100 % synthétique de 10 pages."""
3+
from __future__ import annotations
4+
5+
import csv
6+
import json
7+
import re
8+
import zipfile
9+
from html import escape
10+
from pathlib import Path
11+
12+
ROOT = Path(__file__).resolve().parent
13+
SOURCE = ROOT / "source-pages"
14+
FIELDS = [
15+
"record_id",
16+
"title",
17+
"category",
18+
"city",
19+
"price_eur",
20+
"availability",
21+
"updated_at",
22+
"source_url",
23+
]
24+
ROWS = [
25+
("SYN-001", "Lampe de bureau", "Bureau", "Lille", "42.00", "disponible", "2026-07-28"),
26+
("SYN-002", "Support écran", "Bureau", "Rennes", "59.00", "disponible", "2026-07-29"),
27+
("SYN-003", "Clavier compact", "Informatique", "Nantes", "74.50", "rupture", "2026-07-29"),
28+
("SYN-004", "Sacoche ordinateur", "Mobilité", "Lyon", "38.90", "disponible", "2026-07-30"),
29+
("SYN-005", "Hub USB-C", "Informatique", "Paris", "49.90", "disponible", "2026-07-30"),
30+
("SYN-006", "Carnet relié", "Papeterie", "Bordeaux", "18.00", "disponible", "2026-07-30"),
31+
("SYN-007", "Webcam HD", "Informatique", "Toulouse", "69.00", "précommande", "2026-07-31"),
32+
("SYN-008", "Gourde inox", "Mobilité", "Grenoble", "24.90", "disponible", "2026-07-31"),
33+
("SYN-009", "Repose-poignets", "Bureau", "Strasbourg", "21.50", "disponible", "2026-07-31"),
34+
("SYN-010", "Casque antibruit", "Bureau", "Marseille", "84.00", "rupture", "2026-08-01"),
35+
]
36+
37+
38+
def build_sources() -> None:
39+
SOURCE.mkdir(parents=True, exist_ok=True)
40+
for record_id, title, category, city, price, availability, updated_at in ROWS:
41+
html = f"""<!doctype html>
42+
<html lang="fr"><head><meta charset="utf-8"><title>{escape(title)}</title></head>
43+
<body><main class="product" data-record-id="{record_id}">
44+
<h1>{escape(title)}</h1><p class="category">{escape(category)}</p>
45+
<p class="city">{escape(city)}</p><data class="price" value="{price}">{price} €</data>
46+
<p class="availability">{escape(availability)}</p><time datetime="{updated_at}">{updated_at}</time>
47+
</main></body></html>"""
48+
(SOURCE / f"{record_id.lower()}.html").write_text(html, encoding="utf-8")
49+
50+
51+
def text(pattern: str, html: str, field: str) -> str:
52+
match = re.search(pattern, html, flags=re.I | re.S)
53+
if not match:
54+
raise ValueError(f"champ absent: {field}")
55+
return re.sub(r"<[^>]+>", "", match.group(1)).strip()
56+
57+
58+
def extract() -> list[dict[str, str]]:
59+
records = []
60+
for path in sorted(SOURCE.glob("*.html")):
61+
html = path.read_text(encoding="utf-8")
62+
record = {
63+
"record_id": text(r'data-record-id="([^"]+)"', html, "record_id"),
64+
"title": text(r"<h1>(.*?)</h1>", html, "title"),
65+
"category": text(r'class="category">(.*?)</p>', html, "category"),
66+
"city": text(r'class="city">(.*?)</p>', html, "city"),
67+
"price_eur": text(r'class="price" value="([^"]+)"', html, "price_eur"),
68+
"availability": text(r'class="availability">(.*?)</p>', html, "availability"),
69+
"updated_at": text(r'<time datetime="([^"]+)"', html, "updated_at"),
70+
"source_url": f"https://exemple.invalid/catalogue/{path.stem}/",
71+
}
72+
records.append(record)
73+
return records
74+
75+
76+
def write_outputs(records: list[dict[str, str]]) -> None:
77+
with (ROOT / "exemple-extraction.csv").open("w", encoding="utf-8", newline="") as handle:
78+
writer = csv.DictWriter(handle, fieldnames=FIELDS, lineterminator="\n")
79+
writer.writeheader()
80+
writer.writerows(records)
81+
(ROOT / "exemple-extraction.json").write_text(
82+
json.dumps(records, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
83+
)
84+
(ROOT / "journal-execution.txt").write_text(
85+
"STATUT=SUCCES\nPAGES_LUES=10\nLIGNES_PRODUITES=10\nERREURS=0\nDONNEES=SYNTHETIQUES\n",
86+
encoding="utf-8",
87+
)
88+
89+
90+
def bundle() -> None:
91+
files = [
92+
"extracteur_demo.py",
93+
"README.md",
94+
"exemple-extraction.csv",
95+
"exemple-extraction.json",
96+
"journal-execution.txt",
97+
]
98+
with zipfile.ZipFile(ROOT / "exemple-extracteur-donnees-publiques.zip", "w", zipfile.ZIP_DEFLATED) as archive:
99+
for name in files:
100+
archive.write(ROOT / name, arcname=name)
101+
for path in sorted(SOURCE.glob("*.html")):
102+
archive.write(path, arcname=f"source-pages/{path.name}")
103+
104+
105+
if __name__ == "__main__":
106+
build_sources()
107+
records = extract()
108+
write_outputs(records)
109+
bundle()
110+
print(f"OK: {len(records)} pages, {len(FIELDS)} champs, 0 erreur")

‎extracteur-donnees-publiques/exemple/journal-execution.txt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
STATUT=SUCCES
2+
PAGES_LUES=10
3+
LIGNES_PRODUITES=10
4+
ERREURS=0
5+
DONNEES=SYNTHETIQUES

‎extracteur-donnees-publiques/exemple/source-pages/syn-001.html‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
<!doctype html>
2+
<html lang="fr"><head><meta charset="utf-8"><title>Lampe de bureau</title></head>
3+
<body><main class="product" data-record-id="SYN-001">
4+
<h1>Lampe de bureau</h1><p class="category">Bureau</p>
5+
<p class="city">Lille</p><data class="price" value="42.00">42.00 €</data>
6+
<p class="availability">disponible</p><time datetime="2026-07-28">2026-07-28</time>
7+
</main></body></html>

‎extracteur-donnees-publiques/exemple/source-pages/syn-002.html‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
<!doctype html>
2+
<html lang="fr"><head><meta charset="utf-8"><title>Support écran</title></head>
3+
<body><main class="product" data-record-id="SYN-002">
4+
<h1>Support écran</h1><p class="category">Bureau</p>
5+
<p class="city">Rennes</p><data class="price" value="59.00">59.00 €</data>
6+
<p class="availability">disponible</p><time datetime="2026-07-29">2026-07-29</time>
7+
</main></body></html>

‎extracteur-donnees-publiques/exemple/source-pages/syn-003.html‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
<!doctype html>
2+
<html lang="fr"><head><meta charset="utf-8"><title>Clavier compact</title></head>
3+
<body><main class="product" data-record-id="SYN-003">
4+
<h1>Clavier compact</h1><p class="category">Informatique</p>
5+
<p class="city">Nantes</p><data class="price" value="74.50">74.50 €</data>
6+
<p class="availability">rupture</p><time datetime="2026-07-29">2026-07-29</time>
7+
</main></body></html>

0 commit comments

Comments
 (0)