Skip to content

Commit 06d79ea

Browse files
jenningsandersonsethfitzdanabauer
authored
New reference documentation from Pydantic (#222)
* Build docs without schema repo dependency This change restructures the docs build to remove the dependency on the schema repository, enabling faster builds and simpler deployment. Key changes: - Add injectable schema references for dynamic content - Reorganize example files and schema definitions - Optimize image assets (convert large GIFs to MP4s, compress PNGs) - Add staging deployment workflow - Update linting and formatting configurations - Fix package-lock.json for cross-platform compatibility * Replace schema reference with current codegen output Generated from overture-schema-codegen (code-generation branch) using explicit theme selection to exclude internal types: overture-codegen generate \ --theme addresses --theme base --theme buildings \ --theme divisions --theme places --theme transportation \ --format markdown --output-dir docs/schema/reference Cross-theme types (names, perspectives, cartography, scoping, strings, sources, primitives, geometry) now live under types/{subsystem}/ instead of flat at reference root or under Names/. Transportation segments (road, rail, water) and their ~50 referenced types are not yet generated -- the codegen doesn't decompose the Segment discriminated union into individual models. A WIP banner on the schema index notes this gap. Broken link warnings enabled in docusaurus.config.js to surface pre-existing and new issues during builds. * Sync schema reference with current codegen The codegen now outputs a flatter directory structure: types/ split into core/ (names, cartography, scoping, sources, perspectives) and system/ (primitives, string types, ref types). Theme feature-type subdirectories removed (e.g. addresses/address/address.md becomes addresses/address.md). Internal cross-reference links updated to match; no content changes. sidebars.js rewritten to match the new layout. * Sync schema reference Fixes newlines that break tables. * Sync schema reference Include Segment, a union type with complex rendering requirements * Sync schema reference Fix sidebar * Sync schema reference Include model-level validation rules * Sync schema reference Include various minor improvements * Fix syntax error * One sidebar for them all. * Fixing links and build warnings * sidebar and structural changes to support new schema doc work --------- Co-authored-by: Seth Fitzsimmons <sethfitz@amazon.com> Co-authored-by: Dana Bauer <dana.bauer@gmail.com>
1 parent b863991 commit 06d79ea

206 files changed

Lines changed: 13927 additions & 14523 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,4 @@
11
.DS_Store
22
.docusaurus
33
node_modules/
4-
docs/_examples/
5-
docs/_schema/
6-
docs/schema/
7-
docs/schema
8-
static/_schema/
94
build/

.markdownlint.json

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"default": true,
3+
"MD001": false,
4+
"MD003": false,
5+
"MD010": false,
6+
"MD013": false,
7+
"MD024": {
8+
"siblings_only": true
9+
},
10+
"MD025": false,
11+
"MD029": false,
12+
"MD033": false,
13+
"MD036": false,
14+
"MD040": false,
15+
"MD041": false,
16+
"MD046": false
17+
}

.prettierignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
build/
2+
.docusaurus/
3+
node_modules/
4+
package-lock.json
5+
*.md
6+
*.mdx

.prettierrc

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"semi": true,
3+
"singleQuote": true,
4+
"tabWidth": 2,
5+
"trailingComma": "es5",
6+
"printWidth": 100,
7+
"bracketSpacing": true,
8+
"arrowParens": "always",
9+
"proseWrap": "preserve"
10+
}

blog/2024-04-21-welcome.mdx

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,15 @@
11
---
22
title: Welcome to the Overture Maps engineering blog!
3-
authors:
4-
name: Dana Bauer
5-
title: Technical Product Manager, Overture Maps
6-
email: dana@overturemaps.org
3+
authors: dana
74
---
85

96
Welcome to the Overture Maps engineering blog!
107
We're excited to tell the technical stories behind this big, exciting, ambitious, intensely collaborative project we’ve been [working on for more than a year](https://overturemaps.org/looking-forward-to-2024/).
118
The engineers building Overture Maps are eager to connect with our developer community.
129
In the posts we publish in the coming months, you’ll hear from team members who work at Amazon, Meta, Microsoft, TomTom, Esri, Development Seed, Precisely, and more.
10+
1311
<!-- truncate -->
12+
1413
And we’d like to hear from you: [feedback and contributions](https://github.com/OvertureMaps/data/discussions) from folks working with our data and schema are crucial to our success.
1514
Thanks for joining us on this journey.
1615
We’re glad you’re here.

blog/2024-04-22-beta-release.mdx

Lines changed: 28 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,79 +1,84 @@
11
---
22
title: Exploring our beta release
33
description: Highlights from Overture Maps beta release
4-
authors:
5-
name: Dana Bauer
6-
title: Technical Product Manager, Overture Maps
7-
email: dana@overturemaps.org
4+
authors: dana
85
tags:
96
- tools
107
- geoparquet
118
---
9+
1210
import Tabs from '@theme/Tabs';
1311
import TabItem from '@theme/TabItem';
1412

15-
Last week Overture Maps [announced the beta release](https://overturemaps.org/overture-maps-foundation-releases-beta-of-its-first-open-map-dataset/) of our schema and data. After months of hard work and steady improvements, we are nearing production-level stability. In a series of posts over the next few weeks -- starting with this one -- we’ll unpack the highlights and improvements you'll see in this release and beyond.
13+
Last week Overture Maps [announced the beta release](https://overturemaps.org/overture-maps-foundation-releases-beta-of-its-first-open-map-dataset/) of our schema and data. After months of hard work and steady improvements, we are nearing production-level stability. In a series of posts over the next few weeks -- starting with this one -- we’ll unpack the highlights and improvements you'll see in this release and beyond.
1614

1715
<!-- truncate -->
1816

1917
## Making geospatial fast
18+
2019
The first thing you'll notice is how much faster you can [pull Overture Maps data out of the cloud](https://docs.overturemaps.org/getting-data/locally/). We made this possible by working closely with open-source collaborators on the repartitioning and [spatial optimization](https://cloudnativegeo.org/blog/2024/03/coming-soon-geoparquet-1.1/) of our GeoParquet files. Meanwhile, our friends at DuckDB, one of our favorite SQL tools, [added a feature](https://github.com/duckdb/duckdb/pull/10314) that improves the performance of queries with bounding boxes.
2120

2221
<details>
2322

24-
<summary>Query speeds have improved significantly from our February release to the April beta release. This example compares DuckDB queries for buildings in Philadelphia. [See here for more information about our performance testing](https://github.com/opengeospatial/geoparquet/discussions/188#discussioncomment-8909348).</summary>
23+
<summary>
24+
Query speeds have improved significantly from our February release to the April beta release. This
25+
example compares DuckDB queries for buildings in Philadelphia. [See here for more information
26+
about our performance
27+
testing](https://github.com/opengeospatial/geoparquet/discussions/188#discussioncomment-8909348).
28+
</summary>
2529

2630
<Tabs>
2731
<TabItem value="february" label="2024-02-15-alpha.0 release" default>
2832
```
29-
SELECT
33+
SELECT
3034
*
31-
FROM
35+
FROM
3236
read_parquet('s3://overturemaps-us-west-2/release/2024-02-15-alpha.0/theme=buildings/type=building/*', filename=true, hive_partitioning=1)
33-
WHERE
37+
WHERE
3438
bbox.minx > -75.60154
3539
AND bbox.maxx < -74.955763
3640
AND bbox.miny > 39.867004
3741
AND bbox.maxy < 40.137992;
3842
```
39-
820,915 buildings
43+
820,915 buildings
4044
~120s
4145
</TabItem>
4246

43-
47+
4448
<TabItem value="april" label="2024-04-16-beta.0 release" default>
4549
```
46-
SELECT
50+
SELECT
4751
*
48-
FROM
52+
FROM
4953
read_parquet('s3://overturemaps-us-west-2/release/2024-04-16-beta.0/theme=buildings/type=building/*', filename=true, hive_partitioning=1)
50-
WHERE
54+
WHERE
5155
bbox.xmin > -75.60154
5256
AND bbox.xmax < -74.955763
5357
AND bbox.ymin > 39.867004
5458
AND bbox.ymax < 40.137992
5559
```
56-
852,487 buildings
60+
852,487 buildings
5761
~25s
5862
</TabItem>
5963
</Tabs>
64+
6065
</details>
6166

62-
We're continuing to make things faster and easier for users. Along with the folks at Development Seed, an [Overture Maps Foundation member](https://overturemaps.org/about/members/), we're [building special tools for Overture Maps data on top of lonboard](https://developmentseed.org/lonboard/latest/examples/overture-maps/), their Python library for visualizing large geospatial datasets in Jupyter. And recently our friends at Wherobots [took a comprehensive look](https://wherobots.com/overture-maps-data-cloud-native-geoparquet-apache-sedona/) at how our use of GeoParquet makes querying and analyzing our data with Apache Sedona very efficient.
67+
We're continuing to make things faster and easier for users. Along with the folks at Development Seed, an [Overture Maps Foundation member](https://overturemaps.org/about/members/), we're [building special tools for Overture Maps data on top of lonboard](https://developmentseed.org/lonboard/latest/examples/overture-maps/), their Python library for visualizing large geospatial datasets in Jupyter. And recently our friends at Wherobots [took a comprehensive look](https://wherobots.com/overture-maps-data-cloud-native-geoparquet-apache-sedona/) at how our use of GeoParquet makes querying and analyzing our data with Apache Sedona very efficient.
6368

6469
As you can see, we're moving forward with the community to iterate on data, software, and specifications with the shared goal of making geospatial **_fast_**.
6570

6671
## Easier-to-use schema
72+
6773
Another highlight of the beta release is the transition to an easier-to-use schema for our administrative boundary data. We first [explored this idea](https://github.com/OvertureMaps/schema/discussions/117) with the Overture Maps community in February, and after two short months of work, the [new divisions schema and data](https://docs.overturemaps.org/guides/divisions/) are ready to go. Here's a query to `divisions` that grabs geometries for all the countries in the world:
6874

6975
```
7076
SELECT *
7177
FROM read_parquet('s3://overturemaps-us-west-2/release/2024-04-16-beta.0/theme=divisions/type=division_area/*', filename=true, hive_partitioning=1)
7278
WHERE subtype = 'country';
7379
```
74-
75-
You can see that the `divisions` query above is much simpler than a comparable query to `admins`:
7680

81+
You can see that the `divisions` query above is much simpler than a comparable query to `admins`:
7782

7883
```
7984
WITH admins AS (
@@ -87,14 +92,14 @@ WITH admins AS (
8792
ON country_area.locality_id = country_locality.id
8893
WHERE country_locality.locality_type = 'country';
8994
90-
```
95+
```
9196

92-
We plan to [deprecate admins](https://docs.overturemaps.org/release-notes/) by the July release. In the meantime, both `admins` and `divisions` will be available to users.
97+
We plan to [deprecate admins](https://docs.overturemaps.org/release-notes/) by the July release. In the meantime, both `admins` and `divisions` will be available to users.
9398

9499
## Bridges, islands, waterfalls, and more!
100+
95101
We added more rich detail to our `base` layer in this release, including an `infrastructure` type with familiar features from [Facebook’s Daylight map distribution](https://daylightmap.org/). We also added new subtypes and classes for the `land`, `land_use`, and `water` feature types. You'll find a comprehensive listing of the subtypes and classes for each feature type in our [schema reference docs](https://docs.overturemaps.org/schema/reference/base/infrastructure). Ready to make your own map? We have a [tutorial to help you get started](https://docs.overturemaps.org/examples/build-a-map/#13/47.6/-122.33/0/45).
96-
97-
## Stay tuned for more highlights
98-
We'll be back soon with more posts that explore our path from the beta release to production. In the meantime, we invite you to [get started with our data](https://docs.overturemaps.org/getting-data/) and share with us your [comments and feedback](https://github.com/OvertureMaps/data/discussions).
99102

103+
## Stay tuned for more highlights
100104

105+
We'll be back soon with more posts that explore our path from the beta release to production. In the meantime, we invite you to [get started with our data](https://docs.overturemaps.org/getting-data/) and share with us your [comments and feedback](https://github.com/OvertureMaps/data/discussions).

blog/2024-05-16-land-cover.mdx

Lines changed: 19 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
11
---
22
title: Land cover is live
3-
authors:
4-
name: Dana Bauer
5-
title: Technical Product Manager, Overture Maps
6-
email: dana@overturemaps.org
3+
authors: dana
74
tags:
85
- base
96
---
@@ -16,7 +13,7 @@ import useBaseUrl from '@docusaurus/useBaseUrl';
1613

1714
## Mapmakers rejoice!
1815

19-
The [May release](https://docs.overturemaps.org/release/latest) of Overture Maps includes new high-resolution land cover data *and* new cartographic schema properties.
16+
The [May release](https://docs.overturemaps.org/release/latest) of Overture Maps includes new high-resolution land cover data _and_ new cartographic schema properties.
2017

2118
<!-- truncate -->
2219

@@ -26,20 +23,19 @@ Our May release also includes [schema properties](https://docs.overturemaps.org/
2623

2724
## Exploring land cover
2825

29-
In the notebook example below, we'll show you how to extract, process, and visualize land cover data for an area of interest using [lonboard](https://developmentseed.org/lonboard/latest/) and the [Overture Maps Python command-line tool](https://github.com/OvertureMaps/overturemaps-py). We recommend that you consult [this example](https://developmentseed.org/lonboard/latest/examples/overture-maps/ ) in the lonboard docs to better understand the methods used here. You can view and download the complete notebook on [Notebook Sharing Space](https://notebooksharing.space/view/b63f6b3dda1da99c45caf53284fbc508aaae0a43480b43bb0316db636c5e6677).
30-
26+
In the notebook example below, we'll show you how to extract, process, and visualize land cover data for an area of interest using [lonboard](https://developmentseed.org/lonboard/latest/) and the [Overture Maps Python command-line tool](https://github.com/OvertureMaps/overturemaps-py). We recommend that you consult [this example](https://developmentseed.org/lonboard/latest/examples/overture-maps/) in the lonboard docs to better understand the methods used here. You can view and download the complete notebook on [Notebook Sharing Space](https://notebooksharing.space/view/b63f6b3dda1da99c45caf53284fbc508aaae0a43480b43bb0316db636c5e6677).
3127

3228
![land cover](/img/blog/notebook-overture-lc.png)
3329

34-
To follow along, you'll need to have [JupyterLab or Jupyter Notebook](https://jupyter.org/) running and the following dependencies installed:
35-
- [lonboard](https://developmentseed.org/lonboard/latest/#install)
36-
- [overturemaps-py](https://github.com/OvertureMaps/overturemaps-py)
37-
- [pandas](https://pandas.pydata.org/)
38-
- [GeoPandas](https://geopandas.org/en/stable/index.html)
39-
- [Shapely](https://shapely.readthedocs.io/en/stable/index.html)
30+
To follow along, you'll need to have [JupyterLab or Jupyter Notebook](https://jupyter.org/) running and the following dependencies installed:
4031

32+
- [lonboard](https://developmentseed.org/lonboard/latest/#install)
33+
- [overturemaps-py](https://github.com/OvertureMaps/overturemaps-py)
34+
- [pandas](https://pandas.pydata.org/)
35+
- [GeoPandas](https://geopandas.org/en/stable/index.html)
36+
- [Shapely](https://shapely.readthedocs.io/en/stable/index.html)
4137

42-
``` python
38+
```python
4339
import pandas as pd
4440
import geopandas as gpd
4541
import overturemaps
@@ -48,28 +44,28 @@ from lonboard import Map, PolygonLayer
4844
from lonboard.colormap import apply_categorical_cmap
4945
```
5046

51-
``` python
47+
```python
5248
# specify bounding box
5349
bbox = -78.6429, 39.463, -73.7806, 41.6242
5450
```
5551

56-
``` python
52+
```python
5753
# read in Overture Maps land_cover data type
5854
table = overturemaps.record_batch_reader("land_cover", bbox).read_all()
5955
table = table.combine_chunks()
6056
```
6157

62-
``` python
58+
```python
6359
# convert to dataframe
6460
df = table.to_pandas()
6561
```
6662

67-
``` python
63+
```python
6864
# filter for higher resolution land_cover features
6965
df_h = df[df.cartography.apply(lambda x: x['min_zoom'] == 8)]
7066
```
7167

72-
``` python
68+
```python
7369
# create color map for land_cover subtypes, loosely based on natural-color palette: https://www.shadedrelief.com/shelton/c.html
7470
color_map = {
7571
"urban": [167, 162, 186],
@@ -85,12 +81,12 @@ color_map = {
8581
}
8682
```
8783

88-
``` python
84+
```python
8985
# apply color map to land_cover subtypes
9086
colors = apply_categorical_cmap(df_h.subtype, color_map)
9187
```
9288

93-
``` python
89+
```python
9490
# dataframe to geodataframe, set crs
9591
gdf = gpd.GeoDataFrame(
9692
df_h,
@@ -99,7 +95,7 @@ gdf = gpd.GeoDataFrame(
9995
)
10096
```
10197

102-
``` python
98+
```python
10399
# create map layer
104100
layer = PolygonLayer.from_geopandas(
105101
gdf= gdf[['id','subtype', 'cartography', 'geometry']].reset_index(drop=True),
@@ -108,7 +104,7 @@ layer = PolygonLayer.from_geopandas(
108104
)
109105
```
110106

111-
``` python
107+
```python
112108
#render map
113109
view_state = {
114110
"longitude": -76.2,

blog/2024-07-24-explore-site.mdx

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
11
---
22
title: Exploring Overture data, no SQL required
3-
authors:
4-
name: Dana Bauer
5-
title: Technical Product Manager, Overture Maps
6-
email: dana@overturemaps.org
3+
authors: dana
74
tags:
85
- tools
96
---
@@ -17,7 +14,13 @@ Overture has launched a new browser-based tool to give users a no-code option to
1714
<source src={useBaseUrl('/img/blog/explore-philadelphia2.mp4')} type="video/mp4" />
1815
</video>
1916
<figcaption style={{ textAlign: 'center' }}>
20-
<em>Checking out <a href="https://www.associationforpublicart.org/artwork/lion-crushing-a-serpent/">"Lion Crushing a Serpent"</a> in Philadelphia's Rittenhouse Park</em>
17+
<em>
18+
Checking out{' '}
19+
<a href="https://www.associationforpublicart.org/artwork/lion-crushing-a-serpent/">
20+
"Lion Crushing a Serpent"
21+
</a>{' '}
22+
in Philadelphia's Rittenhouse Park
23+
</em>
2124
</figcaption>
2225
</figure>
2326

blog/2024-08-15-preview-august-release.mdx

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
11
---
2-
title: What to expect in Overture’s August release
3-
authors:
4-
name: Dana Bauer
5-
title: Technical Product Manager, Overture Maps
6-
email: dana@overturemaps.org
2+
title: What to expect in Overture's August release
3+
authors: dana
74
tags:
85
- addresses
96
- base

0 commit comments

Comments
 (0)