Skip to content

feat: Enable Full Dataset Support & Improve Documentation #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
henrikvalv3 merged 1 commit into from
Feb 13, 2026
Merged

feat: Enable Full Dataset Support & Improve Documentation #26

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 68 additions & 23 deletions places_insights/notebooks/sample_data_demo/places_insights_sample_data_demo.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,9 @@
"source": [
"### Overall Goal\n",
"\n",
"This notebook serves as a technical introduction for developers and data analysts who have subscribed to the [Places Insights Sample Datasets](https://developers.google.com/maps/documentation/placesinsights/cloud-setup#sample_data).\n",
"This notebook serves as a technical introduction for developers and data analysts who have subscribed to the **[Places Insights Datasets](https://developers.google.com/maps/documentation/placesinsights)**.\n",
"\n",
"Its primary purpose is to demonstrate how to query, aggregate, and visualize Google Maps Platform [Places Insights](https://developers.google.com/maps/documentation/placesinsights) data within a **BigQuery** environment. By running this notebook, you will learn how to transition from raw dataset subscriptions to actionable geospatial insights using Standard SQL and Python.\n",
"Its primary purpose is to demonstrate how to query, aggregate, and visualize Google Maps Platform Places Insights data within a **BigQuery** environment. Whether you are evaluating the product using **Sample Data** or building production workflows with **Full Data**, this notebook will help you transition from raw dataset subscriptions to actionable geospatial insights using Standard SQL and Python.\n",
"\n",
"### Key Technologies Used\n",
"\n",
Expand All @@ -94,7 +94,7 @@
"\n",
"### The Step-by-Step Workflow\n",
"\n",
"1. **Configuration:** The notebook initializes the environment based on your selection of a Sample City (e.g., \"New York City\", \"Tokyo\", \"London\"). It automatically maps your selection to the correct **Analytics Hub Dataset ID** and geographic coordinates.\n",
"1. **Configuration:** The notebook initializes the environment based on your selection of a City (e.g., \"New York City\", \"Tokyo\") and your **Data Source** (Sample or Full). It automatically maps your selection to the correct Analytics Hub Dataset ID and geographic coordinates.\n",
"\n",
"2. **Direct Query Analysis (Radius Search):** We demonstrate how to count places that match specific criteria, such as `primary_type`, `business_status`, and boolean attributes like `allows_dogs`, within a set radius. We utilize the `WITH AGGREGATION_THRESHOLD` clause to compare amenity density across multiple neighborhoods programmatically.\n",
"\n",
Expand All @@ -108,7 +108,7 @@
"\n",
"1. **Prerequisites:**\n",
" * A Google Cloud Project with the **BigQuery API** and **Map Tiles API** enabled.\n",
" * A subscription to at least one of the [Places Insights Sample Datasets](https://developers.google.com/maps/documentation/placesinsights/cloud-setup#sample_data) via Analytics Hub.\n",
" * A subscription to either the [Sample Datasets](https://developers.google.com/maps/documentation/placesinsights/cloud-setup#sample_data) or the [Full Datasets](https://developers.google.com/maps/documentation/placesinsights/cloud-setup#full_data) via Analytics Hub.\n",
"\n",
"2. **Set Up Secrets:** Configure the following keys in the Colab \"Secrets\" tab (the **key icon** on the left menu):\n",
" * `GCP_PROJECT_ID`: Your Google Cloud Project ID.\n",
Expand Down Expand Up @@ -162,17 +162,22 @@
"cell_type": "code",
"source": [
"# @title 2. Configuration\n",
"# @markdown Select a Sample City to demo. This configures the dataset target and map center.\n",
"# @markdown\n",
"# @markdown **Important:** You must be subscribed to the [Sample Dataset](https://developers.google.com/maps/documentation/placesinsights/cloud-setup#sample_data) in your GCP project.\n",
"# @markdown\n",
"# @markdown If you renamed the dataset during the Analytics Hub subscription process, enter the full table name (e.g., `my_project.my_dataset.places_sample`) in the **Custom Table Name** field below. Leave it blank to use the default.\n",
"# @markdown Select a Sample City to demo. This configures the map center and the target BigQuery dataset.\n",
"\n",
"# @markdown ### 1. Select Location\n",
"SAMPLE_LOCATION = \"New York City, USA\" # @param [\"Sydney, Australia\", \"Sao Paulo, Brazil\", \"Toronto, Canada\", \"Paris, France\", \"Berlin, Germany\", \"Mumbai, India\", \"Jakarta, Indonesia\", \"Rome, Italy\", \"Tokyo, Japan\", \"Mexico City, Mexico\", \"Madrid, Spain\", \"Zurich, Switzerland\", \"London, UK\", \"New York City, USA\"]\n",
"\n",
"# @markdown ### 2. Select Data Source\n",
"# @markdown Choose **Sample Data** for evaluation (requires [\"Sample\" subscription](https://developers.google.com/maps/documentation/placesinsights/cloud-setup#sample_data)) or **Full Data** for production (requires [\"Full\" subscription](https://developers.google.com/maps/documentation/placesinsights/cloud-setup#full_data)).\n",
"DATASET_MODE = \"Sample Data\" # @param [\"Sample Data\", \"Full Data\"]\n",
"\n",
"# @markdown ### 3. Custom Override (Optional)\n",
"# @markdown Enter a specific table name (e.g., `my_project.my_dataset.my_table`) to override the default.\n",
"# @markdown\n",
"# @markdown Only use this field if you renamed the dataset or table during the Analytics Hub subscription process. Leave blank to use the default naming convention.\n",
"CUSTOM_TABLE_NAME = \"\" # @param {type:\"string\"}\n",
"\n",
"# Configuration Dictionary mapping selection to Dataset and Center Coordinates\n",
"# Note: These table names assume the default naming convention from Analytics Hub.\n",
"# Configuration Dictionary mapping selection to Country Code and Center Coordinates\n",
"config_map = {\n",
" \"Sydney, Australia\": {\"code\": \"au\", \"center\": (-33.8688, 151.2093)},\n",
" \"Sao Paulo, Brazil\": {\"code\": \"br\", \"center\": (-23.5505, -46.6333)},\n",
Expand All @@ -190,17 +195,26 @@
" \"New York City, USA\": {\"code\": \"us\", \"center\": (40.7580, -73.9855)},\n",
"}\n",
"\n",
"# 1. Retrieve Config\n",
"selected_config = config_map[SAMPLE_LOCATION]\n",
"code = selected_config['code']\n",
"CITY_CENTER_LAT, CITY_CENTER_LNG = selected_config['center']\n",
"\n",
"# Determine Table Name\n",
"# 2. Determine Table Name\n",
"if CUSTOM_TABLE_NAME and CUSTOM_TABLE_NAME.strip():\n",
" # Priority 1: User Custom Override\n",
" DATASET_TABLE = CUSTOM_TABLE_NAME.strip()\n",
" print(f\"⚠️ Using Custom Table Name: {DATASET_TABLE}\")\n",
"else:\n",
" # Default construction\n",
" DATASET_TABLE = f\"places_insights___{selected_config['code']}___sample.places_sample\"\n",
" print(f\"🎯 Target Table (Default): {DATASET_TABLE}\")\n",
" # Priority 2: Auto-Construct based on Mode\n",
" if DATASET_MODE == \"Sample Data\":\n",
" # Pattern: places_insights___us___sample.places_sample\n",
" DATASET_TABLE = f\"places_insights___{code}___sample.places_sample\"\n",
" else:\n",
" # Pattern: places_insights___us.places\n",
" DATASET_TABLE = f\"places_insights___{code}.places\"\n",
"\n",
" print(f\"🎯 Target Table ({DATASET_MODE}): {DATASET_TABLE}\")\n",
"\n",
"print(f\"🌍 Selected Location: {SAMPLE_LOCATION}\")\n",
"print(f\"📍 Center Point: {CITY_CENTER_LAT}, {CITY_CENTER_LNG}\")"
Expand Down Expand Up @@ -292,8 +306,14 @@
{
"cell_type": "code",
"source": [
"# @title 4. Direct Query: Analyzing \"Al Fresco & Dog-Friendly\" Hubs\n",
"# @markdown The following cell runs the BigQuery analysis to find hotspots.\n",
"# @title 4. Direct Query: Identifying \"Al Fresco\" Dining Hubs\n",
"# @markdown This cell demonstrates how to perform a radius search to find specific business types with distinct attributes.\n",
"# @markdown\n",
"# @markdown **The Analysis Workflow:**\n",
"# @markdown 1. **Generate Probe Locations:** Python calculates coordinates for 5 \"neighborhoods\" around the city center.\n",
"# @markdown 2. **Spatial Join (`ST_DWITHIN`):** We query BigQuery to find places within **1,000 meters** of these points.\n",
"# @markdown 3. **Attribute Filtering:** We filter the results for **Operational Restaurants** that specifically feature `outdoor_seating` **OR** `allows_dogs`.\n",
"# @markdown 4. **Privacy Compliance:** The query uses `WITH AGGREGATION_THRESHOLD` to ensure counts are only returned if they meet the privacy threshold (minimum 5).\n",
"\n",
"# --- 1. Generate Candidate Locations (Python) ---\n",
"def get_offset_point(lat, lng, lat_offset, lng_offset):\n",
Expand Down Expand Up @@ -405,7 +425,14 @@
"cell_type": "code",
"source": [
"# @title 5. Visualization: Marker Map\n",
"# @markdown Renders the query results using the initialized map assets.\n",
"# @markdown This cell renders the analysis results on an interactive map.\n",
"# @markdown\n",
"# @markdown **Visualization Details:**\n",
"# @markdown * **Map Engine:** We use [Folium](https://python-visualization.github.io/folium/latest/) to generate the map interface.\n",
"# @markdown * **Base Layer:** The background uses Google Maps Tiles(initialized in Cell 3).\n",
"# @markdown * **Data Overlay:** Markers represent the 5 distinct neighborhoods analyzed in the previous step. They are **color-coded** to highlight the relative density of dining options (Red = Hotspot, Blue = Low Activity).\n",
"# @markdown * **Interactivity:** Click on any marker to see the exact count of matching places.\n",
"\n",
"import folium\n",
"from folium import plugins, Element\n",
"\n",
Expand Down Expand Up @@ -468,7 +495,8 @@
"display(m)"
],
"metadata": {
"id": "F4Zt7xqqJjWB"
"id": "F4Zt7xqqJjWB",
"cellView": "form"
},
"execution_count": null,
"outputs": []
Expand All @@ -477,7 +505,16 @@
"cell_type": "code",
"source": [
"# @title 6. Function Query: Fetch H3 Density Data\n",
"# @markdown We use the [PLACES_COUNT_PER_H3](https://developers.google.com/maps/documentation/placesinsights/place-count-functions/places-count-per-h3) function to retrieve aggregation data.\n",
"# @markdown This cell shows a Grid Search using the [PLACES_COUNT_PER_H3](https://developers.google.com/maps/documentation/placesinsights/place-count-functions/places-count-per-h3) SQL function.\n",
"# @markdown\n",
"# @markdown **The Analysis Workflow:**\n",
"# @markdown 1. **H3 Grid System:** We use [H3](https://h3geo.org/) (Resolution 8) to normalize the city map into uniform hexagonal cells. This is ideal for comparing density across irregular city layouts.\n",
"# @markdown 2. **The Query:** We call the pre-defined BigQuery function `PLACES_COUNT_PER_H3`. This function automatically aggregates places into the grid cells.\n",
"# @markdown 3. **Search Parameters:**\n",
"# @markdown * **Area:** A **5km radius** around the city center.\n",
"# @markdown * **Types:** Broad commercial activity (`restaurant`, `store`, `point_of_interest`).\n",
"# @markdown * **Status:** Only `OPERATIONAL` places.\n",
"# @markdown 4. **Geometry Processing:** The query returns GeoJSON strings for each hexagon, which we parse into a `GeoDataFrame` for visualization.\n",
"\n",
"import json\n",
"from shapely.geometry import shape\n",
Expand Down Expand Up @@ -539,8 +576,15 @@
{
"cell_type": "code",
"source": [
"# @title 7. Visualization: H3 Heatmap\n",
"# @markdown We visualize the density data as a Choropleth layer on the Google Maps basemap, reusing the backend assets.\n",
"# @title 7. Visualization: H3 Density Heatmap\n",
"# @markdown This cell renders the H3 grid data as a Choropleth map overlay.\n",
"# @markdown\n",
"# @markdown **Visualization Details:**\n",
"# @markdown * **Technique (Choropleth):** We map the `count` of places in each H3 hexagon to a color scale.\n",
"# @markdown * **Color Scale:** We use a **Yellow-Orange-Red (`YlOrRd`)** gradient.\n",
"# @markdown * 🟨 **Yellow:** Lower density of places.\n",
"# @markdown * 🟥 **Red:** High density/commercial saturation.\n",
"# @markdown * **Context:** By overlaying this on the map, you can correlate high-density hexagons with specific neighborhoods, transit hubs, or shopping districts.\n",
"\n",
"if 'gdf_h3' not in locals() or gdf_h3 is None or gdf_h3.empty:\n",
" raise ValueError(\"No H3 data found! Please run Cell 6 successfully first.\")\n",
Expand Down Expand Up @@ -574,7 +618,8 @@
"display(m_hex)"
],
"metadata": {
"id": "H_6oq9nrJpev"
"id": "H_6oq9nrJpev",
"cellView": "form"
},
"execution_count": null,
"outputs": []
Expand Down