Skip to content

Commit fb27f3a

Browse files
keyurvakeyurva
authored
Add skill resource pointers to discovery tools and split entity playbooks (#210)
1 parent 7972107 commit fb27f3a

6 files changed

Lines changed: 386 additions & 45 deletions

File tree

‎packages/datacommons-mcp/datacommons_mcp/instructions/agent_api/server.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
Act as a Data Commons Research Assistant. This server provides direct access to a massive, unified knowledge graph of aggregated statistical data from authoritative regional and global sources like the UN, World Bank, and Census Bureau. Use it to transform natural language queries into precise statistical insights by identifying specific indicators and retrieving their observations. It contains historical and recent data points on topics like demographics, economics, health, and environment across various geographic levels. It does not contain information on topics like real-time news, subjective viewpoints, or private corporate data.
22

3-
CRITICAL INSTRUCTION: When the user asks you to perform statistical research, search for indicators, or fetch observations, you MUST first read the skill resource at 'skill://data-commons-researcher/SKILL.md' to load the official research guidelines, place resolution heuristics, and child-sampling playbooks before calling any tools. This playbook contains the required rules and steps to successfully fulfill the query.
3+
CRITICAL INSTRUCTION: When performing statistical research, searching for indicators, or fetching observations, you MUST first read the appropriate **MCP Resource** before calling any tools. Use your platform's standard MCP resource-reading capability to retrieve:
4+
- `skill://data-commons-researcher/SKILL.md` (for single-place queries)
5+
- `skill://data-commons-child-places-researcher/SKILL.md` (for child-places, sub-national breakdowns, or geographic hierarchies)
46

57
Crucially, every data point retrieved must be attributed to its original source provided in the tool output; never present statistics as "known facts" without citing the specific organization or dataset they originated from. Prioritize data integrity and transparency, ensuring that users understand both the metric and the provenance of the information provided.

‎packages/datacommons-mcp/datacommons_mcp/instructions/agent_api/skills/data-commons-child-places-researcher/SKILL.md‎

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
---
2+
name: data-commons-child-places-researcher
3+
description: Guidelines, heuristics, and workflows for concept splitting, parent/child place resolution, sub-national sampling, place type determination, and retrieving statistical observations for child places from Data Commons.
4+
---
5+
6+
## Foundational Knowledge: Data Commons Graph Structure
7+
8+
Data Commons organizes data into two main structural hierarchies. Understanding these is key to choosing your place names and variables:
9+
10+
1. **Topics (Variable Hierarchy)**: A taxonomy of categories (e.g., `Health` -> `Clinical Data` -> `Medical Conditions`). Topics contain sub-topics and individual variables.
11+
2. **Places (Geographic Hierarchy)**: A taxonomy of spatial containment (e.g., `World` -> `Continent` -> `Country` -> `State` -> `County`).
12+
13+
### Data Availability & Efficiency Tips:
14+
15+
* **Direct Containment Efficiency**: Querying the direct child places of a parent (e.g., all counties inside California) is highly optimized and returns faster than querying arbitrary cross-border place sets.
16+
* **Single-Place Routing**: If the user's query asks for statistics about a single specific place (e.g., *"population of France"* or *"GDP of California"*), you MUST read the base skill resource at 'skill://data-commons-researcher/SKILL.md' instead.
17+
18+
---
19+
20+
## 1. The Three-Step Tool Pipeline
21+
22+
When researching statistics across child places within a parent entity, always separate your work into three distinct phases to avoid context bloat:
23+
24+
1. **Discovery (`search_child_indicators`)**: Use this to find candidate variables matching the user's concept that are available at the sub-national/child level.
25+
2. **Assessment (`get_variable_metadata`)**: Pass candidate variables and target child locations to retrieve structural metadata, ensuring the dataset matches the required temporal range, granularity, and source trust.
26+
3. **Retrieval (`get_child_observations`)**: Fetch the actual timeseries arrays across all child places of a specified type once the variables and facets have been qualified.
27+
28+
### CRITICAL: Always validate variable-place combinations first
29+
* You **MUST** call `search_child_indicators` first to verify that the variable exists for the specified child places.
30+
* You **MUST** call `get_variable_metadata` to verify dataset facets (source, dates, coverage) for sampled child places before retrieving heavy observation arrays.
31+
* Only use DCIDs returned by `search_child_indicators` - never guess or assume variable-place combinations.
32+
* This ensures data availability and prevents errors from invalid combinations.
33+
34+
---
35+
36+
## 2. Discovery Heuristics: Concept Splitting & Parameter Tuning
37+
38+
To ensure focused and accurate candidate retrieval when calling `search_child_indicators`:
39+
40+
### A. Concept Extraction & Multi-Query Splitting
41+
42+
* **Search Single Concepts**: Always search for one semantic concept at a time.
43+
* **Split Compound Queries**:
44+
* *Incorrect*: `query="health and unemployment rate"` (Causes search index confusion).
45+
* *Correct*: Split into two separate, sequential tool calls:
46+
1. `search_child_indicators(query="health", ...)`
47+
2. `search_child_indicators(query="unemployment rate", ...)`
48+
49+
### B. Parameter Configuration Guidelines
50+
51+
* **Toggling Topics (`include_topics`)**:
52+
* Set `include_topics=true` (Default) when the user's request is exploratory (e.g., *"What health data do you have for California counties?"*).
53+
* Set `include_topics=false` when targeting a specific dataset or observation (e.g., *"Find the diabetes rate for California counties"*). This reduces the return payload size.
54+
55+
* **Setting Result Limits (`per_search_limit`)**:
56+
* Always stick to the default value of `10` to keep payloads small.
57+
* Do **not** increase the limit unless the user explicitly requests more candidate indicators.
58+
59+
---
60+
61+
## 3. Geographic Place Qualification & Fallback Recovery
62+
63+
Data Commons requires qualified geographic names to avoid database name conflicts.
64+
65+
### A. Core Qualification Rules
66+
67+
* **Never use DCIDs in Search Parameters**: Only pass qualified, human-readable English place names to `parent_place` or `sample_child_places` in `search_child_indicators` (e.g., use `"California"`, not `"geoId/06"`).
68+
* **Always Qualify Naming Ambiguities**: Add parent geographic or administrative context when specifying parent or sample child places:
69+
* *New York*: Differentiate between `"New York City, USA"` and `"New York State, USA"`.
70+
* *Washington*: Differentiate between `"Washington, DC, USA"` and `"Washington State, USA"`.
71+
* *Madrid*: Differentiate between `"Madrid, Spain"` (city) and `"Community of Madrid, Spain"` (autonomous community).
72+
* *London*: Differentiate between `"London, UK"` and `"London, Ontario, Canada"`.
73+
* *Scotland*: Differentiate between `"Scotland, UK"` and `"Scotland County, USA"`.
74+
* **Extracting names from other tools**: If you get place info from another tool, extract and use *only* the readable name, but always qualify it with geographic context.
75+
76+
### B. Child Place Indicator Discovery Rule
77+
* When searching for indicators related to child places within a parent (e.g., states within a country, or counties within a state), you MUST call `search_child_indicators`, passing the parent entity in the `parent_place` parameter and a diverse sample of 5-6 of its child places in the `sample_child_places` list. This ensures the discovery of indicators that have data at the child place level.
78+
79+
### C. Vague & Unqualified Query Fallbacks
80+
* If a user asks a general or vague question about available sub-national data without specifying target indicators (e.g., *"What child data do you have for California?"* or *"What statistics exist for US counties?"*), proactively run a child topic lookup:
81+
* Call: `search_child_indicators(query="", parent_place="California", sample_child_places=["Los Angeles County, CA", "San Francisco County, CA", "Alpine County, CA"], include_topics=true)`.
82+
* Present the returned high-level topics for that place's children to guide the user's focus.
83+
84+
### D. Geographic Resolution Recovery (Troubleshooting)
85+
86+
* If the search tool resolves the wrong parent place:
87+
* Re-run `search_child_indicators` with explicit administrative parameters.
88+
89+
---
90+
91+
## 4. Playbook Recipes & Call Examples
92+
93+
### Recipe: Sampling Child Places & Containment Data
94+
* **Goal**: Check and retrieve data across child places of a parent (e.g., "unemployment rate in Indian states" or "GDP of all countries in the World").
95+
* **Step 1 (Discovery & Sampling)**: Call `search_child_indicators` using a diverse sample of child places to verify variable availability:
96+
* *Example (States in India)*: `search_child_indicators(query="unemployment", parent_place="India", sample_child_places=["Uttar Pradesh, India", "Maharashtra, India", "Tripura, India", "Bihar, India", "Kerala, India"])`
97+
* *Example (Countries in the World)*: `search_child_indicators(query="GDP", parent_place="World", sample_child_places=["USA", "China", "Germany", "Nigeria", "Brazil"])`
98+
* *Example (Administrative Level Sampling for Cities)*: `search_child_indicators(query="population", parent_place="USA", sample_child_places=["New York City, USA", "Los Angeles, USA", "Chicago, USA", "Houston, USA", "Phoenix, USA"])`
99+
* *Example (Administrative Level Sampling for States)*: `search_child_indicators(query="population", parent_place="USA", sample_child_places=["California, USA", "Texas, USA", "Florida, USA", "New York State, USA", "Pennsylvania, USA"])`
100+
* **Proxy Logic rules**:
101+
1. If a sampled child place shows data in `placesWithData` for a variable, assume that variable is available across all child places of that type.
102+
2. If no sampled child place shows data, assume the variable is not available at the child level.
103+
3. **Definitiveness of Child Search**: The results of `search_child_indicators` are absolute and definitive for the targeted child places. If a variable or concept does not appear in the child search results, it is guaranteed not to exist for those child places. Do NOT run follow-up global searches (`search_indicators`) to double-check or verify if the variable exists elsewhere.
104+
4. **No Redundant Single-Place Pings**: If a variable is confirmed via `search_child_indicators` or has child place coverage, proceed directly to `get_child_observations` (using `latest` or a narrow range). Do NOT run redundant single-place `get_observations` calls to verify the variable's active status.
105+
5. Determine the common child place type (e.g. `"State"` or `"Country"`) from the returned `dcidPlaceTypeMappings`.
106+
* **Step 2 (Assessment)**: Verify facets and date ranges for the variable across the child level by passing the resolved DCIDs of the sampled child places:
107+
* `get_variable_metadata(variable_dcids=["unemployment_rate_dcid"], entity_dcids=["resolved_child_dcid_1", "resolved_child_dcid_2", "..."])`
108+
* **Step 3 (Retrieval)**: Query observations for **ALL** child places of the determined type using `get_child_observations`:
109+
* `get_child_observations(variable_dcid="unemployment_rate_dcid", parent_place_dcid="country/IND", child_place_type="State")`
110+
111+
---
112+
113+
## 5. Processing `search_child_indicators` Responses
114+
115+
Always treat results as **candidates**. You must filter, rank, and verify them based on the user's full context.
116+
117+
### A. Response Structure Reference
118+
```json
119+
{
120+
"topics": [
121+
{
122+
"dcid": "dc/t/TopicDcid",
123+
"memberTopics": ["dc/t/SubTopic1", "..."],
124+
"memberVariables": ["dc/v/Variable1", "..."],
125+
"placesWithData": ["geoId/06037", "..."]
126+
}
127+
],
128+
"variables": [
129+
{
130+
"dcid": "dc/v/VariableDcid",
131+
"placesWithData": ["geoId/06037", "geoId/06075", "..."]
132+
}
133+
],
134+
"dcidNameMappings": {
135+
"dc/t/TopicDcid": "Readable Topic Name",
136+
"dc/v/VariableDcid": "Readable Variable Name",
137+
"geoId/06037": "Los Angeles County, CA",
138+
"geoId/06075": "San Francisco County, CA"
139+
},
140+
"dcidPlaceTypeMappings": {
141+
"geoId/06037": ["County"],
142+
"geoId/06075": ["County"]
143+
},
144+
"status": "SUCCESS"
145+
}
146+
```
147+
148+
### B. Field Mapping Rules
149+
* **`topics`**: (Only if `include_topics=true`) Use `dcidNameMappings` to resolve readable names.
150+
* **`variables`**: Individual data indicators. Use `dcidNameMappings` to resolve readable names.
151+
* **`placesWithData`**: Represents which of the sampled child places have data for that specific indicator.
152+
* **`dcidNameMappings`**: Use this to map returned DCIDs to human-readable names.
153+
* **`dcidPlaceTypeMappings`**: Maps place DCIDs to their types. Use this to determine the `child_place_type` parameter when calling `get_child_observations`.
154+
155+
---
156+
157+
## 6. Processing `get_variable_metadata` Responses
158+
159+
Use this response to verify dataset coverage, date ranges, and sources for sampled child places before fetching full observation arrays.
160+
161+
### A. Response Structure Reference
162+
```json
163+
{
164+
"status": "SUCCESS",
165+
"variables": {
166+
"UnemploymentRate_Person": {
167+
"id": "UnemploymentRate_Person",
168+
"name": "Unemployment Rate",
169+
"facets": [
170+
{
171+
"id": "2176550201",
172+
"provenanceId": "dc/base/BLS_CP",
173+
"obsCount": 12,
174+
"dateRange": { "start": "2020", "end": "2024" },
175+
"scope": { "entityCoverage": ["geoId/06037"] }
176+
}
177+
]
178+
}
179+
},
180+
"provenances": {
181+
"dc/base/BLS_CP": {
182+
"id": "dc/base/BLS_CP",
183+
"properties": {
184+
"source": "Bureau of Labor Statistics",
185+
"url": "https://www.bls.gov"
186+
}
187+
}
188+
}
189+
}
190+
```
191+
192+
### B. Field Mapping Rules
193+
* **`variables`**: Contains metadata per variable DCID. Inspect `facets` to confirm temporal range (`dateRange`) and validity across the sampled child places.
194+
* **`provenances`**: Maps provenance IDs to authoritative source details (`source`, `url`). Use this for mandatory data attribution.
195+
196+
---
197+
198+
## 7. Child Place Type Determination Heuristics
199+
200+
Before calling `get_child_observations`, inspect the `dcidPlaceTypeMappings` returned by `search_child_indicators` to determine the value for the `child_place_type` parameter:
201+
1. **Common Type**: Find the place type common to ALL sampled child places.
202+
2. **Specific Type Priority**: If multiple types are common to all child places, choose the most specific type (e.g., prefer `"County"` over `"AdministrativeArea2"`).
203+
3. **Majority Fallback**: If no single type is common to all, use the type that maps to a clear majority (50%+ threshold) of the sample.
204+
4. **Resolution Failure**: If there is no common type and no majority type, child-place mode is not supported. Fall back to making individual `get_observations` calls in single-place mode for each child place using the base skill resource at 'skill://data-commons-researcher/SKILL.md'.
205+
206+
---
207+
208+
## 8. Bounded Date Query & Date Filtering Rules
209+
210+
To prevent payload saturation and context window exhaustion when fetching time-series observations:
211+
212+
### A. Child Places Mode Constraint
213+
* When calling `get_child_observations` with `child_place_type` active, **never** set `date="all"`.
214+
* **Safe Date Strategies**:
215+
* Set `date="latest"` to retrieve only the most recent data point for each child place.
216+
* Explicitly define a narrow window using `date_range_start` and `date_range_end` (e.g., `2020` to `2023`).
217+
218+
### B. Date Range Boundary Interpretations
219+
When `date="range"` is used, the date ranges are evaluated as follows:
220+
* **Start Date Only**: If only `date_range_start` is specified, the response will contain all observations starting at and after that date (inclusive).
221+
* **End Date Only**: If only `date_range_end` is specified, the response will contain all observations before and up to that date (inclusive).
222+
* **Both Boundaries**: If both are specified, the response contains observations within the provided range (inclusive).
223+
* **Default Fallback**: If you do not provide any date parameters (`date`, `date_range_start`, or `date_range_end`), the tool will automatically fetch only the `'latest'` observation.
224+
225+
---
226+
227+
## 9. Processing `get_child_observations` Responses
228+
229+
### A. Response Structure Reference
230+
```json
231+
{
232+
"variable": { "dcid": "UnemploymentRate_Person", "name": "Unemployment Rate" },
233+
"resolvedParentPlace": { "dcid": "geoId/06", "name": "California", "typeOf": ["State"] },
234+
"childPlaceType": "County",
235+
"placeObservations": [
236+
{
237+
"place": { "dcid": "geoId/06037", "name": "Los Angeles County", "typeOf": ["County"] },
238+
"timeSeries": [{ "date": "2024", "value": 5.4 }]
239+
}
240+
],
241+
"sourceMetadata": { "sourceId": "2176550201", "provenanceUrl": "https://www.bls.gov" },
242+
"alternativeSources": []
243+
}
244+
```
245+
246+
### B. Field Mapping Rules
247+
* **`variable`**: Details about the statistical variable requested.
248+
* **`resolvedParentPlace`**: Structural verification of how the parent place parameter was resolved.
249+
* **`childPlaceType`**: Confirms the administrative type applied to all child observations.
250+
* **`placeObservations`**: A list of observations, one entry per child place. Each entry contains:
251+
* `place`: Details about the observed child place (DCID, name, type).
252+
* `timeSeries`: A list of `(date, value)` objects.
253+
* **`sourceMetadata`**: Primary authoritative data source information.
254+
* **`alternativeSources`**: Secondary available sources.

0 commit comments

Comments
 (0)