Skip to content

README.md

Latest commit

 

History

History
286 lines (209 loc) · 11.2 KB

README.md

File metadata and controls

286 lines (209 loc) · 11.2 KB

OpenEcon Data Documentation

OpenEcon Data is a one-stop MCP + API layer for economic data across 10+ providers. This index provides quick access to setup, MCP integration, and technical references.

Quick Links

Table of Contents

  1. User Guides
  2. Data Providers
  3. API Reference
  4. Development
  5. Deployment
  6. Architecture
  7. MCP Quick Start
  8. Troubleshooting

User Guides

Guides to help you get started and use OpenEcon Data effectively.

Guide Description
Getting Started First-time setup and basic usage
Cross-Platform Setup Platform-specific installation (Linux, macOS, Windows)
Testing Guide How to run and write tests
Complex Query Testing Testing multi-provider and complex queries

Data Providers

OpenEcon Data integrates with 10+ economic data providers. Each provider has specific capabilities and data coverage.

Provider Documentation

Provider Description Documentation
FRED Federal Reserve Economic Data (US) API Reference
World Bank Global development indicators API Quick Reference
UN Comtrade International trade flows Trade Data Guide
Statistics Canada Canadian economic data Categorical Data
IMF International financial statistics Regional Queries
BIS Bank for International Settlements Provider Fixes
Eurostat European Union statistics Complete Guide
OECD OECD member countries data Dynamic Discovery
ExchangeRate-API Currency exchange rates Quick Reference
CoinGecko Cryptocurrency prices Quick Reference

Provider Technical Reference

Document Description
SDMX API Research SDMX protocol for IMF, BIS, Eurostat, OECD
Trade Data Reference UN Comtrade usage and HS codes
Eurostat Research Eurostat API technical details

API Reference

Full endpoint documentation: API Reference -- request/response schemas, authentication, conversation flow, rate limiting, circuit breaker behavior.

Core Endpoints:

Endpoint Method Auth Description
/api/health GET No Health check with service status
/api/query POST Optional Process natural language query
/api/query/stream POST Optional Streaming query (SSE) with real-time progress
/api/query/pro POST Optional Pro Mode (AI-generated code execution)
/api/query/pro/stream POST Optional Pro Mode with streaming
/api/export POST No Export data as CSV/JSON/Stata
/api/feedback POST No Submit user feedback
/api/auth/register POST No User registration
/api/auth/login POST No User login
/api/auth/me GET Yes Current user profile
/api/user/history GET Yes Query history
/api/user/history DELETE Yes Clear query history
/api/session/history GET No Anonymous session history
/api/cache/stats GET Yes Cache statistics
/api/cache/clear POST Yes Clear cache
/api/performance/metrics GET No Performance metrics
/api/performance/status GET No System status

MCP Server

OpenEcon Data exposes a Model Context Protocol (MCP) server for AI assistants:

MCP Quick Start

Use one of these endpoints:

  • Hosted: https://data.openecon.ai/mcp
  • Local: http://localhost:3001/mcp

Add to Codex:

codex mcp add openecon-data --url https://data.openecon.ai/mcp
codex mcp get openecon-data

Add to Claude Code:

claude mcp add --transport sse openecon-data https://data.openecon.ai/mcp --scope user
claude mcp get openecon-data

Example prompt:

  • Use query_data to compare US and Canada GDP growth since 2015.

Development

Guides for developers contributing to OpenEcon Data.

Architecture & Design

Document Description
Developer & Contributor Guide Technical setup and contribution workflow
LLM Abstraction LLM provider abstraction layer
Metadata System RAG-based metadata search
FAISS vs ChromaDB Vector search architecture decision
Indicator Resolution Indicator resolution pipeline (current)
Routing Improvements Query routing logic (superseded by UnifiedRouter)
Prompt Architecture LLM prompt design (superseded by UnifiedRouter)

Performance & Optimization

Document Description
FAISS Performance Vector search optimization
FAISS Deployment Production deployment notes
Accuracy Improvements Data accuracy analysis

Agent & AI Integration

Document Description
Agent Instructions AI agent integration guide
LLM Improvements LLM system enhancements

Provider Development

Document Description
Provider Analysis Provider implementation analysis
StatsCan Improvements Statistics Canada 95% accuracy report
Default Time Periods Time period handling

Deployment

Guides for deploying OpenEcon Data to production.

Document Description
Deployment Summary Production deployment overview
Apache Pro Mode Setup Apache2 configuration for Pro Mode

Environment Configuration

See the main CLAUDE.md file for:

  • Required environment variables
  • Production deployment checklist
  • Apache2 configuration details
  • Backend/frontend server management

Architecture

System Overview

OpenEcon Data consists of:

  1. Backend (Python/FastAPI) - API server, LLM integration, data providers
  2. Frontend (React/TypeScript) - Chat interface, data visualization
  3. Supabase - Authentication and query history storage

Data Flow

User Query → LLM Parser → UnifiedRouter → Indicator Selector → Data Provider → Normalizer → Response
                ↓                                                                    ↓
     Conversation Context (Redis)                                          Intent Cache (Redis)

LLM-based routing replaced the old deterministic ProviderRouter and keyword_matcher.py (Phases 1-4 of routing consolidation). The LLM prompt includes a provider capability matrix, and UnifiedRouter makes the final routing decision. Repeat queries hit an intent cache (~72x speedup).

Key Components

Component Location Description
Query Service backend/services/query.py Main orchestration layer
UnifiedRouter backend/routing/unified_router.py LLM-assisted provider routing
OpenRouter Service backend/services/openrouter.py LLM integration
Indicator Selector backend/services/indicator_selector.py Catalog + embedding + LLM indicator resolution
Providers backend/providers/ Data source integrations (FRED, WorldBank, IMF, etc.)
Conversation Manager backend/services/conversation.py Multi-round context with Redis persistence
Cache backend/services/cache.py, redis_cache.py In-memory + Redis distributed caching

Troubleshooting

Common Issues

Query returns no data:

  1. Check if the indicator exists in the provider
  2. Verify date range is valid
  3. Check provider-specific limitations

Authentication errors:

  1. Verify Supabase credentials in .env
  2. Check token expiration
  3. Clear browser localStorage and retry

Provider-specific issues:

Debug Logs

# Backend logs
tail -f /tmp/backend-dev.log

# Check health endpoint
curl http://localhost:3001/api/health

Recent Improvements

Document Description
Provider Testing Summary Latest test results
Eurostat Fixes Eurostat provider improvements
OECD Improvements OECD provider enhancements
IMF Provider Improvements IMF data quality fixes
OECD Rate Limit Fix Rate limiting implementation

Archive

Historical documentation and development logs are available in the archive/ directory.

Contributing

  1. Read the Getting Started guide
  2. Review the Security Policy
  3. Follow coding standards in CLAUDE.md
  4. Submit pull requests to the main branch

Need Help?