Skip to content

AGENTS.md

Latest commit

 

History

History
373 lines (276 loc) · 14.1 KB

AGENTS.md

File metadata and controls

373 lines (276 loc) · 14.1 KB

Agent Instructions for python-openai-demos

This document provides comprehensive instructions for coding agents working on this repository. Following these instructions will help you work more efficiently and avoid common pitfalls.

Overview

This repository contains a collection of Python scripts that demonstrate how to use the OpenAI Responses API (and compatible APIs like Azure OpenAI and Ollama). The repository includes examples of:

  • Basic responses (streaming, async, history)
  • Function calling (basic to advanced multi-function scenarios)
  • Structured outputs using Pydantic models
  • Retrieval-Augmented Generation (RAG) with various complexity levels
  • Prompt engineering and safety features

The scripts are designed to be educational and can run with multiple LLM providers: Azure OpenAI (preferred), OpenAI.com, or local Ollama models.

Code Layout

Python Scripts (Root Directory)

All example scripts are located in the root directory. They follow a consistent pattern of setting up an OpenAI client based on environment variables, then demonstrating specific API features.

Chat Scripts:

  • chat.py - Simple response example
  • chat_stream.py - Streaming responses
  • chat_async.py - Async responses with asyncio.gather examples
  • chat_history.py - Multi-turn chat with message history
  • chat_history_stream.py - Multi-turn chat with streaming
  • chat_safety.py - Content safety filter exception handling

Function Calling Scripts:

  • function_calling_basic.py - Single function declaration, prints tool calls (no execution)
  • function_calling_call.py - Executes the function once if the model requests it
  • function_calling_extended.py - Full round-trip: executes, returns tool output, gets final answer
  • function_calling_errors.py - Same as extended but with robust error handling (malformed JSON args, missing tool, tool exceptions, JSON serialization)
  • function_calling_parallel.py - Shows model requesting multiple tools in one response
  • function_calling_while_loop.py - Conversation loop that keeps executing sequential tool calls until the model produces a final natural language answer (with error handling)

Structured Outputs Scripts:

  • structured_outputs_basic.py - Basic Pydantic model extraction
  • structured_outputs_description.py - Using field descriptions
  • structured_outputs_enum.py - Restricting values with enums
  • structured_outputs_function_calling.py - Combining function calling with Pydantic
  • structured_outputs_nested.py - Nested Pydantic models

RAG Scripts (require requirements-rag.txt):

  • rag_csv.py - RAG with CSV file retrieval
  • rag_multiturn.py - RAG with multi-turn chat interface
  • rag_queryrewrite.py - RAG with query rewriting step
  • rag_documents_ingestion.py - PDF ingestion (uses pymupdf, langchain, creates JSON)
  • rag_documents_flow.py - RAG flow using ingested documents
  • rag_documents_hybrid.py - Hybrid retrieval with vector + keyword search + RRF + reranking

Other Scripts:

  • prompt_engineering.py - Prompt engineering examples
  • reasoning.py - Reasoning examples
  • chained_calls.py - Chained API calls
  • retrieval_augmented_generation.py - Alternative RAG example

Spanish Directory

The spanish/ directory contains Spanish translations of most example scripts and a Spanish README. The Python scripts are functionally equivalent to their English counterparts.

Data Directory

The data/ directory contains PDF files used by the RAG document ingestion examples:

  • Aphideater_hoverfly.pdf
  • California_carpenter_bee.pdf
  • Centris_pallida.pdf
  • Western_honey_bee.pdf

Infrastructure Files (infra/)

Bicep Infrastructure as Code:

  • infra/main.bicep - Defines Azure OpenAI resource provisioning with GPT-4o and embedding models
  • infra/main.parameters.json - Parameters for Bicep deployment

Environment Setup Scripts:

  • infra/write_dot_env.sh - Shell script to create .env from azd environment (Linux/Mac)
  • infra/write_dot_env.ps1 - PowerShell script to create .env from azd environment (Windows)

These scripts are automatically run by azd provision via the azure.yaml postprovision hooks.

Configuration Files

Python Configuration:

  • pyproject.toml - Ruff and Black linter/formatter configuration (line-length: 120, target: py311)
  • requirements.txt - Core dependencies: azure-identity, openai>=1.108.1, python-dotenv, langchain-text-splitters
  • requirements-rag.txt - RAG-specific dependencies: pymupdf4llm, lunr, sentence-transformers
  • requirements-dev.txt - Development dependencies: includes requirements.txt + requirements-rag.txt + pre-commit, ruff, black

Pre-commit Configuration:

  • .pre-commit-config.yaml - Pre-commit hooks for check-yaml, end-of-file-fixer, trailing-whitespace, ruff, and black

Azure Developer CLI:

  • azure.yaml - Defines azd hooks (postprovision scripts)

Environment Variables:

  • .env.sample - Example .env file showing all possible configurations
  • .env.sample.azure - Azure-specific example
  • .env.sample.ollama - Ollama example
  • .env.sample.openai - OpenAI.com example

GitHub Workflows (.github/workflows/)

python.yaml - Linting and Formatting Check:

  • Runs on: push to main, pull requests to main
  • Uses: uv for virtual environment and dependency installation
  • Runs: uv run ruff check . and uv run black . --check --verbose
  • Important: The CI uses uv but local development typically uses standard pip

Dev Container Files (.devcontainer/)

  • .devcontainer/devcontainer.json - Default dev container (Azure OpenAI setup with azd)
  • .devcontainer/Dockerfile - Base Python 3.12 image, installs all requirements-dev.txt
  • .devcontainer/ollama/ - Ollama variant
  • .devcontainer/openai/ - OpenAI.com variant

All dev containers install all dependencies from requirements-dev.txt which includes base, RAG, and dev tools.

Running the Code

Environment Setup

ALWAYS follow these steps in order:

  1. Create a Python virtual environment (if not in a Dev Container or Codespaces):

    python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

  2. Install base dependencies:

    python -m pip install -r requirements.txt

  3. For RAG examples, ALSO install RAG dependencies:

    python -m pip install -r requirements-rag.txt

  4. For development (linting, formatting), install dev dependencies:

    python -m pip install -r requirements-dev.txt

    Note: requirements-dev.txt includes both requirements.txt and requirements-rag.txt via -r directives.

Configuring LLM Provider

The scripts read environment variables from a .env file. Create one based on your provider:

Option 1: Azure OpenAI (recommended)

For agents: Check if Azure OpenAI environment variables are already configured:

if [ -n "$AZURE_OPENAI_ENDPOINT" ] && [ -n "$AZURE_OPENAI_CHAT_DEPLOYMENT" ]; then
    echo "Azure OpenAI available - required environment variables are set"
else
    echo "Azure OpenAI not available - missing AZURE_OPENAI_ENDPOINT or AZURE_OPENAI_CHAT_DEPLOYMENT"
fi

If Azure OpenAI variables are not available, an administrator would need to provision Azure resources using:

azd auth login
azd provision

This creates real Azure resources that incur costs. The .env file would be created automatically with all needed variables after provisioning.

Option 2: OpenAI.com (requires API key and costs)

For agents: Check if OpenAI.com API key is available:

if [ -n "$OPENAI_API_KEY" ]; then
    echo "OpenAI.com available - OPENAI_API_KEY is set"
else
    echo "OpenAI.com not available - OPENAI_API_KEY not found"
fi

If OPENAI_API_KEY is available, ensure API_HOST=openai and OPENAI_MODEL are also set (e.g., gpt-4o-mini).

Option 3: Ollama (requires local Ollama installation)

For agents: Check if Ollama is installed and running:

if command -v ollama &> /dev/null; then
    echo "Ollama command found"
    ollama list
    if [ $? -eq 0 ]; then
        echo "Ollama is running and available"
    else
        echo "Ollama installed but not running"
    fi
else
    echo "Ollama not installed"
fi

If Ollama is available, configure the environment:

# API_HOST=ollama
# OLLAMA_ENDPOINT=http://localhost:11434/v1
# OLLAMA_MODEL=llama3.1

Important: If running in a Dev Container, use http://host.docker.internal:11434/v1 instead of localhost.

Running Scripts

After environment setup and .env configuration, run any script:

python chat.py
python function_calling_basic.py
python rag_csv.py  # Requires requirements-rag.txt to be installed

For Spanish versions:

python spanish/chat.py

Running Tests

Linting with Ruff

# After installing requirements-dev.txt
ruff check .

To auto-fix issues:

ruff check . --fix

Formatting with Black

Check formatting:

black . --check --verbose

Auto-format:

black .

Pre-commit Hooks

Install pre-commit hooks to automatically run checks before commits:

pre-commit install

Run manually:

pre-commit run --all-files

Integration Tests

The repository has limited automated testing via GitHub Actions. Changes to scripts should be manually verified by running them:

python chat.py
python spanish/chat.py

Note: Most scripts are demonstration scripts, not unit-tested. Changes to scripts should be manually verified by running them.

Important Notes and Gotchas

Environment Variables

  • All scripts default to API_HOST=azure if no .env file is present and no environment variable is set.
  • Scripts use load_dotenv(override=True) which means .env values override environment variables.

Model Compatibility

  • Function calling scripts require models that support tools. Not all models support this:
    • ✅ Supported: gpt-4o, gpt-4o-mini, gpt-5.4, and many others
    • ❌ Not supported: Older models, some local Ollama models
  • If a script fails with a function calling error, check if your model supports the tools parameter.

RAG Scripts Dependencies

  • RAG scripts require requirements-rag.txt to be installed. They will fail with import errors if these dependencies are missing.
  • The rag_documents_ingestion.py script creates rag_ingested_chunks.json (large file, ~6MB) which is used by rag_documents_flow.py and rag_documents_hybrid.py.
  • The ingestion process requires the PDF files in the data/ directory.

Dev Container Behavior

  • Dev containers automatically install all dependencies from requirements-dev.txt during build.
  • Azure-focused dev container includes azure-cli and azd features.
  • The default dev container expects Azure OpenAI configuration.

Linting and Formatting

  • Line length is 120 characters (configured in pyproject.toml).
  • Target Python version is 3.11 (though 3.12 is used in dev containers).
  • Ruff is configured to check: E (errors), F (pyflakes), I (isort), UP (pyupgrade).
  • Black and Ruff should both pass for CI to succeed.

CI/CD Workflow Differences

  • CI uses uv for faster installation: uv venv .venv && uv pip install -r requirements-dev.txt
  • Local development typically uses pip: python -m venv .venv && python -m pip install -r requirements-dev.txt
  • Both approaches work, but uv is faster for CI.

Azure Provisioning

  • Running azd provision creates real Azure resources that incur costs.
  • The infrastructure creates:
    • Azure OpenAI account with gpt-4o deployment (capacity: 30)
    • Text embedding deployment text-embedding-3-small (capacity: 30)
  • The postprovision hook automatically creates the .env file.
  • Always run azd down when done to avoid ongoing charges.

File Organization

  • Spanish translations are in spanish/ directory with their own README and scripts.
  • Spanish scripts are functionally identical to English versions, just translated.
  • Both English and Spanish directories have their own rag_ingested_chunks.json files.

Common Errors and Solutions

Error: ImportError: No module named 'pymupdf4llm'

  • Solution: Install RAG dependencies: python -m pip install -r requirements-rag.txt

Error: KeyError: 'AZURE_OPENAI_ENDPOINT'

  • Solution: Your .env file is missing required Azure variables, or API_HOST is set to azure but you haven't configured Azure. Run azd provision or configure Azure properly.

Error: openai.APIError: content_filter

  • This is expected behavior for chat_safety.py - it's demonstrating content filtering.
  • The script catches this error and prints a message.

Error: Function calling not supported

  • Solution: Use a model that supports tools, such as gpt-4o, gpt-4o-mini, or gpt-5.4.

Error: azd command not found

Agent Workflow Recommendations

When making code changes:

  1. Always install dependencies first: python -m pip install -r requirements-dev.txt
  2. Run linters before making changes to understand baseline: ruff check . and black . --check
  3. Make minimal, surgical changes to the relevant scripts.
  4. Run linters again after changes: ruff check . and black . (auto-fix)
  5. Manually test the changed script: 
    python your_modified_script.py
  6. Check that Spanish translations are updated if applicable.

Trust These Instructions

These instructions have been validated by:

  • Reviewing all documentation (README.md, CONTRIBUTING.md)
  • Examining all configuration files (pyproject.toml, .pre-commit-config.yaml, azure.yaml)
  • Analyzing all CI/CD workflows (.github/workflows/)
  • Inspecting all infrastructure files (infra/)
  • Testing environment setup and linting commands
  • Verifying dependency installation processes

If you encounter discrepancies between these instructions and the actual repository state, the repository may have been updated. In that case:

  1. Re-examine the relevant documentation or configuration files
  2. Update these instructions if needed
  3. Otherwise, trust these instructions and only explore further if they are incomplete or proven incorrect