|
| 1 | +# Lightspeed Core Stack Development Guide |
| 2 | + |
| 3 | +## Project Overview |
| 4 | +Lightspeed Core Stack (LCS) is an AI-powered assistant built on FastAPI that provides answers using LLM services, agents, and RAG databases. It integrates with Llama Stack for AI operations. |
| 5 | + |
| 6 | +## Development Environment |
| 7 | +- **Python**: Check `pyproject.toml` for supported Python versions |
| 8 | +- **Package Manager**: uv (use `uv run` for all commands) |
| 9 | +- **Required Commands**: |
| 10 | + - `uv run make format` - Format code (black + ruff) |
| 11 | + - `uv run make verify` - Run all linters (black, pylint, pyright, ruff, docstyle, check-types) |
| 12 | + |
| 13 | +## Code Architecture & Patterns |
| 14 | + |
| 15 | +### Project Structure |
| 16 | +``` |
| 17 | +src/ |
| 18 | +├── app/ # FastAPI application |
| 19 | +│ ├── endpoints/ # REST API endpoints |
| 20 | +│ └── main.py # Application entry point |
| 21 | +├── a2a_storage/ # A2A protocol persistent storage |
| 22 | +│ ├── context_store.py # Abstract base class for context stores |
| 23 | +│ ├── in_memory_context_store.py # In-memory implementation |
| 24 | +│ ├── sqlite_context_store.py # SQLite implementation |
| 25 | +│ ├── postgres_context_store.py # PostgreSQL implementation |
| 26 | +│ └── storage_factory.py # Factory for creating stores |
| 27 | +├── authentication/ # Authentication modules (k8s, jwk, noop, rh-identity) |
| 28 | +├── authorization/ # Authorization middleware & resolvers |
| 29 | +├── cache/ # Conversation cache implementations |
| 30 | +├── quota/ # Quota limiter and token usage tracking |
| 31 | +├── metrics/ # Prometheus metrics |
| 32 | +├── runners/ # Agent runners |
| 33 | +├── models/ # Pydantic models |
| 34 | +│ ├── config.py # Configuration classes |
| 35 | +│ ├── requests.py # Request models |
| 36 | +│ ├── responses.py # Response models |
| 37 | +│ └── database/ # Database models |
| 38 | +├── utils/ # Utility functions |
| 39 | +├── client.py # Llama Stack client wrapper (Singleton) |
| 40 | +├── configuration.py # Config management (Singleton) |
| 41 | +└── constants.py # Shared constants |
| 42 | +``` |
| 43 | + |
| 44 | +### Coding Standards |
| 45 | + |
| 46 | +#### Imports & Dependencies |
| 47 | +- Use absolute imports for internal modules: `from authentication import get_auth_dependency` |
| 48 | +- FastAPI dependencies: `from fastapi import APIRouter, HTTPException, Request, status, Depends` |
| 49 | +- Llama Stack imports: `from llama_stack_client import AsyncLlamaStackClient` |
| 50 | +- **ALWAYS** check `pyproject.toml` for existing dependencies before adding new ones |
| 51 | +- **ALWAYS** verify current library versions in `pyproject.toml` rather than assuming versions |
| 52 | +- Check `constants.py` for shared constants before defining new ones |
| 53 | + |
| 54 | +#### Module Standards |
| 55 | +- All modules start with descriptive docstrings explaining purpose |
| 56 | +- Use `logger = get_logger(__name__)` from `log.py` for module logging |
| 57 | +- Package `__init__.py` files contain brief package descriptions |
| 58 | +- Central `constants.py` for shared constants with descriptive comments |
| 59 | +- Type aliases defined at module level for clarity |
| 60 | + |
| 61 | +#### Configuration |
| 62 | +- All config uses Pydantic models extending `ConfigurationBase` |
| 63 | +- Base class sets `extra="forbid"` to reject unknown fields |
| 64 | +- Use `@field_validator` and `@model_validator` for custom validation |
| 65 | +- Type hints: `Optional[FilePath]`, `PositiveInt`, `SecretStr` |
| 66 | + |
| 67 | +#### Function Standards |
| 68 | +- **Documentation**: All functions require docstrings with brief descriptions |
| 69 | +- **Type Annotations**: Complete type annotations for parameters and return types |
| 70 | + - Use `typing_extensions.Self` for model validators |
| 71 | + - Union types: `str | int` (modern syntax) |
| 72 | + - Optional: `Optional[Type]` |
| 73 | +- **Naming**: Use snake_case with descriptive, action-oriented names (get_, validate_, check_) |
| 74 | +- **Return Values**: **CRITICAL** - Avoid in-place parameter modification anti-patterns: |
| 75 | + ```python |
| 76 | + # ❌ BAD: Modifying parameter in-place |
| 77 | + def process_data(input_data: Any, result_dict: dict) -> None: |
| 78 | + result_dict[key] = value # Anti-pattern |
| 79 | + |
| 80 | + # ✅ GOOD: Return new data structure |
| 81 | + def process_data(input_data: Any) -> dict: |
| 82 | + result_dict = {} |
| 83 | + result_dict[key] = value |
| 84 | + return result_dict |
| 85 | + ``` |
| 86 | +- **Async Functions**: Use `async def` for I/O operations and external API calls |
| 87 | +- **Error Handling**: |
| 88 | + - Use FastAPI `HTTPException` with appropriate status codes for API endpoints |
| 89 | + - Handle `APIConnectionError` from Llama Stack |
| 90 | + |
| 91 | +#### Logging Standards |
| 92 | +- Use `from log import get_logger` and module logger pattern: `logger = get_logger(__name__)` |
| 93 | +- Standard log levels with clear purposes: |
| 94 | + - `logger.debug()` - Detailed diagnostic information |
| 95 | + - `logger.info()` - General information about program execution |
| 96 | + - `logger.warning()` - Something unexpected happened or potential problems |
| 97 | + - `logger.error()` - Serious problems that prevented function execution |
| 98 | + |
| 99 | +#### Class Standards |
| 100 | +- **Documentation**: All classes require descriptive docstrings explaining purpose |
| 101 | +- **Naming**: Use PascalCase with descriptive names and standard suffixes: |
| 102 | + - `Configuration` for config classes |
| 103 | + - `Error`/`Exception` for custom exceptions |
| 104 | + - `Resolver` for strategy pattern implementations |
| 105 | + - `Interface` for abstract base classes |
| 106 | +- **Pydantic Models**: Extend `ConfigurationBase` for config, `BaseModel` for data models |
| 107 | +- **Abstract Classes**: Use ABC for interfaces with `@abstractmethod` decorators |
| 108 | +- **Validation**: Use `@model_validator` and `@field_validator` for Pydantic models |
| 109 | +- **Type Hints**: Complete type annotations for all class attributes |
| 110 | + |
| 111 | +#### Docstring Standards |
| 112 | +- Follow Google Python docstring conventions: https://google.github.io/styleguide/pyguide.html |
| 113 | +- Required for all modules, classes, and functions |
| 114 | +- Include brief description and detailed sections as needed: |
| 115 | + - `Parameters:` for function parameters |
| 116 | + - `Returns:` for return values |
| 117 | + - `Raises:` for exceptions that may be raised |
| 118 | + - `Attributes:` for class attributes (Pydantic models) |
| 119 | + |
| 120 | + |
| 121 | +## Testing Framework |
| 122 | + |
| 123 | +### Test Structure |
| 124 | +``` |
| 125 | +tests/ |
| 126 | +├── unit/ # Unit tests (pytest) |
| 127 | +├── integration/ # Integration tests (pytest) |
| 128 | +└── e2e/ # End-to-end tests (behave) |
| 129 | + └── features/ # Gherkin feature files |
| 130 | +``` |
| 131 | + |
| 132 | +### Testing Framework Requirements |
| 133 | +- **Required**: Use pytest for all unit and integration tests |
| 134 | +- **Forbidden**: Do not use unittest - pytest is the standard for this project |
| 135 | +- **E2E Tests**: Use behave (BDD) framework for end-to-end testing |
| 136 | + |
| 137 | +### Unit Tests (pytest) |
| 138 | +- **Fixtures**: Use `conftest.py` for shared fixtures |
| 139 | +- **Mocking**: `pytest-mock` for AsyncMock objects |
| 140 | +- **Common Pattern**: |
| 141 | + ```python |
| 142 | + @pytest.fixture(name="prepare_agent_mocks") |
| 143 | + def prepare_agent_mocks_fixture(mocker): |
| 144 | + mock_client = mocker.AsyncMock() |
| 145 | + mock_agent = mocker.AsyncMock() |
| 146 | + mock_agent._agent_id = "test_agent_id" |
| 147 | + return mock_client, mock_agent |
| 148 | + ``` |
| 149 | +- **Auth Mock**: `MOCK_AUTH = ("mock_user_id", "mock_username", False, "mock_token")` |
| 150 | +- **Coverage**: Unit tests require 60% coverage, integration 10% |
| 151 | + |
| 152 | +### E2E Tests (behave) |
| 153 | +- **Framework**: Behave (BDD) with Gherkin feature files |
| 154 | +- **Step Definitions**: In `tests/e2e/features/steps/` |
| 155 | +- **Common Steps**: Service status, authentication, HTTP requests |
| 156 | +- **Test List**: Maintained in `tests/e2e/test_list.txt` |
| 157 | + |
| 158 | +### Test Commands |
| 159 | +```bash |
| 160 | +uv run make test-unit # Unit tests with coverage |
| 161 | +uv run make test-integration # Integration tests |
| 162 | +uv run make test-e2e # End-to-end tests |
| 163 | +``` |
| 164 | + |
| 165 | +## Quality Assurance |
| 166 | + |
| 167 | +### Required Before Completion |
| 168 | +1. `uv run make format` - Auto-format code |
| 169 | +2. `uv run make verify` - Run all linters |
| 170 | +3. Create unit tests for new code |
| 171 | +4. Ensure tests pass |
| 172 | + |
| 173 | +### Linting Tools |
| 174 | +- **black**: Code formatting |
| 175 | +- **pylint**: Static analysis (`source-roots = "src"`) |
| 176 | +- **pyright**: Type checking (excludes `src/authentication/k8s.py`) |
| 177 | +- **ruff**: Fast linter |
| 178 | +- **pydocstyle**: Docstring style |
| 179 | +- **mypy**: Additional type checking |
| 180 | + |
| 181 | +### Security |
| 182 | +- **bandit**: Security issue detection |
| 183 | +- Never commit secrets/keys |
| 184 | +- Use environment variables for sensitive data |
| 185 | + |
| 186 | +## Key Dependencies |
| 187 | +**IMPORTANT**: Always check `pyproject.toml` for current versions rather than relying on this list: |
| 188 | +- **FastAPI**: Web framework |
| 189 | +- **Llama Stack**: AI integration |
| 190 | +- **Pydantic**: Data validation/serialization |
| 191 | +- **SQLAlchemy**: Database ORM |
| 192 | +- **Kubernetes**: K8s auth integration |
| 193 | + |
| 194 | +## Development Workflow |
| 195 | +1. Use `uv sync --group dev --group llslibdev` for dependencies |
| 196 | +2. Always use `uv run` prefix for commands |
| 197 | +3. **ALWAYS** check `pyproject.toml` for existing dependencies and versions before adding new ones |
| 198 | +4. Follow existing code patterns in the module you're modifying |
| 199 | +5. Write unit tests covering new functionality |
| 200 | +6. Run format and verify before completion |
0 commit comments