Skip to content

Commit 59d9edf

Browse files
paco-valdezpaco-valdez
authored
Merge pull request #13 from cube-js/add-python-api
Add python API
2 parents f863092 + 31e7382 commit 59d9edf

23 files changed

Lines changed: 3810 additions & 42 deletions

‎.pydoc-markdown.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
2+
loaders:
3+
- type: python
4+
search_path: [lkml2cube]
5+
renderer:
6+
type: markdown
7+
render_toc: true
8+
render_module_header: true
9+

‎CLAUDE.md‎

Lines changed: 222 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,31 @@
22

33
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
44

5+
> **🚨 IMPORTANT**: Always check `docs/lkml2cube_*.md` files BEFORE reading source code. Update docstrings and run `python scripts/generate_docs.py` after any code changes.
6+
57
## Project Overview
68

79
lkml2cube is a Python CLI tool that converts LookML models into Cube data models. It uses the `lkml` library to parse LookML files and generates YAML-based Cube definitions.
810

11+
## 📚 Documentation-First Development
12+
13+
**CRITICAL**: Before reading source code, always consult the generated documentation:
14+
15+
1. **Primary Reference**: Check `docs/lkml2cube_*.md` files first
16+
2. **Implementation Details**: Read source code only if docs are insufficient
17+
3. **Always Update**: Maintain docstrings and regenerate docs for any changes
18+
19+
📖 **Documentation Files**:
20+
- `docs/lkml2cube_main.md` - CLI commands and usage
21+
- `docs/lkml2cube_converter.md` - LookMLConverter Python API
22+
- `docs/lkml2cube_parser_cube_api.md` - Module to interact with Cube meta API
23+
- `docs/lkml2cube_parser_loader.md` - File loading and writing utilities
24+
- `docs/lkml2cube_parser_explores.md` - Module to convert LookML explores to Cube `view` definitions
25+
- `docs/lkml2cube_parser_types.md` - Custom YAML types for proper formatting
26+
- `docs/lkml2cube_parser_views.md` - Module to convert LookML views to Cube `cube` definitions
27+
28+
⚠️ **Required Workflow**: When modifying code → Update docstrings → Run `python scripts/generate_docs.py`
29+
930
## Development Commands
1031

1132
### Environment Setup
@@ -15,9 +36,17 @@ lkml2cube is a Python CLI tool that converts LookML models into Cube data models
1536

1637
### Testing
1738
- Tests are located in `tests/` directory
18-
- Main test file: `tests/test_e2e.py`
39+
- Main test files: `tests/test_e2e.py`, `tests/test_converter.py`, `tests/test_explores_command.py`
1940
- Test samples are in `tests/samples/` with both `lkml/` and `cubeml/` subdirectories
2041
- Tests compare generated output against expected YAML files
42+
- `test_converter.py` provides comprehensive unit tests for the `LookMLConverter` class
43+
44+
### Documentation Generation
45+
- **Generate docs**: `python scripts/generate_docs.py`
46+
- **MANDATORY**: Run after any function/method changes
47+
- **Output**: Updates `docs/lkml2cube_*.md` files
48+
- **Fallback**: Uses manual generation if pydoc-markdown fails
49+
- **Optimization**: Optimizes output for LLM consumption
2150

2251
### CLI Usage
2352
The tool provides three main commands:
@@ -36,8 +65,9 @@ The tool provides three main commands:
3665
- `cube_api.py` - Interfaces with Cube meta API, correctly separates cubes vs views
3766
- `types.py` - Custom YAML types for proper formatting
3867

39-
#### Main Entry Point
68+
#### Main Entry Points
4069
- `main.py` - Typer-based CLI with three commands: cubes, views, explores
70+
- `converter.py` - Python API class `LookMLConverter` for programmatic usage
4171
- Uses Rich for console output formatting
4272

4373
### Key Concepts
@@ -67,4 +97,193 @@ Common options across commands:
6797
- `--parseonly` - Shows parsed LookML as Python dict
6898
- `--printonly` - Prints generated YAML to stdout
6999
- `--outputdir` - Directory for output files
70-
- `--rootdir` - Base path for resolving includes
100+
- `--rootdir` - Base path for resolving includes
101+
102+
## Python API Usage
103+
104+
### LookMLConverter Class
105+
The `LookMLConverter` class provides a Python API for programmatic usage without requiring CLI interaction:
106+
107+
```python
108+
from lkml2cube.converter import LookMLConverter
109+
110+
# Initialize with configuration
111+
converter = LookMLConverter(
112+
outputdir="/tmp/output",
113+
rootdir="/lookml/models",
114+
parseonly=False,
115+
printonly=False,
116+
use_explores_name=False
117+
)
118+
119+
# Convert LookML views to Cube definitions
120+
result = converter.cubes("models/orders.lkml")
121+
122+
# Convert LookML explores to Cube definitions with views
123+
result = converter.views("models/explores.lkml")
124+
125+
# Generate LookML explores from Cube meta API
126+
result = converter.explores("https://api.cube.dev/v1/meta", "jwt-token")
127+
```
128+
129+
#### Key Methods
130+
- `cubes(file_path)` - Equivalent to `lkml2cube cubes` command
131+
- `views(file_path)` - Equivalent to `lkml2cube views` command
132+
- `explores(metaurl, token)` - Equivalent to `lkml2cube explores` command
133+
- `set_config(**kwargs)` - Update configuration options
134+
- `get_config()` - Get current configuration
135+
- `validate_files(file_paths)` - Validate that files can be loaded
136+
137+
#### Configuration Management
138+
The converter maintains state and can be reconfigured:
139+
140+
```python
141+
# Update configuration
142+
converter.set_config(parseonly=True, outputdir="/new/path")
143+
144+
# Get current configuration
145+
config = converter.get_config()
146+
```
147+
148+
#### Return Values
149+
All methods return a dictionary with relevant data:
150+
- `parseonly=True`: Returns parsed model structure
151+
- `printonly=True`: Returns YAML output string
152+
- Default: Returns file generation summary
153+
154+
## Documentation and Code Guidelines
155+
156+
### Documentation-First Approach
157+
158+
**IMPORTANT**: Always refer to the generated documentation in the `docs/` directory before reading source code:
159+
160+
1. **First**: Check `docs/lkml2cube_*.md` files for API documentation
161+
2. **Second**: If implementation details are needed, then read the source code
162+
3. **Always**: Maintain and update documentation when making changes
163+
164+
### Documentation Files
165+
166+
The project maintains auto-generated documentation:
167+
- `docs/lkml2cube_main.md` - CLI commands and usage
168+
- `docs/lkml2cube_converter.md` - LookMLConverter Python API
169+
- `docs/lkml2cube_parser_cube_api.md` - Module to interact with Cube meta API
170+
- `docs/lkml2cube_parser_loader.md` - File loading and writing utilities
171+
- `docs/lkml2cube_parser_explores.md` - Module to convert LookML explores to Cube `view` definitions
172+
- `docs/lkml2cube_parser_types.md` - Custom YAML types for proper formatting
173+
- `docs/lkml2cube_parser_views.md` - Module to convert LookML views to Cube `cube` definitions
174+
175+
### Documentation Maintenance Workflow
176+
177+
**MANDATORY**: When adding new functions or modifying existing ones:
178+
179+
1. **Add/Update Google-style Docstrings**:
180+
```python
181+
def my_function(param1: str, param2: int = 5) -> dict:
182+
"""Brief one-line description.
183+
184+
Detailed description if needed.
185+
186+
Args:
187+
param1 (str): Description of param1.
188+
param2 (int, optional): Description of param2. Defaults to 5.
189+
190+
Returns:
191+
dict: Description of return value.
192+
193+
Raises:
194+
ValueError: Description of when this is raised.
195+
196+
Example:
197+
>>> result = my_function("test", 10)
198+
>>> print(result['key'])
199+
'value'
200+
"""
201+
```
202+
203+
2. **Run Documentation Generation**:
204+
```bash
205+
python scripts/generate_docs.py
206+
```
207+
208+
3. **Verify Documentation**:
209+
- Check that `docs/` files are updated
210+
- Ensure docstrings are properly formatted
211+
- Verify examples work correctly
212+
213+
### Google-style Docstring Requirements
214+
215+
All public functions, methods, and classes MUST have Google-style docstrings including:
216+
- **Clear one-line description**
217+
- **Complete parameter documentation with types**
218+
- **Return value descriptions**
219+
- **Raised exceptions**
220+
- **Simple usage examples**
221+
222+
### Documentation Generation Script
223+
224+
The `scripts/generate_docs.py` script:
225+
- Automatically extracts docstrings from source code
226+
- Generates markdown files in `docs/` directory
227+
- Uses pydoc-markdown with manual fallback
228+
- Optimizes output for LLM consumption
229+
- Must be run after any function signature changes
230+
231+
### When to Update Documentation
232+
233+
Run `python scripts/generate_docs.py` when:
234+
- Adding new functions or methods
235+
- Modifying function signatures
236+
- Changing parameter types or defaults
237+
- Adding or removing classes
238+
- Updating docstrings for clarity
239+
240+
### Code Review Checklist
241+
242+
Before committing changes:
243+
- [ ] All new functions have Google-style docstrings
244+
- [ ] Documentation generation script has been run
245+
- [ ] Generated docs reflect the changes
246+
- [ ] Examples in docstrings are accurate
247+
- [ ] Parameter types and descriptions are correct
248+
249+
## 🔒 Enforcement Rules
250+
251+
**MANDATORY REQUIREMENTS**:
252+
253+
1. **Documentation First**: NEVER read source code without first checking the generated documentation in `docs/`
254+
2. **Google-style Docstrings**: ALL public functions, methods, and classes MUST have complete Google-style docstrings
255+
3. **Documentation Generation**: ALWAYS run `python scripts/generate_docs.py` after any code changes
256+
4. **No Exceptions**: These rules apply to ALL code changes, no matter how small
257+
258+
**VIOLATION CONSEQUENCES**:
259+
- Code changes without proper docstrings will be rejected
260+
- Failure to generate documentation will result in incomplete assistance
261+
- Not following documentation-first approach will lead to suboptimal code understanding
262+
263+
**COMPLIANCE VERIFICATION**:
264+
- Check that `docs/` files are updated after changes
265+
- Verify docstrings follow Google-style format exactly
266+
- Ensure examples in docstrings are working and accurate
267+
- Confirm all parameters and return values are documented
268+
269+
---
270+
271+
## 📝 Quick Reference
272+
273+
**Documentation Workflow**:
274+
1. 📖 Check `docs/lkml2cube_*.md` first
275+
2. 📝 Add/update Google-style docstrings
276+
3. 🔄 Run `python scripts/generate_docs.py`
277+
4. ✅ Verify documentation is updated
278+
279+
**Key Files**:
280+
- `docs/lkml2cube_main.md` - CLI documentation
281+
- `docs/lkml2cube_converter.md` - Python API documentation
282+
- `docs/lkml2cube_parser_cube_api.md` - Module to interact with Cube meta API
283+
- `docs/lkml2cube_parser_loader.md` - File loading and writing utilities
284+
- `docs/lkml2cube_parser_explores.md` - Module to convert LookML explores to Cube `view` definitions
285+
- `docs/lkml2cube_parser_types.md` - Custom YAML types for proper formatting
286+
- `docs/lkml2cube_parser_views.md` - Module to convert LookML views to Cube `cube` definitions
287+
- `scripts/generate_docs.py` - Documentation generation script
288+
289+
**Remember**: Documentation first, code second. Always maintain docstrings!

0 commit comments

Comments
 (0)