-
Notifications
You must be signed in to change notification settings - Fork 11
Expand file tree
/
Copy pathmkdocs.yml
More file actions
174 lines (148 loc) · 5.63 KB
/
mkdocs.yml
File metadata and controls
174 lines (148 loc) · 5.63 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
site_name: convtools
theme:
name: material
# highlightjs: true
# hljs_languages:
# - python
custom_dir: docs/overrides
features:
- content.code.copy
- navigation.expand
extra:
analytics:
provider: google
property: G-Q41F2W9QLY
repo_name: westandskif/convtools
repo_url: https://github.com/westandskif/convtools
site_url: https://convtools.readthedocs.io/en/latest/
use_directory_urls: true
nav:
- Welcome: README.md
- Usage:
- Basics: basics.md
- Collections: collections.md
- Conditions and Pipes: conditions_n_pipes.md
- Aggregations: aggregations.md
- Joins: joins.md
- Dates: dates.md
- Mutations: mutations.md
- Exceptions: exceptions.md
- Window functions: window_funcs.md
- Contrib / Tables: contrib_tables.md
- Contrib / Fs: contrib_fs.md
- Benefits: benefits.md
- Changelog: changelog.md
- Breaking changes: breaking_changes.md
- Contributors: contributors.md
- License: license.md
markdown_extensions:
- toc:
permalink: true
- tables
- markdown_include.include:
base_path: docs
- pymdownx.blocks.tab:
alternate_style: true
- pymdownx.blocks.admonition:
- pymdownx.highlight:
use_pygments: true
- pymdownx.inlinehilite:
- pymdownx.superfences:
- pymdownx.snippets:
base_path: .
dedent_subsections: true
- attr_list
exclude_docs: |
examples-md/*.md
performance-md/*.md
watch:
- docs
- src/convtools
hooks:
- docs/hooks.py
plugins:
- search:
- llmstxt:
markdown_description: |
**convtools** lets you declare data transformations in plain Python, then
compiles them into tiny, optimized Python functions at runtime. You keep your
data in native iterables (lists, dicts, generators, CSV streams)—no heavy
container required.
## LLM quick guide
Install with:
```bash
pip install convtools
```
Current documented version: `{convtools_version}`.
Main imports:
```python
from convtools import conversion as c
from convtools.contrib.tables import Table
```
Core model:
- A conversion is a specification, not an immediate computation.
- Build conversion trees with `c.this`, `c.item(...)`, `c.attr(...)`,
`c.iter(...)`, `c.filter(...)`, `c.group_by(...)`, `c.aggregate(...)`,
`c.join(...)`, and other `c.*` helpers.
- Compile reusable functions with `.gen_converter()`; use `.execute(data)`
for one-off calls.
- `debug=True` prints the generated Python function for inspection.
Small runnable examples:
```python
{llms_example_scalar}
```
```python
{llms_example_iter_filter}
```
```python
{llms_example_aggregation}
```
```python
{llms_example_join}
```
```python
{llms_example_table}
```
Table notes:
- `Table` is a streaming helper for CSV-like/tabular data. It is
single-pass, so the underlying iterable is consumed once.
- `Table.into_iter_rows(...)` returns an iterator.
- Terminal writer methods such as `Table.into_csv(...)` and
`Table.into_jsonl(...)` consume the pipeline, write output, and return
`None`.
Common gotchas:
- Use `c.naive(value)` for constants, lookup tables, and helper functions
known when the converter is built.
- Use `c.input_arg("name")` for values supplied each time the generated
function is called.
- Inside `c.iter(...)`, `c.this` refers to the current element, not the
original outer input.
- In joins, use `c.LEFT` and `c.RIGHT` in the join condition. Equi-joins
can use hash-join optimization, but joins may still materialize right-side
data.
- Reducer arguments are evaluated against each input row, and reducers have
documented `None`, `default=`, `where=`, and `initial=` behavior.
sections:
Getting started:
- README.md: "Overview, installation, short examples, and when to use convtools."
- basics.md: "Core conversion model, placeholders, `c.naive`, `c.input_arg`, and generated-code debugging."
- collections.md: "Iteration, filtering, comprehensions, sorting, chunking, uniqueness, windows over iterables, and cumulative transforms."
- conditions_n_pipes.md: "Conditional expressions, boolean logic, pipes, labels, expectations, and debugging helpers."
Core concepts:
- conditions_n_pipes.md: "Flow control and reusable pipeline composition."
- aggregations.md: "`c.aggregate`, `c.group_by`, reducers, reducer defaults, `None` handling, and custom `c.reduce`."
- joins.md: "Sequence joins, `c.LEFT` / `c.RIGHT`, join types, multi-key and non-equi joins, duplicate behavior, and performance notes."
Advanced:
- dates.md: "Date parsing, formatting, truncation, grids, and datetime helpers."
- window_funcs.md: "PostgreSQL-style window functions over Python iterables."
- exceptions.md: "`c.try_`, exception handling, defaults, and conditional re-raise behavior."
- mutations.md: "In-place item and attribute mutation helpers."
Tabular data stream processing:
- contrib_tables.md: "Streaming `Table` pipelines for rows, CSV, JSONL, joins, pivots, and terminal writers."
Buffer reading:
- contrib_fs.md: "File and buffer helpers for custom newline splitting."
Changelogs:
- changelog.md: "Release history."
- breaking_changes.md: "Compatibility and migration notes."
Benefits:
- benefits.md: "Performance rationale, benchmarks, and generated-code advantages."