Commit 8017ff4
Various performance improvements inspired by asyncpg
D1 — Delete global STMTS_CACHE; add per-connection caches:
- Delete src/statement/cache.rs (process-global RwLock<HashMap> was
incorrect for cross-connection Statement reuse and a serialization point)
- PoolConnection: unchanged — deadpool's prepare_cached already correct
- SingleConnection: gains DashMap<String, Statement> per-connection cache;
prepare() consults/inserts it on every prepared query
- dashmap = "6" added to Cargo.toml
D2 — execute_many: build statement once, single GIL pass:
- StatementBuilder::build() called once per execute_many call, not per row
- Remaining rows reuse extracted Vec<Type> in a single GIL pass
- run_pipelined_batch: accepts pre-built Statement, no redundant prepare
- TODO(bind-execute-many) marker left citing asyncpg coreproto.pyx:1022-1092
D3 — COPY records path: 512 KiB BytesMut streaming encoder:
- Replace BinaryCopyInWriter per-row flush (4 KiB) with hand-rolled encoder
flushing at 512 KiB (COPY_BUFFER_SIZE = 524288, matches asyncpg's value)
- Single streaming pass: open copy_in before GIL, encode+flush per row
- Eliminates intermediate Vec<Vec<Py<PyAny>>> materialization
D4 — Cache COPY column-type introspection per (schema, table, columns):
- Both PoolConnection and SingleConnection gain CopyTypeCache (DashMap)
- copy_records_to_table checks cache before issuing PREPARE+DEALLOCATE
D5 — Record pyclass + additive records() method:
- New #[pyclass] Record: Vec<Py<PyAny>> + Arc<RecordDesc> (shared col map)
- Implements __getitem__ (int/str/slice), __len__, __iter__, __repr__,
get(), keys(), values(), items() — matches asyncpg Record surface
- QueryResult::records() returns Vec<Record>; result() unchanged (additive)
- Type stubs in python/psqlpy/_internal/__init__.pyi updated
D6 — Micro-wins:
- T3#7: is_exact_instance dispatch in from_python.rs (GILOnceCell-cached
PyTypeObject pointers for UUID + Decimal replace string name comparison)
- T3#8: ParametersBuilder::prepare early-returns before Python::with_gil
when params are None or empty
- T3#10: per-row scratch Vec cleared between rows in COPY encoder
Tests: 17 new pytest tests (test_record.py + test_copy_records.py extensions)
Lint: ruff D205/PLR2004 suppressed for test files in pyproject.toml
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1 parent 1d77435 commit 8017ff4
19 files changed
Lines changed: 965 additions & 306 deletions
File tree
- python
- psqlpy/_internal
- tests
- src
- connection
- driver
- statement
- value_converter
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| 67 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
104 | 104 | | |
105 | 105 | | |
106 | 106 | | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | | - | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
111 | 113 | | |
112 | 114 | | |
113 | 115 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
20 | 47 | | |
21 | 48 | | |
22 | 49 | | |
| |||
107 | 134 | | |
108 | 135 | | |
109 | 136 | | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
110 | 146 | | |
111 | 147 | | |
112 | 148 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
3 | 5 | | |
4 | 6 | | |
5 | 7 | | |
| |||
172 | 174 | | |
173 | 175 | | |
174 | 176 | | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
0 commit comments