Skip to content

Commit 129854a

Browse files
ci: enable MD031/MD032/MD034/MD040 markdownlint rules
Originally disabled to land the initial config without manual cleanup, but the violations are all fixable and the rules catch real Markdown smell. - MD040 (require code fence language): 0 violations after auto-fix - MD031 (blanks around fences) / MD032 (blanks around lists): fixed by --fix in docs/introduction.md, docs/sqlalchemy.md, CLAUDE.md - MD034 (no bare URLs): wrapped 4 footer URLs in README.md with <...> Remaining intentionally-disabled rules: - MD013 (line length): docs have long URLs and code lines - MD014 ($ command without output): convention in docs/testing.md, README - MD024 siblings_only: cursor docs reuse subsection names per cursor - MD041 (first line H1): MyST `(label)=` ref targets precede the first heading Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 31af950 commit 129854a

5 files changed

Lines changed: 36 additions & 16 deletions

File tree

.markdownlint-cli2.jsonc

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
// https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md
33
"config": {
44
"default": true,
5-
// Line length: docs have long URLs, code blocks, and prose lines
5+
// Docs have long URLs and code lines that would be awkward to wrap
66
"MD013": false,
77
// Match existing style: dash bullets, 2-space nested indent
88
"MD004": { "style": "dash" },
@@ -13,16 +13,10 @@
1313
},
1414
// Cursor docs intentionally repeat subsection names ("Basic usage") under different cursors
1515
"MD024": { "siblings_only": true },
16-
// Don't require fenced code blocks to specify a language
17-
"MD040": false,
18-
// Don't require blank lines around code fences / lists — existing docs intermix them
19-
"MD031": false,
20-
"MD032": false,
21-
// Allow `$ command` style in shell snippets (testing.md, README)
16+
// `$ command` style is intentional in docs/testing.md and README shell snippets
2217
"MD014": false,
23-
// Allow bare URLs
24-
"MD034": false,
25-
// First line need not be a top-level heading (some files start with MyST ref targets)
18+
// MyST `(label)=` ref targets must come before the first heading, so the
19+
// first line of many docs/*.md files is not an H1
2620
"MD041": false
2721
},
2822
"globs": [

CLAUDE.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,12 +33,14 @@ make test-sqla # SQLAlchemy dialect tests
3333
```
3434

3535
Tests require AWS environment variables. Use a `.env` file (gitignored):
36+
3637
```bash
3738
AWS_DEFAULT_REGION=<region>
3839
AWS_ATHENA_S3_STAGING_DIR=s3://<bucket>/<path>/
3940
AWS_ATHENA_WORKGROUP=<workgroup>
4041
AWS_ATHENA_SPARK_WORKGROUP=<spark-workgroup>
4142
```
43+
4244
```bash
4345
export $(cat .env | xargs) && uv run pytest tests/pyathena/test_file.py -v
4446
```
@@ -53,22 +55,24 @@ export $(cat .env | xargs) && uv run pytest tests/pyathena/test_file.py -v
5355
- **Standalone functions** for unit tests of pure logic (converters, parsers, utils): `def test_to_struct_json_formats(input_value, expected):`
5456
- Test file naming mirrors source: `pyathena/parser.py``tests/pyathena/test_parser.py`
5557
- **Fixtures**: Cursor/engine fixtures are defined in `conftest.py` and injected by name (e.g., `cursor`, `engine`, `async_cursor`). Use `indirect=True` parametrization to pass connection options:
58+
5659
```python
5760
@pytest.mark.parametrize("engine", [{"driver": "rest"}], indirect=True)
5861
def test_query(self, engine):
5962
engine, conn = engine
6063
```
64+
6165
- **Parametrize** with `@pytest.mark.parametrize(("input", "expected"), [...])` for data-driven tests
6266
- **Integration tests** (need AWS) use cursor/engine fixtures with real Athena queries; **unit tests** (no AWS) call functions directly with test data
6367

6468
### Markdown Lint
6569

6670
`docs/**/*.md` and project-root `*.md` files are linted with [markdownlint-cli2](https://github.com/DavidAnson/markdownlint-cli2). The config lives at `.markdownlint-cli2.jsonc`. CI runs lint + Sphinx build on PRs that touch docs (`.github/workflows/docs-lint.yaml`).
6771

68-
`markdownlint-cli2` is pinned in `.mise.toml` (along with `node`), so [`mise`](https://mise.jdx.dev/) installs the exact version used in CI. Run locally:
72+
`markdownlint-cli2` is pinned in `.mise.toml`, so [`mise`](https://mise.jdx.dev/) installs the exact version used in CI. Run locally:
6973

7074
```bash
71-
mise install # one-time: installs node + markdownlint-cli2
75+
mise install # one-time: installs markdownlint-cli2
7276
make docs-lint # check
7377
make docs-lint-fix # auto-fix what's possible
7478
```
@@ -88,6 +92,7 @@ Each cursor type lives in its own subpackage (`pandas/`, `arrow/`, `polars/`, `s
8892
### Filesystem (fsspec) Compatibility
8993

9094
`pyathena/filesystem/s3.py` implements fsspec's `AbstractFileSystem`. When modifying:
95+
9196
- Match `s3fs` library behavior where possible (users migrate from it)
9297
- Use `delimiter="/"` in S3 API calls to minimize requests
9398
- Handle edge cases: empty paths, trailing slashes, bucket-only paths

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -77,10 +77,10 @@ Many of the implementations in this library are based on [PyHive](https://github
7777

7878
## Links
7979

80-
- Documentation: https://pyathena.dev/
81-
- PyPI Releases: https://pypi.org/project/PyAthena/
82-
- Source Code: https://github.com/pyathena-dev/PyAthena/
83-
- Issue Tracker: https://github.com/pyathena-dev/PyAthena/issues
80+
- Documentation: <https://pyathena.dev/>
81+
- PyPI Releases: <https://pypi.org/project/PyAthena/>
82+
- Source Code: <https://github.com/pyathena-dev/PyAthena/>
83+
- Issue Tracker: <https://github.com/pyathena-dev/PyAthena/issues>
8484

8585
## Logo
8686

docs/introduction.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,19 +35,22 @@ Extra packages:
3535
PyAthena provides comprehensive support for Amazon Athena's data types and features:
3636

3737
**Core Features:**
38+
3839
- **DB API 2.0 Compliance**: Full PEP 249 compatibility for database operations
3940
- **SQLAlchemy Integration**: Native dialect support with table reflection and ORM capabilities
4041
- **Multiple Cursor Types**: Standard, Pandas, Arrow, Polars, S3FS and Spark cursor implementations
4142
- **Async Support**: Asynchronous query execution for non-blocking operations
4243

4344
**Data Type Support:**
45+
4446
- **STRUCT/ROW Types**: {ref}`Complete support <sqlalchemy>` for complex nested data structures
4547
- **ARRAY Types**: {ref}`Complete support <sqlalchemy>` for ordered collections with automatic Python list conversion
4648
- **MAP Types**: {ref}`Complete support <sqlalchemy>` for key-value dictionary-like data structures
4749
- **JSON Integration**: Seamless JSON data parsing and conversion
4850
- **Performance Optimized**: Smart format detection for efficient data processing
4951

5052
**Additional Features:**
53+
5154
- **Connection Management**: Efficient connection pooling and configuration
5255
- **Result Caching**: Athena query result reuse capabilities
5356
- **Error Handling**: Comprehensive exception handling and recovery

docs/sqlalchemy.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,7 @@ location
130130
value: s3://bucket/path/to/
131131

132132
Example:
133+
133134
```python
134135
Table("some_table", metadata, ..., awsathena_location="s3://bucket/path/to/")
135136
```
@@ -140,6 +141,7 @@ compression
140141
Description: Specifies the compression format.
141142

142143
Value:
144+
143145
- BZIP2
144146
- DEFLATE
145147
- GZIP
@@ -151,6 +153,7 @@ compression
151153
- NONE|UNCOMPRESSED
152154

153155
Example:
156+
154157
```python
155158
Table("some_table", metadata, ..., awsathena_compression="SNAPPY")
156159
```
@@ -161,6 +164,7 @@ row_format
161164
Description: Specifies the row format of the table and its underlying source data if applicable.
162165

163166
Value:
167+
164168
- [DELIMITED FIELDS TERMINATED BY char [ESCAPED BY char]]
165169
- [DELIMITED COLLECTION ITEMS TERMINATED BY char]
166170
- [MAP KEYS TERMINATED BY char]
@@ -169,6 +173,7 @@ row_format
169173
- SERDE 'serde_name'
170174

171175
Example:
176+
172177
```python
173178
Table("some_table", metadata, ..., awsathena_row_format="SERDE 'org.openx.data.jsonserde.JsonSerDe'")
174179
```
@@ -179,6 +184,7 @@ file_format
179184
Description: Specifies the file format for table data.
180185

181186
Value:
187+
182188
- SEQUENCEFILE
183189
- TEXTFILE
184190
- RCFILE
@@ -189,6 +195,7 @@ file_format
189195
- INPUTFORMAT input_format_classname OUTPUTFORMAT output_format_classname
190196

191197
Example:
198+
192199
```python
193200
Table("some_table", metadata, ..., awsathena_file_format="PARQUET")
194201
Table("some_table", metadata, ..., awsathena_file_format="INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'")
@@ -200,11 +207,13 @@ serdeproperties
200207
Description: Specifies one or more custom properties allowed in SerDe.
201208

202209
Value:
210+
203211
```python
204212
{ "property_name": "property_value", "property_name": "property_value", ... }
205213
```
206214

207215
Example:
216+
208217
```python
209218
Table("some_table", metadata, ..., awsathena_serdeproperties={
210219
"separatorChar": ",", "escapeChar": "\\\\"
@@ -217,11 +226,13 @@ tblproperties
217226
Description: Specifies custom metadata key-value pairs for the table definition in addition to predefined table properties.
218227

219228
Value:
229+
220230
```python
221231
{ "property_name": "property_value", "property_name": "property_value", ... }
222232
```
223233

224234
Example:
235+
225236
```python
226237
Table("some_table", metadata, ..., awsathena_tblproperties={
227238
"projection.enabled": "true",
@@ -239,6 +250,7 @@ bucket_count
239250
Value: Integer value greater than or equal to 0
240251

241252
Example:
253+
242254
```python
243255
Table("some_table", metadata, ..., awsathena_bucket_count=5)
244256
```
@@ -268,6 +280,7 @@ partition
268280
Value: True / False
269281

270282
Example:
283+
271284
```python
272285
Column("some_column", types.String, ..., awsathena_partition=True)
273286
```
@@ -279,6 +292,7 @@ partition_transform
279292
Only has an effect for ICEBERG tables and when partition is set to true for the column.
280293

281294
Value:
295+
282296
- year
283297
- month
284298
- day
@@ -287,6 +301,7 @@ partition_transform
287301
- truncate
288302

289303
Example:
304+
290305
```python
291306
Column("some_column", types.Date, ..., awsathena_partition=True, awsathena_partition_transform='year')
292307
```
@@ -301,6 +316,7 @@ partition_transform_bucket_count
301316
Value: Integer value greater than or equal to 0
302317

303318
Example:
319+
304320
```python
305321
Column("some_column", types.String, ..., awsathena_partition=True, awsathena_partition_transform='bucket', awsathena_partition_transform_bucket_count=5)
306322
```
@@ -315,6 +331,7 @@ partition_transform_truncate_length
315331
Value: Integer value greater than or equal to 0
316332

317333
Example:
334+
318335
```python
319336
Column("some_column", types.String, ..., awsathena_partition=True, awsathena_partition_transform='truncate', awsathena_partition_transform_truncate_length=5)
320337
```
@@ -327,6 +344,7 @@ cluster
327344
Value: True / False
328345

329346
Example:
347+
330348
```python
331349
Column("some_column", types.String, ..., awsathena_cluster=True)
332350
```

0 commit comments

Comments
 (0)