Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 30 additions & 29 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,13 @@ jobs:
name: Creating source release
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4.1.7
- uses: actions/checkout@v6.0.2

- name: Setting up Python
uses: actions/setup-python@v5
- name: Setting up uv
uses: astral-sh/setup-uv@v5
with:
python-version: 3.13

- name: Installing python build dependencies
run: |
pip install uv
python-version: "3.13"
enable-cache: "false"

- name: Building source distribution
run: |
Expand All @@ -34,8 +31,8 @@ jobs:
- name: Ensuring documentation builds
run: |
cd docs && uv run make clean html
- uses: actions/upload-artifact@v4.6.0

- uses: actions/upload-artifact@v6.0.0
with:
name: dist-sdist
path: dist/*.tar.gz
Expand All @@ -48,28 +45,29 @@ jobs:
fail-fast: true
matrix:
os: [ubuntu-latest, windows-latest, macos-14]
py: ["cp39", "cp310", "cp311", "cp312", "cp313"]
py: ["cp39", "cp310", "cp311", "cp312", "cp313", "cp314"]

steps:
- uses: actions/checkout@v4.1.7
- uses: actions/checkout@v6.0.2

- name: Setting up Python
uses: actions/setup-python@v5
- name: Setting up uv
uses: astral-sh/setup-uv@v5
with:
python-version: "3.13"
enable-cache: "false"

- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v3.7.0
with:
platforms: all

- name: Build & test wheels
uses: pypa/cibuildwheel@v2.22.0
uses: pypa/cibuildwheel@v3.3.1
env:
CIBW_BUILD: "${{ matrix.py }}-*"

- uses: actions/upload-artifact@v4.6.0
- uses: actions/upload-artifact@v6.0.0
with:
name: dist-${{ matrix.os }}-${{ matrix.py }}
path: ./wheelhouse/*.whl
Expand All @@ -79,34 +77,37 @@ jobs:
needs: [build_wheels, sdist]
if: github.event_name == 'release'
runs-on: ubuntu-latest
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v4.1.8
- uses: actions/download-artifact@v7.0.0
with:
pattern: dist-*
merge-multiple: true
path: dist

- uses: pypa/gh-action-pypi-publish@v1.4.2
- name: Installing uv
uses: astral-sh/setup-uv@v5
with:
user: ${{ secrets.PYPI_USERNAME }}
password: ${{ secrets.PYPI_PASSWORD }}
python-version: "3.13"
enable-cache: "false"

- name: Publishing to PyPI
run: uv publish dist/*

build_documentation:
name: Building & uploading documentation.
needs: [upload_all]
if: github.event_name == 'release'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4.1.7
- uses: actions/checkout@v6.0.2

- name: Setting up Python
uses: actions/setup-python@v5
- name: Setting up uv
uses: astral-sh/setup-uv@v5
with:
python-version: 3.13

- name: Installing python build dependencies
run: |
pip install uv
python-version: "3.13"
enable-cache: "false"

- name: Installing release dependencies.
run: |
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## 7.03

- Update to upstream simdjson 4.2.4.
- Started building CPython 3.14 binary wheels.

## 7.0.2

- Remove self-dependency in the pyproject.toml, fixing poetry installs (#130)
Expand Down
35 changes: 33 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
![PyPI - License](https://img.shields.io/pypi/l/pysimdjson.svg?style=flat-square)
![Tests](https://github.com/TkTech/pysimdjson/workflows/Run%20tests/badge.svg)
[![tests](https://github.com/TkTech/pysimdjson/actions/workflows/release.yml/badge.svg)](https://github.com/TkTech/pysimdjson/actions/workflows/release.yml)

# pysimdjson

Expand All @@ -10,7 +10,8 @@ pysimdjson safe to use anywhere.
Bindings are currently tested on OS X, Linux, and Windows for Python version
3.9 to 3.12.

## 📝 Documentation

## 📈 Documentation

The latest documentation can be found at https://pysimdjson.tkte.ch.

Expand All @@ -24,3 +25,33 @@ found in its sister project [json_benchmark][].

[simdjson]: https://github.com/lemire/simdjson
[json_benchmark]: https://github.com/tktech/json_benchmark

## 📈 Building Locally

To build pysimdjson from source, you'll need Python 3.9+ and a C++ compiler.

1. **Install uv** (if not already installed):
```bash
pip install uv
```

2. **Clone the repository**:
```bash
git clone https://github.com/TkTech/pysimdjson.git
cd pysimdjson
```

3. **Set up the development environment**:
```bash
uv sync --group dev
```

4. **(Optional) Run tests**:
```bash
uv run --group dev --reinstall-package pysimdjson pytest
```

5. **(Optional) Build documentation**:
```bash
uv run --group dev make -C docs html
```
33 changes: 15 additions & 18 deletions docs/contrib/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@ dependencies, use:

.. code::

pip install -e ".[test]"
uv sync --group dev

To run the tests, just type ``pytest``. To also run some slow integration
tests, use ``pytest --runslow``.
To run the tests, use ``uv run --group dev --reinstall-package pysimdjson
pytest``. To also run the slow integration tests, use ``uv run --group dev
--reinstall-package pysimdjson pytest --runslow``.

To properly test builds on Windows, you need both a recent version of Visual
Studio as well as VS2015, patch 3. Older versions of CPython required portable
Expand All @@ -19,31 +20,27 @@ Use the `Developer Command Prompt`_ to easily switch between versions.
Cythonize
---------

pysimdjson is written using `Cython`_. However, by default ``setup.py`` will
use the already-generated ``csimdjson.cpp`` instead of regenerating it. This
is to avoid making Cython an install-time requirement.

To force the usage of Cython, use ``BUILD_WITH_CYTHON``:
pysimdjson is written using `Cython`_. The extension module is declared in
``pyproject.toml`` and built by setuptools. ``uv sync`` installs the project in
editable mode by default:

.. code::

BUILD_WITH_CYTHON=1 python setup.py develop

This will cause Cython to regenerate the ``csimdjson.cpp`` from the
``csimdjson.pyx`` and ``csimdjson.pxd`` files.
uv sync --group dev

To build pysimdjson with support for linetracing and coverage, use ``BUILD_FOR_DEBUG``:
Python source changes are visible immediately. When you change Cython or C++
sources, force uv to rebuild and reinstall the extension:

.. code::

BUILD_WITH_CYTHON=1 BUILD_FOR_DEBUG=1 python setup.py develop
uv sync --group dev --reinstall-package pysimdjson

pysimdjson will also reuse the generated .so file if you build it more than
once, so to force Cython to rebuild it, use ``FORCE_REBUILD``:
To build pysimdjson with support for linetracing and coverage, use
``BUILD_FOR_DEBUG``:

.. code::

BUILD_WITH_CYTHON=1 FORCE_REBUILD=1 python setup.py develop
BUILD_FOR_DEBUG=1 uv sync --group dev --reinstall-package pysimdjson

Benchmarks
----------
Expand All @@ -57,4 +54,4 @@ correctly.
.. _Developer Command Prompt: https://docs.microsoft.com/en-us/dotnet/
framework/tools/developer-command-prompt-for-vs
.. _Cython: https://cython.readthedocs.io/en/latest/
.. _json_benchmark: https://github.com/tktech/json_benchmark
.. _json_benchmark: https://github.com/tktech/json_benchmark
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ Binary wheels are available for the following:
| CPython 3.12 | Yes | Yes | Yes | Yes | Yes |
+--------------+-------+-------+-------+---------+---------+

When binary wheels are not available, a C++11 (or better) compiler is required
When binary wheels are not available, a C++17 (or better) compiler is required
when installing in order to build the underlying simdjson library.

If you would prefer to always install pysimdjson from source even when
Expand Down
42 changes: 33 additions & 9 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,22 +1,45 @@
[build-system]
requires = ["setuptools>=74.1", "Cython"]
requires = ["setuptools>=82.0.1", "Cython"]
build-backend = "setuptools.build_meta"

[project]
name = "pysimdjson"
version = "7.0.2"
description = "Add your description here"
version = "7.0.3"
description = "Python bindings for the simdjson project, a SIMD-accelerated JSON parser"
readme = "README.md"
license = "MIT"
license-files = ["LICENSE"]
requires-python = ">=3.9"
dependencies = [
classifiers = [
"Programming Language :: Python :: 3",
"Operating System :: OS Independent",
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
]
authors = [
{ name = "Tyler Kennedy", email = "tk@tkte.ch" }
]
keywords = [
"json",
"parser",
"simd"
]

[project.urls]
Homepage = "https://github.com/tktech/pysimdjson"
Repository = "https://github.com/tktech/pysimdjson"

[tool.uv.sources]
pysimdjson = { workspace = true }

[tool.setuptools]
ext-modules = [
{ name = "csimdjson", sources = ["simdjson/simdjson.cpp", "simdjson/util.cpp", "simdjson/csimdjson.pyx"], py-limited-api = true },
[[tool.setuptools.ext-modules]]
name = "csimdjson"
sources = ["simdjson/simdjson.cpp", "simdjson/util.cpp", "simdjson/csimdjson.pyx"]
define-macros = [
[
"SIMDJSON_IMPLEMENTATION_FALLBACK",
"1"
]
]

[tool.setuptools.packages.find]
Expand All @@ -40,12 +63,13 @@ dev = [
before-test = "pip install pytest pytest-benchmark"
test-command = "pytest {project}/tests"
test-skip = "*_arm64 *_universal2:arm64"
# This should be part of ext-modules but is blocked by setuptools issue #4810.
environment = { CPPFLAGS="-DSIMDJSON_IMPLEMENTATION_FALLBACK=1" }

[tool.cibuildwheel.linux]
archs = ["auto", "aarch64", "ppc64le"]

[tool.cibuildwheel.macos]
environment = { CXXFLAGS="-std=c++11" }
archs = ["x86_64", "universal2"]

[tool.cibuildwheel.windows]
environment = { CXXFLAGS="/std:c++11" }
21 changes: 14 additions & 7 deletions simdjson/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,24 @@
Array,
Object,
MAXSIZE_BYTES,
PADDING
PADDING,
VERSION
)
except ImportError:
raise RuntimeError('Unable to import low-level simdjson bindings.')

_ALL_IMPORTS = [
Parser,
Array,
Object,
MAXSIZE_BYTES,
PADDING
__all__ = [
'Parser',
'Array',
'Object',
'MAXSIZE_BYTES',
'PADDING',
'VERSION',
'load',
'loads',
'dump',
'dumps',
'JSONEncoder',
]


Expand Down
Loading
Loading