Refactored test_lex.json to use test_lex.json for test case data.

[rob-ross]

Extracted the list of Case objects to the file test_lex.json. Updated test_lex.py to use the json file.

[rob-ross]

I understand the failing checks and will rectify them.

[rob-ross]

Held up because my imports aren't pretty enough lol. Ok, I don't know what it wants me to do. I hit "Optimize Imports" in my IDE but that doesn't seem up to the task. Are there project guidelines for how it should be formatted?

[jg-rp]

We use Ruff to lint and format all Python files. Among other things, Ruff enforces:

• Black, which has rules about line breaks and indentation.
• isort with force-single-line = true, which ensures all Python imports are grouped and sorted in a consistent way.

Ruff can be integrated with some editors. Personally I run it via Hatch on the command line, as illustrated in the new contributing guide.

If you're working on a Hatch project, you'll probably want to install Hatch globally. I like to use pipx for this.

[rob-ross]

Finally :)

[jg-rp]

Hi @rob-ross,

I think we can simplify it a bit by getting rid of the dataclass and not worry about mapping to token kinds defined in jsonpath.token. After all, if a unknown token kind appears in test_lex.json, we'll soon know about it when comparing "got" with "want".

For example:

import json
import operator
from typing import Any
from typing import Dict

import pytest

from jsonpath import DEFAULT_ENV
from jsonpath import JSONPathSyntaxError
from jsonpath.token import Token

with open("tests/test_lex.json", encoding="UTF-8") as fd:
    CASES = json.load(fd)["tests"]


@pytest.mark.parametrize("case", CASES, ids=operator.itemgetter("description"))
def test_default_lexer(case: Dict[str, Any]) -> None:
    tokens = list(DEFAULT_ENV.lexer.tokenize(case["path"]))
    want = [Token(**token) for token in case["want"]]
    assert tokens == want


def test_illegal_token() -> None:
    with pytest.raises(JSONPathSyntaxError):
        list(DEFAULT_ENV.lexer.tokenize("%"))

No dataclass does means we'd get a KeyError instead of a TypeError if any of the test cases in test_lext.json is malformed. And, with the example above, we'd get a TypeError instead of a KeyError when constructing each Token instance. But I don't think that matters.

What do you think?

[rob-ross]

Sure. Looks fine. I'll work on that.

[rob-ross]

For my own repository, the standard workflow is to delete this branch after a successful merge to avoid confusion. Does this apply to a forked clone as well? I.e., should I delete this branch now?

[jg-rp]

should I delete this branch now?

Yes, I would, as the branch is short-lived and you probably won't be coming back to it.