Skip to content

Make engine testing local#1160

Merged
AttackingOrDefending merged 8 commits into
lichess-bot-devs:masterfrom
MarkZH:local-test-bots
Dec 6, 2025
Merged

Make engine testing local#1160
AttackingOrDefending merged 8 commits into
lichess-bot-devs:masterfrom
MarkZH:local-test-bots

Conversation

@MarkZH
Copy link
Copy Markdown
Collaborator

@MarkZH MarkZH commented Dec 3, 2025

Type of pull request:

  • Bug fix
  • Feature
  • Other

Description:

Stops downloading chess engines every time a test is run. All engine types are simulated locally.

Related Issues:

N/A

Checklist:

  • I have read and followed the contribution guidelines.
  • I have added necessary documentation (if applicable).
  • The changes pass all existing tests.

Screenshots/logs (if applicable):

@MarkZH
Copy link
Copy Markdown
Collaborator Author

MarkZH commented Dec 3, 2025

Problems so far:

  1. Mypy doesn't like the from test_games import scholars_mate in the new engine files: xboard_engine.py, uci_engine.py, buggy_engine.py, and homemade.py.
  2. External moves test is failing when lichess.org returns a 429 error.

@AttackingOrDefending, any ideas for fixing (1)? For (2), can we detect connection errors and skip tests?

@AttackingOrDefending
Copy link
Copy Markdown
Member

AttackingOrDefending commented Dec 3, 2025

For (1):
engine_wrapper.py:

def create_engine(engine_config: Configuration, game: model.Game | None = None) -> EngineWrapper:
    """
    Create the engine.

    Use in a with-block to automatically close the engine when exiting the game.

    :param engine_config: The options for the engine.
    :return: An engine. Either UCI, XBoard, or Homemade.
    """
    cfg = engine_config.engine
    engine_path = cfg.name
    engine_type = cfg.protocol
    commands = []
    if cfg.interpreter:
        commands.append(cfg.interpreter)
        commands.extend(cfg.interpreter_options)
    else:
        engine_path = os.path.abspath(os.path.join(cfg.dir, engine_path))
    commands.append(engine_path)
    if cfg.engine_options:
        for k, v in cfg.engine_options.items():
            commands.append(f"--{k}={v}" if v is not None else f"--{k}")

    stderr = None if cfg.silence_stderr else subprocess.DEVNULL

    Engine: type[UCIEngine | XBoardEngine | MinimalEngine]
    if engine_type == "xboard":
        Engine = XBoardEngine
    elif engine_type == "uci":
        Engine = UCIEngine
    elif engine_type == "homemade":
        Engine = get_homemade_engine(cfg.name)
    else:
        raise ValueError(
            f"    Invalid engine type: {engine_type}. Expected xboard, uci, or homemade.")
    options = remove_managed_options(cfg.lookup(f"{engine_type}_options") or Configuration({}))
    logger.debug(f"Starting engine: {commands}")
    return Engine(commands, options, stderr, cfg.draw_or_resign, game, cwd=cfg.working_dir)

test_bot.py:

python = sys.executable  # Otherwise you have to install chess on the global python

...

        CONFIG["engine"]["dir"] = "test_bot"  # can be removed
        CONFIG["engine"]["name"] = "test_bot.uci_engine"
        CONFIG["engine"]["interpreter"] = python
        CONFIG["engine"]["interpreter_options"] = ["-m"]

And change all imports to: from .test_games import scholars_mate

For (2): Yes, just skip them.

@MarkZH MarkZH marked this pull request as ready for review December 5, 2025 03:57
@MarkZH
Copy link
Copy Markdown
Collaborator Author

MarkZH commented Dec 5, 2025

I ended up using if typing.TYPE_CHECKING: to get around the differences between how Python and Mypy find imports.

Anyway, this is ready for review and/or merging.

@MarkZH
Copy link
Copy Markdown
Collaborator Author

MarkZH commented Dec 5, 2025

The external move tests can be fixed in another PR.

@AttackingOrDefending
Copy link
Copy Markdown
Member

I think python = "python" if platform == "win32" else "python3" should be changed to python = sys.executable, as tests hang if chess isn't installed on the global python.

@AttackingOrDefending AttackingOrDefending merged commit 206e858 into lichess-bot-devs:master Dec 6, 2025
17 of 23 checks passed
@MarkZH MarkZH deleted the local-test-bots branch December 6, 2025 21:42
r0950336 pushed a commit to r0950336/lichess-bot that referenced this pull request Dec 31, 2025
* Make UCI and XBoard tests local

No need to download any chess engines.

Mypy does not see test_games.py

* Delete unused import

* Delete unneeded external files and opponent arg

* Add fix for imports to satisfy mypy

* Use sys.executable for portability

* Break up over-complicated line

* Delete trivial testing engine
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants