Skip to content

Add scheduled CI workflow to catch Neovim API deprecations #33

Description

@zhisme

Problem

Current CI runs busted unit tests that mock all vim.* globals. This means the test suite never actually calls real Neovim APIs, so it cannot detect:

  • Deprecated API calls (Neovim 0.10+ emits warnings via vim.deprecate())
  • Removed or renamed functions in new Neovim releases
  • Changed function signatures or return values

Today this is verified manually. A breaking change in Neovim nightly becomes a stable release ~6 months later — catching it early gives time to fix before users hit it.

Proposed Solution

Two pieces:

1. Smoke test script

A minimal script that loads the plugin inside real nvim --headless and exercises the core code paths (setup, keymap registration, format/config validation). Fails if:

  • Neovim emits deprecation warnings to stderr
  • Plugin fails to load or setup() errors
  • Keymaps fail to register

This is intentionally not a full integration test suite (see #14) — it's a lightweight compatibility check that catches API-level breakage without requiring a full test harness like plenary.

2. Scheduled GitHub Actions workflow

  • Neovim version matrix: [v0.7.0, stable, nightly]
    • v0.7.0 — minimum supported version (first to have vim.keymap.set)
    • stable — current release, what most users run
    • nightly — catches upcoming deprecations before they ship
  • Trigger: schedule: cron biweekly + on push/PR (so it also runs during development)
  • Action: use rhysd/action-setup-vim or direct tarball download to install each Neovim version, then run the smoke test

The nightly job is allowed to fail (it's informational), but stable and minimum version must pass.

Implementation Plan

  1. Add tests/smoke_test.lua — a script meant to run via nvim --headless -l tests/smoke_test.lua

    • Calls require('copy_with_context').setup() with default config
    • Verifies keymaps are registered (vim.fn.maparg)
    • Checks a custom config with all options
    • Captures deprecation warnings by overriding vim.deprecate and fails if any fire
    • Exits with non-zero code on failure
  2. Add .github/workflows/compat.yml — scheduled compatibility workflow

    • Cron: every 2 weeks
    • Matrix: [v0.7.0, stable, nightly]
    • Steps: install Neovim, run smoke test, report
    • nightly marked continue-on-error: true
  3. Add a make smoke target for local use

Relation to Other Issues

This is complementary to #14 (integration tests). #14 is about full behavioral testing inside Neovim (plenary/mini.test). This issue is narrower — a fast compatibility canary that catches API deprecations across Neovim versions.

Metadata

Metadata

Assignees

Labels

featureNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions