Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,18 @@ This file should adhere to [Keep a Changelog](https://keepachangelog.com/en/1.1.

This project should adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html), though some earlier releases may be incompatible with the SemVer standard.

## [1.0.1] - 2026-03-19

### Fixed
- `--name radicale` (and other lowercase names) failed to find servers in the caldav test registry after the caldav library renamed its server entries to capitalised names (`Radicale`, `Xandikos`). The registry lookup is now case-insensitive.
- `--name` registry lookup silently returned nothing when the caldav-server-tester's own `tests/` package shadowed the caldav project's `tests/test_servers` in `sys.modules` or via the CWD entry in `sys.path`. The registry is now loaded via `importlib` using the explicit file path, bypassing `sys.path` resolution.

### Documentation
* Updated USAGE.md
* `--format text` section to reflect current multi-line output format and actual support-level values; added `unknown` status
* added guide for contributing a new server profile to `caldav/compatibility_hints.py`
* added guide for storing checker results in `~/.config/caldav/calendar.conf` (named profile, inline features, and base+overrides patterns)

## [1.0.0] - 2026-03-15

Considering this tool as "production ready" now - even though it's still lots of corner cases to be tested.
Expand Down
144 changes: 133 additions & 11 deletions USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,25 +56,36 @@ Note that the only difference between `--name` and `--config-section` is that `-

Human-readable summary. Without `--verbose`, only features deviating from the CalDAV standard are shown. With `--verbose`, all checked features are shown.

Each feature is reported as a block with up to three lines:

```
Server: radicale (http://localhost:5232/)
caldav library version: 1.5.0

Feature compatibility (non-verbose: showing only deviations from the standard):
[no] search.time-range.alarm
[quirk] search.unlimited-time-range

## search.time-range.alarm
Feature support level found: unsupported

## search.unlimited-time-range
Feature support level found: quirk
Extra check information:
behaviour=accepts-but-ignores-end-date
Description of the feature: Whether the server supports CalDAV REPORT search without an end date in a time-range filter
```

Status markers:
The **"Extra check information"** line describes the *specific behaviour observed* during testing — for example, what the server actually did when a feature was exercised (e.g. `behaviour=delayed-deletion`, `behaviour=mkcol-required`). This is distinct from the **"Description of the feature"** line, which gives the general definition of what the feature covers.

Support levels:

| Marker | Meaning |
|--------------|----------------------------------------------------------|
| `[ok]` | Full support |
| `[no]` | Unsupported (silently ignored by the server) |
| `[quirk]` | Supported but needs special client-side handling |
| `[fragile]` | Unreliable / intermittent |
| `[broken]` | Server behaves incorrectly |
| `[error]` | Server returns an error (ungraceful failure) |
| Value | Meaning |
|----------------|--------------------------------------------------------------------|
| `full` | Full, standard-compliant support |
| `unsupported` | Not supported (server silently ignores or rejects the operation) |
| `quirk` | Supported but requires special client-side handling |
| `fragile` | Unreliable or intermittent behaviour |
| `broken` | Server behaves incorrectly (wrong results, data loss, etc.) |
| `unknown` | Could not be determined (e.g. preconditions for the test not met) |

### `--format json` / `--format yaml`

Expand Down Expand Up @@ -103,6 +114,55 @@ file in the caldav project (or your own config):
}
```

## Contributing a server profile to the caldav library

If your CalDAV server is not yet listed in
[`caldav/compatibility_hints.py`](https://github.com/python-caldav/caldav/blob/master/caldav/compatibility_hints.py),
you can use this tool to produce a ready-made profile and submit it upstream.

**Step 1 — run the tester and capture the hints output:**

```
caldav-server-tester --caldav-url https://example.com/dav \
--caldav-username alice \
--caldav-password secret \
--format hints > myserver_hints.py
```

The output is a Python dict literal containing every feature the tester
observed, e.g.:

```python
{
'create-calendar': {'support': 'full'},
'search.time-range.alarm': {'support': 'unsupported'},
'search.unlimited-time-range': {'support': 'quirk', 'behaviour': 'accepts-but-ignores-end-date'},
...
}
```

**Step 2 — add the profile to `compatibility_hints.py`:**

In a fork of the [caldav repository](https://github.com/python-caldav/caldav),
open `caldav/compatibility_hints.py` and add a module-level variable near the
other server profiles (look for variables like `radicale`, `baikal`, `xandikos`):

```python
myserver = {
'search.time-range.alarm': {'support': 'unsupported'},
'search.unlimited-time-range': {'support': 'quirk', 'behaviour': 'accepts-but-ignores-end-date'},
# ... paste the non-full entries from the hints output
}
```

It is conventional to strip entries where `support` is `full` (the default),
keeping only deviations. Add a short comment above the dict describing the
server and the version it was tested against.

**Step 3 — open a pull request** against `python-caldav/caldav` on GitHub.
Include the raw `--format text --verbose` output as supporting evidence in the
PR description so maintainers can verify the findings.

## Diffing expected vs observed

When you have an existing `compatibility_hints` configuration for a server
Expand All @@ -116,6 +176,68 @@ The `--diff` flag appends a section to the report listing every feature where
the observed support level differs from what the configured hints said to
expect.

## Storing results in your caldav config file

The `~/.config/caldav/calendar.conf` (YAML or JSON) supports a `features` key
in each section that tells the caldav client library which workarounds to
apply. There are two ways to populate it.

### Using a named profile

If your server already has a profile in `caldav/compatibility_hints.py` (e.g.
`radicale`, `baikal`, `xandikos`, `synology`, …), simply name it:

```yaml
myserver:
caldav_url: https://example.com/dav
caldav_username: alice
caldav_password: secret
features: radicale
```

### Using inline features from the tester

If there is no profile for your server yet, run the tester and copy the
`features` block from the YAML output directly into your config:

```
caldav-server-tester --caldav-url https://example.com/dav \
--caldav-username alice \
--caldav-password secret \
--format yaml
```

The output contains a `features:` mapping. Paste it under your config section:

```yaml
myserver:
caldav_url: https://example.com/dav
caldav_username: alice
caldav_password: secret
features:
search.time-range.alarm:
support: unsupported
search.unlimited-time-range:
support: quirk
behaviour: accepts-but-ignores-end-date
```

### Extending a named profile with local overrides

If your server is close to a known profile but differs on a few features, use
the `base` key to inherit that profile and then override only what differs:

```yaml
myserver:
caldav_url: https://example.com/dav
caldav_username: alice
caldav_password: secret
features:
base: radicale
search.time-range.alarm:
support: full
```

## Safety

For servers that supports `MKCALENDAR`, a dedicated calendar will be
Expand Down
54 changes: 45 additions & 9 deletions src/caldav_server_tester/caldav_server_tester.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,44 @@ def _find_caldav_test_registry():
]

for root in candidates:
if (root / "tests" / "test_servers" / "__init__.py").exists():
if str(root) not in sys.path:
sys.path.insert(0, str(root))
try:
from tests.test_servers import get_registry

return get_registry()
except ImportError:
pass
ts_init = root / "tests" / "test_servers" / "__init__.py"
if not ts_init.exists():
continue
## Use importlib to load directly from the file path, bypassing
## sys.path resolution entirely. A plain `from tests.test_servers
## import …` fails when another tests/ directory (e.g. this project's
## own tests/ via CWD or editable install) appears earlier in sys.path.
import importlib.util

tests_spec = importlib.util.spec_from_file_location(
"tests",
str(root / "tests" / "__init__.py"),
submodule_search_locations=[str(root / "tests")],
)
ts_spec = importlib.util.spec_from_file_location(
"tests.test_servers",
str(ts_init),
submodule_search_locations=[str(ts_init.parent)],
)
if tests_spec is None or ts_spec is None:
continue

_saved = {k: sys.modules.pop(k) for k in list(sys.modules) if k == "tests" or k.startswith("tests.")}
try:
tests_mod = importlib.util.module_from_spec(tests_spec)
sys.modules["tests"] = tests_mod
tests_spec.loader.exec_module(tests_mod) # type: ignore[union-attr]

ts_mod = importlib.util.module_from_spec(ts_spec)
sys.modules["tests.test_servers"] = ts_mod
ts_spec.loader.exec_module(ts_mod) # type: ignore[union-attr]

return ts_mod.get_registry()
except Exception:
for k in list(sys.modules):
if k == "tests" or k.startswith("tests."):
del sys.modules[k]
sys.modules.update(_saved)

return None

Expand Down Expand Up @@ -206,6 +235,13 @@ def check_server_compatibility(
registry = _find_caldav_test_registry()
if registry is not None:
server = registry.get(name)
if server is None:
## Case-insensitive fallback — registry names may be capitalised
## (e.g. "Radicale") while users naturally type lowercase
for s in registry.all_servers():
if s.name.lower() == name.lower():
server = s
break
if server is not None:
_check_server(
server,
Expand Down
66 changes: 65 additions & 1 deletion tests/test_cli.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
"""Tests for the CLI (caldav_server_tester.py click application)"""

import sys
import types
from unittest.mock import MagicMock, patch

from click.testing import CliRunner

from caldav_server_tester.caldav_server_tester import check_server_compatibility
from caldav_server_tester.caldav_server_tester import (
_find_caldav_test_registry,
check_server_compatibility,
)


class TestCliConfigSection:
Expand Down Expand Up @@ -134,3 +139,62 @@ def test_name_in_registry_uses_registry(self) -> None:
["--name", "knownserver"],
)
mock_check.assert_called_once()

def test_find_registry_works_when_tests_shadowed_in_sys_modules(self) -> None:
"""Registry discovery must succeed even when sys.modules['tests'] points elsewhere.

When the CLI is installed and run, its own tests/ package ends up in
sys.modules before _find_caldav_test_registry() runs. That used to
shadow the caldav project's tests/test_servers and cause the function
to return None.
"""
# Only meaningful if caldav is checked out as source (has tests/test_servers)
from pathlib import Path

import caldav

caldav_root = Path(caldav.__file__).parent.parent
if not (caldav_root / "tests" / "test_servers" / "__init__.py").exists():
return # skip — caldav is not a source checkout

# Inject a fake conflicting 'tests' module that has no test_servers attr
fake_tests = types.ModuleType("tests")
fake_tests.__path__ = ["/some/unrelated/tests"]

saved_tests = {k: sys.modules.pop(k) for k in list(sys.modules) if k == "tests" or k.startswith("tests.")}
sys.modules["tests"] = fake_tests
try:
registry = _find_caldav_test_registry()
finally:
for k in list(sys.modules):
if k == "tests" or k.startswith("tests."):
del sys.modules[k]
sys.modules.update(saved_tests)

assert registry is not None, (
"_find_caldav_test_registry() returned None even though caldav source is available; "
"the 'tests' shadowing bug was not fixed"
)

def test_name_lookup_is_case_insensitive(self) -> None:
"""--name radicale should match a registry entry named 'Radicale'"""
runner = CliRunner()
mock_server = MagicMock()
mock_server.name = "Radicale"

mock_registry = MagicMock()
mock_registry.get.return_value = None # exact match fails
mock_registry.all_servers.return_value = [mock_server]

with (
patch(
"caldav_server_tester.caldav_server_tester._find_caldav_test_registry",
return_value=mock_registry,
),
patch("caldav_server_tester.caldav_server_tester._check_server") as mock_check,
):
runner.invoke(
check_server_compatibility,
["--name", "radicale"],
)
mock_check.assert_called_once()
Loading