Skip to content

Commit fb2484c

Browse files
committed
Refactor backend into layered architecture with instance locking, expanded SQL dialect support, and docs
- Replace monolithic client.py files with DBBackend abstract + domain mixins (_base, _lock, _profiles, _settings, _tickets) and a thin DBClient facade - Push persistence logic into model classes as to_model()/from_model()/put_model(); remove convert.py - Add instance locking (InstanceLockModel + backend implementations) to prevent multiple bot instances sharing one DB; raise InstanceAlreadyRunningError on conflict - Auto-inject async driver suffix in SQLDatabaseConfig; add postgresql/mysql/mariadb extras with dialect-specific upsert paths - Add CacheNotReadyError, DatabaseOperationError, InstanceAlreadyRunningError - Add docs infrastructure (zensical, mkdocstrings); rewrite model docstrings in Google style with inline attribute docs and [Name][] cross-references - Bump ruff 0.15.9, SQLAlchemy 2.0.49, beanie 2.0.1; add python-dotenv; move Alembic config into pyproject.toml; remove isort (using solely ruff) Signed-off-by: Taku <45324516+Taaku18@users.noreply.github.com>
1 parent 5a96c9b commit fb2484c

105 files changed

Lines changed: 6019 additions & 3356 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.pre-commit-config.yaml

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ repos:
55
- id: check-ast
66
- id: check-builtin-literals
77
- id: check-case-conflict
8-
- id: check-docstring-first
98
- id: check-illegal-windows-names
109
- id: check-executables-have-shebangs
1110
- id: check-json
@@ -26,13 +25,8 @@ repos:
2625
- id: sort-simple-yaml
2726
- id: trailing-whitespace
2827

29-
- repo: https://github.com/pycqa/isort
30-
rev: 8.0.1
31-
hooks:
32-
- id: isort
33-
3428
- repo: https://github.com/astral-sh/ruff-pre-commit
35-
rev: v0.15.6
29+
rev: v0.15.9
3630
hooks:
3731
- id: ruff-check
3832
args: [ --fix ]

CLAUDE.md

Lines changed: 65 additions & 85 deletions
Original file line numberDiff line numberDiff line change
@@ -1,85 +1,65 @@
1-
# CLAUDE.md
2-
3-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4-
5-
## Project Overview
6-
7-
Modmail v5 is a complete rewrite of the Modmail Discord bot (alpha/WIP). It manages support tickets between users and Discord server staff via DM-based modmail. Requires Python 3.14+, uses `uv` as the package manager.
8-
9-
## Commands
10-
11-
### Setup
12-
```bash
13-
# Install with SQLite backend
14-
uv sync --locked --compile-bytecode --no-default-groups --extra speed --extra sqlite
15-
16-
# Install with MongoDB backend
17-
uv sync --locked --compile-bytecode --no-default-groups --extra speed --extra mongodb
18-
```
19-
20-
### Run
21-
```bash
22-
uv run python start.py
23-
# or
24-
python -m modmail
25-
```
26-
27-
### Lint & Format
28-
```bash
29-
uv run pre-commit run --all-files # Run all pre-commit hooks
30-
uv run ruff check --fix modmail tests
31-
uv run ruff format modmail tests
32-
uv run isort modmail tests
33-
```
34-
35-
### Type Check
36-
```bash
37-
uv run pyright
38-
```
39-
40-
### Tests (Not Yet Implemented)
41-
```bash
42-
uv run pytest
43-
uv run pytest tests/test_utils.py # Run a single test file
44-
uv run pytest -k "test_name" # Run a specific test
45-
uv run pytest --cov=modmail --cov-report html
46-
uv run tox # Run all tox environments (lint, type, py3.14)
47-
```
48-
49-
## Architecture
50-
51-
### Module Structure
52-
```
53-
modmail/
54-
├── backends/ # Database abstraction layer
55-
│ ├── common/ # Abstract base class & shared models
56-
│ ├── mongodb/ # Beanie ODM implementation
57-
│ └── sql/ # SQLAlchemy + aiosqlite implementation
58-
├── config/ # Pydantic config models + YAML loader
59-
├── core/ # Bot class, translator, permissions
60-
│ └── internals/ # Cog base, command types, UI views, embeds
61-
├── cogs/
62-
│ ├── modmail/ # Core ticket logic (commands + listeners)
63-
│ └── utility/ # about, status, profile commands
64-
└── locales/ # Fluent FTL translation files (en, de)
65-
```
66-
67-
### Key Design Patterns
68-
69-
**Database Abstraction**: `DBClientBase` in `backends/common/client_base.py` defines the interface. Both SQL and MongoDB clients implement this. The active backend is selected via `config.yaml`.
70-
71-
**Ticket Flow**:
72-
1. User DMs bot → `cogs/modmail/listeners/dm_receive.py` intercepts
73-
2. `core/internals/staff_guild.py` creates a channel or forum thread in the staff server
74-
3. Database stores ticket state via the active backend client
75-
4. Staff reply via commands in `cogs/modmail/commands/`
76-
77-
**Localization**: Fluent FTL format (`locales/en/main.ftl`). All user-facing strings go through `core/translator.py`. Commands use `LazyHybridCommand` (`core/internals/command.py`) supporting both prefix and slash commands.
78-
79-
**Cog Base**: Custom `Cog` class in `core/internals/cog.py` with enhanced send/reply helpers and access to the translator.
80-
81-
### Config
82-
Copy `config.yaml.example``config.yaml`. Required fields: `bot.token`, `bot.staff_server_id`, `log_url`. Database backend is configured under the `database` key.
83-
84-
### Testing Notes
85-
Tests are currently undergoing a rewrite. Pyright runs in strict mode but excludes the `tests/` directory. Pre-commit hooks enforce isort + ruff formatting on every commit.
1+
## Overview
2+
3+
Modmail is a Discord DM-based support ticket bot. It routes user DMs into staff channels/threads and supports interchangeable MongoDB and SQL backends. The codebase is async-first and structured as a Python package under `modmail/`.
4+
5+
## Structure
6+
7+
- Core runtime and Discord integration live in `modmail/core/` (bot, permissions, translator, internal helpers).
8+
- Persistence is abstracted behind `modmail/backends/common/` (`DBBackend`, `DBClient`, shared models) with concrete implementations in `modmail/backends/mongodb/` and `modmail/backends/sql/`.
9+
- Configuration models and loading logic live in `modmail/config/`.
10+
- Discord-facing behavior is in `modmail/cogs/modmail/` (modmail flows) and `modmail/cogs/utility/` (utility commands).
11+
- Localization resources are in `modmail/locales/<lang>/main.ftl`, wired via `modmail/core/translator.py`.
12+
- Support modules: `modmail/logging.py`, `modmail/utils.py`, `modmail/errors.py`, `modmail/enum.py`, and `modmail/__main__.py` as entrypoint.
13+
14+
## Tickets and Data
15+
16+
- DMs are received by listeners in `modmail/cogs/modmail/listeners/`, which create or look up tickets via `StaffGuild` and the database client.
17+
- Staff interact through commands in `modmail/cogs/modmail/commands/` (e.g. setup, reply, close).
18+
- Ticket, message, user/profile, settings, activity, and instance-lock models are defined in `modmail/backends/common/models/` and mirrored in both backends. When changing persistence, update **both** Mongo and SQL implementations and keep `DBBackend` the single interface.
19+
20+
## Configuration and Environment
21+
22+
- Configuration is loaded from YAML via `modmail/config/loader.py` into Pydantic models under `modmail/config/models/`.
23+
- Use configuration (not hardcoded values) for bot token, DB URIs, guild/channel IDs, and locale. Never commit real secrets; only example configs belong in VCS.
24+
25+
## Docs and Docstrings
26+
27+
Docs are built with **Zensical** + **mkdocstrings-python** (griffe). Config: `zensical.toml`. The `docs/` directory contains `:::` stubs — source docstrings are the API reference. Use **Google style**.
28+
29+
Attribute descriptions — write for the *user*, not the implementer:
30+
- No trivial descriptions: `bot_id: The bot ID.` is bad; `bot_id: Discord application ID.` is the minimum.
31+
- No implementation details (ORM internals, storage format, index notes, loading strategy) — these belong in the class docstring, not per-attribute.
32+
- No semicolons; use parenthetical style e.g. `(``None`` if unset)`.
33+
- Stay within the 115-char line limit.
34+
35+
Cross-references — use `[Name][]` (**not** RST `:class:`Name`` — griffe renders it as literal text):
36+
- `` [`ClassName`][] ``, `` [`DBBackend`][modmail.backends.common.db_backend.DBBackend] ``, `` [`discord.Guild`][] ``
37+
38+
Unrecognized section names become admonition boxes: `Note:`, `Warning:`, `Tip:`, `Danger:`, etc.
39+
40+
## General Behaviour
41+
42+
- Be conservative with tokens and tool calls: read only what's needed, don't re-read files already in context, avoid unnecessary exploration.
43+
- Use web search freely — it's cheap and preferred over guessing at external API behaviour.
44+
45+
## Workflows for Claude Code
46+
47+
When working in this repo:
48+
49+
1. Restate the goal and identify relevant modules using the structure above.
50+
2. Consult the corresponding code/docstrings; for external packages or unfamiliar problems, search public API docs or the web before coding.
51+
3. Propose a short plan listing affected files, then implement in small, reviewable changes.
52+
4. Use the existing patterns:
53+
- Go through `DBClient` for persistence.
54+
- Use enums from `modmail/enum.py` and errors from `modmail/errors.py`.
55+
- Keep async I/O non-blocking and follow existing cog patterns.
56+
- Use Python **3.14+ syntax** in line with the existing code.
57+
5. Run tools before considering work done:
58+
- `uv run ruff check --fix && uv run ruff format`
59+
- `uv run pyright`
60+
61+
## Quirks and Warnings
62+
63+
- Tests in this project are currently **obsolete**; do not use them as a source of truth.
64+
- Treat migrations and instance-lock logic with care; avoid destructive schema or data changes without explicit human confirmation and a migration plan.
65+
- No suppression comments: never add `# noqa`, `# type: ignore`, or `# pyright: ignore`; leave errors for a human to decide.

README.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,12 @@
99
1010
1. Copy `config.yaml.example` and rename it to `config.yaml`, then edit it with your settings
1111
2. Install [uv](https://docs.astral.sh/uv/getting-started/installation/), then run `uv sync --locked --compile-bytecode --no-default-groups --extra speed --extra DBTYPE` to install the dependencies
12-
- Replace `DBTYPE` with `mongodb` or `sqlite`, only these two are supported at the moment
12+
- Replace `DBTYPE` with one of the supported database backends:
13+
- `mongodb` — MongoDB
14+
- `sqlite` — SQLite (local file, no server needed)
15+
- `postgresql` — PostgreSQL *(untested)*
16+
- `mysql` — MySQL *(untested)*
17+
- `mariadb` — MariaDB *(untested)*
1318
3. Start the bot with `uv run python start.py`
1419

1520
Please note that the database structure may change at any time, and database migrations between v5 alpha development versions are not available.

config.yaml.example

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -88,12 +88,19 @@ database_type: "mongodb"
8888

8989
# Configurations for the SQL database type. Only required if database_type is sql.
9090
sql_config:
91-
# Required: The SQL connection URI, supported SQL databases are:
92-
# * SQLite
93-
# Example for SQLite: "sqlite+aiosqlite:///modmail.db"
94-
# Example for PostgreSQL (does not work yet): "postgresql+asyncpg://user:password@localhost/modmail"
95-
# For SQLite, in-memory database is not supported, you must use a file.
96-
uri: "sqlite+aiosqlite:///modmail.db"
91+
# Required: The SQL connection URI.
92+
# The async driver suffix is injected automatically, so you may omit it.
93+
# Supported dialects and example URIs:
94+
# SQLite (local file, no server needed):
95+
# "sqlite:///modmail.db"
96+
# PostgreSQL:
97+
# "postgresql://user:password@localhost/modmail"
98+
# MySQL:
99+
# "mysql://user:password@localhost/modmail"
100+
# MariaDB:
101+
# "mariadb://user:password@localhost/modmail"
102+
# Note: SQLite in-memory databases are not supported; you must use a file.
103+
uri: "sqlite:///modmail.db"
97104

98105
# Configurations for the MongoDB database type. Only required if database_type is mongodb.
99106
mongodb_config:

docs/index.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Modmail
2+
3+
A Discord DM-based support ticket bot, built for reliability and flexibility.
4+
5+
Modmail v5 routes user DMs into dedicated staff channels or forum threads, keeping support organized and auditable. It ships two interchangeable database backends and a full Fluent i18n system.
6+
7+
## Key features
8+
9+
- **Dual-mode tickets** — forum-thread or channel-per-ticket layout
10+
- **Swappable backends** — MongoDB (Beanie ODM) or SQL (SQLAlchemy + Alembic), selected at runtime
11+
- **Hybrid commands** — prefix and slash commands from the same definition via `LazyHybridCommand`
12+
- **Fluent i18n** — all user-facing strings in FTL translation files, zero hardcoded text
13+
- **Strict config** — Pydantic models validated at startup, clear errors on misconfiguration
14+
- **Instance locking** — distributed lock prevents split-brain when multiple bot processes start
15+
16+
## API reference
17+
18+
| Section | Description |
19+
|-----------------------------------------|-------------------------------------------------------------------------------------|
20+
| [Backends](reference/backends/index.md) | DB abstraction layer — `DBClient`, `DBBackend`, and the MongoDB/SQL implementations |
21+
| [Cogs](reference/cogs/index.md) | Ticket commands, reply/close workflow, and event listeners |
22+
| [Config](reference/config.md) | Pydantic settings models and YAML loader |
23+
| [Core](reference/core.md) | Bot class, translator, permission system, and shared internals |
24+
| [Misc](reference/misc.md) | Enums, errors, logging, and utilities |
25+
26+
!!! tip "Setup & configuration"
27+
See the [repository](https://github.com/modmail-dev/modmail) for installation instructions, `config.yaml` reference, and deployment guides.
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# DBBackend
2+
3+
::: modmail.backends.common.db_backend
4+
options:
5+
filters:
6+
- "!^_[^_]"
7+
- "!^__all__$"
8+
- "!^logger$"
9+
- "^_try_insert_lock$"
10+
- "^_read_lock$"
11+
- "^_try_takeover_lock$"
12+
- "^_delete_lock$"
13+
- "^_update_heartbeat$"
14+
- "^_acquire_instance_lock$"
15+
- "^_connect$"
16+
- "^_disconnect$"
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# DBClient
2+
3+
::: modmail.backends.common.db_client
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Common
2+
3+
Shared interfaces, base classes, and models used by all backend implementations.
4+
5+
| Page | Contents |
6+
|-------------------------|------------------------------------------------------------------------------------------------------------------------|
7+
| [DBBackend](backend.md) | [`DBBackend`][modmail.backends.common.db_backend.DBBackend]{ data-preview } abstract base and instance-lock primitives |
8+
| [DBClient](client.md) | [`DBClient`][modmail.backends.common.db_client.DBClient]{ data-preview } — the public API and in-memory cache layer |
9+
| [Models](models.md) | Shared Pydantic models for settings, profiles, tickets, and messages |
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Models
2+
3+
Shared Pydantic models used by all database backend implementations.
4+
5+
::: modmail.backends.common.models.settings_model
6+
7+
::: modmail.backends.common.models.activity_model
8+
9+
::: modmail.backends.common.models.profile_model
10+
11+
::: modmail.backends.common.models.ticket_model
12+
13+
::: modmail.backends.common.models.ticket_user_model
14+
15+
::: modmail.backends.common.models.ticket_message_model
16+
17+
::: modmail.backends.common.models.ticket_dm_message_model
18+
19+
::: modmail.backends.common.models.instance_lock_model

docs/reference/backends/index.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Backends
2+
3+
The database abstraction layer consists of two collaborating abstractions:
4+
5+
- [`DBClient`][modmail.backends.common.db_client.DBClient]{ data-preview } — owns all in-memory caches and the public API consumed by the rest of the bot. Delegates persistence work to a [`DBBackend`][modmail.backends.common.db_backend.DBBackend]{ data-preview }.
6+
- [`DBBackend`][modmail.backends.common.db_backend.DBBackend]{ data-preview } — abstract base class for backend implementations, composed from four domain mixins (lock, settings, profiles, tickets).
7+
8+
The active backend is selected at startup from `config.yaml` and is never swapped at runtime.
9+
10+
## Implementations
11+
12+
| Backend | Module | Storage |
13+
|-----------------------------|----------------------------|-----------------------------------------------------------|
14+
| [Common](common/index.md) | `modmail.backends.common` | Shared interfaces and models used by both implementations |
15+
| [MongoDB](mongodb/index.md) | `modmail.backends.mongodb` | Beanie ODM on top of Motor / MongoDB |
16+
| [SQL](sql/index.md) | `modmail.backends.sql` | SQLAlchemy 2 async + Alembic |
17+
18+
!!! note "Instance lock"
19+
All backends implement a distributed instance lock (`_lock` mixin + [`InstanceLockModel`][modmail.backends.common.models.instance_lock_model.InstanceLockModel]{ data-preview }) to prevent two processes from running against the same database simultaneously.

0 commit comments

Comments
 (0)