|
1 | 1 | # Skill: Create Extension Module |
2 | 2 |
|
3 | | -Scaffold a new karrio extension module that hooks into core without modifying the core modules. |
| 3 | +Scaffold a new karrio extension module that hooks into core without modifying core code. Karrio is modular by design — `modules/core`, `modules/graph`, `modules/admin` form the server core, `modules/connectors/*` are carrier plugins, and everything else (`modules/orders`, `modules/data`, `modules/documents`, `modules/events`, `modules/pricing`, `modules/manager`, …) is an auto-discovered extension module following the pattern below. |
4 | 4 |
|
5 | 5 | ## When to Use |
6 | 6 |
|
7 | | -- Creating new domain logic (e.g., a new resource, workflow, or integration) |
8 | | -- Extending GraphQL schema with new types/mutations |
| 7 | +- New domain logic (a new resource, workflow, or integration) |
| 8 | +- Extending the GraphQL schema with new types / mutations |
9 | 9 | - Adding REST endpoints for new resources |
10 | | -- Registering signal handlers or pre-processing hooks at startup |
| 10 | +- Registering signal handlers, hook functions, or Huey tasks at startup |
| 11 | + |
| 12 | +**Do NOT use this pattern for carrier connectors** — they live in `modules/connectors/*` and are scaffolded via `./bin/cli sdk add-extension`. See `.claude/skills/carrier-integration/SKILL.md` and `.claude/rules/carrier-integration.md`. |
11 | 13 |
|
12 | 14 | ## Prerequisites |
13 | 15 |
|
14 | | -- Read `.claude/rules/extension-patterns.md` first. |
15 | | -- Study existing karrio extension modules for reference: |
16 | | - - `modules/orders/karrio/server/orders/` — REST + GraphQL + signals |
17 | | - - `modules/events/karrio/server/events/` — webhooks, Huey task registration |
18 | | - - `modules/documents/karrio/server/documents/` — auto-discovered REST URLs |
19 | | - - `modules/graph/karrio/server/graph/schemas/base/` — canonical GraphQL module layout |
| 16 | +Read these first: |
| 17 | + |
| 18 | +- `.claude/rules/extension-patterns.md` — namespace-package caveats, dependency-direction rule, hook points table |
| 19 | +- `.claude/skills/django-rest-api/SKILL.md` — REST view / router / serializer conventions |
| 20 | +- `.claude/skills/django-graphql/SKILL.md` — Strawberry GraphQL conventions |
| 21 | + |
| 22 | +Study these existing modules — they are the canonical examples, use them as templates: |
| 23 | + |
| 24 | +- `modules/orders/karrio/server/orders/` — REST + GraphQL + signals (the most complete extension module reference) |
| 25 | +- `modules/documents/karrio/server/documents/` — REST + Huey tasks for document generation |
| 26 | +- `modules/events/karrio/server/events/` — webhook delivery + Huey task registration |
| 27 | +- `modules/data/karrio/server/data/` — import / export module with REST views |
| 28 | +- `modules/pricing/karrio/server/pricing/` — admin-scoped pricing with markups and fees |
20 | 29 |
|
21 | 30 | ## Steps |
22 | 31 |
|
23 | | -### 1. Create Module Directory Structure |
| 32 | +### 1. Create the module directory |
24 | 33 |
|
25 | 34 | ```bash |
26 | 35 | mkdir -p modules/<name>/karrio/server/<name> |
27 | 36 | mkdir -p modules/<name>/karrio/server/settings |
28 | 37 | ``` |
29 | 38 |
|
30 | | -Create `__init__.py` files at each level with namespace package declarations: |
| 39 | +Optional sub-packages (create only what you need): |
31 | 40 |
|
32 | | -```python |
33 | | -# modules/<name>/karrio/__init__.py |
34 | | -__path__ = __import__("pkgutil").extend_path(__path__, __name__) |
35 | | - |
36 | | -# modules/<name>/karrio/server/__init__.py |
37 | | -__path__ = __import__("pkgutil").extend_path(__path__, __name__) |
| 41 | +```bash |
| 42 | +mkdir -p modules/<name>/karrio/server/<name>/serializers |
| 43 | +mkdir -p modules/<name>/karrio/server/<name>/tests |
| 44 | +mkdir -p modules/<name>/karrio/server/<name>/migrations |
| 45 | +mkdir -p modules/<name>/karrio/server/graph/schemas/<name> # only if adding GraphQL |
| 46 | +mkdir -p modules/<name>/karrio/server/admin/schemas/<name> # only if adding admin GraphQL |
38 | 47 | ``` |
39 | 48 |
|
40 | | -### 2. Create AppConfig |
| 49 | +**Namespace-package rule (critical):** the only `__init__.py` files you create are inside the **leaf** directories unique to your module — i.e. `karrio/server/<name>/__init__.py`, `karrio/server/<name>/tests/__init__.py`, `karrio/server/graph/schemas/<name>/__init__.py`, etc. Never create `__init__.py` at `karrio/`, `karrio/server/`, `karrio/server/graph/`, `karrio/server/graph/schemas/`, `karrio/server/admin/`, or `karrio/server/admin/schemas/` — those paths are owned by core modules and must stay as implicit namespace packages. Adding an `__init__.py` there shadows the core package and silently breaks `pkgutil.iter_modules()` discovery. See `.claude/rules/extension-patterns.md` for details. |
| 50 | + |
| 51 | +Verify by looking at `modules/orders/karrio/` — there is no `__init__.py` at `karrio/` or `karrio/server/`, only inside `karrio/server/orders/` and its sub-packages. |
| 52 | + |
| 53 | +### 2. Create the AppConfig |
41 | 54 |
|
42 | 55 | ```python |
43 | 56 | # modules/<name>/karrio/server/<name>/apps.py |
44 | 57 | from django.apps import AppConfig |
| 58 | +from django.utils.translation import gettext_lazy as _ |
| 59 | + |
45 | 60 |
|
46 | 61 | class <Name>Config(AppConfig): |
47 | 62 | name = "karrio.server.<name>" |
| 63 | + verbose_name = _("<Name>") |
48 | 64 | default_auto_field = "django.db.models.BigAutoField" |
49 | 65 |
|
50 | 66 | def ready(self): |
51 | | - # Register hooks here |
52 | | - from karrio.server.<name> import hooks |
53 | | - hooks.register() |
| 67 | + from karrio.server.core import utils |
| 68 | + from karrio.server.<name> import signals # only if you have signals |
| 69 | + |
| 70 | + @utils.skip_on_commands() |
| 71 | + def _init(): |
| 72 | + signals.register_signals() |
| 73 | + |
| 74 | + _init() |
54 | 75 | ``` |
55 | 76 |
|
56 | | -### 3. Create Settings Auto-Discovery |
| 77 | +Karrio conventions (see `modules/orders/karrio/server/orders/apps.py`): |
| 78 | + |
| 79 | +- Wrap registration in `@utils.skip_on_commands()` — this prevents signal / hook registration from running during `migrate`, `collectstatic`, `makemigrations`, etc. Without it you get noisy side-effects (or outright failures) when running management commands on a fresh database. |
| 80 | +- Wrap `verbose_name` in `gettext_lazy` so the admin UI can be translated. |
| 81 | +- Import signals lazily inside `ready()`, never at module top-level — Django is not fully initialized yet when `apps.py` is imported. |
| 82 | + |
| 83 | +### 3. Register signal / hook handlers |
57 | 84 |
|
58 | 85 | ```python |
59 | | -# modules/<name>/karrio/server/settings/<name>.py |
60 | | -from karrio.server.settings.base import * # noqa |
| 86 | +# modules/<name>/karrio/server/<name>/signals.py |
| 87 | +import karrio.server.<name>.models as models |
| 88 | +from django.db.models import signals |
| 89 | +from karrio.server.core import utils |
| 90 | +from karrio.server.core.logging import logger |
| 91 | + |
| 92 | + |
| 93 | +def register_signals(): |
| 94 | + signals.post_save.connect(_on_save, sender=models.Widget) |
| 95 | + signals.post_delete.connect(_on_delete, sender=models.Widget) |
| 96 | + logger.info("Signal registration complete", module="karrio.<name>") |
| 97 | + |
| 98 | + |
| 99 | +@utils.disable_for_loaddata |
| 100 | +def _on_save(sender, instance, created, **kwargs): |
| 101 | + ... |
61 | 102 |
|
62 | | -INSTALLED_APPS += ["karrio.server.<name>"] |
63 | 103 |
|
64 | | -# If module has REST endpoints: |
65 | | -# KARRIO_URLS += ["karrio.server.<name>.urls"] |
| 104 | +@utils.disable_for_loaddata |
| 105 | +def _on_delete(sender, instance, **kwargs): |
| 106 | + ... |
66 | 107 | ``` |
67 | 108 |
|
68 | | -### 4. Add Hook Registration (if extending core behavior) |
| 109 | +`@utils.disable_for_loaddata` prevents signals from firing during fixture loading (test setup, `loaddata`). See the orders module's `signals.py` for a full-featured example that touches related models. |
| 110 | + |
| 111 | +To hook into a core serializer's validation pipeline, append to `pre_process_functions`: |
69 | 112 |
|
70 | 113 | ```python |
71 | | -# modules/<name>/karrio/server/<name>/hooks.py |
72 | | -def register(): |
73 | | - from karrio.server.manager.serializers import ShipmentSerializer |
74 | | - # Append validation/processing hooks |
75 | | - ShipmentSerializer.pre_process_functions.append(your_validator) |
| 114 | +# inside register_signals() or a separate hooks.py |
| 115 | +from karrio.server.manager.serializers import ShipmentSerializer |
| 116 | +from karrio.server.<name> import validators |
| 117 | + |
| 118 | +ShipmentSerializer.pre_process_functions.append(validators.validate_widget_link) |
76 | 119 | ``` |
77 | 120 |
|
78 | | -### 5. Add GraphQL Schema (if needed) |
| 121 | +### 4. Add settings auto-discovery |
79 | 122 |
|
80 | | -For tenant-scoped (base graph): |
81 | | -``` |
82 | | -modules/<name>/karrio/server/graph/schemas/<name>/ |
83 | | -├── __init__.py # Query & Mutation classes |
84 | | -├── types.py |
85 | | -├── mutations.py |
86 | | -└── inputs.py |
| 123 | +```python |
| 124 | +# modules/<name>/karrio/server/settings/<name>.py |
| 125 | +# ruff: noqa: F403, F405, I001 |
| 126 | +from karrio.server.settings.base import * # noqa |
| 127 | + |
| 128 | +INSTALLED_APPS += ["karrio.server.<name>"] |
| 129 | +KARRIO_URLS += ["karrio.server.<name>.urls"] # only if the module has REST endpoints |
87 | 130 | ``` |
88 | 131 |
|
89 | | -For admin-scoped: |
| 132 | +`apps/api/karrio/server/settings/__init__.py` iterates its known module names with `importlib.util.find_spec(...)` and imports `karrio.server.settings.<name>` when the module is installed. Your settings file must follow that exact path — `modules/<name>/karrio/server/settings/<name>.py` — for discovery to work. |
| 133 | + |
| 134 | +If your extension is a completely new module not already listed in `apps/api/karrio/server/settings/__init__.py`, add a matching `find_spec` guard there as a separate PR (this edits core and needs review). Karrio's current guards cover `graph`, `orders`, `data`, `admin`, `huey`, `servicebus`, `main` — check the file when adding a new one. |
| 135 | + |
| 136 | +### 5. Add REST endpoints (if needed) |
| 137 | + |
| 138 | +Follow `.claude/skills/django-rest-api/SKILL.md`. At minimum: |
| 139 | + |
| 140 | +- `modules/<name>/karrio/server/<name>/router.py` — `router = DefaultRouter(trailing_slash=False)` |
| 141 | +- `modules/<name>/karrio/server/<name>/urls.py` — `app_name = "karrio.server.<name>"`, mount `router.urls` at `v1/` |
| 142 | +- `modules/<name>/karrio/server/<name>/views.py` — views extending `karrio.server.core.views.api.GenericAPIView` / `APIView`, with a unique 5-char `ENDPOINT_ID`, `@openapi.extend_schema(...)` annotations, and self-registration via `router.urls.append(path(...))` |
| 143 | + |
| 144 | +### 6. Add GraphQL schemas (if needed) |
| 145 | + |
| 146 | +Follow `.claude/skills/django-graphql/SKILL.md`. Four files under a schemas sub-package: |
| 147 | + |
90 | 148 | ``` |
91 | | -modules/<name>/karrio/server/admin/schemas/<name>/ |
92 | | -├── __init__.py |
93 | | -├── types.py |
94 | | -├── mutations.py |
95 | | -└── inputs.py |
| 149 | +modules/<name>/karrio/server/graph/schemas/<name>/ # tenant-scoped |
| 150 | +├── __init__.py # Query + Mutation classes + extra_types = [] (thin interface) |
| 151 | +├── types.py # @strawberry.type with resolve / resolve_list static methods |
| 152 | +├── inputs.py # @strawberry.input filters + mutation inputs |
| 153 | +└── mutations.py # @strawberry.type mutations with mutate() static methods |
96 | 154 | ``` |
97 | 155 |
|
98 | | -Ensure namespace `__init__.py`: |
99 | | -```python |
100 | | -# modules/<name>/karrio/server/graph/schemas/__init__.py |
101 | | -__path__ = __import__("pkgutil").extend_path(__path__, __name__) |
102 | | -``` |
| 156 | +For admin-scoped (system) schemas use `modules/<name>/karrio/server/admin/schemas/<name>/` instead. `modules/graph/karrio/server/graph/schema.py` auto-discovers both via `pkgutil.iter_modules()` — no registration required. |
| 157 | + |
| 158 | +`extra_types: list = []` is required in every schema `__init__.py` even if empty — `schema.py` reads it unconditionally. |
| 159 | + |
| 160 | +### 7. Add `pyproject.toml` |
103 | 161 |
|
104 | | -### 6. Add pyproject.toml |
| 162 | +Match the orders module's structure (`modules/orders/pyproject.toml`): |
105 | 163 |
|
106 | 164 | ```toml |
| 165 | +[build-system] |
| 166 | +requires = ["setuptools>=61.0"] |
| 167 | +build-backend = "setuptools.build_meta" |
| 168 | + |
107 | 169 | [project] |
108 | | -name = "karrio.server.<name>" |
109 | | -version = "2026.1" |
110 | | -dependencies = ["karrio.server"] |
| 170 | +name = "karrio_server_<name>" |
| 171 | +version = "2026.1.29" # keep in sync with apps/api/karrio/server/VERSION |
| 172 | +description = "Multi-carrier shipping API <name> module" |
| 173 | +readme = "README.md" |
| 174 | +requires-python = ">=3.11" |
| 175 | +license = "LGPL-3.0" |
| 176 | +authors = [ |
| 177 | + {name = "karrio", email = "hello@karrio.io"} |
| 178 | +] |
| 179 | +classifiers = [ |
| 180 | + "Programming Language :: Python :: 3", |
| 181 | +] |
| 182 | +dependencies = [ |
| 183 | + "karrio_server_core", |
| 184 | + "karrio_server_graph", # if adding GraphQL |
| 185 | + "karrio_server_manager", # if importing from manager models |
| 186 | +] |
| 187 | + |
| 188 | +[project.urls] |
| 189 | +Homepage = "https://github.com/karrioapi/karrio" |
| 190 | + |
| 191 | +[tool.setuptools] |
| 192 | +zip-safe = false |
| 193 | +include-package-data = true |
| 194 | + |
| 195 | +[tool.setuptools.package-dir] |
| 196 | +"" = "." |
111 | 197 |
|
112 | 198 | [tool.setuptools.packages.find] |
113 | | -where = ["."] |
| 199 | +exclude = ["tests.*", "tests"] |
| 200 | +namespaces = true |
114 | 201 | ``` |
115 | 202 |
|
116 | | -### 7. Add Tests |
| 203 | +`namespaces = true` is mandatory — karrio uses PEP 420 namespace packages across modules. |
117 | 204 |
|
118 | | -```python |
119 | | -# modules/<name>/karrio/server/<name>/tests.py |
120 | | -from karrio.server.graph.tests import GraphTestCase |
| 205 | +### 8. Add tests |
121 | 206 |
|
122 | | -class Test<Name>Feature(GraphTestCase): |
123 | | - fixtures = ["fixtures"] |
| 207 | +Tests live in a `tests/` **directory** (not a single `tests.py`): |
124 | 208 |
|
125 | | - def test_query(self): |
126 | | - response = self.query(QUERY, operation_name="...") |
127 | | - self.assertResponseNoErrors(response) |
| 209 | +``` |
| 210 | +modules/<name>/karrio/server/<name>/tests/ |
| 211 | +├── __init__.py # re-exports test classes for karrio test discovery |
| 212 | +├── base.py # optional: shared fixture class extending APITestCase / GraphTestCase |
| 213 | +├── test_<resource>.py # REST tests |
| 214 | +└── test_<feature>.py # GraphQL / signal tests |
128 | 215 | ``` |
129 | 216 |
|
130 | | -### 8. Register in Build Requirements |
| 217 | +Use the right base class: |
131 | 218 |
|
132 | | -Add the module to `requirements.build.txt`: |
| 219 | +- REST: `karrio.server.core.tests.APITestCase` (see `modules/core/karrio/server/core/tests/base.py`) |
| 220 | +- GraphQL: `karrio.server.graph.tests.GraphTestCase` (see `modules/graph/karrio/server/graph/tests/base.py`) |
133 | 221 |
|
| 222 | +Both provide `setUpTestData` with a superuser, API token, and seeded carrier connections. |
| 223 | + |
| 224 | +```python |
| 225 | +# modules/<name>/karrio/server/<name>/tests/test_widgets.py |
| 226 | +import json |
| 227 | +from django.urls import reverse |
| 228 | +from rest_framework import status |
| 229 | +from karrio.server.core.tests import APITestCase |
| 230 | + |
| 231 | + |
| 232 | +class TestWidgets(APITestCase): |
| 233 | + def test_create_widget(self): |
| 234 | + url = reverse("karrio.server.<name>:widget-list") |
| 235 | + response = self.client.post(url, {"name": "Hello"}) |
| 236 | + # print(response.data) # DEBUG — remove when passing |
| 237 | + self.assertEqual(response.status_code, status.HTTP_201_CREATED) |
134 | 238 | ``` |
135 | | --e ./modules/<name> |
136 | | -``` |
137 | 239 |
|
138 | | -Without this, the module will NOT be installed in Docker images and schema discovery will silently skip it on staging/production. |
| 240 | +### 9. Register in build / dev / CI |
| 241 | + |
| 242 | +All three files must be updated — missing any of them means the module is silently skipped: |
139 | 243 |
|
140 | | -Also add `karrio.server.<name>.tests` to `bin/run-server-tests` so the test suite runs in CI. |
| 244 | +| File | Purpose | |
| 245 | +| --- | --- | |
| 246 | +| `requirements.build.txt` | Installs the module in prod Docker images. Without this, the module never reaches staging / production. | |
| 247 | +| `requirements.server.dev.txt` | Installs the module in local dev environments. Without this, `./bin/run-server-tests` skips it locally. | |
| 248 | +| `bin/run-server-tests` | Adds `karrio.server.<name>.tests` to the Django test-runner invocation. Without this, tests pass locally but never run in CI. | |
141 | 249 |
|
142 | | -### 9. Install in Development |
| 250 | +Add a line like `-e ./modules/<name>` to both requirements files, and add `karrio.server.<name>.tests \` to the `$KARRIO_TEST --failfast` invocation in `bin/run-server-tests`. |
| 251 | + |
| 252 | +### 10. Install in development |
143 | 253 |
|
144 | 254 | ```bash |
| 255 | +source bin/activate-env |
145 | 256 | pip install -e modules/<name> |
| 257 | +./bin/run-server-tests # full server suite |
| 258 | +karrio test --failfast karrio.server.<name>.tests # just your module |
146 | 259 | ``` |
147 | 260 |
|
148 | 261 | ## Verification |
149 | 262 |
|
150 | 263 | ```bash |
151 | | -# Check module is discovered |
| 264 | +# 1. Module imports |
152 | 265 | python -c "import karrio.server.<name>; print('OK')" |
153 | 266 |
|
154 | | -# Run module tests |
155 | | -karrio test --failfast karrio.server.<name>.tests |
| 267 | +# 2. Settings auto-discovery picked it up (INSTALLED_APPS, KARRIO_URLS) |
| 268 | +python -c "from django.conf import settings; print('karrio.server.<name>' in settings.INSTALLED_APPS)" |
| 269 | + |
| 270 | +# 3. Django migrations generate |
| 271 | +karrio makemigrations <name> |
| 272 | +karrio migrate <name> |
| 273 | + |
| 274 | +# 4. GraphQL schema registration (if added) |
| 275 | +python -c "from karrio.server.graph.schema import schema; print(schema)" |
| 276 | + |
| 277 | +# 5. REST endpoints reachable (if added) |
| 278 | +./bin/start |
| 279 | +curl -H "Authorization: Token <tkn>" http://localhost:5002/api/v1/widgets |
156 | 280 | ``` |
| 281 | + |
| 282 | +If `pkgutil.iter_modules()` silently skips your GraphQL schema, the usual culprits are: |
| 283 | + |
| 284 | +1. A stray `__init__.py` at `modules/<name>/karrio/server/graph/` or `.../schemas/` — delete it. |
| 285 | +2. A circular import between `types.py`, `mutations.py`, and `utils.py` — `schema.py` catches the `ImportError` and moves on (check `karrio.server.core.logging` output). |
| 286 | +3. The module isn't installed — `pip install -e modules/<name>` and confirm in `pip list`. |
| 287 | +4. `requirements.build.txt` missing the `-e ./modules/<name>` line — schema discovery works locally but staging / prod silently omit the module. |
0 commit comments