Skip to content

Commit 14d5e83

Browse files
kesmit13claude
andcommitted
Update docs and plugin/ terminology after collocated→plugin rename
Update ARCHITECTURE.md with plugin/ subpackage in the package tree, three-mode execution diagram, Plugin Server CLI section with all 7 arguments and env variables, and Appendix A/D entries. Update CONTRIBUTING.md with plugin server CLI example for manual testing. Update README.md UDF bullet to mention plugin-mode deployment. Replace remaining "collocated" references in plugin/ source files: logger names (collocated.* → plugin.*), docstrings, and argparse description. Files outside plugin/ (config.py, asgi.py, mmap.py) correctly use "collocated" for their own functionality and are left unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 5de453c commit 14d5e83

File tree

9 files changed

+80
-32
lines changed

9 files changed

+80
-32
lines changed

ARCHITECTURE.md

Lines changed: 65 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,14 @@ singlestoredb/
9999
│ ├── mmap.py # Memory-mapped execution
100100
│ ├── json.py # JSON serialization
101101
│ ├── rowdat_1.py # ROWDAT_1 format
102-
│ └── arrow.py # Apache Arrow format
102+
│ ├── arrow.py # Apache Arrow format
103+
│ └── plugin/ # Plugin UDF server (Unix socket)
104+
│ ├── __main__.py # CLI entry point
105+
│ ├── server.py # Socket server with thread/process pool
106+
│ ├── connection.py # Connection handling
107+
│ ├── control.py # Control protocol
108+
│ ├── registry.py # Function registry & discovery
109+
│ └── wasm.py # WASM component interface
103110
104111
├── ai/ # AI/ML integration
105112
│ ├── chat.py # Chat completion factory
@@ -603,9 +610,9 @@ Located in `singlestoredb/fusion/handlers/`:
603610
## External Functions (UDFs)
604611

605612
The functions module (`singlestoredb/functions/`) enables deploying Python functions
606-
as SingleStore external functions. UDF servers can be deployed as HTTP using
607-
an ASGI application or a collocated socket server that uses mmap files to
608-
transfer data.
613+
as SingleStore external functions. UDF servers can be deployed in three modes: as HTTP using an ASGI application
614+
(remote), a memory-mapped collocated server (lowest latency), or as a plugin
615+
server using Unix sockets with a thread/process pool (CLI-driven).
609616

610617
### Architecture
611618

@@ -629,6 +636,7 @@ transfer data.
629636
├─────────────────────────────────────────────────────────────────────┤
630637
│ asgi.py │ HTTP server via ASGI (Uvicorn; JSON or ROWDAT_1) │
631638
│ mmap.py │ Memory-mapped shared memory (collocated; ROWDAT_1) │
639+
│ plugin/ │ Plugin UDF server (Unix socket + thread/process pool) │
632640
│ json.py │ JSON serialization over HTTP │
633641
│ rowdat_1.py│ ROWDAT_1 binary format │
634642
│ arrow.py │ Apache Arrow columnar format │
@@ -659,6 +667,33 @@ python -m singlestoredb.functions.ext.asgi \
659667
my_functions
660668
```
661669

670+
### Plugin Server CLI
671+
672+
The plugin server can be launched via the CLI for Unix socket-based UDF serving:
673+
674+
```bash
675+
# Launch the plugin server
676+
python -m singlestoredb.functions.ext.plugin \
677+
--plugin-name myfuncs \
678+
--search-path /home/user/libs \
679+
--socket /tmp/my-udf.sock
680+
681+
# Or use the console_scripts entry point
682+
python-udf-server --plugin-name myfuncs
683+
```
684+
685+
**CLI Arguments:**
686+
687+
| Argument | Env Variable | Default | Description |
688+
|----------|-------------|---------|-------------|
689+
| `--plugin-name` | `PLUGIN_NAME` | (required) | Python module to import |
690+
| `--search-path` | `PLUGIN_SEARCH_PATH` | `""` | Colon-separated search dirs for the module |
691+
| `--socket` | `PLUGIN_SOCKET_PATH` | auto-generated | Unix socket path |
692+
| `--n-workers` | `PLUGIN_N_WORKERS` | `0` (CPU count) | Worker threads/processes |
693+
| `--max-connections` | `PLUGIN_MAX_CONNECTIONS` | `32` | Socket backlog |
694+
| `--log-level` | `PLUGIN_LOG_LEVEL` | `info` | Logging level (debug/info/warning/error) |
695+
| `--process-mode` | `PLUGIN_PROCESS_MODE` | `process` | Concurrency mode: `thread` or `process` |
696+
662697
### Type Mapping
663698

664699
The `signature.py` module maps Python types to SQL types:
@@ -682,29 +717,30 @@ The `signature.py` module maps Python types to SQL types:
682717
┌─────────────────────────────────────────────────────────────────────┐
683718
│ SingleStore Database │
684719
└─────────────────────────────────────────────────────────────────────┘
685-
│ │
686-
│ ASGI/HTTP │ Memory-mapped
687-
│ (remote) │ (collocated)
688-
▼ ▼
689-
┌─────────────┐ ┌─────────────┐
690-
│ asgi.py │ │ mmap.py │
691-
│ Uvicorn │ │ Shared │
692-
│ HTTP/2 │ │ Memory │
693-
└─────────────┘ └─────────────┘
694-
│ │
695-
└────────────────────┘
696-
697-
698-
┌─────────────────┐
699-
│ Python UDF │
700-
│ Functions │
701-
└─────────────────┘
720+
│ │
721+
│ ASGI/HTTP │ Memory-mapped │ Plugin
722+
│ (remote) │ (collocated) │ (Unix socket)
723+
▼ ▼
724+
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
725+
│ asgi.py │ │ mmap.py │ │ plugin/ │
726+
│ Uvicorn │ │ Shared │ │ Unix sock │
727+
│ HTTP/2 │ │ Memory │ │ + pool │
728+
└─────────────┘ └─────────────┘ └─────────────┘
729+
│ │
730+
└────────────────────┴────────────────────
731+
732+
733+
┌─────────────────┐
734+
│ Python UDF │
735+
│ Functions │
736+
└─────────────────┘
702737
```
703738

704739
| Mode | File | Use Case |
705740
|------|------|----------|
706741
| ASGI | `asgi.py` | Remote execution via HTTP, scalable |
707742
| Memory-mapped | `mmap.py` | Collocated execution, lowest latency |
743+
| Plugin | `plugin/` | Unix socket server, thread/process pool, CLI-driven |
708744
| JSON | `json.py` | Simple serialization, debugging |
709745
| ROWDAT_1 | `rowdat_1.py` | Binary format, efficient |
710746
| Arrow | `arrow.py` | Columnar format, analytics |
@@ -1043,6 +1079,13 @@ with free_tier.start() as server:
10431079
| `SINGLESTOREDB_MANAGEMENT_TOKEN` | Management API token | None |
10441080
| `SINGLESTORE_LICENSE` | License key for Docker | None |
10451081
| `USE_DATA_API` | Use HTTP API for tests | 0 |
1082+
| `PLUGIN_NAME` | Plugin server: Python module to import | None |
1083+
| `PLUGIN_SEARCH_PATH` | Plugin server: colon-separated module search dirs | `""` |
1084+
| `PLUGIN_SOCKET_PATH` | Plugin server: Unix socket path | auto-generated |
1085+
| `PLUGIN_N_WORKERS` | Plugin server: worker count (0 = CPU count) | `0` |
1086+
| `PLUGIN_MAX_CONNECTIONS` | Plugin server: socket backlog | `32` |
1087+
| `PLUGIN_LOG_LEVEL` | Plugin server: logging level | `info` |
1088+
| `PLUGIN_PROCESS_MODE` | Plugin server: `thread` or `process` | `process` |
10461089

10471090
### B. Cursor Type Matrix
10481091

@@ -1096,4 +1139,5 @@ Feature options:
10961139
| Management API | `singlestoredb/management/workspace.py` |
10971140
| Fusion handlers | `singlestoredb/fusion/handler.py` |
10981141
| UDF decorator | `singlestoredb/functions/decorator.py` |
1142+
| Plugin UDF server | `singlestoredb/functions/ext/plugin/server.py` |
10991143
| Test fixtures | `singlestoredb/pytest.py` |

CONTRIBUTING.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,10 @@ pytest -v --cov=singlestoredb.connection singlestoredb/tests/test_connection.py
182182
# Test UDF functionality
183183
pytest singlestoredb/tests/test_udf.py
184184

185+
# Manual testing of the plugin UDF server
186+
python -m singlestoredb.functions.ext.plugin \
187+
--plugin-name myfuncs --search-path /path/to/modules
188+
185189
# Test against specific server (skips Docker)
186190
SINGLESTOREDB_URL=admin:pass@localhost:3306 pytest -v singlestoredb/tests
187191

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ analytics and vector search.
1818
- **Vector Store**: Pinecone-compatible vector database API for similarity search
1919
applications with built-in connection pooling
2020
- **User-Defined Functions**: Deploy Python functions as SingleStore UDFs with
21-
automatic type mapping
21+
automatic type mapping (HTTP/ASGI or plugin-mode via CLI)
2222
- **SQLAlchemy Support**: Integrate with SQLAlchemy through the optional
2323
`sqlalchemy-singlestoredb` adapter
2424
- **Fusion SQL**: Extend SQL with custom client-side command handlers
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
"""High-performance collocated Python UDF server for SingleStoreDB."""
1+
"""High-performance plugin Python UDF server for SingleStoreDB."""

singlestoredb/functions/ext/plugin/__main__.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
"""
2-
CLI entry point for the collocated Python UDF server.
2+
CLI entry point for the plugin Python UDF server.
33
44
Usage::
55
@@ -21,13 +21,13 @@
2121
from .registry import setup_logging
2222
from .server import Server
2323

24-
logger = logging.getLogger('collocated')
24+
logger = logging.getLogger('plugin')
2525

2626

2727
def main(argv: Any = None) -> None:
2828
parser = argparse.ArgumentParser(
2929
prog='python -m singlestoredb.functions.ext.plugin',
30-
description='High-performance collocated Python UDF server',
30+
description='High-performance plugin Python UDF server',
3131
)
3232
parser.add_argument(
3333
'--plugin-name',

singlestoredb/functions/ext/plugin/connection.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
if TYPE_CHECKING:
2828
from .server import SharedRegistry
2929

30-
logger = logging.getLogger('collocated.connection')
30+
logger = logging.getLogger('plugin.connection')
3131

3232
# Protocol constants
3333
PROTOCOL_VERSION = 1

singlestoredb/functions/ext/plugin/control.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
if TYPE_CHECKING:
1616
from .server import SharedRegistry
1717

18-
logger = logging.getLogger('collocated.control')
18+
logger = logging.getLogger('plugin.control')
1919

2020

2121
@dataclass

singlestoredb/functions/ext/plugin/registry.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
This module contains the core FunctionRegistry class (moved from
55
wasm/udf_handler.py) plus standalone call_function() and
66
describe_functions_json() helpers. Both the WASM handler and the
7-
collocated server use these directly.
7+
plugin server use these directly.
88
"""
99
import inspect
1010
import json
@@ -450,7 +450,7 @@ def call_function(
450450
"""Call a registered UDF by name using the C accelerator or fallback.
451451
452452
This is the hot-path function used by both the WASM handler and
453-
the collocated server.
453+
the plugin server.
454454
"""
455455
if name not in registry.functions:
456456
raise ValueError(f'unknown function: {name}')

singlestoredb/functions/ext/plugin/server.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
from .connection import handle_connection
2929
from .registry import FunctionRegistry
3030

31-
logger = logging.getLogger('collocated.server')
31+
logger = logging.getLogger('plugin.server')
3232

3333

3434
def _read_pipe_message(fd: int) -> Optional[bytes]:
@@ -151,7 +151,7 @@ def _build_fresh_registry(self) -> FunctionRegistry:
151151

152152

153153
class Server:
154-
"""Collocated UDF server with Unix socket + thread pool."""
154+
"""Plugin UDF server with Unix socket + thread pool."""
155155

156156
def __init__(self, config: Dict[str, Any]) -> None:
157157
self.config = config

0 commit comments

Comments
 (0)