Skip to content

Commit 7896048

Browse files
huangyiirenehuangyiirene
authored
Merge pull request #80 from objectstack-ai/copilot/add-database-support-options
2 parents 6a10f1e + a447d14 commit 7896048

File tree

14 files changed

+2869
-5
lines changed

14 files changed

+2869
-5
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,16 @@ jobs:
1010
build:
1111
runs-on: ubuntu-latest
1212
timeout-minutes: 15
13+
services:
14+
redis:
15+
image: redis
16+
ports:
17+
- 6379:6379
18+
options: >-
19+
--health-cmd "redis-cli ping"
20+
--health-interval 10s
21+
--health-timeout 5s
22+
--health-retries 5
1323
strategy:
1424
matrix:
1525
node-version: [18.x, 20.x]

‎README.md‎

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -112,22 +112,39 @@ main();
112112

113113
ObjectQL isolates the "What" (Query) from the "How" (Execution).
114114

115-
### SQL Driver (`@objectql/driver-sql`)
115+
### Official Drivers
116+
117+
#### SQL Driver (`@objectql/driver-sql`)
116118

117119
* Supports PostgreSQL, MySQL, SQLite, SQL Server.
118120
* **Smart Hybrid Mode:** Automatically maps defined fields to SQL columns and undefined fields to a `_jsonb` column. This gives you the speed of SQL with the flexibility of NoSQL.
119121

120-
### Mongo Driver (`@objectql/driver-mongo`)
122+
#### Mongo Driver (`@objectql/driver-mongo`)
121123

122124
* Native translation to Aggregation Pipelines.
123125
* High performance for massive datasets.
124126

125-
### SDK / Remote Driver (`@objectql/sdk`)
127+
#### SDK / Remote Driver (`@objectql/sdk`)
126128

127129
* **The Magic:** You can run ObjectQL in the **Browser**.
128130
* Instead of connecting to a DB, it connects to an ObjectQL Server API.
129131
* The API usage remains exactly the same (`repo.find(...)`), but it runs over HTTP.
130132

133+
### Extensibility
134+
135+
ObjectQL's driver architecture supports custom database implementations. Potential databases include:
136+
137+
* **Key-Value Stores:** Redis, Memcached, etcd
138+
* **Search Engines:** Elasticsearch, OpenSearch, Algolia
139+
* **Graph Databases:** Neo4j, ArangoDB
140+
* **Time-Series:** InfluxDB, TimescaleDB
141+
* **Cloud Databases:** DynamoDB, Firestore, Cosmos DB
142+
143+
**Want to add support for your database?** Check out:
144+
* [Driver Extensibility Guide](./docs/guide/drivers/extensibility.md) - Lists potential databases and their implementation complexity
145+
* [Implementing Custom Drivers](./docs/guide/drivers/implementing-custom-driver.md) - Step-by-step guide with code examples
146+
* [Redis Driver Example](./packages/drivers/redis/) - Reference implementation
147+
131148
---
132149

133150
## 🛠️ Validation & Logic

‎docs/guide/drivers/extensibility.md‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
# Driver Extensibility Guide
2+
3+
ObjectQL is designed to support multiple database backends through its **Driver** abstraction layer. This guide explains how to extend ObjectQL with additional database types.
4+
5+
## Current Official Drivers
6+
7+
ObjectQL currently provides official support for:
8+
9+
| Driver | Package | Databases Supported | Status |
10+
|--------|---------|---------------------|--------|
11+
| **SQL Driver** | `@objectql/driver-sql` | PostgreSQL, MySQL, SQLite, SQL Server | ✅ Production Ready |
12+
| **MongoDB Driver** | `@objectql/driver-mongo` | MongoDB 4.0+ | ✅ Production Ready |
13+
| **SDK/Remote Driver** | `@objectql/sdk` | HTTP-based ObjectQL servers | ✅ Production Ready |
14+
15+
## Potential Database Types for Extension
16+
17+
ObjectQL's Driver interface can theoretically support any database system. Here are common database types that could be implemented:
18+
19+
### Key-Value Stores
20+
21+
| Database | Use Case | Implementation Complexity |
22+
|----------|----------|--------------------------|
23+
| **Redis** | Caching, real-time data, pub/sub | 🟢 Low - Simple key-value operations |
24+
| **Memcached** | Distributed caching | 🟢 Low - Basic get/set operations |
25+
| **etcd** | Configuration management, service discovery | 🟡 Medium - Hierarchical keys |
26+
27+
### Document Databases
28+
29+
| Database | Use Case | Implementation Complexity |
30+
|----------|----------|--------------------------|
31+
| **CouchDB** | Multi-master replication, offline-first | 🟡 Medium - Similar to MongoDB |
32+
| **Firebase/Firestore** | Real-time sync, mobile apps | 🟡 Medium - Cloud-native features |
33+
| **RavenDB** | .NET integration, ACID transactions | 🟡 Medium - Advanced indexing |
34+
35+
### Wide Column Stores
36+
37+
| Database | Use Case | Implementation Complexity |
38+
|----------|----------|--------------------------|
39+
| **Cassandra** | High availability, time-series data | 🔴 High - Distributed architecture |
40+
| **HBase** | Hadoop ecosystem, big data | 🔴 High - Complex data model |
41+
| **DynamoDB** | AWS-native, serverless | 🟡 Medium - Single-table design patterns |
42+
43+
### Search Engines
44+
45+
| Database | Use Case | Implementation Complexity |
46+
|----------|----------|--------------------------|
47+
| **Elasticsearch** | Full-text search, analytics | 🟡 Medium - Query DSL mapping |
48+
| **OpenSearch** | Fork of Elasticsearch, AWS managed | 🟡 Medium - Similar to Elasticsearch |
49+
| **Algolia** | Hosted search, real-time indexing | 🟢 Low - REST API based |
50+
| **Meilisearch** | Typo-tolerant search | 🟢 Low - Simple REST API |
51+
52+
### Graph Databases
53+
54+
| Database | Use Case | Implementation Complexity |
55+
|----------|----------|--------------------------|
56+
| **Neo4j** | Social networks, recommendation engines | 🔴 High - Cypher query language |
57+
| **ArangoDB** | Multi-model (graph + document) | 🟡 Medium - AQL query language |
58+
| **OrientDB** | Multi-model graph database | 🟡 Medium - SQL-like syntax |
59+
60+
### Time-Series Databases
61+
62+
| Database | Use Case | Implementation Complexity |
63+
|----------|----------|--------------------------|
64+
| **InfluxDB** | Metrics, IoT, monitoring | 🟡 Medium - Time-based queries |
65+
| **TimescaleDB** | PostgreSQL extension for time-series | 🟢 Low - SQL compatible |
66+
| **Prometheus** | Monitoring and alerting | 🟡 Medium - PromQL query language |
67+
68+
### NewSQL Databases
69+
70+
| Database | Use Case | Implementation Complexity |
71+
|----------|----------|--------------------------|
72+
| **CockroachDB** | Distributed SQL, PostgreSQL compatible | 🟢 Low - PostgreSQL protocol |
73+
| **TiDB** | MySQL compatible, horizontal scaling | 🟢 Low - MySQL protocol |
74+
| **YugabyteDB** | PostgreSQL compatible, cloud-native | 🟢 Low - PostgreSQL protocol |
75+
76+
### Cloud-Native Databases
77+
78+
| Database | Use Case | Implementation Complexity |
79+
|----------|----------|--------------------------|
80+
| **AWS DynamoDB** | Serverless, auto-scaling | 🟡 Medium - NoSQL patterns |
81+
| **Google Cloud Firestore** | Real-time sync, mobile | 🟡 Medium - Document-based |
82+
| **Azure Cosmos DB** | Multi-model, global distribution | 🟡 Medium - Multiple APIs |
83+
| **Supabase** | PostgreSQL-as-a-service | 🟢 Low - PostgreSQL protocol |
84+
| **PlanetScale** | MySQL-compatible, serverless | 🟢 Low - MySQL protocol |
85+
86+
### Columnar Databases
87+
88+
| Database | Use Case | Implementation Complexity |
89+
|----------|----------|--------------------------|
90+
| **ClickHouse** | OLAP, analytics, data warehousing | 🔴 High - Column-oriented queries |
91+
| **Apache Druid** | Real-time analytics | 🔴 High - Complex aggregations |
92+
93+
### Embedded Databases
94+
95+
| Database | Use Case | Implementation Complexity |
96+
|----------|----------|--------------------------|
97+
| **LevelDB** | Embedded key-value store | 🟢 Low - Simple operations |
98+
| **RocksDB** | High-performance embedded DB | 🟢 Low - LevelDB-compatible |
99+
| **LMDB** | Memory-mapped key-value store | 🟢 Low - Fast read operations |
100+
101+
## Implementation Complexity Guide
102+
103+
- 🟢 **Low Complexity** (1-2 weeks): Database has SQL compatibility or simple REST API, straightforward query mapping
104+
- 🟡 **Medium Complexity** (3-6 weeks): Custom query language, moderate feature mapping required
105+
- 🔴 **High Complexity** (2-3 months): Distributed systems, complex data models, significant architectural differences
106+
107+
## Choosing a Database to Implement
108+
109+
When deciding which database to add support for, consider:
110+
111+
### 1. **Use Case Alignment**
112+
- Does the database solve a specific problem for ObjectQL users?
113+
- Does it complement existing drivers?
114+
115+
### 2. **Community Demand**
116+
- Is there active interest in this database?
117+
- Are there existing GitHub issues requesting it?
118+
119+
### 3. **Technical Feasibility**
120+
- How well does the database's data model map to ObjectQL's abstraction?
121+
- Does it support required operations (CRUD, filters, sorting)?
122+
123+
### 4. **Maintenance Burden**
124+
- Is the database actively maintained?
125+
- Does it have a stable API?
126+
- Is there good documentation?
127+
128+
### 5. **Ecosystem Maturity**
129+
- Are there quality Node.js/TypeScript clients?
130+
- Is the client library actively maintained?
131+
132+
## Recommended First Extensions
133+
134+
Based on community needs and implementation complexity, we recommend prioritizing:
135+
136+
1. **Redis Driver** - High demand, simple implementation, excellent for caching
137+
2. **Elasticsearch Driver** - Popular for search features, clear use case
138+
3. **DynamoDB Driver** - AWS ecosystem, serverless applications
139+
4. **ClickHouse Driver** - Analytics and reporting use cases
140+
141+
## Getting Started
142+
143+
To implement a custom driver:
144+
145+
1. Review the [Driver Implementation Guide](./implementing-custom-driver.md)
146+
2. Study existing driver implementations:
147+
- [`@objectql/driver-sql`](../../../packages/drivers/sql/src/index.ts) - SQL databases
148+
- [`@objectql/driver-mongo`](../../../packages/drivers/mongo/src/index.ts) - MongoDB
149+
3. Review the [Driver Interface](../../../packages/foundation/types/src/driver.ts)
150+
4. Follow the [Driver Testing Guide](./testing-drivers.md) (coming soon)
151+
152+
## Community Drivers
153+
154+
We encourage the community to create and maintain third-party drivers for additional databases. If you've implemented a driver, please:
155+
156+
1. Follow the ObjectQL driver conventions
157+
2. Include comprehensive tests
158+
3. Document configuration and usage
159+
4. Submit a PR to add your driver to this list
160+
161+
### Publishing Community Drivers
162+
163+
Name your package following the convention: `@your-org/objectql-driver-<database>`
164+
165+
Example: `@acme/objectql-driver-redis`
166+
167+
## Need Help?
168+
169+
- 📖 Read the [Driver Implementation Guide](./implementing-custom-driver.md)
170+
- 💬 Join the [ObjectQL Discord](https://discord.gg/objectql) (if available)
171+
- 🐛 [Open a GitHub Issue](https://github.com/objectstack-ai/objectql/issues)
172+
- 📧 Contact the maintainers
173+
174+
## Related Documentation
175+
176+
- [SQL Driver Documentation](./sql.md)
177+
- [MongoDB Driver Documentation](./mongo.md)
178+
- [Driver Interface Reference](../../../packages/foundation/types/src/driver.ts)

0 commit comments

Comments
 (0)