Skip to content

Commit 0f8e569

Browse files
committed
feat/added_new_async_driver
- Added the Async SQLite driver - Added the Async Postgres driver - Added the Async UnqLite driver
1 parent 199914e commit 0f8e569

9 files changed

Lines changed: 188 additions & 7 deletions

File tree

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,5 @@
22
/.idea
33
/tmp
44
/samples
5-
test.py
5+
test.py
6+
energy_members.unqlite

CHANGELOG.txt

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
Change Log
22
===============
3-
VERSION="0.0.15"
4-
LAST_UPDATE="11/11/2025"
3+
VERSION="0.0.16"
4+
LAST_UPDATE="12/11/2025"
55
------------------
6-
- Added the Async SQLite driver requirement
6+
- Readme updates
7+
- Added the Async SQLite driver
8+
- Added the Async Postgres driver
9+
- Added the Async UnqLite driver
710
------------------

OntologicalFramework/OntologyToAPI-BusinessModel.ttl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ xsd:duration rdf:type rdfs:Datatype .
6565
owl:InverseFunctionalProperty ;
6666
rdfs:domain :BusinessModel ;
6767
rdfs:range <http://www.cedri.com/SmartLEM-Metadata#Metadata> ;
68-
rdfs:comment "The metadata required to acheive some goal of a business model" .
68+
rdfs:comment "The metadata required to achieve some goal of a business model" .
6969

7070

7171
#################################################################

OntologyToAPI/core/Connectors/IndentifyConnector.py

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,22 @@
22
from OntologyToAPI.core.Connectors.Stateless.APIConnection import APIConnection
33
from OntologyToAPI.core.Connectors.Stateless.MYSQLConnection import MySQLConnection
44
from OntologyToAPI.core.Connectors.Stateless.MongoDBConnection import MongoDBConnection
5+
from OntologyToAPI.core.Connectors.Stateless.SQLITEConnection import SQLiteConnection
6+
from OntologyToAPI.core.Connectors.Stateless.POSTGRESQLConnection import PostgreSQLConnection
7+
from OntologyToAPI.core.Connectors.Stateless.UNQLITEConnection import UnQLiteConnection
58

69
SUPPORTED_CONNECTIONS = {
710
"API": APIConnection,
11+
"SOCKET": SocketConnection,
12+
13+
# SQL Databases
814
"MYSQL": MySQLConnection,
15+
"SQLITE": SQLiteConnection,
16+
"POSTGRESQL": PostgreSQLConnection,
17+
18+
# NoSQL Databases
919
"MONGODB": MongoDBConnection,
10-
"Socket": SocketConnection
20+
"UNQLITE": UnQLiteConnection
1121
}
1222

1323
def identifyConnector(CommunicationTechnology, args):
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
from sqlalchemy import text, create_engine
2+
from sqlalchemy.ext.asyncio import create_async_engine
3+
from datetime import datetime
4+
5+
async def datetime_handler(row, keys):
6+
return {
7+
key: (val.isoformat() if isinstance(val, datetime) else val)
8+
for key, val in zip(keys, row)
9+
}
10+
11+
class PostgreSQLConnection:
12+
def __init__(self, args):
13+
self.engine = create_async_engine(args["hasConnectionString"])
14+
15+
async def exec_query(self, query: str):
16+
async with self.engine.connect() as connection:
17+
result = await connection.execute(text(query))
18+
rows = result.fetchall()
19+
keys = result.keys()
20+
return [await datetime_handler(row, keys) for row in rows]
21+
22+
async def exec_insert(self, query: str):
23+
async with self.engine.begin() as connection:
24+
await connection.execute(text(query))
25+
result = await connection.execute(text("SELECT LAST_INSERT_ID() AS id"))
26+
row = result.fetchone()
27+
return row.id
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
from sqlalchemy import text, create_engine
2+
from sqlalchemy.ext.asyncio import create_async_engine
3+
from datetime import datetime
4+
5+
async def datetime_handler(row, keys):
6+
return {
7+
key: (val.isoformat() if isinstance(val, datetime) else val)
8+
for key, val in zip(keys, row)
9+
}
10+
11+
class SQLiteConnection:
12+
def __init__(self, args):
13+
self.engine = create_async_engine(args["hasConnectionString"])
14+
15+
async def exec_query(self, query: str):
16+
async with self.engine.connect() as connection:
17+
result = await connection.execute(text(query))
18+
rows = result.fetchall()
19+
keys = result.keys()
20+
return [await datetime_handler(row, keys) for row in rows]
21+
22+
async def exec_insert(self, query: str):
23+
async with self.engine.begin() as connection:
24+
await connection.execute(text(query))
25+
result = await connection.execute(text("SELECT LAST_INSERT_ID() AS id"))
26+
row = result.fetchone()
27+
return row.id
Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
import asyncio
2+
import json
3+
import logging
4+
from urllib.parse import urlparse, unquote
5+
from typing import Any, Dict, List
6+
from unqlite import UnQLite
7+
8+
class UnQLiteConnection:
9+
def __init__(self, args: Dict[str, str]):
10+
raw = args.get("hasConnectionString")
11+
if not raw:
12+
raise ValueError("Missing 'hasConnectionString' in args.")
13+
parsed = urlparse(raw)
14+
15+
if parsed.scheme not in ("unqlite+asyncio",):
16+
raise ValueError(f"Unsupported scheme: {parsed.scheme}. Use 'unqlite+asyncio'.")
17+
18+
path = unquote(parsed.netloc or "")
19+
if not path:
20+
raise ValueError(f"Empty database path in connection string.")
21+
if path == "/:memory:":
22+
db_path = ":memory:"
23+
else:
24+
db_path = path
25+
26+
try:
27+
self._db = UnQLite(db_path)
28+
except Exception as e:
29+
raise Exception(f"Could not open UnQLite database at '{db_path}': {e}")
30+
31+
# ---------- public API ----------
32+
33+
async def exec_query(self, collection_query: str) -> List[Dict[str, Any]]:
34+
collection, filt = self._parse_collection_query(collection_query)
35+
docs = await asyncio.to_thread(self._get_collection_docs, collection)
36+
return self._filter_docs(docs, filt)
37+
38+
# ---------- internals (sync) ----------
39+
40+
def _parse_collection_query(self, collection_query: str):
41+
try:
42+
collection, json_str = collection_query.split(".", 1)
43+
filt = json.loads(json_str)
44+
if not isinstance(filt, dict):
45+
raise ValueError("Filter JSON must be an object.")
46+
return collection, filt
47+
except Exception as e:
48+
raise ValueError(f"Invalid collection_query format; expected 'collection.{{...}}'. Error: {e}")
49+
50+
def _get_collection_docs(self, collection: str) -> List[Dict[str, Any]]:
51+
try:
52+
if collection not in self._db:
53+
return []
54+
data = self._db[collection]
55+
if isinstance(data, (bytes, str)):
56+
data = json.loads(data)
57+
if isinstance(data, list):
58+
return [d for d in data if isinstance(d, dict)]
59+
if isinstance(data, dict):
60+
return [data]
61+
return []
62+
except Exception as e:
63+
logging.error(f"Error reading collection '{collection}': {e}")
64+
return []
65+
66+
def _persist_collection(self, collection: str, docs: List[Dict[str, Any]]) -> None:
67+
try:
68+
self._db[collection] = json.dumps(docs, separators=(",", ":"))
69+
except Exception as e:
70+
logging.error(f"Error writing collection '{collection}': {e}")
71+
raise
72+
73+
def _filter_docs(self, docs: List[Dict[str, Any]], filt: Dict[str, Any]) -> List[Dict[str, Any]]:
74+
def match(doc: Dict[str, Any]) -> bool:
75+
for key, cond in filt.items():
76+
val = doc.get(key, None)
77+
if isinstance(cond, dict):
78+
for op, rhs in cond.items():
79+
if op == "$gt" and not (val is not None and val > rhs): return False
80+
if op == "$lt" and not (val is not None and val < rhs): return False
81+
if op == "$eq" and not (val == rhs): return False
82+
# extend here with $gte, $lte, $ne, $in, etc.
83+
else:
84+
if val != cond:
85+
return False
86+
return True
87+
return [d for d in docs if match(d)]

README.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@ The ontological framework is composed of four main modules:
2121
- **ExternalCode Ontology Module:** This module has all the technical details to connect to an external code, it also adds the possibility to dynamically require python packages.
2222
- **Communications Ontology Module:** This module describes the communication technologies that can be used to fetch the data of some metadata in multiple forms (e. g).
2323

24+
> A full manual on how to extend your own ontologies using the OntologyToAPI framework is still in development, but you can check the examples provided at the samples repository to get started at https://github.com/JCGCosta/OntologyToAPISamples
25+
26+
2427
> From now on you must be ready to go and create your own ontological specification importing the [Ontology Modules](https://github.com/JCGCosta/OntologyToAPI/tree/master/OntologicalFramework) and extending it. You can do this by using the Protégé ontology editor (https://protege.stanford.edu/). Or if you prefer you can use any text editor to create your ontology files in the supported formats (.ttl, .rdf, .owl).
2528
2629
### Step 1: Installing the Package
@@ -31,7 +34,6 @@ pip install -U ontologytoapi
3134

3235
### Step 2: Running
3336

34-
- If you want to do a quick test we provided some .ttl samples at the following repository: https://github.com/JCGCosta/OntologyToAPISamples
3537
- With your metadata and business models ontologies implemented you can generate your API by having the following python file as an entry point:
3638

3739
```python
@@ -50,3 +52,25 @@ if __name__ == "__main__":
5052
api_app = APIGen.generate_api_routes()
5153
uvicorn.run(api_app, host="127.0.0.1", port=5000)
5254
```
55+
56+
## Supported communication technologies are (Currently):
57+
58+
#### Stateful Connections
59+
- "SOCKET" - For Socket connections using asyncio streams
60+
61+
#### Stateless Connections
62+
- "API" - For REST APIs using requests driver
63+
- "MYSQL" - For MySQL Databases using aiomysql driver
64+
- "SQLITE" - For SQLite Databases using aiosqlite driver
65+
- "POSTGRESQL" - For PostgreSQL Databases using asyncpg driver
66+
- "MONGODB" - For MongoDB Databases using motor driver
67+
- "UNQLITE" - For UnQLite Databases using unqlite+asyncio driver
68+
69+
## Next Steps:
70+
71+
Next steps involve extending the support for new communication technologies.
72+
- "FILE" - For File operations using aiofiles driver
73+
- "WEBSOCKET" - For WebSocket connections using websockets driver
74+
- "MQTT" - For MQTT connections using asyncio-mqtt driver
75+
- "REDIS" - For Redis Databases using aioredis driver
76+
- "CASSANDRA" - For Cassandra Databases using cassandra-driver with asyncio support

requirements.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ python-multipart~=0.0.20
1010

1111
# Dependencies for async operations in specific databases
1212

13+
asyncio~=4.0.0 # Core async library
14+
asyncpg~=0.27.0 # Async PostgreSQL driver
1315
aiomysql~=0.2.0 # Async MySQL driver
1416
aiosqlite~=0.21.0 # Async SQLite driver
1517
motor~=3.7.1 # Async MongoDB driver

0 commit comments

Comments
 (0)