Skip to content

Commit f36c1af

Browse files
wenermewenerme
committed
docs: add Tencent CLS FDW documentation
- Add catalog page (docs/catalog/tencent_cls.md) with full usage guide - Add to mkdocs.yaml navigation - Add to README.md FDW table - Add FDW README.md
1 parent e70aa51 commit f36c1af

4 files changed

Lines changed: 225 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
| [Slack](./wasm-wrappers/fdw/slack_fdw) | A Wasm FDW for [Slack](https://www.slack.com/) | ✅ | ❌ |
3535
| [Snowflake](./wasm-wrappers/fdw/snowflake_fdw) | A Wasm FDW for [Snowflake](https://www.snowflake.com/) | ✅ | ✅ |
3636
| [Stripe](./wrappers/src/fdw/stripe_fdw) | A FDW for [Stripe](https://stripe.com/) API | ✅ | ✅ |
37+
| [Tencent CLS](./wrappers/src/fdw/tencent_cls_fdw) | A FDW for [Tencent Cloud Log Service](https://www.tencentcloud.com/products/cls) | ✅ | ❌ |
3738

3839
### Warning
3940

‎docs/catalog/tencent_cls.md‎

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
---
2+
source:
3+
documentation:
4+
author: 陈杨文(https://github.com/wenerme)
5+
tags:
6+
- native
7+
- community
8+
---
9+
10+
# Tencent CLS
11+
12+
[Tencent Cloud Log Service (CLS)](https://www.tencentcloud.com/products/cls) is a centralized log management service for collection, storage, search, and analysis.
13+
14+
The Tencent CLS Wrapper allows you to query CLS log data within your Postgres database using CQL (CLS Query Language).
15+
16+
## Preparation
17+
18+
Before you can query CLS, you need to enable the Wrappers extension and store your credentials in Postgres.
19+
20+
### Enable Wrappers
21+
22+
Make sure the `wrappers` extension is installed on your database:
23+
24+
```sql
25+
create extension if not exists wrappers with schema extensions;
26+
```
27+
28+
### Enable the Tencent CLS Wrapper
29+
30+
Enable the `tencent_cls_wrapper` FDW:
31+
32+
```sql
33+
create foreign data wrapper tencent_cls_wrapper
34+
handler tencent_cls_fdw_handler
35+
validator tencent_cls_fdw_validator;
36+
```
37+
38+
### Store your credentials (optional)
39+
40+
By default, Postgres stores FDW credentials inside `pg_catalog.pg_foreign_server` in plain text. Anyone with access to this table will be able to view these credentials. Wrappers is designed to work with [Vault](https://supabase.com/docs/guides/database/vault), which provides an additional level of security for storing credentials. We recommend using Vault to store your credentials.
41+
42+
```sql
43+
-- Save your Tencent Cloud SecretId in Vault
44+
select vault.create_secret(
45+
'AKIDxxxxxxxxxxxxxxxx',
46+
'tencent_secret_id',
47+
'Tencent Cloud SecretId for CLS'
48+
);
49+
50+
-- Save your Tencent Cloud SecretKey in Vault
51+
select vault.create_secret(
52+
'xxxxxxxxxxxxxxxxxxxxxxxx',
53+
'tencent_secret_key',
54+
'Tencent Cloud SecretKey for CLS'
55+
);
56+
```
57+
58+
### Connecting to CLS
59+
60+
We need to provide Postgres with the credentials to connect to Tencent CLS. We can do this using the `create server` command:
61+
62+
=== "With Vault"
63+
64+
```sql
65+
create server tencent_cls_server
66+
foreign data wrapper tencent_cls_wrapper
67+
options (
68+
secret_id_id '<secret_id vault id>',
69+
secret_key_id '<secret_key vault id>',
70+
region 'ap-shanghai'
71+
);
72+
```
73+
74+
=== "Without Vault"
75+
76+
```sql
77+
create server tencent_cls_server
78+
foreign data wrapper tencent_cls_wrapper
79+
options (
80+
secret_id 'AKIDxxxxxxxxxxxxxxxx',
81+
secret_key 'xxxxxxxxxxxxxxxxxxxxxxxx',
82+
region 'ap-shanghai'
83+
);
84+
```
85+
86+
#### Server options
87+
88+
| Option | Required | Description |
89+
|--------|----------|-------------|
90+
| `secret_id` | Yes* | Tencent Cloud SecretId (plain text) |
91+
| `secret_id_id` | Yes* | Vault secret UUID or name for SecretId |
92+
| `secret_key` | Yes* | Tencent Cloud SecretKey (plain text) |
93+
| `secret_key_id` | Yes* | Vault secret UUID or name for SecretKey |
94+
| `region` | No | Tencent Cloud region (default: `ap-shanghai`) |
95+
| `endpoint` | No | Custom CLS API endpoint |
96+
| `max_response_size` | No | Max response body size in bytes (default: 10MB) |
97+
98+
\* Either `secret_id` or `secret_id_id` is required. Same for `secret_key` / `secret_key_id`.
99+
100+
## Creating Foreign Tables
101+
102+
The CLS Wrapper maps CLS log records to Postgres rows. You define columns based on the fields in your logs.
103+
104+
```sql
105+
create foreign table cls_logs (
106+
ts timestamptz, -- log timestamp (from CLS Time field)
107+
log text, -- raw LogJson string
108+
source text, -- log source
109+
topic_id text, -- CLS topic ID
110+
topic_name text, -- CLS topic name
111+
file_name text, -- log file name
112+
host_name text, -- host name
113+
_result text, -- full CLS result JSON (meta column)
114+
-- Add any custom columns matching your LogJson fields:
115+
level text,
116+
message text,
117+
service text
118+
)
119+
server tencent_cls_server
120+
options (
121+
topic 'my-log-topic',
122+
query '*',
123+
limit '100'
124+
);
125+
```
126+
127+
### Table options
128+
129+
| Option | Required | Description |
130+
|--------|----------|-------------|
131+
| `topic` | Yes | CLS topic name (auto-resolved to topic ID) or topic ID |
132+
| `topic_id` | Yes | Alternative to `topic` — direct topic ID |
133+
| `query` | No | Default CQL query expression (default: `*`) |
134+
| `syntax_rule` | No | `0` for Lucene, `1` for CQL (default: `1`) |
135+
| `limit` | No | Max rows to return (default: `1000`) |
136+
137+
### Column mapping
138+
139+
| Column name | Source | Description |
140+
|-------------|--------|-------------|
141+
| `ts` | `Time` (milliseconds) | Converted to `timestamptz` |
142+
| `log` | `LogJson` | Raw JSON string |
143+
| `source` | `Source` | Log source |
144+
| `topic_id` | `TopicId` | Topic ID |
145+
| `topic_name` | `TopicName` | Topic name |
146+
| `file_name` | `FileName` | File name |
147+
| `host_name` | `HostName` | Host name |
148+
| `_result` | Full result | Complete CLS result as JSON |
149+
| *other* | `LogJson.{name}` | Extracted from LogJson by column name |
150+
151+
## Query Pushdown
152+
153+
The CLS Wrapper supports WHERE clause pushdown for efficient log querying:
154+
155+
```sql
156+
-- Timestamp range pushdown (required for efficient queries)
157+
select * from cls_logs
158+
where ts >= now() - interval '1 hour'
159+
and ts < now();
160+
161+
-- Direct CQL query via _query pseudo-column
162+
select * from cls_logs
163+
where ts >= now() - interval '1 hour'
164+
and _query = 'level:ERROR AND service:gateway';
165+
```
166+
167+
The `query` table option serves as a base filter that is automatically combined with WHERE conditions using `AND`.
168+
169+
## Examples
170+
171+
### Basic log query
172+
173+
```sql
174+
select ts, log, source
175+
from cls_logs
176+
where ts >= now() - interval '1 hour'
177+
limit 10;
178+
```
179+
180+
### Query with preset filter
181+
182+
```sql
183+
-- Create a table preset to error logs
184+
create foreign table cls_errors (
185+
ts timestamptz,
186+
log text,
187+
source text,
188+
level text,
189+
message text
190+
)
191+
server tencent_cls_server
192+
options (
193+
topic 'my-log-topic',
194+
query 'level:ERROR'
195+
);
196+
197+
-- Just query it — base filter is applied automatically
198+
select ts, message, source
199+
from cls_errors
200+
where ts >= now() - interval '30 minutes';
201+
```
202+
203+
### Raw CQL query
204+
205+
```sql
206+
select ts, log
207+
from cls_logs
208+
where ts >= now() - interval '1 hour'
209+
and _query = 'status:>=500 AND method:POST';
210+
```

‎mkdocs.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ nav:
3030
- S3 Vectors: "catalog/s3vectors.md"
3131
- Stripe: "catalog/stripe.md"
3232
- SQL Server: "catalog/mssql.md"
33+
- Tencent CLS: "catalog/tencent_cls.md"
3334
- Wasm:
3435
- catalog/wasm/index.md
3536
- Cal.com: "catalog/cal.md"

‎wrappers/src/fdw/tencent_cls_fdw/README.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Tencent CLS Foreign Data Wrapper
2+
3+
This is a foreign data wrapper for [Tencent Cloud Log Service (CLS)](https://www.tencentcloud.com/products/cls). It is developed using [Wrappers](https://github.com/supabase/wrappers) and supports log data querying via CQL.
4+
5+
## Documentation
6+
7+
[https://fdw.dev/catalog/tencent_cls/](https://fdw.dev/catalog/tencent_cls/)
8+
9+
## Changelog
10+
11+
| Version | Date | Notes |
12+
| ------- | ---------- | ---------------------------------------------- |
13+
| 0.1.0 | 2026-03-27 | Initial version |

0 commit comments

Comments
 (0)