Skip to content

Commit 1416535

Browse files
authored
✨ graphql service endpoint for catalogd (#2100)
* initial vibe * integration with existing http service endpoint Signed-off-by: grokspawn <jordan@nimblewidget.com> * allow POST method only for graphql handler * functional, caching dynamic graphql server * claude-based storage/service interface division, missing preconditions * caehe on unpack; fix image verification policy Signed-off-by: grokspawn <jordan@nimblewidget.com> * schema-agnostic approaches with working tests Signed-off-by: Jordan <jordan@nimblewidget.com> * asciicast demo Signed-off-by: Jordan <jordan@nimblewidget.com> * review updates Signed-off-by: grokspawn <jordan@nimblewidget.com> --------- Signed-off-by: grokspawn <jordan@nimblewidget.com> Signed-off-by: Jordan <jordan@nimblewidget.com>
1 parent 67f22c2 commit 1416535

23 files changed

Lines changed: 3748 additions & 145 deletions

cmd/catalogd/main.go

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -365,12 +365,27 @@ func run(ctx context.Context) error {
365365
return err
366366
}
367367

368-
localStorage = &storage.LocalDirV1{
369-
RootDir: storeDir,
370-
RootURL: baseStorageURL,
371-
EnableMetasHandler: features.CatalogdFeatureGate.Enabled(features.APIV1MetasHandler),
368+
var metasMode storage.MetasHandlerMode
369+
if features.CatalogdFeatureGate.Enabled(features.APIV1MetasHandler) {
370+
metasMode = storage.MetasHandlerEnabled
371+
} else {
372+
metasMode = storage.MetasHandlerDisabled
372373
}
373374

375+
var graphqlMode storage.GraphQLQueriesMode
376+
if features.CatalogdFeatureGate.Enabled(features.GraphQLCatalogQueries) {
377+
graphqlMode = storage.GraphQLQueriesEnabled
378+
} else {
379+
graphqlMode = storage.GraphQLQueriesDisabled
380+
}
381+
382+
localStorage = storage.NewLocalDirV1(
383+
storeDir,
384+
baseStorageURL,
385+
metasMode,
386+
graphqlMode,
387+
)
388+
374389
// Config for the catalogd web server
375390
catalogServerConfig := serverutil.CatalogServerConfig{
376391
ExternalAddr: cfg.externalAddr,
Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
# Catalog queries using GraphQL
2+
3+
!!! warning "Alpha Feature"
4+
The GraphQL endpoint is an **alpha feature** controlled by the `GraphQLCatalogQueries` feature gate.
5+
The API and behavior may change in future releases.
6+
7+
After you [add a catalog of extensions](../../tutorials/add-catalog.md) to your cluster, you can query the catalog using GraphQL for flexible, structured queries with precise field selection.
8+
9+
## Prerequisites
10+
11+
* You have added a ClusterCatalog of extensions, such as [OperatorHub.io](https://operatorhub.io), to your cluster.
12+
* The `GraphQLCatalogQueries` feature gate is enabled in catalogd.
13+
14+
!!! note
15+
By default, Catalogd is installed with TLS enabled for the catalog webserver.
16+
The following examples will show this default behavior, but for simplicity's sake will ignore TLS verification in the curl commands using the `-k` flag.
17+
18+
You also need to port forward the catalog server service:
19+
20+
``` terminal
21+
kubectl -n olmv1-system port-forward svc/catalogd-service 8443:443
22+
```
23+
24+
## GraphQL Endpoint
25+
26+
The GraphQL endpoint is available at:
27+
28+
```
29+
https://localhost:8443/catalogs/<catalog-name>/api/v1/graphql
30+
```
31+
32+
All queries must be sent as **HTTP POST** requests with a JSON body containing a `query` field.
33+
34+
## Understanding GraphQL Field Names
35+
36+
**IMPORTANT**: GraphQL field names are automatically generated from catalog schema names.
37+
38+
### Naming Convention
39+
40+
Schema names are converted to GraphQL field names using this process:
41+
42+
1. Remove dots and special characters: `olm.bundle``olmbundle`
43+
2. Convert to lowercase: `OLM.Bundle``olmbundle`
44+
3. Append 's' for pluralization: `olmbundle``olmbundles`
45+
46+
**Examples:**
47+
48+
| Schema Name | GraphQL Field Name |
49+
|-------------|-------------------|
50+
| `olm.bundle` | `olmbundles` |
51+
| `olm.package` | `olmpackages` |
52+
| `olm.channel` | `olmchannels` |
53+
| `helm.chart` | `helmcharts` |
54+
55+
### Discovering Available Fields
56+
57+
To find the exact field names available for your catalog, use GraphQL introspection:
58+
59+
``` terminal
60+
curl -k -X POST 'https://localhost:8443/catalogs/operatorhubio/api/v1/graphql' \
61+
-H "Content-Type: application/json" \
62+
-d '{
63+
"query": "{ __schema { queryType { fields { name description } } } }"
64+
}' | jq
65+
```
66+
67+
This returns all available query fields for the catalog, including the automatically generated schema-based fields.
68+
69+
!!! warning "Pluralization Limitations"
70+
The current implementation appends 's' to schema names for pluralization. This may not produce grammatically correct English plurals in all cases (e.g., `index``indexs` instead of `indices`). When creating custom schemas, use singular nouns that pluralize well with a simple 's' suffix.
71+
72+
## Basic Queries
73+
74+
### Catalog Summary
75+
76+
Get an overview of schemas and object counts in the catalog:
77+
78+
``` terminal
79+
curl -k -X POST 'https://localhost:8443/catalogs/operatorhubio/api/v1/graphql' \
80+
-H "Content-Type: application/json" \
81+
-d '{
82+
"query": "{ summary { totalSchemas schemas { name totalObjects totalFields } } }"
83+
}' | jq
84+
```
85+
86+
### Query Bundles
87+
88+
List bundles with specific fields:
89+
90+
``` terminal
91+
curl -k -X POST 'https://localhost:8443/catalogs/operatorhubio/api/v1/graphql' \
92+
-H "Content-Type: application/json" \
93+
-d '{
94+
"query": "{ olmbundles(limit: 5, offset: 0) { name package image } }"
95+
}' | jq
96+
```
97+
98+
### Query Packages
99+
100+
List packages with metadata:
101+
102+
``` terminal
103+
curl -k -X POST 'https://localhost:8443/catalogs/operatorhubio/api/v1/graphql' \
104+
-H "Content-Type: application/json" \
105+
-d '{
106+
"query": "{ olmpackages(limit: 10) { name description defaultChannel } }"
107+
}' | jq
108+
```
109+
110+
### Query Channels
111+
112+
List channels:
113+
114+
``` terminal
115+
curl -k -X POST 'https://localhost:8443/catalogs/operatorhubio/api/v1/graphql' \
116+
-H "Content-Type: application/json" \
117+
-d '{
118+
"query": "{ olmchannels { name package entries } }"
119+
}' | jq
120+
```
121+
122+
## Advanced Queries
123+
124+
### Pagination
125+
126+
All schema-based queries support pagination via `limit` and `offset` arguments:
127+
128+
``` terminal
129+
curl -k -X POST 'https://localhost:8443/catalogs/operatorhubio/api/v1/graphql' \
130+
-H "Content-Type: application/json" \
131+
-d '{
132+
"query": "{ olmbundles(limit: 10, offset: 20) { name } }"
133+
}' | jq
134+
```
135+
136+
### Nested Field Selection
137+
138+
Select only the fields you need, including array-nested objects:
139+
140+
``` terminal
141+
curl -k -X POST 'https://localhost:8443/catalogs/operatorhubio/api/v1/graphql' \
142+
-H "Content-Type: application/json" \
143+
-d '{
144+
"query": "{ olmpackages { name channels { name entries { name } } } }"
145+
}' | jq
146+
```
147+
148+
**Note:** Non-array nested fields (like `icon`) are returned as JSON strings. To access their data, request the field as a scalar and parse the JSON:
149+
150+
``` terminal
151+
curl -k -X POST 'https://localhost:8443/catalogs/operatorhubio/api/v1/graphql' \
152+
-H "Content-Type: application/json" \
153+
-d '{
154+
"query": "{ olmpackages { name icon } }"
155+
}' | jq '.data.olmpackages[] | .icon | fromjson'
156+
```
157+
158+
### Complex Bundle Properties
159+
160+
Query bundle properties with their type and value fields:
161+
162+
``` terminal
163+
curl -k -X POST 'https://localhost:8443/catalogs/operatorhubio/api/v1/graphql' \
164+
-H "Content-Type: application/json" \
165+
-d '{
166+
"query": "{ olmbundles(limit: 5) { name properties { type value } } }"
167+
}' | jq
168+
```
169+
170+
**Note:** The `properties` field contains an array of objects, each with a `type` string and a `value` field that can contain complex nested data. GraphQL will return the full JSON structure for the `value` field.
171+
172+
## Comparing GraphQL vs Metas Endpoint
173+
174+
| Feature | GraphQL (`/api/v1/graphql`) | Metas (`/api/v1/metas`) |
175+
|---------|---------------------------|------------------------|
176+
| Field selection | Precise - request only needed fields | All fields always returned |
177+
| Query complexity | Rich queries with nested objects | Simple parameter-based filtering |
178+
| Response size | Minimal - only requested data | Full objects always returned |
179+
| Schema discovery | Introspection built-in | External documentation needed |
180+
| Pagination | Built-in `limit` and `offset` | Manual implementation required |
181+
| HTTP Method | POST only | GET supported |
182+
| Feature status | Alpha (feature gate required) | Stable |
183+
184+
**When to use GraphQL:**
185+
- You need specific fields from large objects
186+
- You want to query related data in a single request
187+
- You need structured, typed responses
188+
- You're building a UI or client that benefits from precise data fetching
189+
190+
**When to use Metas endpoint:**
191+
- You need simple, stable API
192+
- You're doing basic filtering by schema/package/name
193+
- You want to use GET requests for caching
194+
- You need guaranteed API stability
195+
196+
## Limitations
197+
198+
1. **Pluralization**: Schema names are pluralized by appending 's', which may not be grammatically correct for all words
199+
2. **Schema naming**: Full schema names (including namespace/prefix) are preserved in field names (`olm.bundle``olmbundles`, not `bundles`)
200+
3. **POST only**: GraphQL endpoint only accepts POST requests, unlike the metas endpoint which supports GET
201+
4. **Alpha stability**: API may change in future releases while in alpha
202+
203+
## Enabling the GraphQL Feature
204+
205+
The GraphQL endpoint is controlled by the `GraphQLCatalogQueries` feature gate. To enable it:
206+
207+
``` yaml
208+
args:
209+
- --feature-gates=GraphQLCatalogQueries=true
210+
```
211+
212+
See [enable webhook support](enable-webhook-support.md) for more details on configuring feature gates.

go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ require (
1717
github.com/google/go-containerregistry v0.21.5
1818
github.com/google/renameio/v2 v2.0.2
1919
github.com/gorilla/handlers v1.5.2
20+
github.com/graphql-go/graphql v0.8.1
2021
github.com/klauspost/compress v1.18.6
2122
github.com/opencontainers/go-digest v1.0.0
2223
github.com/opencontainers/image-spec v1.1.1

go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -280,6 +280,8 @@ github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5T
280280
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA=
281281
github.com/gosuri/uitable v0.0.4 h1:IG2xLKRvErL3uhY6e1BylFzG+aJiwQviDDTfOKeKTpY=
282282
github.com/gosuri/uitable v0.0.4/go.mod h1:tKR86bXuXPZazfOTG1FIzvjIdXzd0mo4Vtn16vt0PJo=
283+
github.com/graphql-go/graphql v0.8.1 h1:p7/Ou/WpmulocJeEx7wjQy611rtXGQaAcXGqanuMMgc=
284+
github.com/graphql-go/graphql v0.8.1/go.mod h1:nKiHzRM0qopJEwCITUuIsxk9PlVlwIiiI8pnJEhordQ=
283285
github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 h1:+ngKgrYPPJrOjhax5N+uePQ0Fh1Z7PheYoUI/0nzkPA=
284286
github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
285287
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.1-0.20210315223345-82c243799c99 h1:JYghRBlGCZyCF2wNUJ8W0cwaQdtpcssJ4CgC406g+WU=

hack/demo/graphql-demo-script.sh

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
#!/usr/bin/env bash
2+
3+
#
4+
# Catalogd Dynamic GraphQL API Demo
5+
#
6+
# This demo showcases the dynamic GraphQL endpoint for querying
7+
# File-Based Catalog (FBC) content. The schema is automatically
8+
# discovered from catalog data -- no manual type definitions needed.
9+
#
10+
11+
set -euo pipefail
12+
trap cleanup SIGINT SIGTERM EXIT
13+
14+
SCRIPTPATH="$(cd -- "$(dirname "$0")" > /dev/null 2>&1; pwd -P)"
15+
SERVER_PID=""
16+
PORT=9376
17+
BASE="http://localhost:${PORT}/catalogs/example-catalog/api/v1/graphql"
18+
19+
cleanup() {
20+
if [[ -n "${SERVER_PID}" ]]; then
21+
kill "${SERVER_PID}" 2>/dev/null || true
22+
wait "${SERVER_PID}" 2>/dev/null || true
23+
fi
24+
if [[ -n "${TMPBIN:-}" && -f "${TMPBIN}" ]]; then
25+
rm -f "${TMPBIN}"
26+
fi
27+
}
28+
29+
gql() {
30+
local query="$1"
31+
curl -s -X POST "${BASE}" \
32+
-H "Content-Type: application/json" \
33+
-d "{\"query\": \"$query\"}" | jq .
34+
}
35+
36+
banner() {
37+
echo ""
38+
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
39+
echo " $1"
40+
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
41+
}
42+
43+
# -- Build and start the demo server --
44+
banner "Building GraphQL demo server"
45+
TMPBIN=$(mktemp /tmp/graphql-demo.XXXXXX)
46+
REPOROOT="$(cd "${SCRIPTPATH}/../.."; pwd)"
47+
(cd "${REPOROOT}" && go build -o "${TMPBIN}" ./hack/demo/graphql-demo-server/)
48+
echo "Built successfully."
49+
50+
echo ""
51+
echo "Starting server on port ${PORT}..."
52+
"${TMPBIN}" 2>/dev/null &
53+
SERVER_PID=$!
54+
sleep 1
55+
echo "Server ready. Catalog loaded: example-catalog (5 packages, 11 bundles)"
56+
57+
# -- 1. Catalog Summary --
58+
banner "1. Discover catalog contents (summary query)"
59+
echo '$ curl ... -d '\''{"query": "{ summary { totalSchemas schemas { name totalObjects totalFields } } }"}'\'''
60+
echo ""
61+
gql "{ summary { totalSchemas schemas { name totalObjects totalFields } } }"
62+
sleep 2
63+
64+
# -- 2. List packages --
65+
banner "2. List packages (field selection: name + defaultChannel only)"
66+
echo '$ curl ... -d '\''{"query": "{ olmpackages { name defaultChannel } }"}'\'''
67+
echo ""
68+
gql "{ olmpackages { name defaultChannel } }"
69+
sleep 2
70+
71+
# -- 3. List bundles with pagination --
72+
banner "3. Browse bundles with pagination (offset 3, limit 4)"
73+
echo '$ curl ... -d '\''{"query": "{ olmbundles(limit: 4, offset: 3) { name package } }"}'\'''
74+
echo ""
75+
gql "{ olmbundles(limit: 4, offset: 3) { name package } }"
76+
sleep 2
77+
78+
# -- 4. Nested properties --
79+
banner "4. Query bundle properties (nested objects)"
80+
echo '$ curl ... -d '\''{"query": "{ olmbundles(limit: 2) { name package properties { type value } } }"}'\'''
81+
echo ""
82+
gql "{ olmbundles(limit: 2) { name package properties { type value } } }"
83+
sleep 2
84+
85+
# -- 5. Related images --
86+
banner "5. Query related images (nested array)"
87+
echo '$ curl ... -d '\''{"query": "{ olmbundles(limit: 1) { name relatedImages { name image } } }"}'\'''
88+
echo ""
89+
gql "{ olmbundles(limit: 1) { name relatedImages { name image } } }"
90+
sleep 2
91+
92+
# -- 6. Channel upgrade graph --
93+
banner "6. Explore channel upgrade graph"
94+
echo '$ curl ... -d '\''{"query": "{ olmchannels(limit: 2) { name package entries { name skipRange } } }"}'\'''
95+
echo ""
96+
gql "{ olmchannels(limit: 2) { name package entries { name skipRange } } }"
97+
sleep 2
98+
99+
# -- 7. Cross-schema query --
100+
banner "7. Multi-schema query in one request"
101+
echo '$ curl ... -d '\''{"query": "{ olmpackages(limit: 3) { name defaultChannel } olmbundles(limit: 3) { name package } }"}'\'''
102+
echo ""
103+
gql "{ olmpackages(limit: 3) { name defaultChannel } olmbundles(limit: 3) { name package } }"
104+
sleep 2
105+
106+
# -- 8. GraphQL introspection --
107+
banner "8. Standard GraphQL introspection (auto-generated types)"
108+
echo '$ curl ... -d '\''{"query": "{ __schema { types { name kind } } }"}'\'' | jq ...'
109+
echo ""
110+
curl -s -X POST "${BASE}" \
111+
-H "Content-Type: application/json" \
112+
-d '{"query": "{ __schema { types { name kind } } }"}' \
113+
| jq '[.data.__schema.types[] | select(.name | startswith("__") | not) | select(.kind == "OBJECT")]'
114+
sleep 2
115+
116+
# -- Done --
117+
banner "Demo complete"
118+
echo ""
119+
echo "Key takeaways:"
120+
echo " - GraphQL schema is auto-discovered from FBC data"
121+
echo " - No code changes needed when FBC schemas evolve"
122+
echo " - Supports field selection, pagination, nested queries"
123+
echo " - Full GraphQL introspection for tooling support"
124+
echo ""

0 commit comments

Comments
 (0)