Skip to content

Commit 6448366

Browse files
committed
drop all sqlite support (and channel priority properties)
1 parent ab08265 commit 6448366

170 files changed

Lines changed: 53 additions & 58966 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

AGENTS.md

Lines changed: 18 additions & 170 deletions
Original file line numberDiff line numberDiff line change
@@ -9,18 +9,15 @@ The Operator Registry is a Kubernetes/OpenShift component that provides operator
99
## Key Components
1010

1111
### Binaries
12-
- **`opm`**: Main CLI tool for generating and updating registry databases and index images
13-
- **`initializer`**: **(Deprecated)** Converts operator manifests to SQLite database format
14-
- **`registry-server`**: **(Deprecated)** Exposes gRPC interface to SQLite databases
15-
- **`configmap-server`**: Parses ConfigMaps into SQLite databases and exposes gRPC interface
12+
- **`opm`**: Main CLI tool for rendering and serving file-based catalogs
1613

1714
### Libraries
1815
- **`pkg/client`**: High-level client interface for gRPC API
1916
- **`pkg/api`**: Low-level client libraries for gRPC interface
2017
- **`pkg/registry`**: Core registry types (Packages, Channels, Bundles)
21-
- **`pkg/sqlite`**: **(Deprecated)** SQLite database interfaces for manifests
2218
- **`pkg/lib`**: External interfaces and standards for operator bundles
2319
- **`pkg/containertools`**: Container tooling integration
20+
- **`pkg/cache`**: Caching backends for file-based catalogs
2421

2522
### Alpha Features
2623
- **`alpha/declcfg`**: Declarative configuration format
@@ -33,7 +30,7 @@ The Operator Registry is a Kubernetes/OpenShift component that provides operator
3330
- **Go 1.24.4**: Minimum Go version required
3431
- **Cobra CLI**: Command-line interface framework
3532
- **gRPC**: Primary API communication protocol
36-
- **SQLite**: Database backend for registry data
33+
- **Declarative Config (FBC)**: File-based catalog format for registry data
3734
- **OCI Images**: Container image format for bundles
3835

3936
### Testing
@@ -93,11 +90,11 @@ podman build -t quay.io/my-namespace/my-bundle:latest -f bundle.Dockerfile .
9390
# IMPORTANT: Bundle images must be published to a registry before they can be consumed
9491
podman push quay.io/my-namespace/my-bundle:latest
9592

96-
# Add bundle to index (deprecated - use file-based catalogs instead)
97-
opm index add --bundles quay.io/my-namespace/my-bundle:latest --from-index quay.io/my-namespace/my-index:latest --tag quay.io/my-namespace/my-index:latest
93+
# Render bundle to declarative config
94+
opm render quay.io/my-namespace/my-bundle:latest --output yaml > catalog.yaml
9895

99-
# Generate index image (deprecated - use file-based catalogs instead)
100-
opm index add --bundles quay.io/my-namespace/my-bundle:latest --tag quay.io/my-namespace/my-index:latest
96+
# Generate catalog from multiple bundles
97+
opm render quay.io/my-namespace/my-bundle:v1.0.0 quay.io/my-namespace/my-bundle:v1.1.0 --output yaml
10198
```
10299

103100
**⚠️ Critical Requirements:**
@@ -117,19 +114,7 @@ model, err := declcfg.ConvertToModel(cfg)
117114
err = declcfg.Write(cfg, "output.yaml")
118115
```
119116

120-
### 4. Database Operations **(Deprecated)**
121-
```go
122-
// Create SQLite database
123-
db, err := sqlite.Open("registry.db")
124-
125-
// Add bundle to database
126-
err = db.AddBundle(bundle)
127-
128-
// Query packages
129-
packages, err := db.ListPackages()
130-
```
131-
132-
### 5. Serving Catalog Content with `opm serve`
117+
### 4. Serving Catalog Content with `opm serve`
133118
The `opm serve` command exposes operator catalog data via a gRPC interface for consumption by OLM.
134119

135120
```bash
@@ -214,121 +199,18 @@ cache, err := cache.New("/cache/dir",
214199
- Cache directory should be persistent for production deployments
215200
- Use `--cache-only` for pre-building cache in CI/CD pipelines
216201

217-
## Deprecated SQLite Commands and Operations
218-
219-
⚠️ **IMPORTANT DEPRECATION NOTICE**: SQLite-based catalogs and their related subcommands are deprecated. Support for them will be removed in a future release. Please migrate your catalog workflows to the new file-based catalog format.
220-
221-
### Deprecated Commands
222-
223-
#### `opm registry` Commands (All Deprecated)
224-
- **`opm registry serve`** - **(Deprecated)** Serve SQLite database via gRPC
225-
- **`opm registry add`** - **(Deprecated)** Add bundles to SQLite database
226-
- **`opm registry rm`** - **(Deprecated)** Remove packages from SQLite database
227-
- **`opm registry prune`** - **(Deprecated)** Prune SQLite database
228-
- **`opm registry prune-stranded`** - **(Deprecated)** Prune stranded bundles from SQLite database
229-
- **`opm registry deprecatetruncate`** - **(Deprecated)** Deprecate bundles in SQLite database
230-
- **`opm registry mirror`** - **(Deprecated)** Mirror SQLite-based catalogs
231-
232-
#### `opm index` Commands (All Deprecated)
233-
- **`opm index add`** - **(Deprecated)** Add bundles to SQLite-based index
234-
- **`opm index rm`** - **(Deprecated)** Delete operators from SQLite-based index
235-
- **`opm index export`** - **(Deprecated)** Export SQLite-based index
236-
- **`opm index prune`** - **(Deprecated)** Prune SQLite-based index
237-
- **`opm index prune-stranded`** - **(Deprecated)** Prune stranded bundles from SQLite-based index
238-
- **`opm index deprecatetruncate`** - **(Deprecated)** Deprecate bundles in SQLite-based index
239-
240-
### Deprecated Libraries and Interfaces
241-
242-
#### `pkg/sqlite` Package (Deprecated)
243-
- **`SQLQuerier`** - **(Deprecated)** Query interface for SQLite databases
244-
- **`SQLLoader`** - **(Deprecated)** Load bundles into SQLite databases
245-
- **`DeprecationAwareLoader`** - **(Deprecated)** Handle bundle deprecations in SQLite
246-
- **`SQLDeprecator`** - **(Deprecated)** Deprecate bundles in SQLite databases
247-
248-
#### `pkg/lib/registry` Package (Deprecated)
249-
- **`RegistryUpdater`** - **(Deprecated)** Update SQLite-based registries
250-
- **`RegistryDeleter`** - **(Deprecated)** Delete from SQLite-based registries
251-
- **`RegistryDeprecator`** - **(Deprecated)** Deprecate bundles in SQLite-based registries
252-
253-
### Migration Path
254-
255-
**From SQLite to File-Based Catalogs:**
256-
257-
1. **Replace `opm registry serve`****Use `opm serve`**
258-
```bash
259-
# Old (deprecated)
260-
opm registry serve --database bundles.db
261-
262-
# New (recommended)
263-
opm serve ./catalog-directory
264-
```
265-
266-
2. **Replace `opm index add`****Use `opm alpha generate dockerfile` + `opm serve`**
267-
```bash
268-
# Old (deprecated)
269-
opm index add --bundles quay.io/my/bundle:latest --tag quay.io/my/index:latest
270-
271-
# New (recommended)
272-
opm alpha generate dockerfile ./catalog-directory
273-
# Build and serve the generated Dockerfile
274-
```
275-
276-
3. **Replace SQLite database operations****Use declarative config files**
277-
```bash
278-
# Old (deprecated) - SQLite database manipulation
279-
opm registry add --database bundles.db --bundles quay.io/my/bundle:latest
280-
281-
# New (recommended) - File-based catalog
282-
# Create catalog files directly in ./catalog-directory/
283-
```
284-
285-
4. **Convert existing SQLite catalogs****Use migration tools**
286-
```bash
287-
# Convert SQLite index image to FBC
288-
opm migrate quay.io/my-namespace/my-index:latest ./fbc-output --output yaml
289-
290-
# Convert SQLite database to FBC
291-
opm render ./bundles.db --output yaml > catalog.yaml
292-
293-
# Convert FBC to a basic catalog template for simpler maintenance
294-
opm alpha render-template basic ./catalog.yaml
295-
```
296-
297-
### Deprecation Warnings
298-
299-
When using deprecated commands, you will see warnings like:
300-
```
301-
DEPRECATION NOTICE:
302-
Sqlite-based catalogs and their related subcommands are deprecated. Support for
303-
them will be removed in a future release. Please migrate your catalog workflows
304-
to the new file-based catalog format.
305-
```
306-
307-
**Action Required**: Plan your migration to file-based catalogs to avoid future compatibility issues.
308-
309-
### Migration from SQLite to File-Based Catalogs (FBC)
202+
## Removed SQLite Support
310203

311-
#### Using `opm migrate`
312-
The `opm migrate` command converts SQLite-based index images or database files to file-based catalogs:
204+
⚠️ **NOTICE**: SQLite-based catalogs and their related commands (`opm registry`, `opm index`) have been removed from this project. All catalog workflows now use the file-based catalog (FBC) format exclusively.
313205

314-
```bash
315-
# Migrate from SQLite index image to FBC
316-
opm migrate quay.io/my-namespace/my-index:latest ./fbc-output --output yaml
317-
318-
# Migrate from SQLite database file to FBC
319-
opm migrate ./bundles.db ./fbc-output --output json
206+
**Migration**: For older versions that supported SQLite-to-FBC migration, refer to previous releases or documentation versions.
320207

321-
# Migrate with specific migration level
322-
opm migrate quay.io/my-namespace/my-index:latest ./fbc-output
323-
```
208+
### Creating File-Based Catalogs
324209

325210
#### Using `opm render`
326-
The `opm render` command can convert various sources to FBC format:
211+
The `opm render` command converts various sources to FBC format:
327212

328213
```bash
329-
# Render from SQLite database to FBC
330-
opm render ./bundles.db --output yaml > catalog.yaml
331-
332214
# Render from bundle images to FBC
333215
opm render quay.io/my/bundle:v1.0.0 quay.io/my/bundle:v1.1.0 --output json
334216

@@ -350,30 +232,6 @@ opm alpha render-template semver ./semver-template.yaml --output json
350232
opm alpha render-template ./template-file.yaml --output yaml
351233
```
352234

353-
#### Combined Migration Workflow
354-
For complex migrations, combine multiple tools:
355-
356-
```bash
357-
# Step 1: Migrate SQLite to FBC
358-
opm migrate quay.io/my-namespace/my-index:latest ./fbc-output --output yaml
359-
360-
# Step 2: Convert to basic template format (if needed)
361-
opm alpha render-template basic ./fbc-output/package.yaml --output yaml > template.yaml
362-
363-
# Step 3: Modify template as needed, then render final FBC
364-
opm alpha render-template basic ./template.yaml --output yaml > final-catalog.yaml
365-
366-
# Step 4: Serve the new FBC catalog
367-
opm serve ./final-catalog.yaml
368-
```
369-
370-
#### Migration Considerations
371-
- **Bundle images must exist** in registries before they can be referenced in FBC
372-
- **Image references are required** - use `--alpha-image-ref-template` if bundle images don't exist yet
373-
- **Migration levels** control which schema transformations are applied
374-
- **Output formats** include JSON (streamable) and YAML (human-readable)
375-
- **Templates provide flexibility** for custom catalog generation workflows
376-
377235
## File Organization
378236

379237
### Key Directories
@@ -448,23 +306,17 @@ func TestMyFunction(t *testing.T) {
448306
### Common Issues
449307
1. **Bundle validation errors**: Check bundle format and annotations
450308
2. **Image pull failures**: Verify image references and registry access
451-
3. **Database corruption**: Rebuild database from source bundles
452-
4. **Template rendering errors**: Check template syntax and schema
453-
5. **Cache integrity failures**: Rebuild cache or disable integrity checks
454-
6. **gRPC connection issues**: Verify port availability and firewall settings
455-
7. **Memory issues with large catalogs**: Use persistent cache directory
456-
8. **Slow startup times**: Pre-build cache with `--cache-only` flag
457-
9. **SQLite deprecation warnings**: Migrate to file-based catalogs using `opm serve`
458-
10. **Hidden commands**: Some deprecated commands are hidden but still accessible
309+
3. **Template rendering errors**: Check template syntax and schema
310+
4. **Cache integrity failures**: Rebuild cache or disable integrity checks
311+
5. **gRPC connection issues**: Verify port availability and firewall settings
312+
6. **Memory issues with large catalogs**: Use persistent cache directory
313+
7. **Slow startup times**: Pre-build cache with `--cache-only` flag
459314

460315
### Debug Commands
461316
```bash
462317
# Validate bundle
463318
opm alpha bundle validate ./bundle
464319

465-
# Inspect index
466-
opm index export --from-index quay.io/my-namespace/my-index:latest
467-
468320
# Debug template rendering
469321
opm alpha render-template --help
470322

@@ -477,10 +329,6 @@ opm serve ./catalog --pprof-addr localhost:6060 --pprof-capture-profiles
477329
# Test gRPC connectivity
478330
grpcurl -plaintext localhost:50051 list
479331
grpcurl -plaintext localhost:50051 api.Registry/ListPackages
480-
481-
# Deprecated SQLite commands (avoid using these)
482-
opm registry serve --database bundles.db # Use 'opm serve' instead
483-
opm index add --bundles quay.io/my/bundle:latest --tag quay.io/my/index:latest # Use file-based catalogs
484332
```
485333

486334
## Contributing Guidelines

Makefile

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,8 @@ $(PROTOC):
2727
null :=
2828
space := $(null) #
2929
comma := ,
30-
# default to json1 for sqlite3 and containers_image_openpgp for containers/image
31-
TAGS := json1,containers_image_openpgp
30+
# default to containers_image_openpgp for containers/image
31+
TAGS := containers_image_openpgp
3232

3333
# Cluster to use for e2e testing
3434
CLUSTER ?= ""
@@ -62,8 +62,8 @@ build: clean $(CMDS) $(OPM)
6262
cross: opm_version_flags=-ldflags "-X '$(PKG)/cmd/opm/version.gitCommit=$(GIT_COMMIT)' -X '$(PKG)/cmd/opm/version.opmVersion=$(OPM_VERSION)' -X '$(PKG)/cmd/opm/version.buildDate=$(BUILD_DATE)'"
6363
cross:
6464
ifeq ($(shell go env GOARCH),amd64)
65-
GOOS=darwin CC=o64-clang CXX=o64-clang++ CGO_ENABLED=1 CGO_LDFLAGS='-Wl,-undefined,dynamic_lookup' $(GO) build $(opm_version_flags) -tags=$(TAGS) -o "bin/darwin-amd64-opm" --ldflags "-extld=o64-clang -extldflags=-Wl,-undefined,dynamic_lookup" ./cmd/opm
66-
GOOS=windows CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ CGO_ENABLED=1 $(GO) build $(opm_version_flags) -tags=$(TAGS) -o "bin/windows-amd64-opm" --ldflags "-extld=x86_64-w64-mingw32-gcc" -buildmode=exe ./cmd/opm
65+
GOOS=darwin CGO_ENABLED=0 $(GO) build $(opm_version_flags) -tags=$(TAGS) -o "bin/darwin-amd64-opm" ./cmd/opm
66+
GOOS=windows CGO_ENABLED=0 $(GO) build $(opm_version_flags) -tags=$(TAGS) -o "bin/windows-amd64-opm" -buildmode=exe ./cmd/opm
6767
endif
6868

6969
.PHONY: static

README.md

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,18 +14,12 @@ contribution. See the [DCO](DCO) file for details.
1414
This project provides the following binaries:
1515

1616
* `opm`, which generates and updates registry databases as well as the index images that encapsulate them.
17-
* `initializer`, which takes as an input a directory of operator manifests and outputs a sqlite database containing the same data for querying
18-
* Deprecated - use `opm registry|index add` instead
19-
* `registry-server`, which takes a sqlite database loaded with manifests, and exposes a gRPC interface to it.
20-
* Deprecated - use `opm registry serve` instead
21-
* `configmap-server`, which takes a kubeconfig and a configmap reference, and parses the configmap into the sqlite database before exposing it via the same interface as `registry-server`.
2217

2318
And libraries:
2419

2520
* `pkg/client` - providing a high-level client interface for the gRPC api.
26-
* `pkg/api` - providing low-level client libraries for the gRPC interface exposed by `registry-server`.
21+
* `pkg/api` - providing low-level client libraries for the gRPC interface.
2722
* `pkg/registry` - providing basic registry types like Packages, Channels, and Bundles.
28-
* `pkg/sqlite` - providing interfaces for building sqlite manifest databases from `ConfigMap`s or directories, and for querying an existing sqlite database.
2923
* `pkg/lib` - providing external interfaces for interacting with this project as an api that defines a set of standards for operator bundles and indexes.
3024
* `pkg/containertools` - providing an interface to interact with and shell out to common container tooling binaries (if installed on the environment)
3125

alpha/action/list.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,7 @@ func (r *ListBundlesResult) WriteColumns(w io.Writer) error {
186186
func indexRefToModel(ctx context.Context, ref string, reg image.Registry) (model.Model, error) {
187187
render := Render{
188188
Refs: []string{ref},
189-
AllowedRefMask: RefDCImage | RefDCDir | RefSqliteImage | RefSqliteFile,
189+
AllowedRefMask: RefDCImage | RefDCDir,
190190
Registry: reg,
191191
}
192192
cfg, err := render.Run(ctx)

alpha/action/migrate.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ func (m Migrate) Run(ctx context.Context) error {
3434
Migrations: m.Migrations,
3535

3636
// Only allow catalogs to be migrated.
37-
AllowedRefMask: RefSqliteImage | RefSqliteFile | RefDCImage | RefDCDir,
37+
AllowedRefMask: RefDCImage | RefDCDir,
3838
}
3939
if m.Registry != nil {
4040
r.Registry = m.Registry

0 commit comments

Comments
 (0)