feat: add elasticsearch backend provider

[houyuwushang]

Description

Adds a first-class Elasticsearch backend provider.

The provider uses Delta Proxy Cache for aggregation-only _search and _msearch requests with size: 0, one inclusive range over the configured timestamp field, and one top-level fixed-interval date_histogram over that field. It normalizes the range in the DPC cache key, rewrites range and histogram bounds for missing extents, and preserves each histogram bucket's JSON, including nested metric aggregations.

The default timestamp field is @timestamp, with backends.<name>.elasticsearch.timestamp_field available for other indices. Query shapes that cannot be merged without changing Elasticsearch semantics fall back to Object Proxy Cache with the exact request body in the key. This includes document-hit searches, calendar or shifted bucket intervals, multiple ranges or top-level aggregations, and unsupported _msearch requests. Write-oriented methods are proxied without caching.

The developer environment includes Elasticsearch, generated recent and older log data, Kibana routed through Trickster, and seeded Kibana data-view/search/dashboard objects.

Fixes #7.

Type of Change

• • Bug fix
• • New feature
• • Optimization
• • Test coverage
• • Documentation
• • Infrastructure

AI Disclosure

• • This contribution DOES NOT include AI-generated changes
• • This contribution DOES include AI-generated changes, and I have reviewed the relevant contributing guidelines.

Validation

• go test ./pkg/backends/elasticsearch/... ./pkg/backends/options ./pkg/backends/providers/... ./pkg/proxy/handlers/... ./pkg/config/validate -count=1
• go test -race ./pkg/backends/elasticsearch/... -count=1
• go test ./pkg/backends/elasticsearch -run '^TestQueryHandlerFallbackCacheSeparatesTimeRanges$' -count=20
• go tool golangci-lint run --timeout 5m ./pkg/backends/elasticsearch/...
• go test -run '^$' ./...
• configuration validation for the full example and developer environment
• clean Docker Compose startup on Linux with Elasticsearch/Kibana; verified Trickster-backed index, count, and date-histogram requests plus the seeded Kibana objects
• git diff --check

[jranson]

@houyuwushang Thanks for this! One thing we'd like to see is an updated developer environment docker-compose.yml that includes:

• a new Elasticsearch backend added to the existing Trickster config yaml
• Elasticsearch containers included in the compose, with the Elasticsearch UX/Frontend configured with Trickster as the Elasticsearch API (instead of pointing the frontend directly at the underlying API)
• Seeded log data to enable out-of-box testing/verification (e.g., logs dynamically generated on environment startup that include recent timestamps to ensure they are properly cached by Trickster - and possibly some older timestamps to exercise time-based cacheable boundaries; refer to the ClickHouse data seeding in the developer env for ideas)

[houyuwushang]

Updated this PR to include the developer Elasticsearch environment requested here:

• added an es1 Elasticsearch backend to the developer Trickster config, using the memory cache and @timestamp field
• added elasticsearch, elasticsearch_seed, and kibana services to the developer docker-compose file
• configured Kibana's ELASTICSEARCH_HOSTS to point at Trickster's es1 API (http://host.docker.internal:8480/es1) rather than the underlying Elasticsearch service
• added a startup seed script that recreates trickster-dev-logs with generated recent and older @timestamp log records for date histogram testing
• documented the Kibana URL and seeded index in the developer environment README

Validated with:

• go run cmd/trickster/main.go -validate-config -config docs/developer/environment/trickster-config/trickster.yaml
• go test ./pkg/backends/elasticsearch/... ./pkg/backends/options ./pkg/backends/providers ./pkg/config/validate -count=1
• go test -run '^$' ./...
• go tool golangci-lint run --timeout 5m ./pkg/backends/elasticsearch/... ./pkg/backends/options ./pkg/backends/providers ./pkg/proxy/handlers/elasticsearch ./pkg/config/validate
• YAML structural check of docs/developer/environment/docker-compose.yml via Python/PyYAML
• git diff --check

I could not run docker compose config or start the compose stack locally because this environment does not have the Docker CLI available.

[jranson]

I could not run docker compose config or start the compose stack locally because this environment does not have the Docker CLI available.

@houyuwushang this comment appears to be from your AI agent. While it's OK to use agentic AI, you will need to install docker locally, run the docker compose and visually verify the new functionality for the developer environment. As reviewers, we are using and verifying changes with the developer environment; and other contributors will expect it to work out of the box without any additional manual steps. My initial tests of the updated docker compose indicate that the new Elasticsearch containers are running, but the Kibana instance prints a lot of errors and does not fully integrate with the trickster backend. For example, the Kibana UX does not show the trickster-dev-logs index in the index list, and also lacks any pre-configured dashboards to demonstrate and exercise the new functionality using the generated logs.

[houyuwushang]

@jranson Thanks, that feedback is fair. To clarify the earlier validation note: the machine I used for the first update did not have a Docker-capable developer environment available, so I should have moved the runtime validation to a suitable environment before posting that update. I have now done that and pushed 4dc4df7 based on the runtime testing.

That update adds:

• host.docker.internal:host-gateway for Kibana so Linux Docker environments can resolve the host Trickster process
• a kibana_seed service that waits for Kibana, then creates the trickster-dev-logs data view, saved search, and Trickster Dev Logs dashboard
• README notes for the seeded Kibana dashboard URL

Additional validation performed on a Docker-capable Linux environment:

• docker compose config
• sh -n docs/developer/environment/docker-compose-data/kibana-config/seeding/seed.sh
• clean compose rebuild/start with fresh volumes for the ES/Kibana developer services
• go run cmd/trickster/main.go -config docs/developer/environment/trickster-config/trickster.yaml --log-level debug
• verified http://127.0.0.1:8480/es1/_cat/indices?v includes trickster-dev-logs
• verified direct Elasticsearch and Trickster-backed _count both return 252 documents for trickster-dev-logs
• verified a date histogram query through http://127.0.0.1:8480/es1/trickster-dev-logs/_search
• verified Kibana status is available
• verified kibana_seed exits 0 and creates the data view, saved search, and dashboard

GitHub checks are green on the updated head as well.