Move JSON ingestion into elasticgraph-json_ingestion via an indexer extension#1302
Open
jwils wants to merge 1 commit into
Open
Move JSON ingestion into elasticgraph-json_ingestion via an indexer extension#1302jwils wants to merge 1 commit into
jwils wants to merge 1 commit into
Conversation
jwils
requested review from
BrianSigafoos-SQ,
bsorbo,
ellisandrews-toast,
jwondrusch,
marcdaniels-toast,
myronmarston and
rossroberts-toast
as code owners
jwils
force-pushed
the
joshuaw/json-ingestion-adapter
branch
2 times, most recently
from
bd53a07 to
bcc1c1a
Compare
jwils
force-pushed
the
joshuaw/ingestion-adapter-seam
branch
from
1f731a7 to
6e17704
Compare
…xtension The JSON ingestion adapter and JSON-schema-based record preparer factory move from elasticgraph-indexer to elasticgraph-json_ingestion, and the JSON ingestion schema definition extension registers an indexer extension in runtime metadata that makes the adapter available at indexing time. Any schema defined with JSON ingestion support automatically gets JSON event ingestion--no settings needed--and elasticgraph-indexer no longer contains (or refers to) any JSON-specific ingestion logic. The be_a_valid_elastic_graph_event matcher moves to elastic_graph/json_ingestion/spec_support/event_matcher, and suites that build indexers from the shared test schema artifacts gain a development dependency on elasticgraph-json_ingestion (whose indexer extension those artifacts now reference).
jwils
force-pushed
the
joshuaw/json-ingestion-adapter
branch
from
bcc1c1a to
4724804
Compare
jwils
force-pushed
the
joshuaw/ingestion-adapter-seam
branch
from
6e17704 to
a9d37cc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
JSON parsing and validation has never conceptually belonged to
elasticgraph-indexer--it should live inelasticgraph-json_ingestionalongside the JSON schema generation, with the indexer staying ingestion-format-neutral soelasticgraph-proto_ingestion(and others) can plug in the same way.What
elasticgraph-json_ingestion(JSONIngestion::IngestionAdapter,JSONIngestion::RecordPreparerFactory)JSONIngestion::SchemaDefinition::APIExtensionregisters aJSONIngestion::IndexerExtensionviaregister_indexer_extension, which contributes the adapter toIndexer#ingestion_adapters. Zero configuration: any schema defined with JSON ingestion support automatically gets JSON event ingestion at indexing time, activated through the schema artifacts' runtime metadataelasticgraph-indexerno longer contains or references any JSON-specific ingestion logic;Indexer#record_preparer_factoryis removed, andoperation_factoryraises a clearConfigErrorwhen a schema has no ingestion extensionsbe_a_valid_elastic_graph_eventmatcher moves toelastic_graph/json_ingestion/spec_support/event_matcher(breaking require-path change)elasticgraph-json_ingestionRemaining follow-up
Indexer::TestSupport::Convertersstill builds JSON-envelope events (json_schema_version); moving it requires touchingelasticgraph-local's runtime fake-data indexing, so it's deferred to a follow-up PR.Verification
script/quick_build(5160 examples, 0 failures, plus site validation with doctests)run_each_gem_specCI mode), all green with 100% coverageStack
Current PR is marked with
->.