Commit 196ddd0
Kiran Kumar Veeravelly
feat(osgi): add opt-in Apache Felix plugin framework
Add an opt-in plugin-loading mode backed by an embedded Apache Felix
OSGi framework, enabled with -Ddata-prepper.plugin.framework=osgi. When
the property is unset, Data Prepper uses the existing classpath loader.
In that legacy mode plugin discovery is functionally identical to before:
the same ServiceLoader providers are found in the same order, the OSGi
framework never starts, no OSGi code path executes, and only a couple of
inert Spring beans are constructed.
Plugins are prepared for OSGi at build time rather than adapted at
runtime. A new standalone, publishable Gradle plugin module
data-prepper-gradle-plugins/osgi-plugin (plugin id
org.opensearch.dataprepper.osgi) bakes an OSGi manifest into the plugin
JAR at build time using bnd (biz.aQute.bnd). The module is publishable so
plugin authors outside this repository can apply it; in-repo plugins opt
in by applying the plugin id. There is no runtime adaptation and no
BundleAdapter. At startup in OSGi mode StaticBundleLoader installs the
pre-built bundle JARs directly, resolves and starts them with fail-fast
behavior, a startup summary log, and Micrometer metrics. A JAR that lacks
an OSGi manifest (no Bundle-SymbolicName) fails fast with an actionable
message pointing the author to the Gradle plugin.
The plugin-framework-osgi module embeds the Felix lifecycle
(FelixPluginManager), bootstraps it in OSGi mode through a Spring-managed
runner (OsgiFrameworkRunner), and exposes a PluginProvider backed by the
OSGi service registry (OsgiPluginRegistry). Baked plugins declare their
@DataPrepperPlugin packages through the DataPrepper-Plugin-Classes
manifest header, which the Gradle plugin generates from
META-INF/data-prepper.plugins.properties, and LegacyPluginBundleActivator
registers those classes as OSGi services.
Data Prepper API packages are exported to bundles with semver version
attributes. The exported package set is generated at build time by a
Gradle task that writes META-INF/osgi-shared-packages.properties listing
every org.opensearch.dataprepper.* package except
org.opensearch.dataprepper.plugins.*, so there is no hand-maintained
list. The version comes from the existing data-prepper-api
DataPrepperVersion, formatted into the OSGi three-part string by a small
package-private toOsgiVersion helper; the separate DataPrepperApiVersion
class was removed. This is consistent with the data-prepper-api
backward-compatibility contract from #6607, so a plugin built on 2.15
resolves on a 2.16 host. The shared set also includes jakarta.validation,
jakarta.validation.constraints, and com.fasterxml.jackson.annotation
(annotations only: jackson-databind and jackson-core are intentionally
not exported, preserving isolation), plus org.slf4j and the framework
package org.opensearch.dataprepper.plugin.osgi so bundles can load
LegacyPluginBundleActivator from the system bundle.
Felix framework storage resolves under
System.getProperty("data-prepper.dir")/data/osgi/ (felix-cache-<pid>),
matching the MaxMindConfig idiom, with a java.io.tmpdir fallback when the
property is unset such as in tests. It does not use the system temp
directory by default.
BundleResolutionErrorTranslator turns Felix resolution failures into
readable diagnostics. BundleHealthCheck verifies framework, bundle, and
classloader isolation. BundleClassLoaderScope manages the thread context
classloader at the plugin invocation boundary so ServiceLoader and SPI
lookups resolve against the bundle classloader. PluginHealthProbe is a
seam for future functional health checks.
The OsgiFrameworkRunner bean is always instantiated by Spring but returns
early in @PostConstruct unless data-prepper.plugin.framework=osgi. The
Felix framework JAR ships on the runtime classpath as a small (<1MB)
library with zero transitive dependencies and is not exercised in legacy
mode.
Scope is limited to classloader isolation and deploy-time validation.
Bundles install once at startup and stop at shutdown, the same lifecycle
as classpath plugins. There is no production hot-reload; PluginHotLoader
is test-scoped only.
Data Prepper-owned system properties use the data-prepper. namespace:
data-prepper.plugin.framework and data-prepper.plugin.bundles.dir. The
genuine OSGi and Felix properties (org.osgi.*, felix.*) are unchanged.
Testing: an integration test OsgiIT in a new integrationTest source set
under plugin-framework-osgi builds a DefaultPluginFactory like
DefaultPluginFactoryIT, loads a test plugin through it, and proves the
load happened through OSGi rather than the classpath. It asserts that
FrameworkUtil.getBundle is non-null, the plugin classloader is a
BundleReference, the classloader is not the application classloader, and
the application classloader throws ClassNotFoundException for the plugin
class because the plugin is supplied only as a bundle JAR and never on
the application classpath. The Gradle plugin has unit and functional
(GradleRunner) tests.
Wiring changes: settings.gradle adds the plugin-framework-osgi module,
the new data-prepper-gradle-plugins/osgi-plugin (consumed via
includeBuild in pluginManagement), the data-prepper-test/osgi-test-plugin
subproject, a Felix version-catalog entry, and the bnd plugin.
build-resources.gradle adds plugin-framework-osgi to coreProjects.
data-prepper-core declares a runtimeOnly dependency on the new module.
PluginProviderLoader is made public with a registerProvider() hook and
caches the framework-mode flag at construction. DefaultPluginFactory
evaluates plugin providers lazily so OSGi-registered providers are
visible. Tests verify these changes, and legacy behavior is unchanged.
The example and test plugin is data-prepper-test/osgi-test-plugin, not a
released data-prepper-plugins module. It demonstrates a minimal
@DataPrepperPlugin built as an OSGi bundle through the new Gradle plugin.
Resolves #6760
Signed-off-by: Kiran Kumar Veeravelly <veeravkk@amazon.com>1 parent 4818896 commit 196ddd0
48 files changed
Lines changed: 5765 additions & 27 deletions
File tree
- data-prepper-core
- data-prepper-gradle-plugins/osgi-plugin
- src
- main/java/org/opensearch/dataprepper/gradle/plugin/osgi
- test/java/org/opensearch/dataprepper/gradle/plugin/osgi
- data-prepper-plugin-framework/src
- main/java/org/opensearch/dataprepper/plugin
- test/java/org/opensearch/dataprepper/plugin
- data-prepper-test/osgi-test-plugin
- src
- main
- java/org/opensearch/dataprepper/plugins/processor/osgitestecho
- resources/META-INF
- test/java/org/opensearch/dataprepper/plugins/processor/osgitestecho
- plugin-framework-osgi
- src
- integrationTest/java/org/opensearch/dataprepper/plugin/osgi
- main/java/org/opensearch/dataprepper/plugin/osgi
- test/java/org/opensearch/dataprepper/plugin/osgi
- fixture
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
| 23 | + | |
23 | 24 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
40 | 41 | | |
41 | 42 | | |
42 | 43 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
0 commit comments