Skip to content

Commit 196ddd0

Browse files
author
Kiran Kumar Veeravelly
committed
feat(osgi): add opt-in Apache Felix plugin framework
Add an opt-in plugin-loading mode backed by an embedded Apache Felix OSGi framework, enabled with -Ddata-prepper.plugin.framework=osgi. When the property is unset, Data Prepper uses the existing classpath loader. In that legacy mode plugin discovery is functionally identical to before: the same ServiceLoader providers are found in the same order, the OSGi framework never starts, no OSGi code path executes, and only a couple of inert Spring beans are constructed. Plugins are prepared for OSGi at build time rather than adapted at runtime. A new standalone, publishable Gradle plugin module data-prepper-gradle-plugins/osgi-plugin (plugin id org.opensearch.dataprepper.osgi) bakes an OSGi manifest into the plugin JAR at build time using bnd (biz.aQute.bnd). The module is publishable so plugin authors outside this repository can apply it; in-repo plugins opt in by applying the plugin id. There is no runtime adaptation and no BundleAdapter. At startup in OSGi mode StaticBundleLoader installs the pre-built bundle JARs directly, resolves and starts them with fail-fast behavior, a startup summary log, and Micrometer metrics. A JAR that lacks an OSGi manifest (no Bundle-SymbolicName) fails fast with an actionable message pointing the author to the Gradle plugin. The plugin-framework-osgi module embeds the Felix lifecycle (FelixPluginManager), bootstraps it in OSGi mode through a Spring-managed runner (OsgiFrameworkRunner), and exposes a PluginProvider backed by the OSGi service registry (OsgiPluginRegistry). Baked plugins declare their @DataPrepperPlugin packages through the DataPrepper-Plugin-Classes manifest header, which the Gradle plugin generates from META-INF/data-prepper.plugins.properties, and LegacyPluginBundleActivator registers those classes as OSGi services. Data Prepper API packages are exported to bundles with semver version attributes. The exported package set is generated at build time by a Gradle task that writes META-INF/osgi-shared-packages.properties listing every org.opensearch.dataprepper.* package except org.opensearch.dataprepper.plugins.*, so there is no hand-maintained list. The version comes from the existing data-prepper-api DataPrepperVersion, formatted into the OSGi three-part string by a small package-private toOsgiVersion helper; the separate DataPrepperApiVersion class was removed. This is consistent with the data-prepper-api backward-compatibility contract from #6607, so a plugin built on 2.15 resolves on a 2.16 host. The shared set also includes jakarta.validation, jakarta.validation.constraints, and com.fasterxml.jackson.annotation (annotations only: jackson-databind and jackson-core are intentionally not exported, preserving isolation), plus org.slf4j and the framework package org.opensearch.dataprepper.plugin.osgi so bundles can load LegacyPluginBundleActivator from the system bundle. Felix framework storage resolves under System.getProperty("data-prepper.dir")/data/osgi/ (felix-cache-<pid>), matching the MaxMindConfig idiom, with a java.io.tmpdir fallback when the property is unset such as in tests. It does not use the system temp directory by default. BundleResolutionErrorTranslator turns Felix resolution failures into readable diagnostics. BundleHealthCheck verifies framework, bundle, and classloader isolation. BundleClassLoaderScope manages the thread context classloader at the plugin invocation boundary so ServiceLoader and SPI lookups resolve against the bundle classloader. PluginHealthProbe is a seam for future functional health checks. The OsgiFrameworkRunner bean is always instantiated by Spring but returns early in @PostConstruct unless data-prepper.plugin.framework=osgi. The Felix framework JAR ships on the runtime classpath as a small (<1MB) library with zero transitive dependencies and is not exercised in legacy mode. Scope is limited to classloader isolation and deploy-time validation. Bundles install once at startup and stop at shutdown, the same lifecycle as classpath plugins. There is no production hot-reload; PluginHotLoader is test-scoped only. Data Prepper-owned system properties use the data-prepper. namespace: data-prepper.plugin.framework and data-prepper.plugin.bundles.dir. The genuine OSGi and Felix properties (org.osgi.*, felix.*) are unchanged. Testing: an integration test OsgiIT in a new integrationTest source set under plugin-framework-osgi builds a DefaultPluginFactory like DefaultPluginFactoryIT, loads a test plugin through it, and proves the load happened through OSGi rather than the classpath. It asserts that FrameworkUtil.getBundle is non-null, the plugin classloader is a BundleReference, the classloader is not the application classloader, and the application classloader throws ClassNotFoundException for the plugin class because the plugin is supplied only as a bundle JAR and never on the application classpath. The Gradle plugin has unit and functional (GradleRunner) tests. Wiring changes: settings.gradle adds the plugin-framework-osgi module, the new data-prepper-gradle-plugins/osgi-plugin (consumed via includeBuild in pluginManagement), the data-prepper-test/osgi-test-plugin subproject, a Felix version-catalog entry, and the bnd plugin. build-resources.gradle adds plugin-framework-osgi to coreProjects. data-prepper-core declares a runtimeOnly dependency on the new module. PluginProviderLoader is made public with a registerProvider() hook and caches the framework-mode flag at construction. DefaultPluginFactory evaluates plugin providers lazily so OSGi-registered providers are visible. Tests verify these changes, and legacy behavior is unchanged. The example and test plugin is data-prepper-test/osgi-test-plugin, not a released data-prepper-plugins module. It demonstrates a minimal @DataPrepperPlugin built as an OSGi bundle through the new Gradle plugin. Resolves #6760 Signed-off-by: Kiran Kumar Veeravelly <veeravkk@amazon.com>
1 parent 4818896 commit 196ddd0

48 files changed

Lines changed: 5765 additions & 27 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

build-resources.gradle

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,5 +19,6 @@ ext.coreProjects = [
1919
project(':data-prepper-test'),
2020
project(':data-prepper-plugin-framework'),
2121
project(':data-prepper-plugin-schema'),
22-
project(':data-prepper-plugin-schema-cli')
22+
project(':data-prepper-plugin-schema-cli'),
23+
project(':plugin-framework-osgi')
2324
]

data-prepper-core/build.gradle

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ dependencies {
3737
implementation project(':data-prepper-logstash-configuration')
3838
implementation project(':data-prepper-pipeline-parser')
3939
implementation project(':data-prepper-plugin-framework')
40+
runtimeOnly project(':plugin-framework-osgi')
4041
testImplementation project(':data-prepper-plugin-framework').sourceSets.test.output
4142
testImplementation project(':data-prepper-plugins:common').sourceSets.test.output
4243
testImplementation project(':data-prepper-plugins:file-source')
Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# Data Prepper OSGi Gradle Plugin
2+
3+
A standalone, publishable Gradle plugin that prepares a Data Prepper plugin JAR
4+
for OSGi consumption by baking an OSGi-compliant manifest into it at build time.
5+
6+
This replaces the runtime JAR-to-bundle adaptation previously performed by
7+
`BundleAdapter` in `plugin-framework-osgi`. By generating the manifest at build
8+
time, plugins are valid OSGi bundles from the moment they are built — no runtime
9+
repackaging is required.
10+
11+
## Plugin ID
12+
13+
```
14+
org.opensearch.dataprepper.osgi
15+
```
16+
17+
## Usage
18+
19+
### Internal Data Prepper plugin projects
20+
21+
In your plugin's `build.gradle`:
22+
23+
```groovy
24+
plugins {
25+
id 'org.opensearch.dataprepper.osgi'
26+
}
27+
```
28+
29+
### External plugin authors
30+
31+
Add the plugin to your `buildscript` dependencies or plugin management:
32+
33+
```groovy
34+
// settings.gradle
35+
pluginManagement {
36+
repositories {
37+
mavenCentral()
38+
// or wherever Data Prepper publishes its artifacts
39+
}
40+
plugins {
41+
id 'org.opensearch.dataprepper.osgi' version '<data-prepper-version>'
42+
}
43+
}
44+
```
45+
46+
Then in your plugin project's `build.gradle`:
47+
48+
```groovy
49+
plugins {
50+
id 'org.opensearch.dataprepper.osgi'
51+
}
52+
```
53+
54+
## Requirements
55+
56+
Your project must include a resource file at:
57+
58+
```
59+
src/main/resources/META-INF/data-prepper.plugins.properties
60+
```
61+
62+
With the following property:
63+
64+
```properties
65+
org.opensearch.dataprepper.plugin.packages=com.example.myplugin
66+
```
67+
68+
The value is a comma-separated list of Java package names that contain your
69+
`@DataPrepperPlugin`-annotated classes. The plugin scans these packages at
70+
bundle activation time to register your plugin classes with the OSGi service
71+
registry.
72+
73+
If this file is absent, the build will fail with a clear error message.
74+
75+
## Generated Manifest Headers
76+
77+
When applied, this plugin produces the following OSGi manifest headers in the
78+
output JAR:
79+
80+
| Header | Value | Derivation |
81+
|--------|-------|------------|
82+
| `Bundle-SymbolicName` | `org.opensearch.dataprepper.plugin.<sanitized-name>` | Project name with non-alphanumeric chars replaced by dots |
83+
| `Bundle-Version` | OSGi-normalized project version | `2.16.0-SNAPSHOT` becomes `2.16.0`; ensures 3-part numeric |
84+
| `Export-Package` | All packages except `*.internal.*` | Computed by bnd from compiled bytecode |
85+
| `Import-Package` | `*` (bnd default) | Computed by bnd from bytecode dependency analysis |
86+
| `Bundle-Activator` | `org.opensearch.dataprepper.plugin.osgi.LegacyPluginBundleActivator` | Fixed; provided by `plugin-framework-osgi` at runtime |
87+
| `DataPrepper-Plugin-Classes` | Comma-separated package names | Read from `data-prepper.plugins.properties` |
88+
89+
## How it works
90+
91+
1. The plugin applies `biz.aQute.bnd.builder` to the consuming project.
92+
2. After project evaluation, it reads `data-prepper.plugins.properties` from the
93+
main resource directories.
94+
3. It configures the jar task's bnd instructions with the headers listed above.
95+
4. At jar time, bnd analyzes the compiled bytecode to compute accurate
96+
`Import-Package` and `Export-Package` values — this is more reliable than the
97+
runtime heuristic approach that `BundleAdapter` used.
98+
99+
## Comparison with BundleAdapter
100+
101+
| Aspect | BundleAdapter (runtime) | This plugin (build time) |
102+
|--------|------------------------|--------------------------|
103+
| When it runs | At application startup | At build time |
104+
| Import-Package | Static list of shared API packages | bnd computes from actual bytecode |
105+
| Export-Package | Discovered by scanning JAR entries | bnd computes from compiled classes |
106+
| Bundle-Version | Always `1.0.0` | Actual project version (OSGi-normalized) |
107+
| Performance | Rewrites JARs at startup | Zero startup cost |
108+
| External plugins | Must be adapted at runtime | Already valid bundles |
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
/*
2+
* Copyright OpenSearch Contributors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*
5+
* The OpenSearch Contributors require contributions made to
6+
* this file be licensed under the Apache-2.0 license or a
7+
* compatible open source license.
8+
*
9+
*/
10+
11+
plugins {
12+
id 'java-gradle-plugin'
13+
id 'maven-publish'
14+
}
15+
16+
group = 'org.opensearch.dataprepper'
17+
18+
repositories {
19+
mavenCentral()
20+
gradlePluginPortal()
21+
}
22+
23+
dependencies {
24+
implementation gradleApi()
25+
implementation 'biz.aQute.bnd:biz.aQute.bnd.gradle:7.0.0'
26+
testImplementation 'org.junit.jupiter:junit-jupiter:5.8.2'
27+
testImplementation 'org.junit.jupiter:junit-jupiter-params:5.8.2'
28+
testImplementation 'org.hamcrest:hamcrest:2.2'
29+
}
30+
31+
gradlePlugin {
32+
plugins {
33+
dataPrepperOsgi {
34+
id = 'org.opensearch.dataprepper.osgi'
35+
implementationClass = 'org.opensearch.dataprepper.gradle.plugin.osgi.DataPrepperOsgiPlugin'
36+
}
37+
}
38+
}
39+
40+
test {
41+
useJUnitPlatform()
42+
}
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
/*
2+
* Copyright OpenSearch Contributors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*
5+
* The OpenSearch Contributors require contributions made to
6+
* this file be licensed under the Apache-2.0 license or a
7+
* compatible open source license.
8+
*/
9+
10+
pluginManagement {
11+
repositories {
12+
mavenCentral()
13+
gradlePluginPortal()
14+
}
15+
}
16+
17+
rootProject.name = 'osgi-plugin'

0 commit comments

Comments
 (0)