Skip to content

Commit 5030acc

Browse files
jackshiraziCopilot
andauthored
[dynamic control] Update README on instructions for using the project as an extension (open-telemetry#2885)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
1 parent c8592f9 commit 5030acc

1 file changed

Lines changed: 220 additions & 0 deletions

File tree

dynamic-control/README.md

Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,226 @@ Adding dynamic control of some specific features of the Java agent.
1010
1111
Current plans and progress is tracked in this [meta issue](https://github.com/open-telemetry/opentelemetry-java-contrib/issues/2416)
1212

13+
## Using this project
14+
15+
This project has now reached a stage where it is usable but subject to change in any part including the API.
16+
17+
## Telemetry Policy
18+
19+
The dynamic control is implemented using [Telemetry Policy](https://github.com/open-telemetry/opentelemetry-specification/pull/4738) prototype.
20+
An abstract outline of dynamic control using telemetry policies is that there is a flow consisting of
21+
22+
```text
23+
Message -> Provider -> Policy -> Policy aggregator -> Implementer
24+
25+
```
26+
27+
A concrete example helps to understand this flow:
28+
29+
1. Message: A message is added (to OpAMP data structure) to specify a change to the trace "sampling rate"
30+
2. Provider: The OpampPolicyProvider reads the message from the OpAMP data structure received via OpAMP protocol
31+
3. Policy: The message is converted into a TraceSamplingRatePolicy, ie the policy that changes the sampling rate,
32+
with the target "traceid rate"
33+
4. Policy aggregator: The policy is combined with any other policy changes already applied or pending,
34+
handling source priority and potential merges of policies
35+
5. Implementer: TraceSamplingRatePolicyImplementer takes the policy with the new traceid sampling rate
36+
and applies it to the sampler
37+
38+
## Quick Unix example
39+
40+
The following would test an agent-enabled app configured to use OpAMP to receive messages
41+
that would dynamically change the trace sampling rate:
42+
43+
```bash
44+
git clone https://github.com/open-telemetry/opentelemetry-java-contrib.git
45+
cd opentelemetry-java-contrib
46+
./gradlew dynamic-control:shadowJar
47+
cp dynamic-control/build/libs/opentelemetry-dynamic-control-*-alpha-SNAPSHOT-all.jar ./opentelemetry-dynamic-control-all.jar
48+
echo "sources:" > config.yaml
49+
echo " - kind: opamp" >> config.yaml
50+
echo " format: jsonkeyvalue" >> config.yaml
51+
echo " location: vendor" >> config.yaml
52+
echo " mappings:" >> config.yaml
53+
echo " - sourceKey: sampling_rate" >> config.yaml
54+
echo " policyType: trace-sampling" >> config.yaml
55+
java -Dotel.javaagent.extensions=./opentelemetry-dynamic-control-all.jar -Dotel.java.experimental.telemetry.policy.init.yaml=./config.yaml -Dotel.opamp.service.url=http://127.0.0.1:4320/v1/opamp -javaagent:path/to/opentelemetry-javaagent.jar -Dotel.service.name=my-service -jar myapp.jar
56+
```
57+
58+
## Building
59+
60+
Building consists of executing `gradlew`, and producing a jar in the
61+
`dynamic-control/build/libs`/`dynamic-control\build\libs` directory
62+
63+
### Gradle execution
64+
65+
You can execute gradlew either from the repo root, or the project root, eg
66+
67+
```bash
68+
git clone https://github.com/open-telemetry/opentelemetry-java-contrib.git
69+
cd opentelemetry-java-contrib
70+
#/REM Unix or windows, execute the gradlew from this directory
71+
./gradlew dynamic-control:<TARGET>
72+
.\gradlew dynamic-control:<TARGET>
73+
```
74+
75+
or
76+
77+
```bash
78+
git clone https://github.com/open-telemetry/opentelemetry-java-contrib.git
79+
cd opentelemetry-java-contrib
80+
cd dynamic-control
81+
#/REM Unix or windows, execute the gradlew from the directory above
82+
../gradlew <TARGET>
83+
..\gradlew <TARGET>
84+
```
85+
86+
Useful values for `<TARGET>` are `jar` for the jar containing the classes from this project,
87+
and `shadowJar` to create a jar containing the classes from this project plus all dependencies.
88+
The latter target will produce a `*-all.jar` jar. The extension is loaded in a separate classloader
89+
so there are unlikely to be classloading conflicts, but it's not impossible. The `jar` target
90+
requires you to ensure you have all required dependencies in the JVM classpath. Using the `shadowJar`
91+
is usually easier and more useful, but has a larger jar size and may bring in classes you won't need.
92+
I'd start with `shadowJar` then switch to `jar` and explicit dependency management if that's needed.
93+
94+
## Using as an extension
95+
96+
The jar from the build needs to be accessible by the agent during startup.
97+
Assume you have put the jar at /path/to/jar (or \path\to\jar) then you need to use
98+
the option `otel.javaagent.extensions` (or OTEL_JAVAAGENT_EXTENSIONS environment variable) to specify where the jar is,
99+
eg `java -Dotel.javaagent.extensions=/path/to/jar ...` when starting the application with the agent,
100+
for the agent to include the extension.
101+
102+
Simply including the extension will not enable anything by default. You also need to configure the extension.
103+
The next sections explain that.
104+
First the general config idea is covered, then a more detailed explanation of the config that can be used.
105+
106+
### Using as a declarative config extension
107+
108+
The declarative config file should include a top level `telemetry_policy/development` node which then
109+
uses the config defined below starting at `sources` for its content config, eg
110+
111+
```yaml
112+
telemetry_policy/development:
113+
sources:
114+
- kind: opamp
115+
format: jsonkeyvalue
116+
location: vendor
117+
mappings:
118+
- sourceKey: sampling_rate
119+
policyType: trace-sampling
120+
121+
```
122+
123+
### Using as an auto-configured extension
124+
125+
You can use either `otel.java.experimental.telemetry.policy.init.yaml`
126+
(or OTEL_JAVA_EXPERIMENTAL_TELEMETRY_POLICY_INIT_YAML environment variable)
127+
to specify the file containing a YAML configuration of the policies,
128+
or `otel.java.experimental.telemetry.policy.init.json`
129+
(or OTEL_JAVA_EXPERIMENTAL_TELEMETRY_POLICY_INIT_JSON environment variable)
130+
to specify the file containing a JSON configuration of the policies.
131+
For example `java -Dotel.javaagent.extensions=/path/to/jar
132+
-Dotel.java.experimental.telemetry.policy.init.yaml=/path/to/yaml ...`
133+
where /path/to/yaml is a file containing the config. eg
134+
135+
```yaml
136+
sources:
137+
- kind: opamp
138+
format: jsonkeyvalue
139+
location: vendor
140+
mappings:
141+
- sourceKey: sampling_rate
142+
policyType: trace-sampling
143+
144+
```
145+
146+
### Config Available
147+
148+
The config tree starts with `sources`. You can configure multiple sources.
149+
150+
Each source must specify:
151+
152+
* `kind`: where policy updates come from. Supported values: `opamp`, `file`, `http` and `custom`
153+
(currently only `opamp` creates an active provider, the others are no-op providers)
154+
* `opamp`: the implemented OpAMP provider expects to read the OpAMP config map,
155+
finding the value at the key given by `location`. The contents of that value are parseable by the capability given in `format`
156+
* `format`: how the source payload is parsed. Supported values currently are `jsonkeyvalue` and `keyvalue`
157+
* `jsonkeyvalue`: expects the contents to be convertible as a string into a single or an array of
158+
simple json objects that are key and value, eg '{ "key": value}' or '[{ "key1": value1}, { "key2": value2}]'
159+
* `keyvalue`: expects the contents to be convertible as a string into one or more
160+
line separated 'key=value' pairs (eg a properties file)
161+
* `mappings`: one or more mappings from source-specific keys to dynamic-control policy types
162+
* `location`: optional source-specific selector. For `opamp`, this is the OpAMP config map key, for example `vendor`
163+
164+
Each mapping must specify:
165+
166+
* `sourceKey`: the key in the source payload, for example `sampling_rate` (this is an arbitrary string defined
167+
by the user or already being used/sent from some source)
168+
* `policyType`: the dynamic-control policy type to update, currently only `trace-sampling` is valid
169+
170+
Currently supported values in summary
171+
172+
```yaml
173+
sources:
174+
- kind: opamp|file|http|custom
175+
format: jsonkeyvalue|keyvalue
176+
location: 'opamp config map key'|'file path'|'http url'|omit row
177+
mappings:
178+
- sourceKey: user-defined-string
179+
policyType: trace-sampling
180+
181+
```
182+
183+
### Policies supported
184+
185+
* `trace-sampling`
186+
* IMPORTANT: if this policy is included in the config, then the sampler installed is overridden
187+
and a consistent sampling sampler is installed
188+
(technically the ComposableSampler.parentThreshold(ComposableSampler.probability()) sampler)
189+
* Expects a value between 0.0 and 1.0 (including both end values), and will apply that sampling rate
190+
to the agent's sampler where 0.0 is 0% head sampling and 1.0 is 100% sampling
191+
192+
### Config example
193+
194+
Working through the following example may be helpful (it assumes the `file` source has been implemented,
195+
this is not yet the case).
196+
197+
```yaml
198+
sources:
199+
- kind: opamp
200+
format: jsonkeyvalue
201+
location: vendor
202+
mappings:
203+
- sourceKey: sampling_rate
204+
policyType: trace-sampling
205+
- kind: file
206+
format: keyvalue
207+
location: /path/to/here.conf
208+
mappings:
209+
- sourceKey: trace_rate
210+
policyType: trace-sampling
211+
- sourceKey: traceid_ratio
212+
policyType: trace-sampling
213+
214+
```
215+
216+
There are two sources. The first expects a message from an OpAMP server (`kind: opamp`),
217+
from which it will access the config map and extract the value at key `vendor` (`location: vendor`).
218+
The value is expected to be this json style key-value (`format: jsonkeyvalue`)
219+
object string (ignoring whitespace and numeric diffs) {"sampling_rate": 0.5} (key defined by `sourceKey: sampling_rate`).
220+
On receipt of this, the message is converted to a trace-sampling policy (`policyType: trace-sampling`)
221+
and the new sampling rate applied to the sampler.
222+
223+
The second source expects a file (`kind: file`) at file path /path/to/here.conf (`location: /path/to/here.conf`)
224+
which when changed will be re-read. The contents are expected to be key=value entries,
225+
one per line (`format: keyvalue`). The only keys recognized are `trace_rate` (`sourceKey: trace_rate`)
226+
and `traceid_ratio` (`sourceKey: traceid_ratio`). When the value changes, the message is converted
227+
to a trace-sampling policy (`policyType: trace-sampling`)
228+
and the new sampling rate applied to the sampler.
229+
230+
Because `opamp` source has higher priority than `file` source, if both sources generate a change
231+
at the same time, the opamp change would be applied and the file change dropped.
232+
13233
## Component owners
14234

15235
* [Jack Shirazi](https://github.com/jackshirazi), Elastic

0 commit comments

Comments
 (0)