Skip to content

Commit 3889174

Browse files
committed
chore: update docs, bump maven version
1 parent 17bf3d8 commit 3889174

7 files changed

Lines changed: 38 additions & 8 deletions

File tree

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,9 @@ A typical `data-pipeline` use-case can be:
1313
* a system performing iterative analysis on an input: the `data-pipeline` data model retains intermediate results from all steps, and each result is tagged with lineage metadata
1414

1515
On top of its core feature-set, complying to the `data-pipeline` model comes with rather nice benefits:
16-
* out-of-the-box support for [micrometer](https://micrometer.io) based metrics (success/failure rates, error tracking, etc.)
16+
* out-of-the-box support for [micrometer](https://micrometer.io) based [metrics](doc/integrations.md#metrics) (success/failure rates, error tracking, etc.) and [tracing](doc/integrations.md#tracing)
1717
* out-of-the-box support for [slf4j](https://www.slf4j.org) log markers (pipeline id, component id, etc.)
18-
* easily pluggable [resilience4j](https://resilience4j.readme.io) based resilience features (retries, time-limiter, etc.)
18+
* easily pluggable [resilience4j](https://resilience4j.readme.io) based [resilience features](doc/integrations.md#resilience4j) (retries, time-limiter, etc.)
1919

2020
## I. Installation
2121

@@ -25,7 +25,7 @@ The library requires Java 17+, in order to use it, add the following in your `po
2525
<dependency>
2626
<groupId>tech.illuin</groupId>
2727
<artifactId>data-pipeline</artifactId>
28-
<version>0.23.4</version>
28+
<version>0.24</version>
2929
</dependency>
3030
```
3131

core/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
<parent>
88
<groupId>tech.illuin</groupId>
99
<artifactId>data-pipeline-parent</artifactId>
10-
<version>0.23.4</version>
10+
<version>0.24</version>
1111
</parent>
1212

1313
<artifactId>data-pipeline</artifactId>

core/src/main/java/tech/illuin/pipeline/builder/PayloadPipelineBuilder.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -356,12 +356,18 @@ public PayloadPipelineBuilder<I> setDefaultSinkErrorHandler(SinkErrorHandler def
356356
return this;
357357
}
358358

359+
/**
360+
* @deprecated Use {@link #observabilityManager()} instead
361+
*/
359362
@Deprecated
360363
public MeterRegistry meterRegistry()
361364
{
362365
return this.observabilityManagerBuilder.meterRegistry();
363366
}
364367

368+
/**
369+
* @deprecated Use {@link #addObservabilityComponent(MeterRegistry)} instead
370+
*/
365371
@Deprecated
366372
public PayloadPipelineBuilder<I> setMeterRegistry(MeterRegistry meterRegistry)
367373
{

core/src/main/java/tech/illuin/pipeline/builder/SimplePipelineBuilder.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -344,12 +344,18 @@ public SimplePipelineBuilder<I> setDefaultSinkErrorHandler(SinkErrorHandler defa
344344
return this;
345345
}
346346

347+
/**
348+
* @deprecated Use {@link #observabilityManager()} instead
349+
*/
347350
@Deprecated
348351
public MeterRegistry meterRegistry()
349352
{
350353
return this.observabilityManagerBuilder.meterRegistry();
351354
}
352355

356+
/**
357+
* @deprecated Use {@link #addObservabilityComponent(MeterRegistry)} instead
358+
*/
353359
@Deprecated
354360
public SimplePipelineBuilder<I> setMeterRegistry(MeterRegistry meterRegistry)
355361
{

doc/integrations.md

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,16 @@
22

33
## Micrometer
44

5+
### Metrics
6+
57
`data-pipeline` automatically manages a number of micrometer meters in order to give insights into the performance and reliability of a pipeline and its components.
68

79
In order to enable activity tracking through micrometer, pass a `io.micrometer.core.instrument.MeterRegistry` to the pipeline builder:
810

911
```java
1012
var pipeline = Pipeline.of("my-pipeline")
1113
// register steps, sinks, etc.
12-
.setMeterRegistry(meterRegistry)
14+
.addObservabilityComponent(meterRegistry)
1315
//.build()
1416
;
1517
```
@@ -51,9 +53,25 @@ Each of these have a number of predefined tags like the `pipeline` name, or a `r
5153
| sink | `pipeline.sink.run.failure` | `counter` | `pipeline` `sink` | Total count of failed sink runs |
5254
| sink | `pipeline.sink.run.error.total` | `counter` | `pipeline` `sink` `error` | Total count of sink exceptions |
5355

56+
### Tracing
57+
58+
`data-pipeline` also manages tracing spans via `micrometer-tracing` on core components:
59+
* pipeline execution
60+
* pipeline phases: initialization, steps and sinks
61+
62+
In order to enable tracing through micrometer, pass a `io.micrometer.tracing.Tracer` to the pipeline builder:
63+
64+
```java
65+
var pipeline = Pipeline.of("my-pipeline")
66+
// register steps, sinks, etc.
67+
.addObservabilityComponent(tracer)
68+
//.build()
69+
;
70+
```
71+
5472
## Prometheus
5573

56-
Given the micrometer integration above, existing meters are trivially converted to prometheus metrics:
74+
Given the micrometer integration [above](#metrics), existing meters are trivially converted to prometheus metrics:
5775
* through `micrometer-registry-prometheus`: [Prometheus integration](https://micrometer.io/docs/registry/prometheus) ; note that there are many alternatives such as [OTLP, Influx, Graphite or Datadog](https://micrometer.io/docs)
5876
* or in a spring-boot application for instance: [Production-ready Features - Prometheus](https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuator.metrics.export.prometheus)
5977

pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
<groupId>tech.illuin</groupId>
88
<artifactId>data-pipeline-parent</artifactId>
9-
<version>0.23.4</version>
9+
<version>0.24</version>
1010
<packaging>pom</packaging>
1111

1212
<modules>

resilience4j/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
<parent>
88
<groupId>tech.illuin</groupId>
99
<artifactId>data-pipeline-parent</artifactId>
10-
<version>0.23.4</version>
10+
<version>0.24</version>
1111
</parent>
1212

1313
<artifactId>data-pipeline-resilience4j</artifactId>

0 commit comments

Comments
 (0)