Skip to content

Commit cb57579

Browse files
committed
docs(examples): standardize annotated file workers
1 parent 8d64d51 commit cb57579

8 files changed

Lines changed: 139 additions & 149 deletions

File tree

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,7 @@ public class Workers {
259259
}
260260
```
261261
262-
**Start workers** with `TaskRunnerConfigurer` or `WorkflowExecutor`:
262+
**Start workers** with `TaskRunnerConfigurer` or `AnnotatedWorkerExecutor`:
263263
264264
```java
265265
// Option 1: Using TaskRunnerConfigurer
@@ -272,9 +272,9 @@ TaskRunnerConfigurer configurer = new TaskRunnerConfigurer.Builder(
272272
.build();
273273
configurer.init();
274274
275-
// Option 2: Using WorkflowExecutor (auto-discovers @WorkerTask annotations)
276-
WorkflowExecutor executor = new WorkflowExecutor(client, 10);
277-
executor.initWorkers("com.mycompany.workers"); // Package to scan for @WorkerTask
275+
// Option 2: Register dependency-injected @WorkerTask instances
276+
AnnotatedWorkerExecutor executor = new AnnotatedWorkerExecutor(taskClient);
277+
executor.initWorkersFromInstances(List.of(new Workers()));
278278
```
279279
280280
**Worker Design Principles:**

docs/file-client.md

Lines changed: 22 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -122,84 +122,54 @@ files.download(workflowId, modelHandle, existing);
122122

123123
The download first goes to a unique sibling `.part` file. Only a complete transfer atomically replaces `current-model.bin`. Failure removes the partial file and preserves the existing destination. A filesystem that cannot perform atomic replacement is rejected explicitly.
124124

125-
## Raw worker example
125+
## Recommended annotated worker pattern
126126

127127
```java
128-
public final class ConvertWorker implements Worker {
128+
public final class ConvertWorker {
129129
private final FileClient files;
130130

131131
public ConvertWorker(FileClient files) {
132132
this.files = files;
133133
}
134134

135-
@Override
136-
public String getTaskDefName() {
137-
return "convert_document";
135+
public static class ConvertInput {
136+
public String document;
138137
}
139138

140-
@Override
141-
public TaskResult execute(Task task) {
142-
TaskResult result = new TaskResult(task);
143-
Path input = null;
144-
Path output = null;
139+
@WorkerTask("convert_document")
140+
public @OutputParam("document") String convert(
141+
ConvertInput input,
142+
@WorkflowInstanceIdInputParam String workflowId) throws IOException {
143+
Path source = Files.createTempFile("document-", ".bin");
144+
Path output = Files.createTempFile("converted-", ".pdf");
145145
try {
146-
String workflowId = task.getWorkflowInstanceId();
147-
String inputHandle = (String) task.getInputData().get("document");
148-
input = Files.createTempFile("document-", ".bin");
149-
output = Files.createTempFile("converted-", ".pdf");
150-
151-
files.download(workflowId, inputHandle, input);
152-
convertToPdf(input, output);
153-
154-
String outputHandle = files.upload(
146+
files.download(workflowId, input.document, source);
147+
convertToPdf(source, output);
148+
return files.upload(
155149
workflowId,
156150
output,
157151
new FileUploadOptions()
158152
.setFileName("converted.pdf")
159-
.setContentType("application/pdf")
160-
.setTaskId(task.getTaskId()));
161-
162-
result.setStatus(TaskResult.Status.COMPLETED);
163-
result.addOutputData("document", outputHandle);
164-
} catch (Exception e) {
165-
result.setStatus(TaskResult.Status.FAILED);
166-
result.setReasonForIncompletion(e.getMessage());
153+
.setContentType("application/pdf"));
167154
} finally {
168-
deleteQuietly(input);
169-
deleteQuietly(output);
155+
Files.deleteIfExists(source);
156+
Files.deleteIfExists(output);
170157
}
171-
return result;
172158
}
173159
}
174160
```
175161

176-
The task input and output are strings. No task-runner file integration is required.
162+
`@WorkerTask` names the `SIMPLE` task, the unannotated POJO receives the full resolved input map, `@WorkflowInstanceIdInputParam` supplies the workflow context required by `FileClient`, and `@OutputParam` maps the returned handle to `output.document`. Thrown exceptions become failed task results.
177163

178-
## Annotated worker example
164+
Register already-constructed worker instances so normal constructor injection continues to work:
179165

180166
```java
181-
public record ConvertInput(String document) {}
182-
183-
@WorkerTask("convert_document")
184-
public @OutputParam("document") String convert(
185-
ConvertInput input,
186-
@WorkflowInstanceIdInputParam String workflowId) throws IOException {
187-
Path source = Files.createTempFile("document-", ".bin");
188-
Path output = Files.createTempFile("converted-", ".pdf");
189-
try {
190-
files.download(workflowId, input.document(), source);
191-
convertToPdf(source, output);
192-
return files.upload(
193-
workflowId,
194-
output,
195-
new FileUploadOptions().setContentType("application/pdf"));
196-
} finally {
197-
Files.deleteIfExists(source);
198-
Files.deleteIfExists(output);
199-
}
200-
}
167+
AnnotatedWorkerExecutor workers = new AnnotatedWorkerExecutor(taskClient);
168+
workers.initWorkersFromInstances(List.of(new ConvertWorker(files)));
201169
```
202170

171+
The annotation value must exactly match both the workflow task's `name` and a registered task definition. The task input and output remain plain strings; no task-runner file integration is required. Implement the lower-level `Worker` interface only when the method-binding model is insufficient and direct access to the full `Task` or `TaskResult` is necessary.
172+
203173
## Retry and security behavior
204174

205175
- A new signed URL is requested before every retry.

examples/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ See [`manifest.json`](manifest.json) for per-example metadata: category, workflo
3030

3131
| Example | Description |
3232
|---|---|
33-
| [media-transcoder](file-storage/media-transcoder/) | Complete `FileClient` workflow covering path and stream uploads, metadata, downloads, raw and annotated workers, handle passing, and automatic multipart selection. |
33+
| [media-transcoder](file-storage/media-transcoder/) | Complete `FileClient` workflow covering path and stream uploads, metadata, downloads, a consistent `@WorkerTask` pattern, handle passing, and automatic multipart selection. |
3434

3535
---
3636

examples/file-storage/media-transcoder/README.md

Lines changed: 60 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,71 @@ This example passes videos, thumbnails, and a JSON manifest through a workflow a
66

77
| Case | Source |
88
|---|---|
9-
| Upload a caller-owned `InputStream` with required filename and content type | [`UploadPrimaryVideoWorker.java`](src/main/java/io/conductor/example/mediatranscoder/workers/UploadPrimaryVideoWorker.java) |
10-
| Download a handle, process the file, and upload a `Path` from an annotated worker | [`TranscodeWorker.java`](src/main/java/io/conductor/example/mediatranscoder/workers/TranscodeWorker.java) |
11-
| Download and upload from a raw `Worker`, including task ID metadata | [`ThumbnailWorker.java`](src/main/java/io/conductor/example/mediatranscoder/workers/ThumbnailWorker.java) |
12-
| Read metadata and download before producing another handle | [`ManifestWorker.java`](src/main/java/io/conductor/example/mediatranscoder/workers/ManifestWorker.java) |
9+
| Upload a caller-owned `InputStream` with required filename and content type | Annotated [`UploadPrimaryVideoWorker.java`](src/main/java/io/conductor/example/mediatranscoder/workers/UploadPrimaryVideoWorker.java) |
10+
| Download a handle, process the file, and upload a `Path` | Annotated [`TranscodeWorker.java`](src/main/java/io/conductor/example/mediatranscoder/workers/TranscodeWorker.java) |
11+
| Download and upload a second derived file in a parallel workflow branch | Annotated [`ThumbnailWorker.java`](src/main/java/io/conductor/example/mediatranscoder/workers/ThumbnailWorker.java) |
12+
| Read metadata and download before producing another handle | Annotated [`ManifestWorker.java`](src/main/java/io/conductor/example/mediatranscoder/workers/ManifestWorker.java) |
1313
| Every public `FileClient` overload in small copyable methods | [`FileClientUsage.java`](src/main/java/io/conductor/example/mediatranscoder/FileClientUsage.java) |
1414
| Pass only handle strings between sequential and parallel tasks | [`media_transcode.json`](src/main/resources/workflow/media_transcode.json) |
1515

1616
Large `Path` uploads use the same call shown in the workers. `FileClient` automatically selects multipart for S3 and Azure when the configured threshold is exceeded; GCS and local storage use one request.
1717

18+
## Annotated worker pattern
19+
20+
Every worker follows the same four-part pattern from [`WorkerTask.java`](../../../conductor-client/src/main/java/com/netflix/conductor/sdk/workflow/task/WorkerTask.java):
21+
22+
1. Inject `FileClient` into a plain Java object.
23+
2. Put the workflow task name on a method with `@WorkerTask`.
24+
3. Receive the resolved task input as a typed object and workflow context with `@WorkflowInstanceIdInputParam`.
25+
4. Return the handle string and use `@OutputParam` to name the task output.
26+
27+
```java
28+
public final class TranscodeWorker {
29+
private final FileClient files;
30+
31+
public TranscodeWorker(FileClient files) {
32+
this.files = files;
33+
}
34+
35+
public static class TranscodeInput {
36+
public String primary_video;
37+
public String resolution;
38+
}
39+
40+
@WorkerTask("transcode_video")
41+
public @OutputParam("output_file") String transcode(
42+
TranscodeInput input,
43+
@WorkflowInstanceIdInputParam String workflowId) throws IOException {
44+
Path source = Files.createTempFile("source-", ".mov");
45+
Path output = Files.createTempFile("transcoded-", ".mp4");
46+
try {
47+
files.download(workflowId, input.primary_video, source);
48+
transcode(source, output, input.resolution);
49+
return files.upload(
50+
workflowId,
51+
output,
52+
new FileUploadOptions().setContentType("video/mp4"));
53+
} finally {
54+
Files.deleteIfExists(source);
55+
Files.deleteIfExists(output);
56+
}
57+
}
58+
}
59+
```
60+
61+
Register dependency-injected instances directly. `AnnotatedWorkerExecutor` discovers the annotated methods, binds task inputs, maps return values to task outputs, and converts thrown exceptions into failed task results:
62+
63+
```java
64+
AnnotatedWorkerExecutor workers = new AnnotatedWorkerExecutor(taskClient);
65+
workers.initWorkersFromInstances(List.of(
66+
new UploadPrimaryVideoWorker(fileClient),
67+
new TranscodeWorker(fileClient),
68+
new ThumbnailWorker(fileClient),
69+
new ManifestWorker(fileClient)));
70+
```
71+
72+
The annotation value must exactly match the `name` of the corresponding `SIMPLE` task and its registered task definition. This application checks all four task definitions before starting the workflow, so a missing worker definition cannot silently leave the workflow queued.
73+
1874
## Prerequisites
1975

2076
- Java 21 and Maven 3.8+

examples/file-storage/media-transcoder/src/main/java/io/conductor/example/mediatranscoder/MediaTranscoderApp.java

Lines changed: 10 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -20,18 +20,16 @@
2020
import java.util.Set;
2121

2222
import com.fasterxml.jackson.databind.ObjectMapper;
23-
import com.netflix.conductor.client.automator.TaskRunnerConfigurer;
2423
import com.netflix.conductor.client.http.ConductorClient;
2524
import com.netflix.conductor.client.http.MetadataClient;
2625
import com.netflix.conductor.client.http.TaskClient;
2726
import com.netflix.conductor.client.http.WorkflowClient;
28-
import com.netflix.conductor.client.worker.Worker;
2927
import com.netflix.conductor.common.metadata.tasks.TaskDef;
3028
import com.netflix.conductor.common.metadata.tasks.TaskDef.RetryLogic;
3129
import com.netflix.conductor.common.metadata.workflow.StartWorkflowRequest;
3230
import com.netflix.conductor.common.metadata.workflow.WorkflowDef;
3331
import com.netflix.conductor.common.run.Workflow;
34-
import com.netflix.conductor.sdk.workflow.executor.task.AnnotatedWorker;
32+
import com.netflix.conductor.sdk.workflow.executor.task.AnnotatedWorkerExecutor;
3533
import io.conductor.example.mediatranscoder.workers.ManifestWorker;
3634
import io.conductor.example.mediatranscoder.workers.ThumbnailWorker;
3735
import io.conductor.example.mediatranscoder.workers.TranscodeWorker;
@@ -69,25 +67,14 @@ public static void main(String[] args) throws Exception {
6967
// 2. Start workers. upload_primary_video runs first inside the workflow and publishes
7068
// the primary video handle; downstream tasks consume it via
7169
// ${upload_primary_video_ref.output.primary_video}.
72-
// TranscodeWorker is an @WorkerTask-annotated bean. Both raw and annotated workers inject
73-
// FileClient explicitly and exchange only opaque handle strings.
74-
TranscodeWorker transcodeBean = new TranscodeWorker(fileClient);
75-
AnnotatedWorker transcodeWorker = new AnnotatedWorker(
76-
"transcode_video",
77-
TranscodeWorker.class.getMethod(
78-
"transcode", TranscodeWorker.TranscodeInput.class, String.class),
79-
transcodeBean);
80-
81-
List<Worker> workers = List.of(
70+
// Every worker is an ordinary dependency-injected object whose work method is annotated
71+
// with @WorkerTask. The executor discovers those methods and handles input/output binding.
72+
AnnotatedWorkerExecutor workerExecutor = new AnnotatedWorkerExecutor(taskClient);
73+
workerExecutor.initWorkersFromInstances(List.of(
8274
new UploadPrimaryVideoWorker(fileClient),
83-
transcodeWorker,
75+
new TranscodeWorker(fileClient),
8476
new ThumbnailWorker(fileClient),
85-
new ManifestWorker(fileClient));
86-
87-
TaskRunnerConfigurer configurer = new TaskRunnerConfigurer.Builder(taskClient, workers)
88-
.withThreadCount(4)
89-
.build();
90-
configurer.init();
77+
new ManifestWorker(fileClient)));
9178
System.out.println("Workers started: upload_primary_video, transcode_video, extract_thumbnail, create_manifest");
9279

9380
// 3. Start workflow — no inputs; upload_primary_video publishes primaryVideo.
@@ -99,7 +86,7 @@ public static void main(String[] args) throws Exception {
9986
String workflowId = workflowClient.startWorkflow(request);
10087
System.out.println("Workflow started: " + workflowId);
10188

102-
// 5. Poll for completion
89+
// 4. Poll for completion.
10390
System.out.println("Waiting for workflow to complete...");
10491
for (int i = 0; i < 30; i++) {
10592
Thread.sleep(2000);
@@ -108,13 +95,13 @@ public static void main(String[] args) throws Exception {
10895
if (workflow.getStatus().isTerminal()) {
10996
System.out.println("Workflow " + workflow.getStatus() + "!");
11097
System.out.println("Output: " + workflow.getOutput());
111-
configurer.shutdown();
98+
workerExecutor.shutdown();
11299
System.exit(workflow.getStatus().isSuccessful() ? 0 : 1);
113100
}
114101
}
115102

116103
System.err.println("Workflow did not complete in 60s");
117-
configurer.shutdown();
104+
workerExecutor.shutdown();
118105
System.exit(1);
119106
}
120107

examples/file-storage/media-transcoder/src/main/java/io/conductor/example/mediatranscoder/workers/ManifestWorker.java

Lines changed: 15 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -17,14 +17,14 @@
1717
import java.nio.file.Files;
1818
import java.nio.file.Path;
1919

20-
import com.netflix.conductor.client.worker.Worker;
21-
import com.netflix.conductor.common.metadata.tasks.Task;
22-
import com.netflix.conductor.common.metadata.tasks.TaskResult;
20+
import com.netflix.conductor.sdk.workflow.task.OutputParam;
21+
import com.netflix.conductor.sdk.workflow.task.WorkerTask;
22+
import com.netflix.conductor.sdk.workflow.task.WorkflowInstanceIdInputParam;
2323
import org.conductoross.conductor.client.FileClient;
2424
import org.conductoross.conductor.client.model.file.FileMetadata;
2525
import org.conductoross.conductor.sdk.file.FileUploadOptions;
2626

27-
public class ManifestWorker implements Worker {
27+
public class ManifestWorker {
2828

2929
private static final String MANIFEST = "{\"video\":\"%s\",\"videoName\":\"%s\",\"videoSize\":%d,\"thumbnail\":\"%s\",\"thumbName\":\"%s\",\"thumbSize\":%d}";
3030
private final FileClient fileClient;
@@ -33,20 +33,20 @@ public ManifestWorker(FileClient fileClient) {
3333
this.fileClient = fileClient;
3434
}
3535

36-
@Override
37-
public String getTaskDefName() {
38-
return "create_manifest";
36+
public static class ManifestInput {
37+
public String transcoded_video;
38+
public String thumbnail;
3939
}
4040

41-
@Override
42-
public TaskResult execute(Task task) {
43-
TaskResult result = new TaskResult(task);
44-
String video = (String) task.getInputData().get("transcoded_video");
45-
String thumb = (String) task.getInputData().get("thumbnail");
41+
@WorkerTask("create_manifest")
42+
public @OutputParam("output_file") String create(
43+
ManifestInput input,
44+
@WorkflowInstanceIdInputParam String workflowId) throws IOException {
45+
String video = input.transcoded_video;
46+
String thumb = input.thumbnail;
4647
Path manifestFile = null;
4748

4849
try {
49-
String workflowId = task.getWorkflowInstanceId();
5050
System.out.println("[create_manifest] Inputs:");
5151
FileMetadata videoMetadata = describe(workflowId, video);
5252
FileMetadata thumbMetadata = describe(workflowId, thumb);
@@ -59,17 +59,12 @@ public TaskResult execute(Task task) {
5959
manifestFile = Files.createTempFile("manifest-", ".json");
6060
Files.writeString(manifestFile, manifest);
6161

62-
FileUploadOptions options = new FileUploadOptions().setContentType("application/json").setTaskId(task.getTaskId());
62+
FileUploadOptions options = new FileUploadOptions().setContentType("application/json");
6363
String uploaded = fileClient.upload(workflowId, manifestFile, options);
64-
result.getOutputData().put("output_file", uploaded);
65-
result.setStatus(TaskResult.Status.COMPLETED);
6664

6765
System.out.println("[create_manifest] Uploaded: " + uploaded);
6866
System.out.println("[create_manifest] Content: " + manifest);
69-
} catch (Exception e) {
70-
result.setStatus(TaskResult.Status.FAILED);
71-
result.setReasonForIncompletion(e.getMessage());
72-
e.printStackTrace();
67+
return uploaded;
7368
} finally {
7469
if (manifestFile != null) {
7570
try {
@@ -78,7 +73,6 @@ public TaskResult execute(Task task) {
7873
}
7974
}
8075
}
81-
return result;
8276
}
8377

8478
private FileMetadata describe(String workflowId, String fileHandleId) throws IOException {

0 commit comments

Comments
 (0)