Skip to content

Commit be364d5

Browse files
committed
add license.
1 parent 5d97107 commit be364d5

1 file changed

Lines changed: 19 additions & 2 deletions

File tree

  • .agent/skills/adding-new-metadata

.agent/skills/adding-new-metadata/SKILL.md

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,21 @@
11
---
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
219
name: adding-new-metadata
320
description: Guide on how to add and propagate new metadata fields in Apache Beam's WindowedValue, extending protos, windmill persistence, and runner interfaces to avoid metadata loss.
421
---
@@ -40,14 +57,14 @@ For the Dataflow streaming runner, metadata must survive serialization to and fr
4057

4158
* **Serialization (Sink):**
4259
* **File:** `runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillSink.java`
43-
* **Action:** Extract the metadata from the `WindowedValue`, and add it to already created ElementMetadata proto builder.
60+
* **Action:** Extract the metadata from the `WindowedValue`, and add it to already created ElementMetadata proto builder.
4461
* **Deserialization (Reader):**
4562
* **Files:** `runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/UngroupedWindmillReader.java` and `WindowingWindmillReader.java`
4663
* **Action:** Extract the metadata from ElementMetadata proto and reconstruct the `WindowedValue` using the updated factory methods/builders that include the metadata. This is incremental work, as plenty of metadata is already extracted from the proto.
4764

4865
## 4. Propagation Across Core Classes
4966

50-
Metadata must be explicitly copied or forwarded whenever a `WindowedValue` is transformed, buffered, or processed
67+
Metadata must be explicitly copied or forwarded whenever a `WindowedValue` is transformed, buffered, or processed.
5168

5269
### DoFn Runners (Java Core)
5370
You must ensure that when a DoFn processes an element and outputs a new element, the appropriate metadata from the *input* is propagated to the *output* (unless explicitly changed by the logic).

0 commit comments

Comments
 (0)