Address review comments: Implement config as per the suggested new design

Signed-off-by: Manisha Yadav <yavmanis@amazon.com>

Author: yavmanis

data-prepper-plugins/multiline-codecs/README.md

@@ -12,46 +12,49 @@ The multiline input codec can be configured with source plugins (e.g. S3 source,
 - **Python tracebacks**: `Traceback` blocks spanning multiple lines
 - **Timestamp-prefixed logs**: Logs where each entry starts with a timestamp and continuation lines don't
 - **Multi-line JSON/XML in logs**: Structured data embedded across multiple lines within log entries
-- **Custom log formats**: Any format where a recognizable pattern marks the start of a new event
+- **Custom log formats**: Any format where a recognizable pattern marks the start or end of a new event
 
 ## Configuration Options
 
+Exactly one of the four pattern fields must be specified:
+
 | Option | Required | Type | Default | Description |
 |---|---|---|---|---|
-| `match` | Yes | String (regex) | - | A regular expression pattern used to identify line boundaries |
-| `negate` | No | Boolean | `false` | When `false`, lines matching the pattern are continuation lines. When `true`, lines NOT matching the pattern are continuation lines |
-| `what` | No | String | `previous` | Whether continuation lines belong to the `previous` or `next` event |
+| `event_start_pattern` | One of four | String (regex) | - | A new event begins at each line matching this pattern |
+| `event_end_pattern` | One of four | String (regex) | - | An event ends at each line matching this pattern (inclusive) |
+| `continuation_line_start_pattern` | One of four | String (regex) | - | Lines matching this pattern are continuations of the previous event |
+| `continuation_line_end_pattern` | One of four | String (regex) | - | Lines matching this pattern are prepended to the next event |
+| `omit_matched_section` | No | Boolean | `false` | When true, the matched portion of the line is omitted from the output |
 | `max_lines` | No | Integer | `500` | Maximum number of lines that can be combined into a single event |
-| `max_length` | No | Integer | `10000` | Maximum character length of a combined multiline event |
-| `line_separator` | No | String | `\n` | Separator string used when joining lines into a single event message |
+| `max_length` | No | Integer | `10000` | Maximum character length of a combined multiline event. Note: a single line exceeding this limit will still be emitted as a complete event without truncation |
+| `line_separator` | No | String | `\n` | Separator string used when joining lines into a single event message. Note: `BufferedReader.readLine()` strips original line endings, so the codec normalizes joined lines using this separator. Set to `""` for no separator |
+| `encoding` | No | String | `UTF-8` | Character encoding to use when reading the input stream |
 
 ## How It Works
 
-The codec reads lines from the input stream and uses the `match` regex to determine event boundaries:
+The codec reads lines from the input stream and uses the configured pattern to determine event boundaries:
 
-1. **`negate=true` + `what=previous`** (most common): A new event starts when a line matches the pattern. Lines that do NOT match are appended to the preceding event.
+1. **`event_start_pattern`** (most common): Each line matching the pattern starts a new event. All subsequent non-matching lines are appended to it.
 
-2. **`negate=false` + `what=previous`**: Lines that match the pattern are appended to the preceding event.
+2. **`event_end_pattern`**: Lines are accumulated until a line matches the pattern. The matching line is included in the current event, and the next line starts a new event.
 
-3. **`negate=true` + `what=next`**: Lines that do NOT match the pattern are prepended to the next matching line.
+3. **`continuation_line_start_pattern`**: Lines matching the pattern are continuations of the previous event. Non-matching lines start new events.
 
-4. **`negate=false` + `what=next`**: Lines that match the pattern are prepended to the next non-matching line.
+4. **`continuation_line_end_pattern`**: Lines matching the pattern are prepended to the next non-matching line's event.
 
 ## Examples
 
-### Java Stack Traces (timestamp-based grouping)
+### Java Stack Traces
 
-Each log entry starts with a timestamp. Lines without a timestamp are continuations of the previous entry.
+Each log entry starts with a timestamp. Lines without a timestamp (stack frames) are part of the previous entry.
 
 ```yaml
 pipeline:
   source:
     s3:
       codec:
         multiline:
-          match: "^\\d{4}-\\d{2}-\\d{2}"
-          negate: true
-          what: previous
+          event_start_pattern: "^\\d{4}-\\d{2}-\\d{2}"
 ```
 
 Input:
@@ -62,51 +65,46 @@ Input:
 2024-01-01 12:00:01 INFO Application recovered
 ```
 
-Result: 2 events
-- Event 1: The ERROR line with its full stack trace grouped together
-- Event 2: The INFO line as a single event
+Result: 2 events (stack trace grouped with its ERROR line)
 
-### Java Stack Traces (pattern-based grouping)
+### Delimiter-Separated Entries
 
-Lines starting with whitespace followed by `at `, `...`, or `Caused by:` are continuations.
+Log entries are separated by a `---` line.
 
 ```yaml
 pipeline:
   source:
     s3:
       codec:
         multiline:
-          match: "^\\s+(at |\\.\\.\\.|Caused by:)"
-          negate: false
-          what: previous
+          event_end_pattern: "^---$"
 ```
 
-### Python Tracebacks
+### Stack Traces (continuation pattern)
+
+Lines starting with whitespace followed by `at ` or `Caused by:` are continuations.
 
 ```yaml
 pipeline:
   source:
     s3:
       codec:
         multiline:
-          match: "^Traceback|^\\s|^\\w+Error"
-          negate: false
-          what: previous
+          continuation_line_start_pattern: "^\\s+(at |\\.\\.\\.|Caused by:)"
 ```
 
-### Log Entries with Preamble (next mode)
+### Omitting Timestamps from Output
 
-Lines starting with whitespace are prepended to the next non-indented line.
+Strip the timestamp from each event's first line:
 
 ```yaml
 pipeline:
   source:
     s3:
       codec:
         multiline:
-          match: "^\\s"
-          negate: false
-          what: next
+          event_start_pattern: "^\\d{4}-\\d{2}-\\d{2}\\s+\\d{2}:\\d{2}:\\d{2}\\s+"
+          omit_matched_section: true
 ```
 
 ## Developer Guide

data-prepper-plugins/multiline-codecs/build.gradle

@@ -7,18 +7,9 @@
  * compatible open source license.
  */
 
-plugins {
-    id 'java'
-}
-
 dependencies {
     implementation project(':data-prepper-api')
     implementation 'com.fasterxml.jackson.core:jackson-annotations'
-    implementation libs.parquet.common
     testImplementation project(':data-prepper-plugins:common')
     testImplementation project(':data-prepper-test:test-event')
 }
-
-test {
-    useJUnitPlatform()
-}