Skip to content

feat: allow parsing stringified json fields#342

Open
martindmtrv wants to merge 5 commits into
caronc:masterfrom
martindmtrv:md/nested-json-parsing
Open

feat: allow parsing stringified json fields#342
martindmtrv wants to merge 5 commits into
caronc:masterfrom
martindmtrv:md/nested-json-parsing

Conversation

@martindmtrv

Copy link
Copy Markdown

Description

Related issue (if applicable): #
This PR adds support for an explicit double-colon ::json modifier inside third-party webhook payload mapping rules.
If a third-party webhook sends a payload containing stringified JSON, you can add ::json to that segment name in your mapping rules to instruct the engine to parse the string in-place. This allows you to traverse into the parsed JSON structure using standard dot-notation and bracket-notation, or map the parsed object directly.

Features

  • Nested & Chained parsing: Walk into deeply nested JSON string fields (e.g., event::json.info::json.title).
  • List element parsing: Works seamlessly with bracket/index notation on lists (e.g., items::json[0].value or items[0]::json.value).
  • Flat field parsing: Parse and map stringified JSON directly (e.g. ?:event::json=body).
  • Literal Match Priority: If a payload key literally contains the ::json string, it is matched directly without traversal/parsing, ensuring 100% backward compatibility.
  • Scalar Type Validation: Enforces that mapped scalar fields (format, type, title, and body) cannot be assigned a dict or list object. If a type mismatch occurs, it fails validation gracefully (logs a warning and returns HTTP 400) to prevent runtime crashes in downstream notification plugins.

Checklist

  • Documentation ticket created (if applicable): N/A (updated README.md directly in this PR)
  • The change is tested and works locally.
  • No commented-out code in this PR.
  • No lint errors (use tox -e lint and optionally tox -e format). (some preexisting html lints exist unrelated to these changes)
  • Test coverage added or updated (use tox -e test).

Testing

Anyone can help test as follows:

# Clone the branch
git clone -b <this.branch-name> https://github.com/caronc/apprise-api.git
cd apprise-api

# Run the unit tests
tox -e test

# Run a local instance of the api at http://localhost:8000
tox -e runserver

Successful Parsing & Mapping: Send a POST request with a nested stringified JSON payload using the ::json modifier in mapping rules:

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"event": "{\"title\": \"CPU spike\", \"state\": \"critical\"}"}' \
  "http://localhost:8000/notify/{KEY}?:event::json.title=title&:event::json.state=type"

Verification error case trying to map json to apprise fields

curl -i -X POST \
  -H "Content-Type: application/json" \
  -d '{"event": "{\"title\": \"CPU spike\"}"}' \
  "http://localhost:8000/notify/{KEY}?:event::json=body"
...
HTTP/1.1 400 Bad Request
Date: Sat, 04 Jul 2026 00:41:31 GMT
Server: WSGIServer/0.2 CPython/3.14.6
Content-Type: application/json
Content-Length: 41

{"error": "Payload field mapping failed"}

@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (d6fea32) to head (22ed640).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##            master      #342   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           13        13           
  Lines         1449      1491   +42     
  Branches       263       276   +13     
=========================================
+ Hits          1449      1491   +42     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@caronc

caronc commented Jul 11, 2026

Copy link
Copy Markdown
Owner

I get the idea, but can you give me a use-case or systme that does this?

I'm trying to wrap my head around a system that is sending webhooks using stringified json inside of a json response block. It doesn't seem like optimal/ideal coding practices. I feel this PR would just add overhead at the cost of very extreme edge cases. Can you share with me the system that is doing this? It may be worth investing in putting a PR or filing an issue against it? I'm not against this approach; i just need further reason/justificaiton as I don't agree with it right now.

Things i spot off hand (besides he overhead of now treating even a simple field mapper (what 99.9999% of people we use it for) into a transformation language (parsed). but not just that, it introduces syntax that can collide with actual field names. I think we need to look for specifically ::json and not catch on ::json-like (basically it catches/matches strings it shouldn't).

Mapping in Apprise starts off with :key=value; it leverages the colon... so when we have more than one element, we have :key=value&:key2=value. Will it cause confusion if the field is a json or different variations? I'm just speculating with this comment at this time (i have no tested it myself).

Thoughts?

@martindmtrv

Copy link
Copy Markdown
Author

My primary use case for the feature today is for Ofelia, currently they post a stringified JSON under one of the fields in their webhook for a slack integration.

mcuadros/ofelia#439 (comment)

Agree this could be something that gets addressed directly in that project itself but figured JSON parsing could be a useful feature that adds more flexibility to apprise to work with all kinds of upstream services regardless of their webhook formatting choices

Regarding the edge cases, I have added some tests for ones I could think of, primarily like you mentioned for any field that ends with "::json" already, we prefer to match exact field names before applying any transformations. There is some tests covering this behavior

Having multiple fields should not conflict either since the double colon syntax since it doesn't change the query string parsing at all, but I can also add some additional tests to be sure mixing JSON and non JSON field mapping works as expected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants