From 5ddf0df2d8137bb9e87080e788b9aadb4957eace Mon Sep 17 00:00:00 2001 From: Dan Mahoney Date: Wed, 17 Jun 2026 16:22:51 -0400 Subject: [PATCH 1/5] feat: add DMARCbis walk mode selection (RFC 9989 DNS Tree Walk) Refactor organizational domain discovery in libopendmarc into three independently-callable walk strategies selectable at runtime via a new DMARCbisWalkMode config option in opendmarc.conf. - Add OPENDMARC_WALK_MODE_{AUTO,PSL,RFC7489,RFC9989} constants and walk_mode field to OPENDMARC_LIB_T (dmarc.h.in) - Add np= and psd= fields to DMARC_POLICY_T (opendmarc_internal.h) - Extract dmarc_query_at(), psd_from_record(), query_dmarc_psl(), query_dmarc_rfc7489_walk(), and query_dmarc_rfc9989_walk() from the monolithic opendmarc_policy_query_dmarc(); dispatch by walk_mode - Parse np= and psd= tags in opendmarc_policy_parse_dmarc() - Wire DMARCbisWalkMode config key in opendmarc-config.h and opendmarc.c - Document DMARCbisWalkMode in opendmarc.conf.sample and opendmarc.conf.5.in - Add DMARCBIS-WALK-NOTES.txt capturing RFC 9989 open questions AUTO mode preserves pre-DMARCbis behaviour (PSL if configured, label-strip walk otherwise). No #ifdef compile-time gates. --- DMARCBIS-WALK-NOTES.txt | 110 ++++++++ libopendmarc/dmarc.h.in | 13 + libopendmarc/opendmarc_internal.h | 2 + libopendmarc/opendmarc_policy.c | 455 ++++++++++++++++++++++-------- opendmarc/opendmarc-config.h | 1 + opendmarc/opendmarc.c | 17 ++ opendmarc/opendmarc.conf.5.in | 31 ++ opendmarc/opendmarc.conf.sample | 25 ++ 8 files changed, 540 insertions(+), 114 deletions(-) create mode 100644 DMARCBIS-WALK-NOTES.txt diff --git a/DMARCBIS-WALK-NOTES.txt b/DMARCBIS-WALK-NOTES.txt new file mode 100644 index 00000000..3fb4349d --- /dev/null +++ b/DMARCBIS-WALK-NOTES.txt @@ -0,0 +1,110 @@ +Open questions and implementation notes for RFC 9989 DNS Tree Walk +=================================================================== + +This file tracks uncertainties identified while reading RFC 9989 and +designing the walk-mode refactor in libopendmarc. Each item references +the RFC section where the ambiguity was found. + +------------------------------------------------------------------------ +1. Walk exhaustion without psd= (SS 4.10, 4.10.1) +------------------------------------------------------------------------ + +If the walk exhausts all labels without finding a record marked psd=n or +psd=y, but valid DMARC records were encountered along the way (records +without a psd= tag), the RFC does not clearly specify what to do. + +SS 4.10.1 says "if the set produced by the DNS Tree Walk contains no +DMARC Policy Record, Mail Receivers MUST NOT apply the DMARC mechanism" +-- but does not define whether records-without-psd= constitute "the set" +or are considered absent for this purpose. + +SS 4.10.1 also contains the paragraph: "If the DMARC Policy Record to be +applied is that of either the Organizational Domain or the PSD and the +Author Domain is a subdomain of that domain, then the Domain Owner +Assessment Policy is taken from the sp= tag (if any) if the Author Domain +exists or the np= tag (if any) if the Author Domain does not exist." +This could be read to imply that records-without-psd= found during the +walk are applicable as org-domain records when the walk exhausts, but +this is not stated explicitly. + +Status: deferred. Current implementation treats walk exhaustion without +psd= as no-record-found. + +------------------------------------------------------------------------ +2. "might" in secondary alignment walks (S 4.10.2) +------------------------------------------------------------------------ + +Section 4.10.2 says: "DNS Tree Walks done to discover an Organizational +Domain for use in Identifier Alignment Evaluation *might* start at any +of the following locations: the Author Domain, the SPF-Authenticated +Identifier, any DKIM-Authenticated Identifier." + +"Might" is not a normative term per RFC 2119. Section 5.3.4 normatively +requires alignment checking by reference to 4.10.2, but the walk itself +is described non-normatively. + +This means implementations are not clearly required to perform separate +tree walks for each authenticated identifier to determine its +organizational domain for relaxed alignment comparison. A strict reading +permits using only the org domain found during policy discovery. + +Status: secondary alignment walks not implemented in this pass. Relaxed +alignment continues to compare org domains using the single walk +performed during policy discovery. + +------------------------------------------------------------------------ +3. Multiple DMARC records at one level (SS 4.10 steps 2 and 6) +------------------------------------------------------------------------ + +The walk algorithm says: "if multiple DMARC Policy Records are returned +for a single target, they are all discarded." + +The current dmarc_dns_get_record() implementation returns only the first +DMARC TXT record found at a name; it cannot detect the presence of +additional records. Full compliance with this requirement would need a +DNS-layer change to count matching records before returning. + +Status: known gap. Current implementation returns the first record; +multi-record discard is not enforced. + +------------------------------------------------------------------------ +4. "One label below" psd=y with the 8-label skip (SS 4.10, 4.10.2) +------------------------------------------------------------------------ + +Section 4.10.2 step 2 says: if psd=y is found at a domain other than the +walk's starting domain, "the Organizational Domain is the domain one label +below this one in the DNS hierarchy." + +When the 8-label skip applies (domains with more than 8 labels jump +directly to 7 labels per S 4.10 step 4), the walk may not have visited +the level immediately below the psd=y domain. "One label below in the +DNS hierarchy" must therefore be reconstructed from the original starting +domain, not inferred from the walk's previous step. + +Status: tracked. Implementation computes "one label below" by finding +the child of the psd=y domain that is an ancestor of the original +starting domain. + +------------------------------------------------------------------------ +5. Fate of records-without-psd= collected during the walk (S 4.10.2) +------------------------------------------------------------------------ + +Section 4.10.2 describes a 3-step selection process that operates on +"domains for which valid DMARC Policy Records were retrieved" -- implying +the walk accumulates records (including those without psd=) and the +selection is made afterward. + +Section 4.10 steps 2 and 6 say to stop when psd= is found but do not +say to discard records found at earlier levels. It is not stated +explicitly whether the policy-discovery walk (S 4.10.1) also accumulates +intermediate records or only uses the record at the stopping point. + +The 4.10.2 selection step 3 ("select the DMARC Policy Record found at +the name with the fewest number of labels") only makes sense if multiple +records were collected. This implies the walk accumulates all valid +single records encountered, stopping only when psd= is hit. + +Status: walk implementation accumulates records. The selection process +from 4.10.2 is applied post-walk. Policy discovery (4.10.1) uses the +psd= stopping record when found; falls back to fewest-labels selection +when the walk exhausts without psd=. diff --git a/libopendmarc/dmarc.h.in b/libopendmarc/dmarc.h.in index 2c47f9bd..11eae5ab 100644 --- a/libopendmarc/dmarc.h.in +++ b/libopendmarc/dmarc.h.in @@ -99,10 +99,23 @@ typedef struct { u_char tld_source_file[MAXPATHLEN]; int nscount; struct sockaddr_in nsaddr_list[MAXNS]; + int walk_mode; /* OPENDMARC_WALK_MODE_* */ } OPENDMARC_LIB_T; #define OPENDMARC_TLD_TYPE_NONE (0) /* Will not use a tld file */ #define OPENDMARC_TLD_TYPE_MOZILLA (1) /* mozilla.org effective_tld_names.dat */ + +/* Walk mode: how to find the organizational domain when the author domain + * has no DMARC record of its own. */ +#define OPENDMARC_WALK_MODE_AUTO (0) /* PSL if configured, label walk if not */ +#define OPENDMARC_WALK_MODE_PSL (1) /* PSL only; no fallback walk */ +#define OPENDMARC_WALK_MODE_RFC7489 (2) /* Label-strip walk, stops before TLD */ +#define OPENDMARC_WALK_MODE_RFC9989 (3) /* RFC 9989 S 4.10 DNS Tree Walk */ + +/* psd= tag values (RFC 9989) */ +#define DMARC_RECORD_PSD_UNSPECIFIED (0) /* psd= absent */ +#define DMARC_RECORD_PSD_N (1) /* psd=n: organizational domain */ +#define DMARC_RECORD_PSD_Y (2) /* psd=y: public suffix domain */ /* * Library one time initialization. diff --git a/libopendmarc/opendmarc_internal.h b/libopendmarc/opendmarc_internal.h index 8d374800..2f12c663 100644 --- a/libopendmarc/opendmarc_internal.h +++ b/libopendmarc/opendmarc_internal.h @@ -169,6 +169,8 @@ typedef struct dmarc_policy_t { int aspf; int p; int sp; + int np; /* RFC 9989: non-existent subdomain policy */ + int psd; /* RFC 9989: DMARC_RECORD_PSD_* */ int pct; int rf; uint32_t ri; diff --git a/libopendmarc/opendmarc_policy.c b/libopendmarc/opendmarc_policy.c index c125b957..2eba1ace 100644 --- a/libopendmarc/opendmarc_policy.c +++ b/libopendmarc/opendmarc_policy.c @@ -729,8 +729,290 @@ opendmarc_policy_query_dmarc_xdomain(DMARC_POLICY_T *pctx, u_char *uri) } } +/* + * dmarc_query_at: query _dmarc.{target}, following CNAMEs up to + * DNS_MAX_RETRIES hops. On success returns bp (== buf) and sets + * *dns_reply to NETDB_SUCCESS; on failure returns NULL. + */ +static char * +dmarc_query_at(const char *target, int *dns_reply, + u_char *buf, size_t bufsz) +{ + char copy[256]; + char * bp; + int loop_count = DNS_MAX_RETRIES; + + (void) strlcpy(copy, "_dmarc.", sizeof copy); + (void) strlcat(copy, target, sizeof copy); + + for (;;) + { + (void) memset(buf, '\0', bufsz); + bp = dmarc_dns_get_record(copy, dns_reply, (char *)buf, bufsz); + if (bp != NULL) + return bp; + /* CNAME: buf holds the redirect target */ + if (*buf == '\0') + return NULL; + (void) strlcpy(copy, (char *)buf, sizeof copy); + if (--loop_count == 0) + return NULL; + } +} + +/* + * psd_from_record: extract psd= value from a raw DMARC record string. + * Returns DMARC_RECORD_PSD_Y, _N, or _UNSPECIFIED. + */ +static int +psd_from_record(const u_char *record) +{ + const u_char *p = record; + + while ((p = (const u_char *)strchr((const char *)p, 'p')) != NULL) + { + /* Must be at start or after a ';' and optional whitespace */ + if (p != record && *(p - 1) != ';' && *(p - 1) != ' ' && *(p - 1) != '\t') + { + p++; + continue; + } + if (strncasecmp((const char *)p, "psd=", 4) == 0) + { + switch (*(p + 4)) + { + case 'y': case 'Y': return DMARC_RECORD_PSD_Y; + case 'n': case 'N': return DMARC_RECORD_PSD_N; + default: return DMARC_RECORD_PSD_UNSPECIFIED; + } + } + p++; + } + return DMARC_RECORD_PSD_UNSPECIFIED; +} + +/* + * query_dmarc_psl: PSL-only strategy (OPENDMARC_WALK_MODE_PSL). + * + * Asks the PSL for the organizational domain of 'domain'. If the PSL + * returns a different domain, queries exactly that one and stops. Does + * not fall through to any label-walking; if the PSL cannot identify a + * boundary (no PSL loaded, or PSL returned the same domain), returns + * DMARC_DNS_ERROR_NO_RECORD. + * + * Sets pctx->organizational_domain on success. + * Fills buf with the raw DMARC record on success. + */ +static OPENDMARC_STATUS_T +query_dmarc_psl(DMARC_POLICY_T *pctx, u_char *domain, + u_char *buf, size_t bufsz, int *dns_reply_out) +{ + u_char tld[256]; + int dns_reply = 0; + + (void) memset(tld, '\0', sizeof tld); + if (opendmarc_get_tld(domain, tld, sizeof tld) != 0) + return DMARC_DNS_ERROR_NO_RECORD; + if (strlen((char *)tld) == 0 || + strcasecmp((char *)tld, (char *)domain) == 0) + return DMARC_DNS_ERROR_NO_RECORD; + + pctx->organizational_domain = strdup((char *)tld); + + if (dmarc_query_at((char *)tld, &dns_reply, buf, bufsz) != NULL) + { + if (dns_reply_out != NULL) + *dns_reply_out = dns_reply; + return DMARC_PARSE_OKAY; + } + if (dns_reply_out != NULL) + *dns_reply_out = dns_reply; + return (dns_reply == TRY_AGAIN || dns_reply == NETDB_INTERNAL) + ? DMARC_DNS_ERROR_TMPERR : DMARC_DNS_ERROR_NO_RECORD; +} + +/* + * query_dmarc_rfc7489_walk: RFC 7489 label-strip walk + * (OPENDMARC_WALK_MODE_RFC7489). + * + * Strips labels from the left one at a time and queries each parent, + * stopping before a bare TLD (a label with no further dot). This is + * the original no-PSL fallback behaviour. + * + * Sets pctx->organizational_domain and pctx->org_domain_from_fallback + * on success. Fills buf with the raw DMARC record on success. + */ +static OPENDMARC_STATUS_T +query_dmarc_rfc7489_walk(DMARC_POLICY_T *pctx, u_char *domain, + u_char *buf, size_t bufsz, int *dns_reply_out) +{ + u_char * cur = domain; + u_char * dot; + int dns_reply = 0; + + while ((dot = (u_char *)strchr((char *)cur, '.')) != NULL) + { + cur = dot + 1; + + /* Stop before bare TLDs (labels with no further dot). */ + if (strchr((char *)cur, '.') == NULL) + break; + + if (dmarc_query_at((char *)cur, &dns_reply, buf, bufsz) != NULL) + { + pctx->organizational_domain = strdup((char *)cur); + pctx->org_domain_from_fallback = 1; + if (dns_reply_out != NULL) + *dns_reply_out = dns_reply; + return DMARC_PARSE_OKAY; + } + } + if (dns_reply_out != NULL) + *dns_reply_out = dns_reply; + return (dns_reply == TRY_AGAIN || dns_reply == NETDB_INTERNAL) + ? DMARC_DNS_ERROR_TMPERR : DMARC_DNS_ERROR_NO_RECORD; +} + +/* + * query_dmarc_rfc9989_walk: RFC 9989 S 4.10 DNS Tree Walk + * (OPENDMARC_WALK_MODE_RFC9989). + * + * Implements the generic walk from RFC 9989 S 4.10 as applied to policy + * discovery per S 4.10.1. Key differences from the RFC 7489 walk: + * + * - 8-query cap: domains with more than 8 labels are shortened to 7 + * before the walk begins, so no more than 8 DNS queries are made. + * - Walks all the way to the TLD apex (no "stop before bare TLD" rule). + * - Collects all valid single-record responses encountered. + * - Stops when a record with psd=n or psd=y is found. + * - Applies the S 4.10.2 selection to determine the org domain. + * + * Known gaps (see DMARCBIS-WALK-NOTES.txt): + * - Multiple records at one level cannot be detected; dmarc_dns_get_record() + * returns only the first match. + * - Walk-exhaustion-without-psd= behaviour is treated as no-record. + * + * Sets pctx->organizational_domain on success. + * Fills buf with the raw DMARC record on success. + */ +static OPENDMARC_STATUS_T +query_dmarc_rfc9989_walk(DMARC_POLICY_T *pctx, u_char *domain, + u_char *buf, size_t bufsz, int *dns_reply_out) +{ + char cur[256]; + char prev[256]; /* level visited before current */ + char best_domain[256]; /* domain of best record so far */ + u_char best_buf[BUFSIZ]; /* record text of best so far */ + int best_psd; + int found_any; + int nqueries; + int nlabels; + int dns_reply = 0; + char * p; + char * bp; + + /* Count labels in the author domain. */ + nlabels = 1; + for (p = (char *)domain; *p != '\0'; p++) + if (*p == '.') nlabels++; + + /* + * Determine starting point per S 4.10.1: + * <= 8 labels: immediate parent (remove leftmost label) + * > 8 labels: shorten until 7 labels remain + */ + (void) strlcpy(cur, (char *)domain, sizeof cur); + if (nlabels > 8) + { + int to_remove = nlabels - 7; + p = cur; + while (to_remove-- > 0) + { + p = strchr(p, '.'); + if (p == NULL) + break; + p++; + } + if (p != NULL && p != cur) + (void) memmove(cur, p, strlen(p) + 1); + } + else + { + p = strchr(cur, '.'); + if (p == NULL) + return DMARC_DNS_ERROR_NO_RECORD; + (void) memmove(cur, p + 1, strlen(p + 1) + 1); + } + + best_psd = DMARC_RECORD_PSD_UNSPECIFIED; + found_any = 0; + nqueries = 0; + (void) strlcpy(prev, (char *)domain, sizeof prev); + (void) memset(best_domain, '\0', sizeof best_domain); + (void) memset(best_buf, '\0', sizeof best_buf); + + for (;;) + { + if (nqueries >= 8 || cur[0] == '\0') + break; + nqueries++; + + bp = dmarc_query_at(cur, &dns_reply, buf, bufsz); + if (bp != NULL) + { + int psd = psd_from_record(buf); + + /* + * Per S 4.10.2 step 3, if the walk exhausts without + * a psd= record, we use the record with the fewest + * labels, which is the last one found (highest in tree). + * Unconditionally overwrite best so the last wins. + */ + (void) strlcpy(best_domain, cur, sizeof best_domain); + (void) strlcpy((char *)best_buf, (char *)buf, sizeof best_buf); + best_psd = psd; + found_any = 1; + + /* psd= record terminates the walk (S 4.10 steps 2, 6). */ + if (psd != DMARC_RECORD_PSD_UNSPECIFIED) + break; + } + + (void) strlcpy(prev, cur, sizeof prev); + + /* Remove leftmost label. */ + p = strchr(cur, '.'); + if (p == NULL) + break; + (void) memmove(cur, p + 1, strlen(p + 1) + 1); + } + + if (dns_reply_out != NULL) + *dns_reply_out = dns_reply; + + if (!found_any) + return (dns_reply == TRY_AGAIN || dns_reply == NETDB_INTERNAL) + ? DMARC_DNS_ERROR_TMPERR : DMARC_DNS_ERROR_NO_RECORD; + + /* + * Apply S 4.10.2 org domain selection: + * Step 1: psd=n -> this level is the org domain. + * Step 2: psd=y -> org domain is one label below (prev). + * Step 3: no psd= -> use best_domain (fewest labels = last found). + */ + if (best_psd == DMARC_RECORD_PSD_N) + pctx->organizational_domain = strdup(best_domain); + else if (best_psd == DMARC_RECORD_PSD_Y) + pctx->organizational_domain = strdup(prev); + else + pctx->organizational_domain = strdup(best_domain); + + (void) strlcpy((char *)buf, (char *)best_buf, bufsz); + return DMARC_PARSE_OKAY; +} + /************************************************************************** -** OPENDMARC_POLICY_QUERY_DMARC -- Look up the _dmarc record for the +** OPENDMARC_POLICY_QUERY_DMARC -- Look up the _dmarc record for the ** specified domain. If not found ** try the organizational domain. ** Parameters: @@ -761,13 +1043,10 @@ opendmarc_policy_query_dmarc_xdomain(DMARC_POLICY_T *pctx, u_char *uri) OPENDMARC_STATUS_T opendmarc_policy_query_dmarc(DMARC_POLICY_T *pctx, u_char *domain) { - u_char buf[BUFSIZ]; - u_char copy[256]; - u_char tld[256]; - u_char * bp = NULL; - int dns_reply = 0; - int tld_reply = 0; - int loop_count = DNS_MAX_RETRIES; + u_char buf[BUFSIZ]; + int dns_reply = 0; + int walk_mode; + OPENDMARC_STATUS_T status; if (pctx == NULL) return DMARC_PARSE_ERROR_NULL_CTX; @@ -779,122 +1058,48 @@ opendmarc_policy_query_dmarc(DMARC_POLICY_T *pctx, u_char *domain) return DMARC_PARSE_ERROR_EMPTY; } - (void) strlcpy((char *)copy, "_dmarc.", sizeof copy); - (void) strlcat((char *)copy, (char *)domain, sizeof copy); - -query_again: + /* Direct query at _dmarc.{domain}; done if a record is found. */ (void) memset(buf, '\0', sizeof buf); - bp = (u_char *)dmarc_dns_get_record((char *)copy, &dns_reply, (char *)buf, sizeof buf); - if (bp != NULL) - { - if (dns_reply != HOST_NOT_FOUND) - goto got_record; - } - /* - * Was a CNAME was found that the resolver did - * not follow on its own? - */ - if (bp == NULL && *buf != '\0') - { - (void) strlcpy((char *)copy, (char *)buf, sizeof copy); - if (--loop_count != 0) - goto query_again; - } + if (dmarc_query_at((char *)domain, &dns_reply, buf, sizeof buf) != NULL) + return opendmarc_policy_parse_dmarc(pctx, domain, buf); - (void) memset(tld, '\0', sizeof tld); - tld_reply = opendmarc_get_tld(domain, tld, sizeof tld); - if (tld_reply != 0) - goto dns_failed; + if (dns_reply == TRY_AGAIN || dns_reply == NETDB_INTERNAL) + return DMARC_DNS_ERROR_TMPERR; - /* - * If the PSL identified an organizational domain distinct from the - * queried domain, try exactly that domain and stop. Per RFC 7489 - * §6.6.3 we look at one domain: the organizational domain. - */ - if (strlen((char *)tld) > 0 && strcasecmp((char *)tld, (char *)domain) != 0) - { - pctx->organizational_domain = (u_char *)strdup((char *)tld); + /* No direct record; walk up the tree using the configured strategy. */ + walk_mode = (Opendmarc_Libp != NULL) + ? Opendmarc_Libp->walk_mode + : OPENDMARC_WALK_MODE_AUTO; - loop_count = DNS_MAX_RETRIES; - (void) strlcpy((char *)copy, "_dmarc.", sizeof copy); - (void) strlcat((char *)copy, (char *)tld, sizeof copy); -query_again2: - (void) memset(buf, '\0', sizeof buf); - bp = (u_char *)dmarc_dns_get_record((char *)copy, &dns_reply, (char *)buf, sizeof buf); - if (bp != NULL) - goto got_record; - /* - * Was a CNAME found that the resolver did not follow on its own? - */ - if (bp == NULL && *buf != '\0') - { - (void) strlcpy((char *)copy, (char *)buf, sizeof copy); - if (--loop_count != 0) - goto query_again2; - } - /* Organizational domain has no DMARC record; do not try further. */ - goto dns_failed; - } - - /* - * No PSL was loaded, or the PSL could not identify an organizational - * domain boundary (it returned the input domain unchanged). Walk up - * the label tree as a best-effort fallback, stopping before bare TLDs. - * This is not strictly per RFC 7489 (which requires a PSL to determine - * the organizational domain), but it handles the common case where no - * PSL is configured and a parent domain has a DMARC record. - * Configure PublicSuffixList in opendmarc.conf for RFC-compliant behavior. - */ + switch (walk_mode) { - u_char *cur = (u_char *)domain; - u_char *dot; + case OPENDMARC_WALK_MODE_PSL: + status = query_dmarc_psl(pctx, domain, buf, sizeof buf, &dns_reply); + break; - while ((dot = (u_char *)strchr((char *)cur, '.')) != NULL) - { - cur = dot + 1; + case OPENDMARC_WALK_MODE_RFC9989: + status = query_dmarc_rfc9989_walk(pctx, domain, buf, sizeof buf, &dns_reply); + break; - /* Stop before bare TLDs (labels with no further dot). */ - if (strchr((char *)cur, '.') == NULL) - break; + case OPENDMARC_WALK_MODE_RFC7489: + status = query_dmarc_rfc7489_walk(pctx, domain, buf, sizeof buf, &dns_reply); + break; - loop_count = DNS_MAX_RETRIES; - (void) strlcpy((char *)copy, "_dmarc.", sizeof copy); - (void) strlcat((char *)copy, (char *)cur, sizeof copy); -query_again3: - (void) memset(buf, '\0', sizeof buf); - bp = (u_char *)dmarc_dns_get_record((char *)copy, &dns_reply, (char *)buf, sizeof buf); - if (bp != NULL) - { - pctx->organizational_domain = (u_char *)strdup((char *)cur); - pctx->org_domain_from_fallback = 1; - goto got_record; - } - /* - * Was a CNAME found that the resolver did not follow on its own? - */ - if (bp == NULL && *buf != '\0') - { - (void) strlcpy((char *)copy, (char *)buf, sizeof copy); - if (--loop_count != 0) - goto query_again3; - } - } + case OPENDMARC_WALK_MODE_AUTO: + default: + /* + * AUTO: use PSL if a TLD file is loaded, otherwise fall back to + * the RFC 7489 label-strip walk. Preserves pre-refactor behaviour. + */ + status = query_dmarc_psl(pctx, domain, buf, sizeof buf, &dns_reply); + if (status != DMARC_PARSE_OKAY) + status = query_dmarc_rfc7489_walk(pctx, domain, buf, sizeof buf, &dns_reply); + break; } -dns_failed: - switch (dns_reply) - { - case HOST_NOT_FOUND: - case NO_DATA: - case NO_RECOVERY: - return DMARC_DNS_ERROR_NO_RECORD; - case TRY_AGAIN: - case NETDB_INTERNAL: - return DMARC_DNS_ERROR_TMPERR; - default: - return DMARC_DNS_ERROR_NO_RECORD; - } -got_record: + if (status != DMARC_PARSE_OKAY) + return status; + return opendmarc_policy_parse_dmarc(pctx, domain, buf); } @@ -1098,6 +1303,28 @@ opendmarc_policy_parse_dmarc(DMARC_POLICY_T *pctx, u_char *domain, u_char *recor return DMARC_PARSE_ERROR_BAD_VALUE; } } + else if (strcasecmp((char *)cp, "np") == 0) + { + /* RFC 9989: non-existent subdomain policy */ + if (strncasecmp((char *)vp, "reject", strlen((char *)vp)) == 0) + pctx->np = DMARC_RECORD_P_REJECT; + else if (strncasecmp((char *)vp, "none", strlen((char *)vp)) == 0) + pctx->np = DMARC_RECORD_P_NONE; + else if (strncasecmp((char *)vp, "quarantine", strlen((char *)vp)) == 0) + pctx->np = DMARC_RECORD_P_QUARANTINE; + else + return DMARC_PARSE_ERROR_BAD_VALUE; + } + else if (strcasecmp((char *)cp, "psd") == 0) + { + /* RFC 9989: public suffix declaration */ + if (strncasecmp((char *)vp, "y", 1) == 0) + pctx->psd = DMARC_RECORD_PSD_Y; + else if (strncasecmp((char *)vp, "n", 1) == 0) + pctx->psd = DMARC_RECORD_PSD_N; + else + return DMARC_PARSE_ERROR_BAD_VALUE; + } else if (strcasecmp((char *)cp, "adkim") == 0) { /* diff --git a/opendmarc/opendmarc-config.h b/opendmarc/opendmarc-config.h index 284bbe65..0e4d67a9 100644 --- a/opendmarc/opendmarc-config.h +++ b/opendmarc/opendmarc-config.h @@ -26,6 +26,7 @@ struct configdef dmarcf_config[] = { "BaseDirectory", CONFIG_TYPE_STRING, FALSE }, { "ChangeRootDirectory", CONFIG_TYPE_STRING, FALSE }, { "CopyFailuresTo", CONFIG_TYPE_STRING, FALSE }, + { "DMARCbisWalkMode", CONFIG_TYPE_STRING, FALSE }, { "DNSTimeout", CONFIG_TYPE_INTEGER, FALSE }, { "DomainWhitelist", CONFIG_TYPE_STRING, FALSE }, { "DomainWhitelistFile", CONFIG_TYPE_STRING, FALSE }, diff --git a/opendmarc/opendmarc.c b/opendmarc/opendmarc.c index b1d7c0fd..e197fd70 100644 --- a/opendmarc/opendmarc.c +++ b/opendmarc/opendmarc.c @@ -193,6 +193,7 @@ struct dmarcf_config char * conf_historyfile; char * conf_rejectstring; char * conf_pslist; + char * conf_walkmode; char * conf_ignorelist; char ** conf_trustedauthservids; char ** conf_ignoredomains; @@ -1333,6 +1334,10 @@ dmarcf_config_load(struct config *data, struct dmarcf_config *conf, &conf->conf_pslist, sizeof conf->conf_pslist); + (void) config_get(data, "DMARCbisWalkMode", + &conf->conf_walkmode, + sizeof conf->conf_walkmode); + if (!conf->conf_dolog) { (void) config_get(data, "Syslog", &conf->conf_dolog, @@ -5341,6 +5346,18 @@ main(int argc, char **argv) sizeof libopendmarc.tld_source_file - 1); } + if (curconf->conf_walkmode != NULL) + { + if (strcasecmp(curconf->conf_walkmode, "PSL") == 0) + libopendmarc.walk_mode = OPENDMARC_WALK_MODE_PSL; + else if (strcasecmp(curconf->conf_walkmode, "RFC7489") == 0) + libopendmarc.walk_mode = OPENDMARC_WALK_MODE_RFC7489; + else if (strcasecmp(curconf->conf_walkmode, "RFC9989") == 0) + libopendmarc.walk_mode = OPENDMARC_WALK_MODE_RFC9989; + else + libopendmarc.walk_mode = OPENDMARC_WALK_MODE_AUTO; + } + if (opendmarc_policy_library_init(&libopendmarc) != 0) { if (curconf->conf_dolog) diff --git a/opendmarc/opendmarc.conf.5.in b/opendmarc/opendmarc.conf.5.in index a4323ee0..c2d2d613 100644 --- a/opendmarc/opendmarc.conf.5.in +++ b/opendmarc/opendmarc.conf.5.in @@ -144,6 +144,37 @@ configuration parameter and the domains listed in the DomainWhiteListFile. The final size will be increased by approximately 20% to increase the efficiency of the hashing algorithm. +.TP +.I DMARCbisWalkMode (string) +Selects the strategy used to find the Organizational Domain when the author +domain has no DMARC record of its own. Accepted values are: +.RS +.TP +.B Auto +Use the Public Suffix List if +.I PublicSuffixList +is configured; otherwise fall back to the RFC 7489 label-strip walk. +This is the default and preserves pre-DMARCbis behaviour. +.TP +.B PSL +Use only the Public Suffix List. No fallback walk is performed if the PSL +cannot identify a boundary. Requires +.I PublicSuffixList +to be configured. +.TP +.B RFC7489 +Label-strip walk per RFC 7489. Strips labels from the left one at a time +and stops before bare TLDs. Does not require a PSL. +.TP +.B RFC9989 +DNS Tree Walk per RFC 9989 section 4.10. Walks upward through the DNS +hierarchy stopping when a record with +.I psd=n +or +.I psd=y +is found. Applies an 8-query cap. Does not require a PSL. +.RE + .TP .I DNSTimeout (integer) Sets the DNS timeout in seconds. A value of 0 causes an infinite wait. diff --git a/opendmarc/opendmarc.conf.sample b/opendmarc/opendmarc.conf.sample index 44c0dd8e..f3c8799b 100644 --- a/opendmarc/opendmarc.conf.sample +++ b/opendmarc/opendmarc.conf.sample @@ -140,6 +140,31 @@ # # DomainWhitelistSize 3000 +## DMARCbisWalkMode (string) +## default (none, behaves as "Auto") +## +## Selects the strategy used to find the Organizational Domain when the +## author domain has no DMARC record of its own. Accepted values: +## +## Auto -- Use PSL if PublicSuffixList is configured; otherwise +## fall back to the RFC 7489 label-strip walk. This is +## the pre-DMARCbis default and preserves existing behaviour. +## +## PSL -- Use the Public Suffix List only. No fallback walk is +## performed if the PSL cannot identify a boundary. Requires +## PublicSuffixList to be configured. +## +## RFC7489 -- Label-strip walk per RFC 7489. Strips labels from the +## left one at a time and stops before bare TLDs. Does not +## require a PSL. +## +## RFC9989 -- DNS Tree Walk per RFC 9989 section 4.10. Walks upward +## through the DNS hierarchy stopping only when a record +## with psd=n or psd=y is found. Applies an 8-query cap. +## Does not require a PSL. +# +# DMARCbisWalkMode Auto + ## DNSTimeout (integer) ## default 5 ## From f1205e935a5931a856d5f286653e19c012856066 Mon Sep 17 00:00:00 2001 From: Dan Mahoney Date: Wed, 17 Jun 2026 16:30:32 -0400 Subject: [PATCH 2/5] fix: discard multiple DMARC records at one name (RFC 7489 S 6.6.3, RFC 9989 S 4.10) dmarc_dns_get_record() previously returned the first "v=DMARC" TXT record found without checking for duplicates, silently violating the rule in both RFCs that multiple valid DMARC records at a single DNS name must all be discarded. - Scan all TXT records in the answer before returning; count those containing "v=DMARC". Return the record only when exactly one is found; return NULL/NO_DATA when zero or more than one are found. - Fix the fake DNS table path with the same count-before-returning logic. - Fix a pre-existing cursor advancement bug: the non-DMARC TXT branch was adding the already-consumed dn_expand name length a second time. Replace ad-hoc cur_ptr arithmetic with an rdata_end pointer set from RDLENGTH. - Add a fake-DNS test case: two v=DMARC1 entries for the same name must yield DMARC_DNS_ERROR_NO_RECORD. --- libopendmarc/opendmarc_dns.c | 156 +++++++++++-------- libopendmarc/tests/test_subdomain_fallback.c | 25 +++ 2 files changed, 120 insertions(+), 61 deletions(-) diff --git a/libopendmarc/opendmarc_dns.c b/libopendmarc/opendmarc_dns.c index d85ad1c8..ca87c88c 100644 --- a/libopendmarc/opendmarc_dns.c +++ b/libopendmarc/opendmarc_dns.c @@ -159,22 +159,35 @@ dmarc_dns_get_record(char *domain, int *reply, char *got_txtbuf, size_t got_txtl /* * Pull the answer from the fake DNS table if there is one. + * Count how many entries match this name and contain "v=DMARC"; + * if more than one is found, discard all per RFC 7489 S 6.6.3 and + * RFC 9989 S 4.10. */ if (fake_dns != NULL) { struct fake_dns_data *cur; + struct fake_dns_data *found_entry = NULL; + int dmarc_count = 0; for (cur = fake_dns; cur != NULL; cur = cur->fdns_next) { - if (strcasecmp(cur->fdns_name, domain) == 0) + if (strcasecmp(cur->fdns_name, domain) == 0 && + strstr(cur->fdns_answer, "v=DMARC") != NULL) { - strncpy(got_txtbuf, cur->fdns_answer, - got_txtlen - 1); - *reply_ptr = NETDB_SUCCESS; - return got_txtbuf; + dmarc_count++; + found_entry = cur; } } + if (dmarc_count == 1) + { + strncpy(got_txtbuf, found_entry->fdns_answer, + got_txtlen - 1); + got_txtbuf[got_txtlen - 1] = '\0'; + *reply_ptr = NETDB_SUCCESS; + return got_txtbuf; + } + *reply_ptr = NO_DATA; return NULL; } @@ -273,78 +286,99 @@ dmarc_dns_get_record(char *domain, int *reply, char *got_txtbuf, size_t got_txtl *reply_ptr = NO_DATA; return NULL; } - while (--acnt >= 0 && cur_ptr < end_ptr) + + /* + * Scan ALL TXT records in the answer section and count those that + * contain "v=DMARC". Per RFC 7489 S 6.6.3 step 5 and RFC 9989 + * S 4.10 steps 2 and 6, if more than one valid DMARC record is found + * at a single name, all are discarded and policy discovery stops. + */ { - if ((answer_len = dn_expand((unsigned char *) &answer_buf, - end_ptr, cur_ptr, namebuf, - sizeof namebuf)) < 0) - { - *reply_ptr = NO_DATA; - return NULL; - } - cur_ptr += answer_len; + int dmarc_count = 0; + u_char saved_buf[BUFSIZ]; + u_char * rdata_end; - if (cur_ptr + INT16SZ + INT16SZ > end_ptr) - { - /* corrupt answer */ - *reply_ptr = NO_DATA; - return NULL; - } - GETSHORT(type, cur_ptr); - GETSHORT(class, cur_ptr); - GETLONG(ttl, cur_ptr); + (void) memset(saved_buf, '\0', sizeof saved_buf); - if (type != T_TXT) + while (--acnt >= 0 && cur_ptr < end_ptr) { - /* skip RRTYPEs we don't know */ - GETSHORT(answer_len, cur_ptr); + if ((answer_len = dn_expand((unsigned char *) &answer_buf, + end_ptr, cur_ptr, namebuf, + sizeof namebuf)) < 0) + { + *reply_ptr = NO_DATA; + return NULL; + } cur_ptr += answer_len; - continue; - } - if (cur_ptr + INT16SZ > end_ptr) - { - /* - * Yikes. No payload length - */ - *reply_ptr = NO_DATA; - return NULL; - } + if (cur_ptr + INT16SZ + INT16SZ > end_ptr) + { + /* corrupt answer */ + *reply_ptr = NO_DATA; + return NULL; + } + GETSHORT(type, cur_ptr); + GETSHORT(class, cur_ptr); + GETLONG(ttl, cur_ptr); + + if (cur_ptr + INT16SZ > end_ptr) + { + *reply_ptr = NO_DATA; + return NULL; + } + GETSHORT(cur_len, cur_ptr); /* RDLENGTH */ + rdata_end = cur_ptr + cur_len; - GETSHORT(cur_len, cur_ptr); + if (rdata_end > end_ptr) + { + *reply_ptr = NO_DATA; + return NULL; + } - if (cur_ptr + cur_len > end_ptr) - { - /* - * If the payload length greater than remaining buffer + if (type != T_TXT) + { + cur_ptr = rdata_end; + continue; + } + + /* + * Decode the TXT RDATA (... chunks) into + * a temporary buffer without modifying cur_ptr directly, + * so we can always advance to rdata_end afterward. */ - *reply_ptr = NO_DATA; - return NULL; - } - (void) memset(got_txtbuf, '\0', got_txtlen); - /* copy the returned record into got_txtbuf */ - got_ptr = (u_char *)got_txtbuf; - gote_ptr = (u_char *)got_txtbuf + got_txtlen -1; - while (cur_len > 0 && got_ptr < gote_ptr) - { - ch = *cur_ptr++; - cur_len--; - while (ch > 0 && got_ptr < gote_ptr) { - *got_ptr++ = *cur_ptr++; - ch--; - cur_len--; + u_char txt_tmp[BUFSIZ]; + u_char * tp = txt_tmp; + u_char * te = txt_tmp + sizeof txt_tmp - 1; + u_char * sp = cur_ptr; + + while (sp < rdata_end && tp < te) + { + ch = (unsigned char)*sp++; + while (ch-- > 0 && sp < rdata_end && tp < te) + *tp++ = *sp++; + } + *tp = '\0'; + + if (strstr((char *)txt_tmp, "v=DMARC") != NULL) + { + dmarc_count++; + if (dmarc_count == 1) + (void) strlcpy((char *)saved_buf, + (char *)txt_tmp, + sizeof saved_buf); + } } + + cur_ptr = rdata_end; } - if (strstr(got_txtbuf, "v=DMARC") != NULL) + + if (dmarc_count == 1) { + (void) strlcpy(got_txtbuf, (char *)saved_buf, got_txtlen); *reply_ptr = NETDB_SUCCESS; return got_txtbuf; } - *got_txtbuf = '\0'; - cur_ptr += cur_len; - cur_ptr += answer_len; - continue; } *reply_ptr = NO_DATA; return NULL; diff --git a/libopendmarc/tests/test_subdomain_fallback.c b/libopendmarc/tests/test_subdomain_fallback.c index 34575246..3b5389a4 100644 --- a/libopendmarc/tests/test_subdomain_fallback.c +++ b/libopendmarc/tests/test_subdomain_fallback.c @@ -61,6 +61,15 @@ main(int argc, char **argv) * Use a distinct domain per test to avoid cross-contamination. */ + /* + * Multi-record discard test (RFC 7489 S 6.6.3 step 5 / RFC 9989 S 4.10): + * two "v=DMARC1" entries for the same name must both be discarded. + */ + opendmarc_dns_fake_record("_dmarc.multi.example", + "v=DMARC1; p=reject"); + opendmarc_dns_fake_record("_dmarc.multi.example", + "v=DMARC1; p=none"); + /* Test domains and their fake records. */ opendmarc_dns_fake_record("_dmarc.parent1.example", "v=DMARC1; p=reject; rua=mailto:dmarc@parent1.example"); @@ -80,6 +89,22 @@ main(int argc, char **argv) "v=DMARC1; p=reject"); /* _dmarc.sub.bcx.com: absent → NO_DATA */ + /* + * === Test 0 === + * Multi-record discard: two "v=DMARC1" records at the same name must + * both be discarded per RFC 7489 S 6.6.3 step 5 / RFC 9989 S 4.10. + */ + pctx = opendmarc_policy_connect_init((u_char *)"1.2.3.4", 0); + if (pctx == NULL) { fprintf(stderr, "connect_init failed\n"); return 1; } + + (void) opendmarc_policy_store_from_domain(pctx, (u_char *)"multi.example"); + status = opendmarc_policy_query_dmarc(pctx, (u_char *)"multi.example"); + + CHECK(status == DMARC_DNS_ERROR_NO_RECORD, + "multi-record discard: two v=DMARC1 records must yield NO_RECORD"); + + pctx = opendmarc_policy_connect_shutdown(pctx); + /* * === Test 1 === * No PSL. Subdomain has no record; parent has p=reject. From 1e7e4c062620ec4f9fb10c36b6dda22d827a0870 Mon Sep 17 00:00:00 2001 From: Dan Mahoney Date: Wed, 17 Jun 2026 16:58:54 -0400 Subject: [PATCH 3/5] feat(survey): check RFC 7489 S 7.1 external reporting consent in pct-survey For every rua= and ruf= URI whose destination domain differs from the policy domain, query {policy-domain}._report._dmarc.{dest-domain} and flag any that lack a v=DMARC1 consent record. - New output columns rua_bad / ruf_bad: comma-separated list of external URIs missing consent, or "-" if none problematic - Async consent-check phase after the main DNS loop, at full concurrency - Same-domain and subdomain URIs skipped (no check needed) - Output buffered until consent phase completes; then written in one pass - Summary log gains rua_ext_bad / ruf_ext_bad columns for trend tracking --- contrib/dmarc-pct-survey.pl | 230 +++++++++++++++++++++++++++++++++--- 1 file changed, 211 insertions(+), 19 deletions(-) diff --git a/contrib/dmarc-pct-survey.pl b/contrib/dmarc-pct-survey.pl index abefe9c5..396ea29d 100755 --- a/contrib/dmarc-pct-survey.pl +++ b/contrib/dmarc-pct-survey.pl @@ -9,14 +9,20 @@ use IO::Uncompress::Unzip qw(unzip $UnzipError); # Query Umbrella top-1M domains for DMARC records, reporting pct=, psd=, and t= usage. +# Also checks RFC 7489 S 7.1 external reporting consent for every rua= and ruf= URI. # # If --input file does not exist, downloads it automatically from Cisco Umbrella. # -# Output (--output or dated file): TSV - run_date, domain, pct_value, psd_value, t_value, full_record -# run_date: ISO 8601 date of this run (YYYY-MM-DD), for multi-run aggregation -# pct_value: numeric value if present, "-" if absent -# psd_value: "y", "n", or "-" if absent -# t_value: "y", "n", or "-" if absent +# Output (--output or dated file): TSV +# run_date: ISO 8601 date of this run (YYYY-MM-DD), for multi-run aggregation +# domain: queried domain +# pct: numeric value if present, "-" if absent +# psd: "y", "n", or "-" if absent +# t: "y", "n", or "-" if absent +# rua_bad: comma-separated external rua= URIs lacking {policy}._report._dmarc.{dest} +# consent records per RFC 7489 S 7.1; "-" if none problematic +# ruf_bad: same for ruf= +# record: full DMARC TXT record # # Extra output (--extra-output or dated file): TSV of _dmarc records found at label # levels above the queried domains that are NOT themselves in the top-1M input. @@ -76,7 +82,6 @@ ); open(my $fh, '<', $infile) or die "Cannot open $infile: $!\n"; -open(my $out, '>:encoding(UTF-8)', $outfile) or die "Cannot open $outfile: $!\n"; # Stats my $n_queued = 0; @@ -92,9 +97,16 @@ my $n_pct100_noop = 0; # p=reject or p=quarantine with pct=100 (a no-op) my $n_none_pct = 0; # p=none with any pct= value (has no effect) my $n_errors = 0; +my $n_rua_ext_bad = 0; # domains with at least one unconsented external rua= +my $n_ruf_ext_bad = 0; # domains with at least one unconsented external ruf= my %has_dmarc; # domains that returned a v=DMARC1 record in the main phase +# Buffered main-phase results: domain => hashref of parsed fields + URI lists. +# Written to output only after the consent-check phase completes. +my %dmarc_found; +my @dmarc_order; # preserve completion order for output + # Parent-walk tracking: domains we directly queried, and unlisted parents to check. my %known_domains; my %extra_parents; # parent_domain => first_triggering_child @@ -106,6 +118,25 @@ my $progress_interval = 10_000; my $next_progress = $progress_interval; +# Extract the host (destination domain) from a rua=/ruf= URI. +# Returns undef for unrecognised schemes. +sub extract_uri_host { + my ($uri) = @_; + if ($uri =~ /^mailto:[^@]+\@([^!>\s]+)/i) { + return lc($1); + } + if ($uri =~ m{^https?://([^/:?\#\s]+)}i) { + return lc($1); + } + return undef; +} + +# Parse a comma-separated tag value (rua= or ruf=) into a list of URI strings. +sub parse_uri_list { + my ($val) = @_; + return map { s/^\s+|\s+$//gr } split(/,/, $val); +} + sub dispatch { my ($domain) = @_; my $qname = "_dmarc.$domain"; @@ -132,17 +163,19 @@ sub harvest { my $pkt = eval { $resolver->bgread($sock) }; unless ($pkt) { $n_errors++; - print $out "# error: $domain\n"; + # record error for output phase + $dmarc_found{"#error:$domain"} = undef; + push @dmarc_order, "#error:$domain"; next; } my $rcode = $pkt->header->rcode; - # NXDOMAIN and NOERROR-with-no-answers are normal (domain has no record) next if $rcode eq 'NXDOMAIN'; next if $rcode eq 'NOERROR' && !($pkt->answer); if ($rcode ne 'NOERROR') { $n_errors++; - print $out "# $rcode: $domain\n"; + $dmarc_found{"#$rcode:$domain"} = undef; + push @dmarc_order, "#$rcode:$domain"; next; } @@ -170,7 +203,6 @@ sub harvest { $n_t_y++ if $t eq 'y'; $n_t_n++ if $t eq 'n'; } - if (($p eq 'reject' || $p eq 'quarantine') && $pct eq '100') { $n_pct100_noop++; } @@ -178,7 +210,19 @@ sub harvest { $n_none_pct++; } - print $out join("\t", $run_date, $domain, $pct, $psd, $t, $txt), "\n"; + # Parse rua= and ruf= URI lists for consent checking. + my @rua_uris = ($txt =~ /\brua=([^;]+)/i) ? parse_uri_list($1) : (); + my @ruf_uris = ($txt =~ /\bruf=([^;]+)/i) ? parse_uri_list($1) : (); + + $dmarc_found{$domain} = { + pct => $pct, + psd => $psd, + t => $t, + txt => $txt, + rua_uris => \@rua_uris, + ruf_uris => \@ruf_uris, + }; + push @dmarc_order, $domain; last; # only evaluate first v=DMARC1 record } } @@ -193,7 +237,8 @@ sub reap_stale { $sel->remove($sock); $n_errors++; $n_done++; - print $out "# timeout: $domain\n"; + $dmarc_found{"#timeout:$domain"} = undef; + push @dmarc_order, "#timeout:$domain"; } } } @@ -262,9 +307,8 @@ sub harvest_extra { } } -print $out join("\t", "run_date", "domain", "pct", "psd", "t", "record"), "\n"; - -print STDERR "Reading $infile, writing $outfile, concurrency=$concurrency, timeout=${timeout}s\n"; +print STDERR "Reading $infile, writing $outfile (buffered until consent checks complete)\n"; +print STDERR "Concurrency=$concurrency, timeout=${timeout}s\n"; while (my $line = <$fh>) { chomp $line; @@ -313,8 +357,6 @@ sub harvest_extra { } } -close($out); - # --- Extra phase: walk parent labels not directly queried --- delete $extra_parents{$_} for keys %known_domains; @@ -348,6 +390,152 @@ sub harvest_extra { close($extra); +# --- Consent check phase (RFC 7489 S 7.1) --- +# +# For every external rua= and ruf= URI found in the main phase, query +# {policy-domain}._report._dmarc.{dest-domain}. A v=DMARC1 record there +# authorises the destination to receive reports on behalf of the policy domain. +# Absence means the address is unconsented and must be ignored per the RFC. + +print STDERR "\nRunning RFC 7489 S 7.1 consent checks ...\n"; + +# %consent_result: "$policy_domain|$dest_domain" => 1 (ok) or 0 (missing/error) +my %consent_result; + +# Build the list of unique (policy, dest) pairs that need checking. +my @consent_queue; +for my $domain (keys %dmarc_found) { + next unless defined $dmarc_found{$domain}; # skip error/timeout sentinels + my $data = $dmarc_found{$domain}; + for my $tag ('rua', 'ruf') { + for my $uri (@{$data->{"${tag}_uris"}}) { + my $dest = extract_uri_host($uri); + next unless defined $dest; + # Same domain (or a subdomain of it): no consent check required. + next if $dest eq $domain || $dest =~ /\.\Q$domain\E$/i; + my $key = "$domain|$dest"; + next if exists $consent_result{$key}; + $consent_result{$key} = undef; # mark pending + push @consent_queue, [$domain, $dest]; + } + } +} + +my $n_consent_total = scalar @consent_queue; +my $n_consent_done = 0; +printf STDERR " %d unique (policy, destination) pairs to check\n", $n_consent_total; + +# Async consent check loop, reusing the same resolver and IO::Select. +my %consent_inflight; # socket => [policy_domain, dest_domain, dispatch_time] +my $csel = IO::Select->new; + +while (@consent_queue || %consent_inflight) { + # Fill up to concurrency + while (@consent_queue && scalar(keys %consent_inflight) < $concurrency) { + my ($pdomain, $dest) = @{shift @consent_queue}; + my $qname = "${pdomain}._report._dmarc.${dest}"; + my $sock = $resolver->bgsend($qname, 'TXT'); + if ($sock) { + $consent_inflight{$sock} = [$pdomain, $dest, time()]; + $csel->add($sock); + } else { + $consent_result{"$pdomain|$dest"} = 0; + $n_consent_done++; + } + } + + my @ready = $csel->can_read(%consent_inflight ? $timeout : 0); + for my $sock (@ready) { + my ($pdomain, $dest) = @{delete $consent_inflight{$sock}}; + $csel->remove($sock); + my $key = "$pdomain|$dest"; + $n_consent_done++; + + my $pkt = eval { $resolver->bgread($sock) }; + if (!$pkt) { + $consent_result{$key} = 0; + next; + } + + my $rcode = $pkt->header->rcode; + if ($rcode eq 'NOERROR') { + my $found = 0; + for my $rr ($pkt->answer) { + next unless $rr->type eq 'TXT'; + if (join('', $rr->txtdata) =~ /^v=DMARC1\b/i) { + $found = 1; + last; + } + } + $consent_result{$key} = $found; + } else { + $consent_result{$key} = 0; # NXDOMAIN or error: not consented + } + } + + # Reap stale consent queries + my $now = time(); + for my $sock (keys %consent_inflight) { + if ($now - $consent_inflight{$sock}[2] > $timeout * 2) { + my ($pdomain, $dest) = @{delete $consent_inflight{$sock}}; + $csel->remove($sock); + $consent_result{"$pdomain|$dest"} = 0; + $n_consent_done++; + } + } + + if ($n_consent_done > 0 && $n_consent_total > 0 && + $n_consent_done % 1000 == 0) { + printf STDERR " consent checks: %d/%d done\n", + $n_consent_done, $n_consent_total; + } +} + +# --- Write main output --- + +open(my $out, '>:encoding(UTF-8)', $outfile) or die "Cannot open $outfile: $!\n"; +print $out join("\t", "run_date", "domain", "pct", "psd", "t", "rua_bad", "ruf_bad", "record"), "\n"; + +for my $key (@dmarc_order) { + # Error/timeout sentinels stored as "#type:domain" + if ($key =~ /^#(.+?):(.+)$/) { + print $out "# $1: $2\n"; + next; + } + + my $domain = $key; + my $data = $dmarc_found{$domain}; + next unless defined $data; + + # Determine unconsented external URIs for rua= and ruf=. + my %bad; + for my $tag ('rua', 'ruf') { + my @bad_uris; + for my $uri (@{$data->{"${tag}_uris"}}) { + my $dest = extract_uri_host($uri); + next unless defined $dest; + next if $dest eq $domain || $dest =~ /\.\Q$domain\E$/i; + my $key2 = "$domain|$dest"; + if (!$consent_result{$key2}) { + push @bad_uris, $uri; + } + } + $bad{$tag} = @bad_uris ? join(',', @bad_uris) : '-'; + } + + $n_rua_ext_bad++ if $bad{rua} ne '-'; + $n_ruf_ext_bad++ if $bad{ruf} ne '-'; + + print $out join("\t", + $run_date, $domain, + $data->{pct}, $data->{psd}, $data->{t}, + $bad{rua}, $bad{ruf}, + $data->{txt}, + ), "\n"; +} + +close($out); + printf STDERR "\nDone. Output written to %s\n", $outfile; printf STDERR " Domains queried : %d\n", $n_done; printf STDERR " Errors : %d\n", $n_errors; @@ -361,12 +549,16 @@ sub harvest_extra { printf STDERR " t=n : %d\n", $n_t_n; printf STDERR " p=reject/quarantine with pct=100 (no-op) : %d\n", $n_pct100_noop; printf STDERR " p=none with pct= (no effect) : %d\n", $n_none_pct; +printf STDERR "RFC 7489 S 7.1 external reporting consent (%s):\n", $outfile; +printf STDERR " Pairs checked : %d\n", $n_consent_total; +printf STDERR " Domains w/ unconsented rua= : %d\n", $n_rua_ext_bad; +printf STDERR " Domains w/ unconsented ruf= : %d\n", $n_ruf_ext_bad; printf STDERR "Parent-label walk (%s):\n", $extrafile; printf STDERR " Unlisted parents checked : %d\n", $n_extra_queued; printf STDERR " Unlisted parents w/DMARC : %d\n", $n_extra_dmarc; my $is_new = !-f $summarylog; open(my $sum, '>>', $summarylog) or die "Cannot open $summarylog: $!\n"; -print $sum join("\t", qw(run_date queried errors have_dmarc pct_total psd_total psd_y psd_n t_total t_y t_n pct100_noop none_pct)), "\n" if $is_new; -print $sum join("\t", $run_date, $n_done, $n_errors, $n_dmarc, $n_pct, $n_psd, $n_psd_y, $n_psd_n, $n_t, $n_t_y, $n_t_n, $n_pct100_noop, $n_none_pct), "\n"; +print $sum join("\t", qw(run_date queried errors have_dmarc pct_total psd_total psd_y psd_n t_total t_y t_n pct100_noop none_pct rua_ext_bad ruf_ext_bad)), "\n" if $is_new; +print $sum join("\t", $run_date, $n_done, $n_errors, $n_dmarc, $n_pct, $n_psd, $n_psd_y, $n_psd_n, $n_t, $n_t_y, $n_t_n, $n_pct100_noop, $n_none_pct, $n_rua_ext_bad, $n_ruf_ext_bad), "\n"; close($sum); From cc47258043e2ca198147b94886e9980b3d6808e5 Mon Sep 17 00:00:00 2001 From: Dan Mahoney Date: Sun, 21 Jun 2026 14:09:34 -0400 Subject: [PATCH 4/5] fix: resolve pointer-sign warnings in DMARCbis walk-mode helpers Casts strdup() results back to u_char * for organizational_domain assignments in query_dmarc_psl, query_dmarc_rfc7489_walk, and query_dmarc_rfc9989_walk, matching the cast convention applied elsewhere in #426. --- libopendmarc/opendmarc_policy.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/libopendmarc/opendmarc_policy.c b/libopendmarc/opendmarc_policy.c index 2eba1ace..d93e8595 100644 --- a/libopendmarc/opendmarc_policy.c +++ b/libopendmarc/opendmarc_policy.c @@ -817,7 +817,7 @@ query_dmarc_psl(DMARC_POLICY_T *pctx, u_char *domain, strcasecmp((char *)tld, (char *)domain) == 0) return DMARC_DNS_ERROR_NO_RECORD; - pctx->organizational_domain = strdup((char *)tld); + pctx->organizational_domain = (u_char *)strdup((char *)tld); if (dmarc_query_at((char *)tld, &dns_reply, buf, bufsz) != NULL) { @@ -860,7 +860,7 @@ query_dmarc_rfc7489_walk(DMARC_POLICY_T *pctx, u_char *domain, if (dmarc_query_at((char *)cur, &dns_reply, buf, bufsz) != NULL) { - pctx->organizational_domain = strdup((char *)cur); + pctx->organizational_domain = (u_char *)strdup((char *)cur); pctx->org_domain_from_fallback = 1; if (dns_reply_out != NULL) *dns_reply_out = dns_reply; @@ -1001,11 +1001,11 @@ query_dmarc_rfc9989_walk(DMARC_POLICY_T *pctx, u_char *domain, * Step 3: no psd= -> use best_domain (fewest labels = last found). */ if (best_psd == DMARC_RECORD_PSD_N) - pctx->organizational_domain = strdup(best_domain); + pctx->organizational_domain = (u_char *)strdup(best_domain); else if (best_psd == DMARC_RECORD_PSD_Y) - pctx->organizational_domain = strdup(prev); + pctx->organizational_domain = (u_char *)strdup(prev); else - pctx->organizational_domain = strdup(best_domain); + pctx->organizational_domain = (u_char *)strdup(best_domain); (void) strlcpy((char *)buf, (char *)best_buf, bufsz); return DMARC_PARSE_OKAY; From 9e78478404b15750f3ea025f77916ac4d1909822 Mon Sep 17 00:00:00 2001 From: Dan Mahoney Date: Sun, 21 Jun 2026 14:11:50 -0400 Subject: [PATCH 5/5] docs: record AUTO-mode fallback scope decision in DMARCbis walk notes Documents that AUTO mode's fallback to the RFC 7489 walk applies whenever the PSL path fails, not only when no PSL boundary was found, which is a deliberate broadening versus pre-refactor behavior. --- DMARCBIS-WALK-NOTES.txt | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/DMARCBIS-WALK-NOTES.txt b/DMARCBIS-WALK-NOTES.txt index 3fb4349d..db4ea8c4 100644 --- a/DMARCBIS-WALK-NOTES.txt +++ b/DMARCBIS-WALK-NOTES.txt @@ -108,3 +108,21 @@ Status: walk implementation accumulates records. The selection process from 4.10.2 is applied post-walk. Policy discovery (4.10.1) uses the psd= stopping record when found; falls back to fewest-labels selection when the walk exhausts without psd=. + +------------------------------------------------------------------------ +6. AUTO mode fallback scope when a PSL boundary is found but unqueryable +------------------------------------------------------------------------ + +In OPENDMARC_WALK_MODE_AUTO, query_dmarc_psl() is tried first. If it +fails for any reason, query_dmarc_rfc7489_walk() is tried next. This +includes the case where the PSL found a real organizational-domain +boundary but a query at that boundary turned up no record, not only the +case where the PSL found no boundary at all (not loaded, or returned the +input domain unchanged). + +This is broader than the pre-refactor (RFC 7489-only) behavior, which +stopped without falling back to the label walk once a PSL boundary was +identified and queried unsuccessfully. + +Status: intentional. AUTO favors finding a record over strict adherence +to the PSL boundary when one was identified but had nothing there.