Skip to content

TIKA-4779: Add audio:bitrate, audio:is-variable-bitrate and audio:has-drm#2953

Open
dschmidt wants to merge 2 commits into
apache:mainfrom
dschmidt:audio-bitrate-vbr-drm
Open

TIKA-4779: Add audio:bitrate, audio:is-variable-bitrate and audio:has-drm#2953
dschmidt wants to merge 2 commits into
apache:mainfrom
dschmidt:audio-bitrate-vbr-drm

Conversation

@dschmidt

Copy link
Copy Markdown
Contributor

Part 2 of TIKA-4779, following up on #2947 with the three audio properties that need parser work rather than plain mapping: audio:bitrate (average, bits per second), audio:is-variable-bitrate and audio:has-drm, defined under Tika's own namespace in Audio.java.

MP3: the parser already walks every audio frame for the duration, and since frame duration does not depend on the bitrate, the plain mean over the frames is the exact average and differing frame bitrates are an exact VBR signal. The one special case is the Xing/Info/VBRI tag frame some encoders write first: it is a valid MPEG frame without audio, often at a different bitrate than the stream, so it is detected and excluded (a CBR file with an Info tag would otherwise be misreported as VBR, there is a fixture proving exactly that). I deliberately did not go the usual route of reading the bitrate from the Xing table itself: the full frame walk already happens for the duration, so the exact mean is free and works for files without any tag frame.

Vorbis: the identification header declares upper/nominal/lower rates. Nominal maps to audio:bitrate; a zero-width bracket (upper == lower, no nominal) also pins an exact rate. Fixed rate means all three declared rates agree, or only a zero-width bracket is declared; anything else counts as variable.

MP4/M4A: a TikaMp4SoundHandler (same extension pattern as TIKA-2861) structurally walks the sample description entries. Protected sample entry formats (drms, enca) set audio:has-drm; these markers are readable by design since DRM encrypts the payload, not the metadata. The average bitrate comes from the esds elementary stream descriptor, located as a child box of the entry (honoring the version-dependent entry layout, QuickTime wave wrappers and the optional ES descriptor fields). I deliberately avoided scanning the stsd bytes for fourcc markers: codec private data or a URL field can legally contain the string sinf, and a fixture covers that false positive.

For files with several audio tracks the properties reflect the last sound track's sample description, consistent with how the existing MP4 audio metadata behaves.

Note: the hdlr dispatch hook in TikaMp4BoxHandler overlaps with #2936; whichever merges second has a trivial rebase.

Tests run against real files where the repo has them (testMP4.m4a with a declared average of 256000, the constant bitrate MP3 fixtures, testVORBIS.ogg copied from the integration tests). Synthetic fixtures cover only what no real specimen exercises: a VBR MP3, a CBR MP3 with an Info tag frame, a drms protected M4A and an M4A whose ES descriptor uses the optional fields.

…s-drm

Part 2 of TIKA-4779, covering the audio properties that need parser work
rather than plain mapping.

MP3: the parser already walks every audio frame for the duration, and
frame duration does not depend on the bitrate, so the plain mean over
the frames is the true average bitrate and differing frame bitrates are
an exact variable bitrate signal. The only special case is the leading
Xing/Info/VBRI tag frame some encoders write: it is a valid MPEG frame
that carries no audio and may use a different bitrate, so it is detected
(via a new peek method on MpegStream) and kept out of the statistics.

Vorbis: the identification header declares upper/nominal/lower rates.
The nominal rate is mapped to audio:bitrate (or the shared bound when a
zero-width bracket is declared without a nominal rate), and the stream
counts as fixed rate when all three declared rates agree or when only a
zero-width bracket is declared.

MP4: a TikaMp4SoundHandler (same pattern as the box handler extensions
from TIKA-2861) walks the sound track's sample description entries.
Protected sample entry formats ('drms', 'enca') set audio:has-drm; the
esds elementary stream descriptor is located structurally below the
entry (honoring the version-dependent entry layout, QuickTime 'wave'
wrappers and the optional ES descriptor fields) and its average bitrate
is mapped to audio:bitrate.

Verified against the real files in the repo where they cover the
behavior: testVORBIS.ogg (copied from the integration tests; a real
quality-mode encoding with nominal 80000 and no bounds), the 192 kbps
testMP3lyrics.mp3 and the 128 kbps ID3 fixtures for constant bitrate,
and testMP4.m4a whose esds declares an average bitrate of 256000.
Synthetic fixtures fill only the gaps the repo has no real specimen
for: a variable bitrate MP3 (frames alternating 128/192 kbps), a
constant bitrate MP3 with an 'Info' tag frame at a different rate, a
DRM-protected M4A ('drms' sample entry) and an M4A whose ES descriptor
uses the optional dependency/URL/OCR fields.
@dschmidt dschmidt marked this pull request as ready for review July 14, 2026 14:35
@tballison

Copy link
Copy Markdown
Contributor

Small nits from my agent:

  1. Audio.HAS_DRM javadoc overstates the implementation. It says "…or a protection scheme ('sinf') box," but the
  code only flags the drms/enca sample-entry formats — it never independently detects a standalone sinf. In
  practice enca entries always carry sinf, so coverage is fine, but the doc implies a check that isn't there. Trim
  the sinf mention or reword to "protected sample entry formats."
  2. Multi-track asymmetry. BITRATE is last-track-wins (documented), but HAS_DRM is any-track-wins (only ever set
  to true). Defensible — arguably more correct for DRM — but the two properties describe "the last track" and "any
  track" respectively. Worth a one-line note in the PR/javadoc if you want it explicit.
  3. M4A with avgBitrate=0 gets no BITRATE even when maxBitrate>0 (some encoders only set max). Acceptable given
  the property is defined as average; just a coverage gap, not a bug.
  4. Vorbis nominal-only ⇒ is-variable-bitrate=true is aggressive but correct in practice (nominal-only is
  quality-mode VBR). Fine as-i

Not sure you need to address any of them. The add vs set, number 2 is ... not clear what the right answer is?

The has-drm doc still mentioned a standalone 'sinf' scan that the
review removed; it now names the protected sample entry formats only.
Both properties now state their multi-track semantics explicitly:
bitrate is per-stream (last sound track), has-drm is file-level (any
protected track sets it).
@dschmidt

dschmidt commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author
  1. was a leftover from an earlier iteration that scanned raw bytes for sinf; the javadoc now names only the protected sample entry formats.
  2. I think the asymmetry is the correct semantics rather than an accident: has-drm answers "does this file contain protected audio anywhere", so any track sets it, while bitrate describes one stream and follows the same last-sound-track convention as the rest of the MP4 audio metadata. Both javadocs now state their multi-track semantics explicitly.
  3. is deliberate: the property is defined as the average, and deriving one from maxBitrate would report a number the encoder never claimed; a separate max-bitrate property would be the clean extension if anyone ever needs it.
  4. agreed, leaving as is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants