freeze format may emit import features with key "import_" rather than "import"

[williballenthin]

need to verify, but this would be unintended. we want they key to be import and then map back to import_ when in python-land.

[williballenthin]

maybe we can introduce some backwards-compatibility handling so we can change the key to "import" without breaking capa.

[williballenthin]

What happens: model_dump_json() is called without by_alias=True on lines 457 and 576 of capa/features/freeze/init.py. Pydantic's default is to serialize using the Python field name, so the JSON key is "import_" (with trailing underscore) instead of "import".

Why it exists: The Python field is named import_ because import is a reserved keyword. The Field(alias="import") is configured correctly, and populate_by_name=True means deserialization accepts both "import" and "import_" — so roundtrip tests pass even though the output key is wrong. This masks the bug.

The same issue affects other aliased fields: FunctionNameFeature.function_name (alias "function name"), ClassFeature.class_ (alias "class"), OperandNumberFeature.operand_number (alias "operand number"), OperandOffsetFeature.operand_offset (alias "operand offset"), and several fields in init.py like base_address (alias "base address"), basic_block/basic_blocks, global_.

Recommendation: The cleanest fix is to add by_alias=True to both model_dump_json() calls (lines 457 and 576), plus all the sorting lambdas that also call model_dump_json(). This ensures the freeze file uses the intended human-readable keys ("import", "function name", "class", etc.) rather than Python-internal names.

Alternatively, you could set model_config = ConfigDict(frozen=True, populate_by_name=True, serialize_by_alias=True) on FeatureModel (Pydantic v2 feature) to make alias serialization the default for all models inheriting from it — but that's a broader change that could affect result documents and other serialization paths. The targeted by_alias=True at the two freeze dump sites is safer and more explicit.

[williballenthin]

Impact of serialize_by_alias=True on FeatureModel

The change would be on FeatureModel only (line 25 of features.py). The container models in init.py and result_document.py inherit from BaseModel/HashableModel/FrozenModel — not FeatureModel — so they are not directly affected. However, when a parent model serializes a nested FeatureModel field, Pydantic respects the child's serialize_by_alias config.

Affected fields (5 total, all in features.py)

┌──────────────────────┬────────────────┬──────────────────┬──────────────────┐
│        Model         │     Field      │ Current JSON key │   Would become   │
├──────────────────────┼────────────────┼──────────────────┼──────────────────┤
│ ImportFeature        │ import_        │ "import_"        │ "import"         │
├──────────────────────┼────────────────┼──────────────────┼──────────────────┤
│ FunctionNameFeature  │ function_name  │ "function_name"  │ "function name"  │
├──────────────────────┼────────────────┼──────────────────┼──────────────────┤
| ClassFeature         │ class_         │ "class_"         │ "class"          │
├──────────────────────┼────────────────┼──────────────────┼──────────────────┤
│ OperandNumberFeature │ operand_number │ "operand_number" │ "operand number" │
├──────────────────────┼────────────────┼──────────────────┼──────────────────┤
│ OperandOffsetFeature │ operand_offset │ "operand_offset" │ "operand offset" │
└──────────────────────┴────────────────┴──────────────────┴──────────────────┘

Serialization sites impacted

1. Freeze format output (capa/features/freeze/init.py)

• Lines 457, 576: freeze.model_dump_json() — the two main dump points. Nested Feature fields would serialize using aliases. This is the desired fix.
• 10 sorting lambdas (lines 362, 372, 385, 398, 411, 473, 483, 497, 510, 524): feature.model_dump_json() — sort keys would change, but that's cosmetic (affects reproducible output ordering, not correctness).

2. Feature comparison (capa/features/common.py:161-162)

• Feature.lt uses model_dump_json() for ordering. Key names would change, altering sort order of features. Functionally fine — only needs to be consistent, not canonical.

3. Result document JSON output (nested Feature fields)

• capa/render/json.py:21 — ResultDocument.model_dump_json()
• capa/main.py:1013 — result_doc.model_dump_json()
• capa/ida/plugin/form.py:1309 — resdoc_cache.model_dump_json()
• scripts/bulk-process.py:148 — doc.model_dump()
• scripts/bulk-process.py:215, scripts/proto-to-results.py:75 — more ResultDocument dumps
• tests/test_result_document.py:252,260,267 — test assertions on JSON output

In all these, Feature models are nested inside StatementNode.child and FeatureNode.feature. The Feature portion of the JSON would change from "import_" to "import", etc. This is also the correct/desired behavior — but it means the result document JSON schema changes too, not just freeze files.

4. Already using by_alias=True (would become redundant but harmless)

• capa/render/vverbose.py:167 — child.model_dump(by_alias=True)
• capa/render/vverbose.py:227 — feature.model_dump(by_alias=True)
• capa/ida/plugin/model.py:560 — feature.dict(by_alias=True)
• tests/test_result_document.py:319 — meta.model_dump(by_alias=True) (this one is on a Metadata model, not FeatureModel, so unrelated)

5. Manual workarounds that could be removed

• capa/render/vverbose.py:220-224 — special-cases ImportFeature.import_ and ClassFeature.class_ because the alias-based lookup wouldn't work without by_alias. With serialize_by_alias=True, the generic path on line 227 would handle these correctly.

Bottom line

The FeatureModel ConfigDict approach is clean and correct — it fixes both freeze files and result document JSON in one shot. The blast radius is well-contained: only the 5 aliased fields change their serialization keys, and everywhere that consumes these already expects the alias form (that's why the aliases exist). The by_alias=True calls become redundant but harmless, and the manual workarounds in vverbose.py become removable dead code.

[williballenthin]

2. HashableModel/BaseModel subclasses in capa/features/freeze/init.py

5 aliased fields on container models:

  ┌───────────────────────────────┬──────────────────┬────────────────┐
  │             Field             │ Current JSON key │  Intended key  │
  ├───────────────────────────────┼──────────────────┼────────────────┤
  │ BasicBlockFeature.basic_block │ "basic_block"    │ "basic block"  │
  ├───────────────────────────────┼──────────────────┼────────────────┤
  │ FunctionFeatures.basic_blocks │ "basic_blocks"   │ "basic blocks" │
  ├───────────────────────────────┼──────────────────┼────────────────┤
  │ StaticFeatures.global_        │ "global_"        │ "global"       │
  ├───────────────────────────────┼──────────────────┼────────────────┤
  │ DynamicFeatures.global_       │ "global_"        │ "global"       │
  ├───────────────────────────────┼──────────────────┼────────────────┤
  │ Freeze.base_address           │ "base_address"   │ "base address" │
  └───────────────────────────────┴──────────────────┴────────────────┘

Serialization paths: same freeze model_dump_json() calls. These inherit from HashableModel/BaseModel, not FeatureModel, so the FeatureModel ConfigDict fix alone wouldn't help — these need their own fix.

3. FrozenModel subclasses in capa/render/result_document.py

7 aliased fields across two models:

  ┌─────────────────────────────────────┬──────────────────────────┬──────────────────────────┐
  │                Field                │     Current JSON key     │       Intended key       │
  ├─────────────────────────────────────┼──────────────────────────┼──────────────────────────┤
  │ MaecMetadata.analysis_conclusion    │ "analysis_conclusion"    │ "analysis-conclusion"    │
  ├─────────────────────────────────────┼──────────────────────────┼──────────────────────────┤
  │ MaecMetadata.analysis_conclusion_ov │ "analysis_conclusion_ov" │ "analysis-conclusion-ov" │
  ├─────────────────────────────────────┼──────────────────────────┼──────────────────────────┤
  │ MaecMetadata.malware_family         │ "malware_family"         │ "malware-family"         │
  ├─────────────────────────────────────┼──────────────────────────┼──────────────────────────┤
  │ MaecMetadata.malware_category       │ "malware_category"       │ "malware-category"       │
  ├─────────────────────────────────────┼──────────────────────────┼──────────────────────────┤
  │ MaecMetadata.malware_category_ov    │ "malware_category_ov"    │ "malware-category-ov"    │
  ├─────────────────────────────────────┼──────────────────────────┼──────────────────────────┤
  │ RuleMetadata.attack                 │ "attack"                 │ "att&ck"                 │
  ├─────────────────────────────────────┼──────────────────────────┼──────────────────────────┤
  │ RuleMetadata.is_subscope_rule       │ "is_subscope_rule"       │ "capa/subscope-rule"     │
  └─────────────────────────────────────┴──────────────────────────┴──────────────────────────┘

Serialization paths: capa/render/json.py:21, capa/main.py:1013, capa/ida/plugin/form.py:1309, scripts/bulk-process.py:148,215, scripts/proto-to-results.py:75.

[williballenthin]

The web UI is hardcoded to the current (buggy) Python field names:

• rdocParser.js:508 — feature.operand_offset (not feature["operand offset"])
• rdocParser.js:510 — feature.class_ (not feature["class"])
• rdocParser.js:512 — feature.import_ (not feature["import"])
• rdocParser.js:115 — rule.meta.attack (not rule.meta["att&ck"])
• rdocParser.js:268 — finfo.matched_basic_blocks (not finfo["matched basic blocks"])
• Test fixtures in rdocParser.test.js also use matched_basic_blocks