feat: IW3 d3dbsp dumping / loading

[michaeloliverx]

[WIP] This needs cleanup and review, especially around structure/readability, but the core IW3 d3dbsp loader/dumper path is now working and producing linker_pc-compatible output.

Scope

IW3 raw .d3dbsp files use IBSP version 22 with named/typed chunks. The current loader/dumper work targets this v22 CoD4 Mod Tools format.

Radiant still has internal raw BSP/world loading code embedded in the executable. IW3xRadiant exposes that path, and the stock asserts/sanity checks have been useful for validating generated BSPs. Also the CoD4 decomp still has raw bsp loading code: https://github.com/SwagSoftware/KisakCOD/blob/1d8ff8d579187354ff595cce9477f2f78d5f2382/src/qcommon/cm_load_obj.cpp#L343

Current status

My primary testing map has been mp_backlot because the devs gave the full map source for it in the mod tools. The original cod4map.exe BSP is not expected to be byte-identical to the dumped fixed point. linker_pc consumes and normalizes parts of the source BSP into runtime assets, then OAT dumps the canonical post-link representation.

• An OAT dumped mp_backlot.d3dbsp and linked again by linker_pc works
• An OAT linked mp_backlot with an original cod4map.exe produced d3dbsp loaded from disk works
• An OAT dumped d3dbsp from many stockmaps loads into IW3xRadiant successfully

Given the same source .d3dbsp produced by cod4map.exe, the dumped raw BSP output is now byte-identical between the stock linker and OAT:

stock d3dbsp -> linker_pc -> OAT unlink
stock d3dbsp -> OAT link  -> OAT unlink

Both dumped BSPs have 35 chunks and all 35 / 35 are byte-identical:

0  MATERIALS
1  LIGHTMAPS
2  LIGHTGRID_ENTRIES
3  LIGHTGRID_COLORS
4  PLANES
5  BRUSHSIDES
6  BRUSHSIDE_EDGE_COUNTS
7  BRUSHEDGES
8  BRUSHES
9  LAYERED_TRI_SOUPS
10 LAYERED_VERTS
11 LAYERED_INDICES
25 CELLS
27 NODES
28 LEAFS
29 LEAFBRUSHES
31 COLLISIONVERTS
32 COLLISIONTRIS
33 COLLISION_EDGE_WALKABLE
34 COLLISIONBORDERS
35 COLLISIONPARTITIONS
36 COLLISIONAABBS
37 MODELS
39 ENTITIES
41 REFLECTION_PROBES
43 PRIMARY_LIGHTS
44 LIGHTGRID_HEADER
45 LIGHTGRID_ROWS
47 SIMPLE_TRI_SOUPS
48 SIMPLE_VERTS
49 SIMPLE_INDICES
51 SIMPLE_AABBTREES
52 LIGHT_REGION_COUNTS
53 LIGHT_REGION_HULLS
54 LIGHT_REGION_AXE

Source BSP vs Dumped BSP

16 chunks byte-identical
13 chunks same-size byte diffs
6 chunks size diffs
4 source chunks omitted from dumped fixed point

Byte-identical source chunks:

2  LIGHTGRID_ENTRIES
3  LIGHTGRID_COLORS
4  PLANES
5  BRUSHSIDES
6  BRUSHSIDE_EDGE_COUNTS
7  BRUSHEDGES
31 COLLISIONVERTS
32 COLLISIONTRIS
33 COLLISION_EDGE_WALKABLE
34 COLLISIONBORDERS
36 COLLISIONAABBS
44 LIGHTGRID_HEADER
45 LIGHTGRID_ROWS
52 LIGHT_REGION_COUNTS
53 LIGHT_REGION_HULLS
54 LIGHT_REGION_AXES

Source chunks not emitted in the dumped fixed point:

19 PORTALVERTS
24 LAYERED_AABBTREES
26 PORTALS
42 VERTEX_LAYER_DATA

These chunks are also omitted by the linker_pc-origin dump, so this is not an OAT-only omission.

Notes on the omitted chunks:

19 PORTALVERTS / 26 PORTALS
    Raw portal/PVS-style world portal data emitted by cod4map.exe.
    The linked runtime GfxWorld does not preserve enough source portal lump
    data to dump these byte-for-byte, and the canonical fixed-point BSP omits
    them.

24 LAYERED_AABBTREES
    Source layered AABB tree data. The fixed-point BSP uses the simple
    AABB-tree path for this map, so the layered tree lump is not emitted.

42 VERTEX_LAYER_DATA
    Extra payload for the layered geometry path. For the simple/unlayered
    path, linker_pc uses a small dummy runtime vertex-layer buffer instead of
    preserving the raw lump.

Resources

• https://github.com/SwagSoftware/KisakCOD - CM_LoadMapFromBsp
• https://github.com/SE2Dev/D3DBSP_Converter
• https://github.com/7894752/cod2map
• https://github.com/riicchhaarrd/iw3_linker
• linker_pc.exe
• CoD4Radiant.exe
• cod4map.exe

AI Disclaimer

AI assistance was heavily used during reverse engineering and implementation, especially for the loader. The output has been validated against linker_pc and runtime/Radiant loading tests described above.

[Nikolai2390]

1. After testing the IW3 D3DBSP export, i noticed that "dyn_model" entities are currently missing from the exported d3dbsp files.

2. As a separate feature suggestion, it could be useful to support map specific entity overrides. For example, if a .map file exists in {mapname}/maps/override/, its entities could be used in place of the dumped entities when generating the D3DBSP.

[michaeloliverx]

1. After testing the IW3 D3DBSP export, i noticed that "dyn_model" entities are currently missing from the exported d3dbsp files.

Missed those, added now :)

[michaeloliverx]

Relates to #532