Skip to content

BENCH-001: Number format benchmarks #479

@gHashTag

Description

@gHashTag

Goal

Produce systematic, reproducible benchmarks comparing Trinity ternary number format against standard formats (f16, bf16).

Checklist

  • Error curves: Trinity ternary vs f16 vs bf16 (quantization MSE)
  • Dynamic range table: min/max representable values
  • LUT cost: ternary add/mul vs f16 add/mul on XC7A100T
  • Inference benchmark: same model, same weights, f16 vs ternary
  • PPL comparison: f16 baseline vs ternary HSLM
  • Power/throughput on FPGA

Context

Response to critique: format claims need empirical validation. 142 φ-formulas for physical constants are verified (79/79 tests), but connection between φ and optimal quantization spacing is unproven. Benchmarks > formulas.

Deliverables

  1. Plots: quantization MSE vs value range for ternary/f16/bf16
  2. Table: min/max representable, subnormal denormals, special values
  3. Synthesis report: LUT/FF counts for add/mul in both formats
  4. Inference speed: tokens/sec or images/sec
  5. PPL on validation set: same model, different number formats
  6. Power measurement: W at 100MHz, throughput: ops/sec

Metadata

Metadata

Assignees

No one assigned

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions