XLS backend

[vasdommes]

Description

This PR adds XLS backend. It is based on PR #1343, with most of the code rewritten and new features added.

Google XLS is an open-source (Apache 2) High Level Syntesis toolchain that produces an RTL (Verilog or SystemVerilog) design from a high-level description (DSLX or C++).

Adding XLS as a new hls4ml backend allows to generate RTL without vendor-specific dependencies and benefit from the developments that XLS brings to HLS field.

XLS workflow

XLS backend performs the following transformations:

• write(): hls4ml representation -> DSLX project
• compile(): DSLX -> XLS IR -> Optimized XLS IR
• build(): Optimized XLS IR -> (System)Verilog -> IP

DSLX -> IR -> (System)Verilog conversion is done by XLS.
IP is generated by Vivado. One can choose another vendor and generate IP from Verilog file manually.

XLS features

XLS backend supports the following layers: Input, ApplyAlpha, BatchNormalization, Dense, Conv1D, DepthwiseConv1D, Conv2D, DepthwiseConv2D, Pooling1D, Pooling2D, GlobalPooling1D, GlobalPooling2D, Merge, Concatenate, Dot, Activation, HardActivation, ParametrizedActivation, PReLU, Reshape, Softmax, Transpose, TernaryTanh.

You can override default codegen options as follows:

config = hls4ml.utils.config_from_keras_model(model)
# This sets hls_model.config['XLSCodegenFlags']
hls_model = hls4ml.converters.convert_from_keras_model(
    model, hls_config=config, backend='XLS',
    xls_codegen_flags={'delay_model': 'asap7', 'generator': 'pipeline', 'use_system_verilog': False}
)

DSLX standard library has only signed FixedPoint type (similar to ap_fixed). Thus, unsigned types are not supported.

Currently, XLS backend implements only IOType: io_parallel. Strategy is ignored.
All operations are fully unrolled.

io_stream could be implemented via DSLX procs. @calad0i and I are going to work on that after finishing this PR.

Other changes

I made some minor changes in non-XLS code:

• Fixed test_softmax.py does not test argmax and latency implementations; latency fails #1443, since it was needed to test all softmax implementations in XLS.
• Added hook to ModelGraph to call custom backend.get_top_function(). This is needed for XLS because it uses optimized XLS IR file instead of .so library generated by other backends.
• Updated docs/ir/attributes.rst. Aside from adding XLS, this commit some other missing layers and attributes.

Dependencies

XLS backend uses xls-python to access XLS API. It is enabled by dependency group xls:

pip install hls4ml[xls]

xls-python comes with batteries (libxls.so and DSLX standard library) included, no separate XLS installation is required.
The code has been tested for the version xls-python=0.1.9875.

Known issues

XLS doesn't work with Dense layer imported from PyTorch Linear layer because of shape mismatch: PyTorch stores Linear weights as (out_features, in_features), while hls4ml Dense layers use the Keras-style layout (in_features, out_features).

Repro: add XLS backend to test_pytorch_api.py/test_squeeze and run the test.

Note that the weights in this test are constant, and other backends flatten them without checking shape.
So, it is unclear whether they handle this situation correctly or not.

Type of change

• Documentation update
• New feature (non-breaking change which adds functionality)

Tests

📝 Please describe the tests that you ran to verify your changes.

• Provide instructions so we can reproduce.
• Please also list any relevant details for your test configuration.

XLS has been added to the following tests:
test_activations.py, test_auto_precision.py, test_binary_cnn.py, test_causalpadding.py, test_depthconv1d.py, test_depthconv2d.py, test_keras_api.py, test_keras_v3_api.py, test_merge.py, test_multi_dense.py, test_pointwiseconv.py, test_pooling.py, test_pytorch_api.py, test_reshape.py, test_sepconv1d.py, test_sepconv2d.py, test_softmax.py.

Test Configuration

Add xls dependency, e.g.

pip install .[da,testing,testing-keras2,sr,optimization,xls]"
# or: pip install .[da,testing,testing-keras3,sr,xls]"

and run tests, e.g.:

pytest test/pytest --randomly-dont-reset-seed -k XLS

Notes on performance

Some test cases are very slow for XLS (e.g. ~30 minutes vs ~10 seconds on other backends).
This happens because XLS generates (in model.compile()) and uses (in model.predict()) an optimized XLS IR code, where all loops are fully unrolled. The resulting file can be huge and thus slow for the likes of Conv2D.

During development, I made test faster by reducing dimensions in some tests.
For example, in test_keras_api.py/test_conv2d I replaced

input_shape = (28, 28, 3)
filters=32

with

input_shape = (14, 14, 3)
filters=8

I haven't pushed such changes, but that could be one of the ways of speeding things up.

UPD: I reduced XLS dimensions in many tests, see 8578a62 and 75a04fb.

Checklist

• I have read the guidelines for contributing.
• I have commented my code, particularly in hard-to-understand areas.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings.
• I have installed and run pre-commit on the files I edited or added.
• I have added tests that prove my fix is effective or that my feature works.

[vasdommes]

I reduced XLS dimensions in many tests to make them fast enough, but the test https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75768174 failed due to 2hr timeout.

$ pytest $PYTESTFILE -rA --cov-report xml --cov-report term --cov=hls4ml --junitxml=report.xml --randomly-seed=42 --randomly-dont-reorganize --randomly-dont-reset-seed
============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.0.2, pluggy-1.6.0
pytest-randomly: reseed with 42
Using --randomly-seed=42
rootdir: /builds/fastmachinelearning/hls4ml
configfile: pyproject.toml
plugins: anyio-4.12.0, cov-7.0.0, randomly-4.0.1
collected 227 items / 1 skipped
test_einsum_dense.py ..................                                  [  7%]
test_keras_v3_api.py ...........s....................................... [ 30%]
.Terminated
WARNING: step_script could not run to completion because the timeout was exceeded. For more control over job and script timeouts see: https://docs.gitlab.com/ee/ci/runners/configure_runners.html#set-script-and-after_script-timeouts
ERROR: Job failed: execution took longer than 2h0m0s seconds

That's surprising - on my laptop, test_einsum_dense finishes in 3 min and test_keras_v3_api - in 16 min (including 4 min for XLS).

[calad0i]

This timeout looks like CI issue, not something real in the code.

[vloncar]

If it is not doing synthesis (and it seems that it skips that path), what does it do that takes a long time?

[calad0i]

If it is not doing synthesis (and it seems that it skips that path), what does it do that takes a long time?

I feel like that the slowness is not due to the tests themselves, more likely the runner itself stalls on random tests one in a while...

https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75680057
https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75627913
https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75482738
https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75329237
https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75320437

[vloncar]

Hmm, seems that all tests always run only on -05, other nodes are never used. Gitlab reports them as "stale". Turns out they are not configured right, always reporting as -05, but only the real -05 is used. I've reconfigured 06 and 08 to be separate, they are in the pool now. Sadly 07 doesn't respond, will need to ping an openlab admin to configure that one. Hopefully then it will be a bit less crowded and tests will run faster

[Copilot]

Pull request overview

Adds an experimental Google XLS backend to hls4ml, enabling generation of (System)Verilog from hls4ml models via DSLX → XLS IR → optimized IR → codegen, and integrates it into docs, packaging extras, CI, and the pytest backend matrix.

Changes:

• Introduces new XLSBackend and XLSWriter, plus XLS-specific optimizer passes and IR/JIT-based simulation support.
• Adds an XLS template firmware tree (templates/xls/**) implementing core nnet utilities in DSLX and Vivado build scripts.
• Extends tests/CI and documentation to include the new backend and optional dependency group (hls4ml[xls]).

Reviewed changes

Copilot reviewed 52 out of 54 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/pytest/test_softmax.py	Adds XLS to backend matrix; updates softmax implementation selection and data generation
test/pytest/test_sepconv2d.py	Adds XLS case and reduces dimensions for XLS runtime
test/pytest/test_sepconv1d.py	Adds XLS case
test/pytest/test_reshape.py	Adds XLS backend + io_parallel-only skip
test/pytest/test_pytorch_api.py	Adds XLS backend + io_parallel-only skips (notes failing squeeze case)
test/pytest/test_pooling.py	Adds XLS backend; reduces dimensions for XLS runtime
test/pytest/test_pointwiseconv.py	Adds XLS backend; reduces dimensions for XLS runtime
test/pytest/test_multi_dense.py	Adds XLS backend + io_parallel-only skip
test/pytest/test_merge.py	Adds XLS backend + io_parallel-only skips across merge ops
test/pytest/test_keras_v3_api.py	Adds XLS backend; reduces dimensions for XLS runtime
test/pytest/test_keras_api.py	Adds XLS backend; reduces dimensions for XLS runtime
test/pytest/test_depthconv2d.py	Adds XLS case
test/pytest/test_depthconv1d.py	Adds XLS case
test/pytest/test_causalpadding.py	Adds XLS backend + io_parallel-only skip
test/pytest/test_binary_cnn.py	Adds XLS case and reduces dimensions for XLS runtime
test/pytest/test_auto_precision.py	Adds XLS backend; backend-dependent shapes/fixtures to reduce XLS runtime
test/pytest/test_activations.py	Adds XLS backend + io_parallel-only skip; XLS-specific LUT sizing tweak
test/pytest/synthesis_helpers.py	Skips synthesis for XLS backend
test/pytest/ci-template.yml	Adds xls extra deps to CI environments
pyproject.toml	Adds optional-dependencies.xls (xls-python)
hls4ml/writer/xls_writer.py	New writer to emit DSLX project and copy XLS templates
hls4ml/writer/init.py	Registers XLSWriter
hls4ml/templates/xls/firmware/nnet_utils/transpose.x	New DSLX transpose utilities
hls4ml/templates/xls/firmware/nnet_utils/reshape.x	New DSLX reshape utilities
hls4ml/templates/xls/firmware/nnet_utils/merge.x	New DSLX merge/concat/dot utilities
hls4ml/templates/xls/firmware/nnet_utils/lookup_table.x	New DSLX lookup table implementation
hls4ml/templates/xls/firmware/nnet_utils/depthwise_conv.x	New DSLX depthwise conv utilities
hls4ml/templates/xls/firmware/nnet_utils/dense.x	New DSLX dense utilities
hls4ml/templates/xls/firmware/nnet_utils/data_format.x	New DSLX data-format helpers
hls4ml/templates/xls/firmware/nnet_utils/conv1d.x	New DSLX conv1d utilities
hls4ml/templates/xls/firmware/nnet_utils/batchnorm.x	New DSLX batchnorm utilities
hls4ml/templates/xls/firmware/nnet_utils/activations.x	New DSLX activations/softmax utilities
hls4ml/templates/xls/firmware/myproject.x	New top-level DSLX template
hls4ml/templates/xls/firmware/layer.x	New per-layer DSLX template
hls4ml/templates/xls/constraints.xdc	New Vivado constraints template
hls4ml/templates/xls/build_prj.tcl	New Vivado build script for XLS-generated SV
hls4ml/report/xls_report.py	New Vivado utilization report parser for XLS builds
hls4ml/report/init.py	Exposes parse_xls_report
hls4ml/model/graph.py	Adds backend hook for custom top-function retrieval
hls4ml/backends/xls/xls_types.py	New XLS/DSLX type/value helpers and LUT builder helpers
hls4ml/backends/xls/xls_backend.py	New XLS backend: flows, DSLX→IR compile, JIT predict hook, SV codegen + Vivado build
hls4ml/backends/xls/passes/build_tables.py	New XLS optimizer pass to generate LUTs
hls4ml/backends/xls/passes/build_attr.py	New XLS optimizer pass to build writer-facing layer attrs
hls4ml/backends/xls/init.py	Initializes XLS backend package
hls4ml/backends/init.py	Registers XLSBackend
docs/intro/status.rst	Lists XLS as experimental backend in support matrix
docs/intro/setup.rst	Documents pip install hls4ml[xls]
docs/index.rst	Adds XLS backend docs page to toctree
docs/backend/xls.rst	New XLS backend documentation page
.gitignore	Ignores .idea/

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[calad0i]

Looks like bit exactness issue when data is defined as unsigned type? Consider warn instead of raise since it is a bit-exactness compromise instead of something unrecoverable.

[vasdommes]

Currently we don't have a warning/error for constants like weights and biases (allow_unsigned = True).
But we throw an error (which should be a warning, as you suggest) e.g. when input/output variable type is unsigned.

@jmitrevs noted that we could even skip all warnings and simply mention in the docs that all unsigned types are converted to signed.
In that case, I could simplify the code a bit by removing the allow_unsigned parameter (21 usage) .

What do you think? Are such warnings helpful?

[calad0i]

Why do you need a new precision type here? The ${BACKEND}WhateverType in other backends are converted by metaclass to add write method, but this one seems to be a plain thin wrapper?

[vasdommes]

The class XLSFixedPointType and its __str__() method plays a role similar to ACFixedPrecisionDefinition and its cpp_definition() method.

I could try and refactor the code to make it similar to *Definition / *Converter / TransformTypes logic used in other backends.