You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[hls4ml](https://fastmachinelearning.org/hls4ml/) is a Python package developed by the [Fast Machine Learning Lab](https://fastmachinelearning.org/). It's primary purpose is to create firmware implementations of machine learning (ML) models to be run on FPGAs. The package interfaces with a high-level synthesis (HLS) backend (i.e. Xilinx Vivado HLS) to transpile the ML model into hardware description language (HDL). The primary hls4ml documentation, including API reference pages, is located [here](https://fastmachinelearning.org/hls4ml/).
# Print full list of example models if you want to explore more
41
+
hls4ml.utils.fetch_example_list()
42
+
43
+
# Use Vivado HLS to synthesize the model
44
+
# This might take several minutes
45
+
hls_model.build()
46
+
47
+
# Print out the report if you want
48
+
hls4ml.report.read_vivado_report('my-hls-test')
49
+
50
+
```
51
+
52
+
### More resources
53
+
14
54
The main hls4ml tutorial code is kept on [GitHub](https://github.com/fastmachinelearning/hls4ml-tutorial). Users are welcome to walk through the notebooks at their own pace. There is also a set of slides linked to the [README](https://github.com/fastmachinelearning/hls4ml-tutorial/blob/master/README.md).
15
55
16
56
That said, there have been several cases where the hls4ml developers have given live demonstrations and tutorials. Below is a non-exhaustive list of tutorials given in the last few years (newest on top).
QONNX (Quantized ONNX) introduces three new custom operators -- [`Quant`](docs/qonnx-custom-ops/quant_op.md), [`BipolarQuant`](docs/qonnx-custom-ops/bipolar_quant_op.md), and [`Trunc`](docs/qonnx-custom-ops/trunc_op.md) -- in order to represent arbitrary-precision uniform quantization in [ONNX](onnx.md). This enables:
12
+
13
+
* Representation of binary, ternary, 3-bit, 4-bit, 6-bit or any other quantization.
14
+
* Quantization is an operator itself, and can be applied to any parameter or layer input.
15
+
* Flexible choices for scaling factor and zero-point granularity.
16
+
* Quantized values are carried using standard `float` datatypes to remain [ONNX](onnx.md) protobuf-compatible.
17
+
18
+
This repository contains a set of Python utilities to work with QONNX models, including but not limited to:
19
+
20
+
* executing QONNX models for (slow) functional verification
21
+
* shape inference, constant folding and other basic optimizations
22
+
* summarizing the inference cost of a QONNX model in terms of mixed-precision MACs, parameter and activation volume
23
+
* Python infrastructure for writing transformations and defining executable, shape-inferencable custom ops
24
+
* (experimental) data layout conversion from standard ONNX NCHW to custom QONNX NHWC ops
Copy file name to clipboardExpand all lines: content/resources/fpga_resources/index.md
+1-169Lines changed: 1 addition & 169 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,172 +10,4 @@ To programm an FPGA one has two main options. The first one is to use a Hardware
10
10
11
11
When it comes to vendors there are two big onces: Xilinx (part of AMD) and Altera (part of Intel). Both vendors provide there own tooling to simulate, synthesis and debug the design. Xilinx FPGAs are used for CMS and they have Vivado for design, simulation, synthesis, and debugging tasks and Vitis for software development for Xilinx FPGAs and SoCs. Intel FPGAs are programmed using Quartus Prime. For HLS tools they come with Vivado HLS (Xilinx) and HLS Compiler (Intel).
12
12
13
-
To simplify the pipeline from a trained model to an implementation on the FPGA CMS is supporting different tools, which will be explained in the flowing in more detail.
hls4ml is a Python library designed to bring machine learning inference to FPGAs by leveraging high-level synthesis (HLS). The idea is to convert trained machine learning models from popular open-source frameworks (such as PyTorch, Tensorflow, Keras etc.) into FPGA-compatible firmware, tailored to specific needs.
24
-
25
-
As the project is actively evolving the hls4ml team is always looking for people trying there tools.
QONNX (Quantized ONNX) introduces three new custom operators -- [`Quant`](docs/qonnx-custom-ops/quant_op.md), [`BipolarQuant`](docs/qonnx-custom-ops/bipolar_quant_op.md), and [`Trunc`](docs/qonnx-custom-ops/trunc_op.md) -- in order to represent arbitrary-precision uniform quantization in ONNX. This enables:
121
-
122
-
* Representation of binary, ternary, 3-bit, 4-bit, 6-bit or any other quantization.
123
-
* Quantization is an operator itself, and can be applied to any parameter or layer input.
124
-
* Flexible choices for scaling factor and zero-point granularity.
125
-
* Quantized values are carried using standard `float` datatypes to remain ONNX protobuf-compatible.
126
-
127
-
This repository contains a set of Python utilities to work with QONNX models, including but not limited to:
128
-
129
-
* executing QONNX models for (slow) functional verification
130
-
* shape inference, constant folding and other basic optimizations
131
-
* summarizing the inference cost of a QONNX model in terms of mixed-precision MACs, parameter and activation volume
132
-
* Python infrastructure for writing transformations and defining executable, shape-inferencable custom ops
133
-
* (experimental) data layout conversion from standard ONNX NCHW to custom QONNX NHWC ops
[High Granularity Quantization (HGQ)](https://github.com/calad0i/HGQ/) is a library that performs gradient-based automatic bitwidth optimization and quantization-aware training algorithm for neural networks to be deployed on FPGAs. By laveraging gradients, it allows for bitwidth optimization at arbitrary granularity, up to per-weight and per-activation level.
Conversion of models made with HGQ library is fully supported. The HGQ models are first converted to proxy model format, which can then be parsed by hls4ml bit-accurately. Below is an example of how to create a model with HGQ and convert it to hls4ml model.
from hls4ml.converters import convert_from_keras_model
169
-
170
-
trace_minmax(model, x_train, cover_factor=1.0)
171
-
proxy = to_proxy_model(model, aggressive=True)
172
-
173
-
model_hls = convert_from_keras_model(
174
-
proxy,
175
-
backend='vivado',
176
-
output_dir=...,
177
-
part=...
178
-
)
179
-
```
180
-
181
-
An interactive example of HGQ can be found in the [kaggle notebook](https://www.kaggle.com/code/calad0i/small-jet-tagger-with-hgq-1). Full documentation can be found at [calad0i.github.io/HGQ](https://calad0i.github.io/HGQ/>).
13
+
To simplify the pipeline from a trained model to an implementation on the FPGA CMS is supporting different tools, which are explained in the inference section ([hls4ml](../../inference/hls4ml.md), [conifer](../../inference/conifer.md), [qonnx](../../inference/qonnx.md)). Furthermore, tools for quantize aware training are used (QKeras, [HGQ](../../training/HGQ.md)).
Text taken and adopted from the HGQ [README.md](https://github.com/calad0i/HGQ/blob/master/README.md).
8
+
9
+
[High Granularity Quantization (HGQ)](https://github.com/calad0i/HGQ/) is a library that performs gradient-based automatic bitwidth optimization and quantization-aware training algorithm for neural networks to be deployed on FPGAs. By laveraging gradients, it allows for bitwidth optimization at arbitrary granularity, up to per-weight and per-activation level.
Conversion of models made with HGQ library is fully supported. The HGQ models are first converted to proxy model format, which can then be parsed by hls4ml bit-accurately. Below is an example of how to create a model with HGQ and convert it to hls4ml model.
from hls4ml.converters import convert_from_keras_model
36
+
37
+
trace_minmax(model, x_train, cover_factor=1.0)
38
+
proxy = to_proxy_model(model, aggressive=True)
39
+
40
+
model_hls = convert_from_keras_model(
41
+
proxy,
42
+
backend='vivado',
43
+
output_dir=...,
44
+
part=...
45
+
)
46
+
```
47
+
48
+
An interactive example of HGQ can be found in the [kaggle notebook](https://www.kaggle.com/code/calad0i/small-jet-tagger-with-hgq-1). Full documentation can be found at [calad0i.github.io/HGQ](https://calad0i.github.io/HGQ/>).
0 commit comments