Coyote accelerator backend by bo3z · Pull Request #1347 · fastmachinelearning/hls4ml

bo3z · 2025-07-28T10:18:28Z

Description

📝 This PR introduces a new accelerator backend, CoyoteAccelerator, which leverages the open-source Coyote shell for deploying models on a PCI-attached FPGA.

Generally, Coyote offers several advantages, when compared to some other shells, including:

Networking support, so the backend can easily be extended to support distributed inference. Also interesting for in-network ML.
GPU - FPGA integration, so models can be executed on a combination of hardware
Dynamic reconfiguration, which could allow run-time reconfiguration of models
Multi-tenancy, so multiple models could be deployed concurrently.

The backend is briefly described in Section 9.7 of the paper: https://arxiv.org/pdf/2504.21538.

Type of change

New feature (non-breaking change which adds functionality)
A new research paper code implementation

Tests

This backend was compared agains a modified* version of the VivadoAccelerator backend: the backend was modified to run HLS synthesis with Vitis instead of Vivado (also using Vitis templates and optimizers), while the rest of the backend infrastructure (drivers, data movers remained the same since they also work in newer version of Vivado). Results are attached below - clearly indicating an advantage in Coyote, for two reasons (1) optimised data movement, bypassing card memory and (2) optimised host-side library (Python, C++).

In principle, the correct test would be to compare against VitisAccelerator (#991), but only after the io_parallel issues are resolved. However, the expectation is that the result will stay mostly the same, sine the underlying platform requires a data copy between host and card memory.

Will add some more results, also for io-stream CNN, and comparisons to VitisAccelerator.

Figure above: comparison of CoyoteAccelerator with modified Vivado Accelerator for the UNSW-NB15 dataset in io_parallel.

Checklist

I have read the guidelines for contributing.
I have commented my code, particularly in hard-to-understand areas.
I have made corresponding changes to the documentation.
My changes generate no new warnings.
I have installed and run pre-commit on the files I edited or added.
I have added tests that prove my fix is effective or that my feature works.

…-backend (fastmachinelearning#1347) Merge branch 'init_interval_fix_zeropad_maxpooling' into coyote-accelerator-and-pooling

JanFSchulte

A few misc comments based on trying to run the CoyoteAccelerator for a dummy model. Right now, i am stuck with a python import error:

which is puzzling because I do have jinja2 installed in my environment and the same import works fine in an interactive python session.

Also, can you fix the pre-commit issues?

JanFSchulte · 2026-01-21T20:12:45Z

+        )
+
+        if not os.path.exists(f'{model.config.get_output_dir()}/build/{model.config.get_project_name()}_cyt_hw'):
+            os.mkdir(f'{model.config.get_output_dir()}/build/{model.config.get_project_name()}_cyt_hw')


I think this needs to use os.makedirs() because the build folder doesn't exist already.

Will change.

vloncar

Looks good. I have mostly questions for better understanding and minor nitpicks that I don't feel are crucial, rather optional.

vloncar · 2026-01-23T15:15:19Z

+        if len(X.shape) == 1:
+            X = np.array([X])
+        if not (isinstance(X.dtype, float) or isinstance(X.dtype, np.float32)):
+            logging.warning('CoyoteOverlay only supports (for now) floating-point inputs; casting input data to float')


Have we completely switched to logging module or we still use warnings. Do these two play along nicely?

I am not sure.

I see three instances in the code:

In some cases the function warn(...) is used.

In some other cases, a normal print is used, e.g., print('WARNING:...')

Some code (mostly mine), uses logging.warning.

I've never had issues with logging.warning but happy to change as needed.

vloncar · 2026-01-23T15:30:19Z

+#include "defines.h"
+#include "host_libs.hpp"
+
+#include <boost/program_options.hpp>


Does this require us to have Boost library installed on the host or that comes with Coyote?

Yes, it's required, and, no it's not installed with Coyote.

One can argue that Boost shouldn't be needed for something as simple as CLI parsing, but Coyote relies internally on Boost for some other functionality (inter-process mutexes).

Currently, Coyote's compilation flow (i.e. CMake) will check if Boost is installed and throw an error if not. To me this sounds okay, as Boost is a fairly straightforward library to install.

JanFSchulte · 2026-02-12T14:57:22Z

Just confirming that installing jinja2 system-wide solved my issues and I can now successfully run synthesis and create bit files using this back end.

bo3z · 2026-02-13T17:56:46Z

That's good to now. At some point, I should update Coyote to make sure that jina isn't required system-wide.

If it also works for inference, that's great.

FIFO Optimization fix and add aclk_freq as parameter

CoyoteAccelerator: fix dtype check in CoyoteOverlay

bo3z · 2026-07-05T20:15:53Z

Should be mostly ready to merge now. Think all the comments were addressed.

JanFSchulte · 2026-07-08T18:19:21Z

Still need to fix the pre-commit checks ;) The "line too long" ones can't be automatically fixed.

bo3z added 8 commits July 25, 2025 12:36

Initialize CoyoteAccelerator Backend

72aa3ea

CoyoteAccelerator backend hardware modules

896665b

Init Coyote submodule

4be45d1

CoyoteAccelerator backend software modules

ecde593

CoyoteAccelerator backend writer + build scripts

be28016

CoyoteAccelerator backend Python Overlay for neural network inference

2c600a6

Minor fixes and documentation updates

3055a0b

Remove unnecessary sleep when polling which reduces performance

9554271

vloncar reviewed Aug 5, 2025

View reviewed changes

Comment thread hls4ml/templates/coyote/host_libs.cpp

JanFSchulte added this to the v1.3.0 milestone Nov 5, 2025

Merge branch 'main' into coyote-accelerator

e0714b7

JanFSchulte reviewed Jan 21, 2026

View reviewed changes

vloncar reviewed Jan 23, 2026

View reviewed changes

bo3z added the feature New hls4ml feature label Jan 30, 2026

bo3z mentioned this pull request Feb 9, 2026

Vivado HLS generating multiple output streams #417

Closed

LordGash and others added 12 commits February 24, 2026 14:41

FIFO Optimization fix

d58bf73

Update coyote_accelerator_writer.py

789e093

Update coyote_accelerator_backend.py

9a6b0fe

Update FIFO depth optimization

d35820f

Merge pull request #6 from LordGash/coyote

d4a6a2f

FIFO Optimization fix and add aclk_freq as parameter

CoyoteAccelerator: fix overlay floating dtype check

14c0ecf

Merge pull request #8 from lorenzo-as/fix/coyote-overlay-dtype-check

f73f399

CoyoteAccelerator: fix dtype check in CoyoteOverlay

Rename CoyoteAccelerator to Coyote & minor doc fixes

77be539

Address PR comments

9630158

Merge branch 'main' into coyote-accelerator

657cef2

Update submodule

fea86b0

Correctly parse HLS clock uncertainty

770d3dc

Fix SW compile errors for new version of Coyote

2dd6984

Uh oh!

Conversation

bo3z commented Jul 28, 2025

Description

Type of change

Tests

Checklist

Uh oh!

Uh oh!

JanFSchulte left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

JanFSchulte Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

bo3z Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vloncar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vloncar Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

bo3z Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

vloncar Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

bo3z Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JanFSchulte commented Feb 12, 2026

Uh oh!

bo3z commented Feb 13, 2026

Uh oh!

bo3z commented Jul 5, 2026

Uh oh!

JanFSchulte commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

JanFSchulte left a comment •

edited

Loading