Ahmed Allam, Youssef Mansour, and Mohamed Shalan
Department of Computer Science and Engineering, The American University in Cairo
Cairo, Egypt
{ ahmedeallam, youssef-mansour, mshalan } @aucegypt.edu
Abstract -Large Language Models (LLMs) have demonstrated remarkable capabilities in Register Transfer Level (RTL) design, enabling high-quality code generation from natural language descriptions. However, LLMs alone face significant limitations in real-world hardware design workflows, including the inability to execute code, lack of debugging capabilities, and absence of long-term memory. To address these challenges, we present ASIC-Agent, an autonomous system designed specifically for digital ASIC design tasks. ASIC-Agent enhances base LLMs with a multi-agent architecture incorporating specialized subagents for RTL generation, verification, OpenLane hardening, and Caravel chip integration, all operating within a comprehensive sandbox environment with access to essential hardware design tools. The system leverages a vector database containing documentation, API references, error knowledge, and curated insights from the open-source silicon community. To evaluate ASIC-Agent's performance, we introduce ASIC-Agent-Bench, the first benchmark specifically designed to assess agentic systems in hardware design tasks. We evaluate ASIC-Agent with various base LLMs, providing quantitative comparisons and qualitative insights into agent behavior across different design scenarios. Our results demonstrate that ASIC-Agent, when powered by Claude 4 Sonnet, successfully automates a broad range of ASIC design tasks spanning varying levels of complexity, showing the potential of significantly accelerating the ASIC design workflow. Our work is open-source and publicly available on Github 1 .
Index Terms -LLM-Aided Hardware Design, ASIC Design Automation, Agent Systems, Benchmarking LLM Agents
Digital Application-Specific Integrated Circuits (ASICs) are pivotal to the functionality of modern electronic systems, delivering tailored performance and efficiency across diverse applications, from consumer electronics to high-performance computing. The design of digital ASICs, however, remains a complex and resource-intensive endeavor. Stages such as Register Transfer Level (RTL) modeling, verification, synthesis, and physical design demand significant manual effort, with RTL modeling and verification being particularly laborintensive and prone to errors. As hardware complexity escalates, the need for automation in these processes becomes increasingly critical to reduce development time, minimize errors, and accelerate time-to-market.
The emergence of Large Language Models (LLMs) has opened new avenues for automation in code generation, driven by advancements in artificial intelligence. LLMs have demonstrated remarkable success in producing high-quality code for
1 https://github.com/AUCOHL/ASIC-Agent-Bench
software languages like Python, Java, and C++ from natural language descriptions [1]-[3], hinting at their potential to transform hardware design workflows. In the context of digital ASIC design, LLMs have shown promise in generating RTL code, such as Verilog, thereby alleviating some of the burdens of manual design tasks [4], [5]. However, applying LLMs directly to hardware design reveals significant limitations: they cannot execute code, lack real-time debugging capabilities, and do not possess long-term memory-features essential for navigating the intricacies of ASIC development.
Notably, progress in software engineering agents, such as SWE-agent [6] and OpenHands [7], has demonstrated the potential of autonomous systems in automating complex coding tasks. Inspired by these developments, we present ASICAgent, an autonomous multi-agent system tailored for digital ASIC design, addressing the unique challenges of hardware development. ASIC-Agent builds upon the generative strengths of LLMs by integrating a multi-agent architecture featuring specialized sub-agents dedicated to key tasks: RTL generation, verification, OpenLane hardening, and Caravel chip integration. These sub-agents operate within a robust sandbox environment equipped with critical open-source hardware design tools, including iverilog, yosys, OpenLane, and more, enabling them to interact dynamically with the design process. Additionally, ASIC-Agent leverages a vector database containing documentation, API references, a curated error knowledge base, and insights from the open-source silicon community, enhancing its ability to tackle complex design challenges.
While there are several benchmarks for RTL and hardware, such as VerilogEval [8] and RTLLM [9], they are designed for simpler, non-agentic tasks, typically focusing on generating single-file or standalone Verilog modules. These benchmarks fail to capture the complexity of real-world ASIC design workflows, which often involve multi-file contexts, dynamic tool interactions, and iterative debugging-capabilities inherent to agentic systems. To address this gap, we introduce ASIC-Agent-Bench, the first benchmark specifically tailored to evaluate agentic systems in hardware design tasks. ASICAgent-Bench comprises a diverse set of real-world, openended design scenarios, offering a standardized and rigorous framework to assess the performance of agentic systems comprehensively.
In this paper, we make two primary contributions. First, we introduce ASIC-Agent, an innovative framework that in- tegrates large language models (LLMs) with a multi-agent architecture to overcome the constraints inherent to using standalone LLMs in hardware design. Second, we present ASIC-Agent-Bench, a critical benchmarking tool that enables rigorous quantitative evaluation and fosters further advancements in agent-based systems for ASIC design. We conduct extensive evaluations of ASIC-Agent across multiple base LLMs, offering both quantitative performance comparisons and qualitative analyses of agent behaviors in varied design scenarios.
Autonomous agents have grown increasingly important in software engineering, utilizing large language models (LLMs) to automate code generation, debugging, and issue resolution. Systems like SWE-agent [6] and OpenHands [7] interpret natural language prompts to produce functional code in simulated development environments, effectively handling code repositories and tests across multiple files. While software agents have advanced significantly, hardware agents remain in earlier stages of development, particularly for Register Transfer Level (RTL) design in Verilog. Current approaches include standalone RTL-focused LLMs like ChipNeMo [10] and RTLCoder [5], which generate Verilog but lack execution capabilities and integration with broader design flows. Hardware agent systems such as VerilogCoder [11] employ task planning and AST-based debugging but are limited to modulelevel generation without ASIC synthesis capabilities. Similarly, MAGE [12] uses multiple agents for RTL and testbench generation with recursive debugging, yet it excludes critical physical design stages. Existing approaches-both standalone LLMs and specialized hardware agents-cannot manage the complete ASIC development process, highlighting the need for a more comprehensive system to streamline complex ASIC workflows.
ASIC-Agent is a specialized autonomous system for digital ASIC design, built upon the foundation of OpenHands (formerly known as OpenDevin) [7] and the CodeAct system [13]. While leveraging OpenHands' proven infrastructure for autonomous software development, ASIC-Agent extends this framework specifically for hardware design workflows, transforming it into a platform capable of handling the unique challenges of ASIC design.
ASIC-Agent employs a multi-agent architecture designed to decompose the complex ASIC design workflow into specialized tasks while maintaining cohesive progress toward design objectives. This architecture consists of a central main agent and specialized sub-agents, each with distinct capabilities, specialized tools, and knowledge bases tailored to their specific roles across the ASIC design process. Each agent is equipped with a system prompt template optimized for its specific task domain, ensuring focused and effective performance.
-
Main Agent: The Main LLM Agent serves as the central agent and primary interface for the entire system. This agent directly handles RTL generation, synthesizing Verilog code from natural language specifications with appropriate module interfaces, signal declarations, and behavioral logic. The agent performs linting and static analysis on generated code to identify potential syntax errors, design rule violations, and suboptimal coding patterns before verification. It maintains a global state of the project including design constraints, specifications, and progress across all stages of the workflow. The Main Agent employs specialized tools for RTL development including linters such as Verilator and iverilog, ensuring code quality and adherence to best practices. Through in-context learning and prompted reasoning, the Main Agent adaptively plans the design trajectory, creating a comprehensive RTL implementation before initiating verification, hardening, and integration processes when appropriate.
-
Verification Agent: The Verification Agent specializes in comprehensive functional validation of RTL designs, ensuring the correctness, performance, and robustness of digital hardware implementations. This agent generates test environments, conducts simulations, and analyzes results to validate designs against their specifications. While supporting multiple verification methodologies, we found Python-based testbench development with cocotb offers significant advantages over traditional HDL approaches. This choice leverages LLMs' superior Python proficiency compared to hardware description languages. The cocotb framework enables sophisticated test scenarios with complex stimulus generation and reference model comparison [14]. This Python foundation allows the agent to implement advanced verification constructs like matrix multiplication and neural network operations that would be cumbersome in HDLs. The agent interfaces with simulation tools such as Icarus Verilog and Verilator, collecting detailed waveform data. Upon detecting verification failures, the agent performs root-cause analysis and provides actionable feedback with suggested fixes.
-
Hardening Agent: The Hardening Agent specializes in transforming functionally verified RTL designs into physical layout implementations through the open-source OpenLane 2 ASIC implementation flow [15]. This agent's function is generating appropriate OpenLane configuration files with parameters tailored to the specific design requirements. It selects and configures critical flow variables based on design analysis and performance targets. The agent executes the OpenLane flow and monitors its progress. A key innovation in ASICAgent is the integration of a specialized OpenLane debugging tool that the agent can invoke when flow errors occur. This tool utilizes a dedicated LLM designed specifically for analyzing OpenLane logs, which examines each step's output files and error logs to pinpoint failure causes. The debugging tool presents findings in a structured format that makes complex error patterns interpretable. The OpenLane Agent excels in optimization through iterative refinement, repeatedly executing the flow with varied parameters, observing performance metrics (timing, power, area), and adjusting configurations
Fig. 1: ASIC-Agent System Architecture
accordingly until design constraints are met.
- Caravel Integration Agent: The Caravel Integration Agent specializes in integrating hardened user designs into the Caravel SoC harness, which is the standard framework for projects submitted to open-source silicon initiatives like the Google-sponsored Open MPW program. This agent utilizes specialized knowledge of the Caravel architecture including its predefined interfaces, memory map, pin assignments, and integration requirements. The agent generates wrapper modules and interconnect logic to bridge the user design with Caravel's standardized interfaces, ensuring proper signal routing, clock domain crossing management, and reset synchronization. It implements memory-mapped registers as needed for control and status monitoring via the Wishbone bus.
ASIC-Agent operates within a specialized runtime environment that provides a comprehensive set of tools and interfaces tailored to hardware design tasks. This infrastructure enables agents to interact with hardware design tools, manipulate files, and execute complex design workflows autonomously.
-
- Runtime Environment: The system runs in an isolated Docker container that provides a controlled and reproducible environment for hardware design tasks. This sandbox contains pre-installed EDA tools including Icarus Verilog, Verilator, Yosys, and OpenLane, enabling immediate execution of design, verification, and synthesis tasks.
-
- Tool Integration: ASIC-Agent implements an AgentComputer Interface (ACI) architecture similar to those used in SWE-Agent [6] and OpenHands [7] systems, but extensively customized for hardware design workflows. This interface provides structured commands and feedback mechanisms that allow the LLM agents to reliably interact with design tools, and manage complex hardware design tasks. The hardwarespecific ACI includes specialized functions for running and using hardware EDA tools, analyzing HDL, parsing, and other capabilities that traditional software-focused ACIs lack.
- lint_verilog : Performs static analysis on Verilog code, identifying potential issues before simulation. This function executes automatically after each Verilog file modification, providing immediate feedback on code quality and potential design issues
- simulate_verilog : Automates testbench execution with appropriate simulator configurations
- parse_verilog : Generates abstract syntax tree (AST) representations of HDL code using PyVerilog, enabling sophisticated code analysis and debugging
- run_openlane : Executes OpenLane flows with appropriate configuration management
- view_openlane_metrics : Extracts and analyzes key performance metrics from OpenLane runs
- query_opensource_ips : Interfaces with the IPM (IP Package Manager) to search and retrieve open-source hardware IPs relevant to the current task
- query_docs : Performs contextual searches across hardware-specific documentation using the external knowledge base
ASIC-Agent's capabilities are enhanced by a comprehensive external knowledge infrastructure extending beyond LLMs' parametric knowledge, employing vector databases and RAG to provide contextual, domain-specific information for specialized ASIC design challenges. Central to its debugging capabilities is a vector database of error patterns and solutions created from the Open Source Silicon Design Community's discussions, which captures conversations, classifies useful threads, and extracts structured information to identify potential causes and solutions through semantic similarity matching. A separate vector database of open-source IP blocks, indexed by functionality and specifications, integrates with IPM through the IPs Marketplace to enable quick identification of suitable pre-verified components, accelerating development by leveraging existing open-source modules.
The system incorporates documentation for essential tools including OpenLane, Caravel, and cocotb, enabling semantic queries in natural language to receive relevant configuration options or usage examples, which transforms technical documentation into an accessible knowledge resource. ASICAgent implements specialized RAG architecture optimized for hardware design workflows, employing an agentic approach with multi-hop information retrieval that allows synthesis of information from multiple documents and connects concepts, tools, and design patterns. This enhanced architecture combines LLMs' reasoning capabilities with targeted domain knowledge, maintaining accuracy in specialized hardware design tasks while addressing the limitations of context windows.
To establish a rigorous and comprehensive framework for evaluating agents' performance in real-world hardware design tasks, we present the ASIC-Agent-Bench benchmark, inspired by the approach of TheAgentCompany benchmark [16], specifically developed to address critical limitations in existing RTL benchmarks. This benchmark is specifically designed for ASIC design agents, assessing not only the LLM's effectiveness in interpreting and generating hardware design solutions but also its ability to leverage external tools for synthesis, verification, and optimization-advancing beyond conventional code generation.
-
Diversity of Task Domains: ASIC-Agent-Bench evaluates agents across a range of ASIC design tasks. Unlike benchmarks focused solely on module generation, it assesses the agent's ability to integrate RTL designs into the Caravel SoC template, a pre-built system-on-chip for user-defined IP integration. The benchmark also includes hardening tasks using the OpenLane flow, which automates ASIC implementation from RTL to GDSII. This diverse nature of tasks ensures a robust evaluation of the agent's capability to synthesize, verify, and implement complex ASIC components within an opensource design ecosystem.
-
Varying Levels of Complexity: The benchmark tasks span a broad complexity spectrum, from basic combinational logic to advanced processor architectures, ensuring a rigorous evaluation of an ASIC design agent's capabilities. Complexity classification is based on four key factors: (1) the inclusion of sequential logic and state elements, (2) the sophistication of data processing and control mechanisms, (3) architectural depth, encompassing pipelining and multi-stage operations, and (4) the integration of RTL modules into Efabless Caravel SoC and execution of the OpenLane RTL-to-GDSII flow.
-
LLM-Powered Evaluation Framework: Traditional benchmarks impose rigid constraints on agents (e.g., implementing a module as a single file with a specific name) and require integration with a fixed testbench. ASICAgent-Bench removes these limitations, granting agents full autonomy to carry out open-ended tasks. This approach mirrors real-world workflows by evaluating the agent's ability to structure tasks, develop test frameworks, and complete ASIC implementation independently. Given the nature of these tasks, conventional rule-based grading is insufficient. Instead, ASIC-Agent-Bench employs an LLM judge to assess agent performance, enabling more flexible and context-aware evaluation.
An LLM can effectively serve as a judge because evaluation tasks are more structured and tractable than generation. The evaluation process does not write or execute test cases but qualitatively assesses the codebase against predefined checkpoints while the evaluation script separately verifies the successful execution of testbenches. This approach ensures a scalable and adaptive evaluation methodology, overcoming the rigidity of conventional template-driven frameworks.
- Checkpoint-Based Grading with Partial Credit: The benchmark framework uses a partial credit system to account for the complexity and multi-stage progression of tasks. Since agents may not always produce complete solutions, checkpoints systematically assess incremental progress, awarding credit for correctly executed intermediate steps.
Tasks are structured into three components
- a) Prompt: The problem statement or instructions provided to the agent.
b) Checkpoints: Evaluation criteria are based on agent outputs and their expected properties (e.g., a testbench for a counter must include a wrap-around case). Checkpoints ensure objective verification by being actionable, testable, and tied to observable agent deliverables, such as the generated codebase and terminal stdout.
The Checkpoints follow these Design Principles:
-
- Observable Artifacts: Each checkpoint must correspond to explicit, verifiable outputs such as workspace content (e.g. source code files) and testbenches execution result.
-
- Atomic Verification: Checkpoints should be evaluated using binary (yes/no) to ensure unambiguous assessment (e.g., 'Does the testbench cover random inputs ? Y/N').
-
- LLM-Evaluator Compatibility: Criteria should emphasize the presence of required elements (e.g., 'Code includes assertions for overflow conditions') rather than qualitative assessments (e.g., 'Code is well-structured'), ensuring alignment with automated evaluation methods.
- c) Evaluation Methodology: In the benchmark, RTL generation tasks have key milestones such as implementing the top module and testbenches and verifying their content, and the execution of testbench simulations by the evaluation script, which verifies both the output and the successful completion of the simulation. For tasks involving OpenLane flow execution, milestones include the generation of the config.json file and the successful production of the GDSII layout.
The LLM-based evaluator assesses the codebase by comparing it to predefined checkpoints, assigning a score based on the number of checkpoints successfully met. The LLM judge is fixed as gemini-2.5-pro for consistency. Figure 2 provides an overview of the evaluation process.
Fig. 2: ASIC-Agent-Bench Evaluation Process
The final score is computed as a weighted sum of individual milestone scores, where weights are assigned based on the relative importance of each milestone to the overall task objective.
To ensure the reliability of our evaluation, human reviewers analyzed whether the evaluation agent's results accurately reflected the quality of the generated agent's output in solving the task problem. Through iterative refinement, we enhanced the evaluator to ensure fair and logical grading. This process refined the scoring to match human expectations, ensuring that the final score accurately represents task completeness and correctness.
To contextualize the contributions of our benchmark, we compare it against prominent existing benchmarks in hardware design tasks: RTLLM [9], RTL-Repo [17], and VerilogEval [8]. Our benchmark uniquely introduces three key advancements over existing benchmarks in hardware design: (1) Flexibility and open-endedness, allowing autonomous decisionmaking in task execution rather than enforcing rigid constraints like predefined module names; (2) Agentic Evaluation, as the first benchmark explicitly assessing agentic systems in hardware design-where agents independently plan, utilize external tools, and iteratively debug their multi-file codebases-while other benchmarks focus solely on single-file RTL module generation; and (3) Hardening and Integration, incorporating tasks that evaluate physical layout generation like integration with Caravel SoC-capabilities that are critical to the ASIC design process that are absent from prior benchmarks.
The experimental evaluation summarized in Table I highlights the significant influence of the underlying large language model (LLM) on the ASIC-Agent's performance across multiple tasks. Three distinct LLM backbones-Claude 4 Sonnet [18], GPT-4.1 [19], and Gemini 2.5 Pro [20]-are compared using three key metrics: score (percentage performance), the number of steps taken, and cost (in dollars).
On average, the ASIC-Agent employing Claude 4 Sonnet achieves the highest performance score (88%), though at the highest cost but with a moderate number of steps per task. Remarkably, Claude 4 Sonnet's performance on hard tasks (75.17%) surpasses GPT-4.1 and Gemini's performance on medium tasks, demonstrating its exceptional capability to tackle complex problems. The model shows consistent excellence across all difficulty levels, with performance increasing from 75.17% on hard tasks to 96.67% on easy tasks.
In contrast, GPT-4.1 achieves the lowest average score (60.8%) but compensates with the most cost-effective operation and fewest steps, making it suitable for budget-constrained applications despite lower accuracy. Gemini 2.5 Pro strikes a middle ground with a moderate score (71.45%) at intermediate cost, while requiring a similar number of steps to Claude 4 Sonnet. Notably, Gemini 2.5 Pro demonstrates strong performance on easy tasks (93.67%) that nearly matches Claude 4 Sonnet, but struggles significantly on hard (52.17%) and medium (57.80%) tasks, suggesting it may be better suited for less complex ASIC design challenges.
a) Debugging capabilities: The agent exhibited strong debugging capabilities, significantly reducing the time engineers spend on manual debugging. The process involved iteratively refining the model to pass testbench-defined test
TABLE I: Comparative Performance Analysis of ASIC-Agent on Tasks Using Different Base LLM Models
| ASIC-Agent with Different Base LLMs | ASIC-Agent with Different Base LLMs | ASIC-Agent with Different Base LLMs | ASIC-Agent with Different Base LLMs | ASIC-Agent with Different Base LLMs | ASIC-Agent with Different Base LLMs | ASIC-Agent with Different Base LLMs | ASIC-Agent with Different Base LLMs | ASIC-Agent with Different Base LLMs | |
|---|---|---|---|---|---|---|---|---|---|
| Task | Claude 4 Sonnet | Claude 4 Sonnet | Claude 4 Sonnet | GPT-4.1 | GPT-4.1 | GPT-4.1 | Gemini 2.5 Pro | Gemini 2.5 Pro | Gemini 2.5 Pro |
| Score (%) | Steps # | Cost ($) | Score (%) | Steps # | Cost ($) | Score (%) | Steps # | Cost ($) | |
| Neural Network Accelerator | 70 | 60 | 8.97 | 31 | 63 | 3.79 | 8 | 28 | 3.49 |
| RISC-V Processor Core | 85 | 54 | 6.91 | 74 | 46 | 2.29 | 87 | 49 | 4.26 |
| AES Encryption Core | 70 | 65 | 9.21 | 27 | 60 | 5.50 | 27 | 59 | 5.58 |
| Matrix Multiplier Core | 73 | 35 | 3.41 | 50 | 20 | 0.83 | 65 | 43 | 2.92 |
| IEEE-754 Floating Point Unit | 97 | 49 | 8.35 | 13 | 99 | 10.11 | 37 | 99 | 12.64 |
| UART | 56 | 79 | 15.68 | 60 | 32 | 1.99 | 89 | 42 | 5.41 |
| Pipelined Multiplier | 68 | 88 | 18.44 | 23 | 40 | 2.21 | 94 | 69 | 14.36 |
| Wishbone Bridge Bug Fix | 86 | 10 | 0.67 | 80 | 13 | 0.50 | 40 | 16 | 0.93 |
| Memory Controller Debugging | 100 | 61 | 5.79 | 40 | 24 | 0.98 | 20 | 14 | 0.72 |
| Adder DPI Validation | 100 | 16 | 1.06 | 100 | 12 | 0.42 | 100 | 16 | 0.66 |
| Finite State Machine | 100 | 59 | 7.65 | 25 | 9 | 0.35 | 35 | 67 | 8.82 |
| Karnaugh Map Solver | 80 | 17 | 1.32 | 60 | 12 | 0.48 | 80 | 10 | 0.46 |
| 8-bit Barrel Shifter | 100 | 14 | 0.98 | 30 | 11 | 0.39 | 92 | 15 | 0.76 |
| Carry-Lookahead Adder | 100 | 16 | 1.17 | 100 | 12 | 0.46 | 100 | 13 | 0.63 |
| D Flip-Flop | 100 | 9 | 0.62 | 100 | 11 | 0.32 | 100 | 8 | 0.68 |
| Counter | 100 | 14 | 0.92 | 90 | 11 | 0.38 | 90 | 12 | 0.48 |
| Edge Detector | 100 | 13 | 0.86 | 100 | 10 | 0.35 | 100 | 23 | 1.02 |
| UART Integration Caravel | 100 | 43 | 4.22 | 28 | 99 | 6.02 | 90 | 91 | 7.94 |
| IPM Management Caravel | 100 | 11 | 0.68 | 100 | 4 | 0.13 | 100 | 11 | 0.44 |
| GPIO Integration Caravel | 75 | 17 | 1.27 | 85 | 4 | 0.14 | 75 | 10 | 0.66 |
| Average | 88 | 37 | 4.91 | 60.8 | 30 | 1.88 | 71.45 | 35 | 3.64 |
Claude 4 Sonnet
GPT-4.1
Gemini 2.5 Pro
Fig. 3: Performance of Different Base LLMs with ASIC-Agent by Task Difficulty
cases, resolving syntax errors, managing environment configurations, and addressing linting violations, among other tasks. ASIC-Agent with Claude demonstrated the highest efficiency, requiring the fewest steps to resolve issues.
b) Iterative Design Optimization: In the physical layout flow, the agent showed proficiency in optimizing PPA (Power, Performance, Area) metrics and successfully resolved violations such as timing, antenna, and DRC by iteratively refining OpenLane's configuration variables and modifying the original RTL until the design constraints and specifications were met.
- c) Python-based verification: We observed that our ASIC agent, when using Cocotb, enables a more efficient, flexible, and comprehensive verification process. This allows for better test development and execution compared to Verilog. We attribute this to LLM proficiency in Python and the abstractions provided by Cocotb.
- d) Linting Errors Handling: The agent frequently encounters linting errors when processing Verilog files. However, the ability to resolve these errors varies across different underlying LLMs. ASIC-Agent with Claude demonstrates adaptive problem-solving by effectively diagnosing and correcting linting errors, particularly in medium-easy tasks. In contrast, other models often remain stuck even with medium tasks, repeatedly failing to resolve the same errors across multiple iterations.
- e) Vector Database Utilization: ASIC-Agent usually utilizes the vector database and its RAG implementation when encountering challenges, such as OpenLane flow errors, linting issues, and Caravel-specific problems. When stuck, it queries the database to diagnose errors, and identify potential solutions and best practices, improving its effectiveness in accurately completing the tasks.
In this paper, we introduce ASIC-Agent, a multi-agent system that overcomes LLM limitations in digital ASIC design by integrating specialized sub-agents. To evaluate its effectiveness, we present ASIC-Agent-Bench, a benchmark for agentic hardware design. Our findings highlight ASIC-Agent's ability to enhance ASIC development and streamline workflows.
- [1] T. Zheng, G. Zhang, T. Shen, X. Liu et al. , 'Opencodeinterpreter: Integrating code generation with execution and refinement,' https://arxiv.org/abs/2402.14658 , 2024.
- [2] E. Nijkamp, B. Pang, H. Hayashi, L. Tu et al. , 'Codegen: An open large language model for code with multi-turn program synthesis,' in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https://openreview.net/forum?id=iaYcJKpY2B
- [3] A. Lozhkov, R. Li, L. B. Allal, F. Cassano et al. , 'Starcoder 2 and the stack v2: The next generation,' 2024.
- [4] S. Thakur, B. Ahmad, H. Pearce, B. Tan et al. , 'Verigen: A large language model for verilog code generation,' ACM Trans. Des. Autom. Electron. Syst. , vol. 29, no. 3, apr 2024. [Online]. Available: https://doi.org/10.1145/3643681
- [5] S. Liu, W. Fang, Y. Lu, J. Wang et al. , 'Rtlcoder: Fully open-source and efficient llm-assisted rtl code generation technique,' 2024. [Online]. Available: https://arxiv.org/abs/2312.08617
- [6] J. Yang, C. E. Jimenez, A. Wettig, K. Lieret et al. , 'Swe-agent: Agent-computer interfaces enable automated software engineering,' 2024. [Online]. Available: https://arxiv.org/abs/2405.15793
- [7] X. Wang, B. Li, Y. Song, F. F. Xu et al. , 'Openhands: An open platform for ai software developers as generalist agents,' 2024. [Online]. Available: https://arxiv.org/abs/2407.16741
- [8] M. Liu, N. Pinckney, B. Khailany, and H. Ren, 'Verilogeval: Evaluating large language models for verilog code generation,' 2023.
- [9] Y. Lu, S. Liu, Q. Zhang, and Z. Xie, 'Rtllm: An open-source benchmark for design rtl generation with large language model,' in Proceedings of the 29th Asia and South Pacific Design Automation Conference , ser. ASPDAC '24. IEEE Press, 2024, p. 722-727. [Online]. Available: https://doi.org/10.1109/ASP-DAC58780.2024.10473904
- [10] M. Liu, T.-D. Ene, R. Kirby, C. Cheng et al. , 'Chipnemo: Domainadapted llms for chip design,' 2024.
- [11] C.-T. Ho, H. Ren, and B. Khailany, 'Verilogcoder: Autonomous verilog coding agents with graph-based planning and abstract syntax tree (ast)-based waveform tracing tool,' 2025. [Online]. Available: https://arxiv.org/abs/2408.08927
- [12] Y. Zhao, H. Zhang, H. Huang, Z. Yu et al. , 'Mage: A multi-agent engine for automated rtl code generation,' 2024.
- [13] X. Wang, Y. Chen, L. Yuan, Y. Zhang et al. , 'Executable code actions elicit better llm agents,' 2024. [Online]. Available: https://arxiv.org/abs/2402.01030
- [14] D. N. Gadde, S. Kumari, and A. Kumar, 'Effective design verification constrained random with python and cocotb,' 2024. [Online]. Available: https://arxiv.org/abs/2407.10312
- [15] A. A. Ghazy and M. Shalan, 'Openlane: The open-source digital asic implementation flow,' 2020. [Online]. Available: https://api. semanticscholar.org/CorpusID:233404245
- [16] F. F. Xu, Y. Song, B. Li, Y. Tang et al. , 'Theagentcompany: Benchmarking llm agents on consequential real world tasks,' 2024. [Online]. Available: https://arxiv.org/abs/2412.14161
- [17] A. Allam and M. Shalan, 'Rtl-repo: A benchmark for evaluating llms on large-scale rtl design projects,' 2024. [Online]. Available: https://arxiv.org/abs/2405.17378
- [18] [Online]. Available: https://www-cdn.anthropic.com/ fed9cc193a14b84131812372d8d5857f8f304c52/Model Card Claude 3 Addendum.pdf
- [19] OpenAI. [Online]. Available: https://openai.com/index/gpt-4-1/
- [20] K. Kavukcuoglu, 'Gemini 2.5: Our most intelligent ai model,' Mar 2025. [Online]. Available: https://blog.google/technology/google-deepmind/ gemini-model-thinking-updates-march-2025/#gemini-2-5-thinking
This section demonstrates a full task trajectory, detailing the agent's actions, observations, and both intermediate and final outputs.
The agent was assigned the task of designing and testing a Four-Bit Pipelined Multiplier Implementation. The process involved implementing the Verilog module, verifying its correctness, and hardening the design using the OpenLane flow.
-
- Implementation and Linting: The agent began by implementing the Verilog module and performing linting to ensure syntax correctness. The module was free of syntax errors, allowing the agent to proceed to testing.
-
- Initial Testing with Cocotb: The agent started by querying the documentation database to ensure it followed best practices to create a Cocotb testbench. The agent then executed a self-checking Cocotb test using a Makefile script. During testing, a failing test case was encountered where the output did not match the expected result.
-
- Debugging and Refinement: To address the failing test, the agent modified the testbench file to initialize output signals properly and did modifications to the Verilog file as well. However, this introduced additional issues. The agent iteratively refined the implementation, incorporating debugging information to analyze the problem further. This process revealed a synchronization issue in the pipeline, which the agent resolved, leading to all test cases passing successfully.
-
- OpenLane Flow Execution: With a verified design, the agent proceeded to harden the multiplier. It first queried the vector database, then generated the necessary configuration file config.json , and successfully ran the OpenLane flow.
-
- Final Analysis: Upon completing the flow, the agent analyzed key metrics, including timing, area, power, DRC and LVS errors, to ensure the hardening process was successful with no critical errors or violations.
The agent successfully designed, verified, and hardened the Four-Bit Pipelined Multiplier. The final implementation passed all test cases and was successfully synthesized through OpenLane, demonstrating a structured and resilient problemsolving process.
Verilog Four-Bit Pipelined Multiplier Implementation and Testing
Step 1: Implement a Four-Bit Pipelined Multiplier
Design a 4-bit unsigned pipelined multiplier in Verilog with the following interface:
- 'clk'
- 'reset'
- 'a' (4-bit input)
- 'b' (4-bit input)
- 'product' (8-bit output)
- 'valid' (output)
The multiplier should have a three-cycle pipeline.
Step 2: Create a Self-Checking Testbench
- Develop a comprehensive testbench for the multiplier module with assertions that covers all possible cases.
- If a test case fails, the assertion should terminate the testbench execution.
Ensure the following points are addressed within the testbench:
Pipeline Functionality:
- Verify that the product is correctly calculated after exactly three clock cycles.
- Include test cases to test pipeline functionality including stalling and valid signals.
Input Combinations:
- Test all possible input combinations.
Step 3: run test.sh File Creation
- Create a shell script run test.sh that contains the command to only run the testbench.
- Execute the run test.sh to make sure it successfully runs the testbench.
Step 4: OpenLane Flow
- Harden the design using the OpenLane flow.
- Ensure the hardening process is power, performance, and area (PPA) efficient, leading to the successful generation of the final GDSII.
I'll help you complete this task step by step. Let's start with implementing the 4-bit pipelined multiplier. First, let me create the Verilog module for the multiplier. I'll name it 'pipelined multiplier.v':
edit /workspace/pipelined_multiplier.v
---/workspace/pipelined_multiplier.v +++ /workspace/pipelined_multiplier.v @@ -0,0 +1,8 @@ +module pipelined_multiplier ( + input wire clk, + input wire reset, + input wire [3:0] a, + input wire [3:0] b,
Fig. 4: Trajectory for task Pipelined Multiplier .
+ output reg [7:0] product, + output reg valid +); @@ -1,0 +10,46 @@ + // Pipeline registers + reg [3:0] a_reg1, b_reg1; + reg [3:0] a_reg2, b_reg2; + reg [7:0] partial_product; + reg valid_reg1, valid_reg2; + + // Stage 1: Register inputs + always @(posedge clk or posedge reset) begin + if (reset) begin + a_reg1 <= 4'b0; + b_reg1 <= 4'b0; + valid_reg1 <= 1'b0; + end else begin + a_reg1 <= a; + b_reg1 <= b; + valid_reg1 <= 1'b1; + end + end + + // Stage 2: Calculate partial product and register + always @(posedge clk or posedge reset) begin + if (reset) begin + a_reg2 <= 4'b0; + b_reg2 <= 4'b0; + partial_product <= 8'b0; + valid_reg2 <= 1'b0; + end else begin + a_reg2 <= a_reg1; + b_reg2 <= b_reg1; + partial_product <= a_reg1 * b_reg1; + valid_reg2 <= valid_reg1; + end + end + + // Stage 3: Register final product + always @(posedge clk or posedge reset) begin + if (reset) begin + product <= 8'b0; + valid <= 1'b0; + end else begin + product <= partial_product; + valid <= valid_reg2; + end + end + +endmodule
Let's lint the Verilog code to ensure there are no issues:
lint_result, line_num = lint_verilog('/workspace/pipelined_multiplier.v') print(lint_result)
SUCCESS: No linting errors found.
Pipelined Multiplier
Pipelined Multiplier
Great! The Verilog code passed linting. Now let's create a cocotb testbench to verify the functionality. First, let me query the documentation database to ensure I follow best practices for cocotb:
print(query_docs_db("cocotb testbench best practices and examples"))
To create effective cocotb testbenches, follow these best practices and examples:
- Directory Structure: Organize your testbench files under a dedicated directory, such as
verilog/dv/cocotb, with separate files for firmware (.c) and Python testbench (.py). This helps maintain clarity and organization. ↪ → ↪ → - Firmware Creation: Write firmware in C to run on the Caravel management SoC. This firmware should configure GPIOs and set specific values, which the testbench will later verify. ↪ → ↪ →
- Python Test Bench: Use Python to create the testbench. Import necessary cocotb libraries and define your test as a coroutine using
@cocotb.test(). Configure the test environment and monitor signals to verify expected outcomes. ↪ → ↪ → - Running Tests: Use a Makefile to automate test execution or run tests manually using the
caravel\_cocotbcommand. Ensure that your environment is correctly set up with necessary paths and dependencies. ↪ → ↪ → - Debugging and Verification: After running tests, check log files for pass/fail results and debug any issues by examining the firmware and testbench code. ↪ →
Example: ``` python @cocotb.test() async def gpio_test(dut): caravelEnv = await test_configure(dut) await caravelEnv.release_csb() await caravelEnv.wait_mgmt_gpio(1) gpios_value_str = caravelEnv.monitor_gpio(37, 0).binstr expected_gpio_value = 0x8F if caravelEnv.monitor_gpio(37, 0).integer == expected_gpio_value: cocotb.log.info(f"[TEST] Pass: GPIO value is '{hex(expected_gpio_value)}'") else: cocotb.log.error(f"[TEST] Fail: GPIO value is not as expected") ```
These practices ensure a structured approach to writing cocotb testbenches, facilitating effective simulation and verification of ASIC designs within the Caravel environment. ↪ →
Now I'll create a cocotb testbench for our pipelined multiplier. First, let's create a directory structure and the necessary files:
mkdir -p /workspace/test
Let's create the test file:
edit /workspace/test/test_pipelined_multiplier.py
---/workspace/test/test_pipelined_multiplier.py +++ /workspace/test/test_pipelined_multiplier.py @@ -0,0 +1,5 @@ +import cocotb +from cocotb.clock import Clock +from cocotb.triggers import RisingEdge, Timer +from cocotb.regression import TestFactory +from itertools import product @@ -1,0 +7,74 @@ +async def reset_dut(dut): + dut.reset.value = 1 + await Timer(10, units='ns') + dut.reset.value = 0 + await Timer(10, units='ns') + +@cocotb.test() +async def test_pipelined_multiplier(dut): + """Test all possible input combinations and verify pipeline functionality""" + + # Start the clock + clock = Clock(dut.clk, 10, units="ns") + cocotb.start_soon(clock.start()) + + # Reset the DUT + await reset_dut(dut) + + # Dictionary to store expected results + expected_results = {} + current_cycle = 0 + + # Test all possible input combinations (4-bit × 4-bit = 256 cases) + for a, b in product(range(16), range(16)): + await RisingEdge(dut.clk) + + # Set inputs + dut.a.value = a + dut.b.value = b + + # Store expected result for checking after pipeline delay + expected_results[current_cycle + 3] = (a * b, 1) # (expected product, expected valid) ↪ → + + # Check results from 3 cycles ago + if current_cycle >= 3: + expected_product, expected_valid = expected_results[current_cycle] + + if dut.valid.value != expected_valid: + raise cocotb.result.TestFailure( + f"Valid signal incorrect at cycle {current_cycle}. " + f"Expected {expected_valid}, got {dut.valid.value}" + ) + + if dut.valid.value == 1: + if dut.product.value != expected_product: + raise cocotb.result.TestFailure( + f"Product incorrect at cycle {current_cycle}. " + f"For inputs from 3 cycles ago, expected {expected_product}, " + f"got {dut.product.value}" + ) + + current_cycle += 1 + + # Check the final 3 cycles for remaining pipeline results
for _ in range(3): await RisingEdge(dut.clk) current_cycle += 1 if current_cycle in expected_results: expected_product, expected_valid = expected_results[current_cycle] if dut.valid.value != expected_valid: raise cocotb.result.TestFailure( f"Valid signal incorrect at cycle {current_cycle}. " f"Expected {expected_valid}, got {dut.valid.value}" ) if dut.valid.value == 1: if dut.product.value != expected_product: raise cocotb.result.TestFailure( f"Product incorrect at cycle {current_cycle}. " f"Expected {expected_product}, got {dut.product.value}" ) dut._log.info("All test cases passed successfully!")
Now let's create a Makefile to run the cocotb tests:
edit /workspace/test/Makefile
---/workspace/test/Makefile +++ /workspace/test/Makefile @@ -0,0 +1,2 @@ +SIM ?= icarus +TOPLEVEL_LANG ?= verilog @@ -1,0 +4,5 @@ +VERILOG_SOURCES = $(PWD)/../pipelined_multiplier.v +TOPLEVEL = pipelined_multiplier +MODULE = test_pipelined_multiplier + +include $(shell cocotb-config --makefiles)/Makefile.sim
Now let's create the run test.sh script:
edit /workspace/run_test.sh
---/workspace/run_test.sh +++ /workspace/run_test.sh @@ -1 +1,2 @@ -+#!/bin/bash +cd test && make
Pipelined Multiplier
Pipelined Multiplier
Pipelined Multiplier
Pipelined Multiplier
Let's make the script executable:
chmod +x /workspace/run_test.sh
Now let's run the tests:
cd /workspace && ./run_test.sh
Pipelined Multiplier
make -f Makefile results.xml make[1]: Entering directory '/workspace/test' mkdir -p sim_build /usr/bin/iverilog -o sim_build/sim.vvp -D COCOTB_SIM=1 -s pipelined_multiplier -g2012 -f sim_build/cmds.f /workspace/test/../pipelined_multiplier.v ↪ → MODULE=test_pipelined_multiplier TESTCASE= TOPLEVEL=pipelined_multiplier TOPLEVEL_LANG=verilog \ ↪ → /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp ↪ → 0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable) 0.00ns INFO cocotb Running tests with cocotb v1.9.2 0.00ns INFO cocotb Seeding Python random module with 1741821210 0.00ns INFO cocotb.regression Found test test_pipelined_multiplier.test_pipelined_multiplier ↪ → 0.00ns INFO cocotb.regression running test_pipelined_multiplier (1/1) /workspace/test/test_pipelined_multiplier.py:51: DeprecationWarning: TestFailure is deprecated, use an `` assert `` statement instead ↪ → raise cocotb.result.TestFailure( 50.00ns INFO cocotb.regression test_pipelined_multiplier failed Traceback (most recent call last): File "/workspace/test/test_pipelined_multiplier.py", line 51, in test_pipelined_multiplier ↪ → raise cocotb.result.TestFailure( cocotb.result.TestFailure: Product incorrect at cycle 3. Expected 0, got xxxxxxxx 50.00ns INFO cocotb.regression ** TESTS=1 PASS=0 FAIL=1 SKIP=0 **
make[1]: Leaving directory '/workspace/test'
It seems we have a failing test case. The issue is that we need to initialize the output signals properly in the testbench. Let me modify the testbench:
edit /workspace/test/test_pipelined_multiplier.py
---/workspace/test/test_pipelined_multiplier.py +++ /workspace/test/test_pipelined_multiplier.py @@ -22,0 +23,8 @@
Pipelined Multiplier
+ + # Initialize inputs + dut.a.value = 0 + dut.b.value = 0 + + # Wait for 3 cycles after reset to ensure pipeline is cleared + for _ in range(3): + await RisingEdge(dut.clk)
Let's run the tests again:
cd /workspace && ./run_test.sh
Pipelined Multiplier
make -f Makefile results.xml make[1]: Entering directory '/workspace/test' MODULE=test_pipelined_multiplier TESTCASE= TOPLEVEL=pipelined_multiplier TOPLEVEL_LANG=verilog \ ↪ → /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp ↪ → 0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable) 0.00ns INFO cocotb Running tests with cocotb v1.9.2 0.00ns INFO cocotb Seeding Python random module with 1741821222 0.00ns INFO cocotb.regression Found test test_pipelined_multiplier.test_pipelined_multiplier ↪ → 0.00ns INFO cocotb.regression running test_pipelined_multiplier (1/1) /workspace/test/test_pipelined_multiplier.py:59: DeprecationWarning: TestFailure is deprecated, use an `` assert `` statement instead ↪ → raise cocotb.result.TestFailure( 250.00ns INFO cocotb.regression test_pipelined_multiplier failed Traceback (most recent call last): File "/workspace/test/test_pipelined_multiplier.py", line 59, in test_pipelined_multiplier ↪ → raise cocotb.result.TestFailure( cocotb.result.TestFailure: Product incorrect at cycle 20. Expected 1, got 00000000 make[1]: Leaving directory '/workspace/test'
We still have an issue. Let me modify the Verilog code to fix the issue:
edit /workspace/pipelined_multiplier.v
---/workspace/pipelined_multiplier.v +++ /workspace/pipelined_multiplier.v @@ -12 +11,0 @@ -reg [3:0] a_reg2, b_reg2; @@ -14,0 +14 @@
Pipelined Multiplier
Pipelined Multiplier
+ wire [7:0] mult_result;
edit /workspace/pipelined_multiplier.v
---/workspace/pipelined_multiplier.v +++ /workspace/pipelined_multiplier.v @@ -29 +29,4 @@ -// Stage 2: Calculate partial product and + // Combinational multiplication + assign mult_result = a_reg1 * b_reg1; + + // Stage 2: Register partial product @@ -32,2 +34,0 @@ -a_reg2 <= 4'b0; -b_reg2 <= 4'b0; @@ -37,3 +38 @@ -a_reg2 <= a_reg1; -b_reg2 <= b_reg1; -partial_product <= a_reg1 * b_reg1; + partial_product <= mult_result;
Let's run the tests again:
cd /workspace && ./run_test.sh
Pipelined Multiplier
Pipelined Multiplier
make -f Makefile results.xml make[1]: Entering directory '/workspace/test' /usr/bin/iverilog -o sim_build/sim.vvp -D COCOTB_SIM=1 -s pipelined_multiplier -g2012 -f sim_build/cmds.f /workspace/test/../pipelined_multiplier.v ↪ → MODULE=test_pipelined_multiplier TESTCASE= TOPLEVEL=pipelined_multiplier TOPLEVEL_LANG=verilog \ ↪ → /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp ↪ → 0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable) 0.00ns INFO cocotb Running tests with cocotb v1.9.2 0.00ns INFO cocotb Seeding Python random module with 1741821242 0.00ns INFO cocotb.regression Found test test_pipelined_multiplier.test_pipelined_multiplier ↪ → 0.00ns INFO cocotb.regression running test_pipelined_multiplier (1/1) /workspace/test/test_pipelined_multiplier.py:59: DeprecationWarning: TestFailure is deprecated, use an `` assert `` statement instead ↪ → raise cocotb.result.TestFailure( 250.00ns INFO cocotb.regression test_pipelined_multiplier failed Traceback (most recent call last):
register
File "/workspace/test/test_pipelined_multiplier.py", line 59, in test_pipelined_multiplier ↪ → raise cocotb.result.TestFailure( cocotb.result.TestFailure: Product incorrect at cycle 20. Expected 1, got 00000000 make[1]: Leaving directory '/workspace/test'
Let me modify the testbench to better handle the pipeline stages:
edit /workspace/test/test_pipelined_multiplier.py
Pipelined Multiplier
Pipelined Multiplier
---/workspace/test/test_pipelined_multiplier.py +++ /workspace/test/test_pipelined_multiplier.py @@ -32,3 +32,2 @@ -# Dictionary to store expected results -expected_results = {} -current_cycle = 0 + # Test all possible input combinations (4-bit × 4-bit = 256 cases) + pipeline = [] # Store (a, b) pairs in pipeline @@ -36 +34,0 @@ -# Test all possible input combinations (4-bit × 4-bit = 256 cases) @@ -44,2 +42,2 @@ -# Store expected result for checking after pipeline delay -expected_results[current_cycle + 3] = (a * b, 1) # (expected product, expected valid) ↪ → + # Add current inputs to pipeline + pipeline.append((a, b)) @@ -47,3 +45,4 @@ -# Check results from 3 cycles ago -if current_cycle >= 3: -expected_product, expected_valid = expected_results[current_cycle] + # Check output if pipeline is full + if len(pipeline) > 3: + old_a, old_b = pipeline.pop(0) # Get oldest values + expected = old_a * old_b @@ -51,13 +50,10 @@ -if dut.valid.value != expected_valid: -raise cocotb.result.TestFailure( -f"Valid signal incorrect at cycle {current_cycle}. " -f"Expected {expected_valid}, got {dut.valid.value}" -) --if dut.valid.value == 1: -if dut.product.value != expected_product: -raise cocotb.result.TestFailure( -f"Product incorrect at cycle {current_cycle}. " -f"For inputs from 3 cycles ago, expected {expected_product}, " -f"got {dut.product.value}" -) + assert dut.valid.value == 1, f"Valid signal should be 1" + assert dut.product.value == expected, \ + f"Product incorrect. For inputs a={old_a}, b={old_b}, " \ + f"expected {expected}, got {dut.product.value}" + + # Check remaining values in pipeline + while pipeline: + await RisingEdge(dut.clk)
+ old_a, old_b = pipeline.pop(0) + expected = old_a * old_b @@ -65,22 +61,4 @@ -current_cycle += 1 --# Check the final 3 cycles for remaining pipeline results -for _ in range(3): -await RisingEdge(dut.clk) -current_cycle += 1 --if current_cycle in expected_results: -expected_product, expected_valid = expected_results[current_cycle] --if dut.valid.value != expected_valid: -raise cocotb.result.TestFailure( -f"Valid signal incorrect at cycle {current_cycle}. " -f"Expected {expected_valid}, got {dut.valid.value}" -) --if dut.valid.value == 1: -if dut.product.value != expected_product: -raise cocotb.result.TestFailure( -f"Product incorrect at cycle {current_cycle}. " -f"Expected {expected_product}, got {dut.product.value}" -) + assert dut.valid.value == 1, f"Valid signal should be 1" + assert dut.product.value == expected, \ + f"Product incorrect. For inputs a={old_a}, b={old_b}, " \ + f"expected {expected}, got {dut.product.value}"
Let's run the tests again:
cd /workspace && ./run_test.sh
Pipelined Multiplier
make -f Makefile results.xml make[1]: Entering directory '/workspace/test' MODULE=test_pipelined_multiplier TESTCASE= TOPLEVEL=pipelined_multiplier TOPLEVEL_LANG=verilog \ ↪ → /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp ↪ → 0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable) 0.00ns INFO cocotb Running tests with cocotb v1.9.2 0.00ns INFO cocotb Seeding Python random module with 1741821271 0.00ns INFO cocotb.regression Found test test_pipelined_multiplier.test_pipelined_multiplier ↪ → 0.00ns INFO cocotb.regression running test_pipelined_multiplier (1/1) 250.00ns INFO cocotb.regression test_pipelined_multiplier failed Traceback (most recent call last): File "/workspace/test/test_pipelined_multiplier.py", line 51, in test_pipelined_multiplier ↪ → assert dut.product.value == expected, \ AssertionError: Product incorrect. For inputs a=1, b=1, expected 1, got 00000000 make[1]: Leaving directory '/workspace/test'
Let me fix the Verilog code to properly handle the pipeline stages:
edit /workspace/pipelined_multiplier.v
---/workspace/pipelined_multiplier.v +++ /workspace/pipelined_multiplier.v @@ -10,5 +10,5 @@ -// Pipeline registers -reg [3:0] a_reg1, b_reg1; -reg [7:0] partial_product; -reg valid_reg1, valid_reg2; -wire [7:0] mult_result; + // Pipeline stage registers + reg [3:0] a_pipe1, b_pipe1; + reg [3:0] a_pipe2, b_pipe2; + reg [7:0] product_pipe2; + reg valid_pipe1, valid_pipe2; @@ -19,3 +19,3 @@ -a_reg1 <= 4'b0; -b_reg1 <= 4'b0; -valid_reg1 <= 1'b0; + a_pipe1 <= 4'b0; + b_pipe1 <= 4'b0; + valid_pipe1 <= 1'b0; @@ -23,3 +23,3 @@ -a_reg1 <= a; -b_reg1 <= b; -valid_reg1 <= 1'b1; + a_pipe1 <= a; + b_pipe1 <= b; + valid_pipe1 <= 1'b1; @@ -29,4 +29 @@ -// Combinational multiplication -assign mult_result = a_reg1 * b_reg1; --// Stage 2: Register partial product + // Stage 2: Perform multiplication and @@ -35,2 +32,4 @@ -partial_product <= 8'b0; -valid_reg2 <= 1'b0; + a_pipe2 <= 4'b0; + b_pipe2 <= 4'b0; + product_pipe2 <= 8'b0; + valid_pipe2 <= 1'b0; @@ -38,2 +37,4 @@ -partial_product <= mult_result; -valid_reg2 <= valid_reg1; + a_pipe2 <= a_pipe1; + b_pipe2 <= b_pipe1; + product_pipe2 <= a_pipe1 * b_pipe1; + valid_pipe2 <= valid_pipe1; @@ -49,2 +50,2 @@ -product <= partial_product; -valid <= valid_reg2; + product <= product_pipe2; + valid <= valid_pipe2;
register
Let's run the tests again:
cd /workspace && ./run_test.sh
make -f Makefile results.xml make[1]: Entering directory '/workspace/test' /usr/bin/iverilog -o sim_build/sim.vvp -D COCOTB_SIM=1 -s pipelined_multiplier -g2012 -f sim_build/cmds.f /workspace/test/../pipelined_multiplier.v ↪ → MODULE=test_pipelined_multiplier TESTCASE= TOPLEVEL=pipelined_multiplier TOPLEVEL_LANG=verilog \ ↪ → /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp ↪ → 0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable) 0.00ns INFO cocotb Running tests with cocotb v1.9.2 0.00ns INFO cocotb Seeding Python random module with 1741821290 0.00ns INFO cocotb.regression Found test test_pipelined_multiplier.test_pipelined_multiplier ↪ → 0.00ns INFO cocotb.regression running test_pipelined_multiplier (1/1) 250.00ns INFO cocotb.regression test_pipelined_multiplier failed Traceback (most recent call last): File "/workspace/test/test_pipelined_multiplier.py", line 51, in test_pipelined_multiplier ↪ → assert dut.product.value == expected, \ AssertionError: Product incorrect. For inputs a=1, b=1, expected 1, got 00000000 make[1]: Leaving directory '/workspace/test'
Let me modify the testbench to better handle the pipeline stages and timing:
edit /workspace/test/test_pipelined_multiplier.py
---/workspace/test/test_pipelined_multiplier.py +++ /workspace/test/test_pipelined_multiplier.py @@ -34,0 +35,6 @@ + # First, fill the pipeline + for _ in range(3): + await RisingEdge(dut.clk) + pipeline.append((0, 0)) # Add initial values + + # Now test all combinations @@ -41,0 +48,10 @@ + # Get expected output from pipeline + old_a, old_b = pipeline.pop(0) + expected = old_a * old_b + + # Check output + assert dut.valid.value == 1, f"Valid signal should be 1" + assert dut.product.value == expected, \
Pipelined Multiplier
Pipelined Multiplier
+ f"Product incorrect. For inputs a={old_a}, b={old_b}, " \ + f"expected {expected}, got {dut.product.value}" + @@ -44,10 +59,0 @@ --# Check output if pipeline is full -if len(pipeline) > 3: -old_a, old_b = pipeline.pop(0) # Get oldest values -expected = old_a * old_b --assert dut.valid.value == 1, f"Valid signal should be 1" -assert dut.product.value == expected, \ -f"Product incorrect. For inputs a={old_a}, b={old_b}, " -f"expected {expected}, got {dut.product.value}"
Let's run the tests again:
cd /workspace && ./run_test.sh
\
Pipelined Multiplier
Pipelined Multiplier
make -f Makefile results.xml make[1]: Entering directory '/workspace/test' MODULE=test_pipelined_multiplier TESTCASE= TOPLEVEL=pipelined_multiplier TOPLEVEL_LANG=verilog \ ↪ → /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp ↪ → 0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable) 0.00ns INFO cocotb Running tests with cocotb v1.9.2 0.00ns INFO cocotb Seeding Python random module with 1741821316 0.00ns INFO cocotb.regression Found test test_pipelined_multiplier.test_pipelined_multiplier ↪ → 0.00ns INFO cocotb.regression running test_pipelined_multiplier (1/1) 280.00ns INFO cocotb.regression test_pipelined_multiplier failed Traceback (most recent call last): File "/workspace/test/test_pipelined_multiplier.py", line 54, in test_pipelined_multiplier ↪ → assert dut.product.value == expected, \ AssertionError: Product incorrect. For inputs a=1, b=1, expected 1, got 00000000 make[1]: Leaving directory '/workspace/test'
Let me modify the testbench to better handle the timing:
edit /workspace/test/test_pipelined_multiplier.py
---/workspace/test/test_pipelined_multiplier.py +++ /workspace/test/test_pipelined_multiplier.py @@ -32,0 +33 @@
Pipelined Multiplier
Pipelined Multiplier
+ test_cases = list(product(range(16), range(16))) @@ -35,2 +36,6 @@ -# First, fill the pipeline -for _ in range(3): + # First, fill the pipeline with initial test cases + for i in range(3): + a, b = test_cases[i] + dut.a.value = a + dut.b.value = b + pipeline.append((a, b)) @@ -38 +42,0 @@ -pipeline.append((0, 0)) # Add initial values @@ -40,3 +44,3 @@ -# Now test all combinations -for a, b in product(range(16), range(16)): -await RisingEdge(dut.clk) + # Now test all remaining combinations + for i in range(3, len(test_cases)): + a, b = test_cases[i] @@ -59,0 +64,2 @@ + + await RisingEdge(dut.clk) @@ -62 +68 @@ -while pipeline: + for _ in range(3): @@ -64,7 +70,8 @@ -old_a, old_b = pipeline.pop(0) -expected = old_a * old_b --assert dut.valid.value == 1, f"Valid signal should be 1" -assert dut.product.value == expected, \ -f"Product incorrect. For inputs a={old_a}, b={old_b}, " \ -f"expected {expected}, got {dut.product.value}" + if pipeline: + old_a, old_b = pipeline.pop(0) + expected = old_a * old_b + + assert dut.valid.value == 1, f"Valid signal should be 1" + assert dut.product.value == expected, \ + f"Product incorrect. For inputs a={old_a}, b={old_b}, " \ + f"expected {expected}, got {dut.product.value}"
Let's run the tests again:
cd /workspace && ./run_test.sh
Pipelined Multiplier
Pipelined Multiplier
make -f Makefile results.xml make[1]: Entering directory '/workspace/test' MODULE=test_pipelined_multiplier TESTCASE= TOPLEVEL=pipelined_multiplier TOPLEVEL_LANG=verilog \ ↪ → /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp ↪ → 0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable) 0.00ns INFO cocotb Running tests with cocotb v1.9.2 0.00ns INFO cocotb Seeding Python random module with 1741821344
0.00ns INFO cocotb.regression Found test test_pipelined_multiplier.test_pipelined_multiplier ↪ → 0.00ns INFO cocotb.regression running test_pipelined_multiplier (1/1) 240.00ns INFO cocotb.regression test_pipelined_multiplier failed Traceback (most recent call last): File "/workspace/test/test_pipelined_multiplier.py", line 58, in test_pipelined_multiplier ↪ → assert dut.product.value == expected, \ AssertionError: Product incorrect. For inputs a=1, b=1, expected 1, got 00000000 make[1]: Leaving directory '/workspace/test'
Let me modify the testbench to better handle the timing and add more debug information:
edit /workspace/test/test_pipelined_multiplier.py
---/workspace/test/test_pipelined_multiplier.py +++ /workspace/test/test_pipelined_multiplier.py @@ -3 +3 @@ -from cocotb.triggers import RisingEdge, Timer +from cocotb.triggers import RisingEdge, Timer, FallingEdge @@ -9 +9,4 @@ -await Timer(10, units='ns') + dut.a.value = 0 + dut.b.value = 0 + await Timer(20, units='ns') + await RisingEdge(dut.clk) @@ -11 +14,2 @@ -await Timer(10, units='ns') + await RisingEdge(dut.clk) + await FallingEdge(dut.clk) @@ -24,3 +28,3 @@ -# Initialize inputs -dut.a.value = 0 -dut.b.value = 0 + # Test simple sequence first + test_sequence = [(1, 1), (2, 2), (3, 3), (4, 4)] + pipeline = [] @@ -28,11 +32,2 @@ -# Wait for 3 cycles after reset to ensure pipeline is cleared -for _ in range(3): -await RisingEdge(dut.clk) --# Test all possible input combinations (4-bit × 4-bit = 256 -test_cases = list(product(range(16), range(16))) -pipeline = [] # Store (a, b) pairs in pipeline --# First, fill the pipeline with initial test cases -for i in range(3): -a, b = test_cases[i] + # Fill pipeline with initial values + for a, b in test_sequence[:3]: @@ -42,0 +38 @@ + await FallingEdge(dut.clk) @@ -44,5 +40,3 @@ -# Now test all remaining combinations
Pipelined Multiplier
cases)
-for i in range(3, len(test_cases)): -a, b = test_cases[i] --# Set inputs + # Process remaining values + for a, b in test_sequence[3:]: + # Set new inputs @@ -52 +46 @@ -# Get expected output from pipeline + # Check result from 3 cycles ago @@ -56 +50,7 @@ -# Check output + # Wait for result to appear + await RisingEdge(dut.clk) + await Timer(1, units='ns') # Small delay to let signals settle + + dut._log.info(f"Testing a={old_a}, b={old_b}, expected={expected}") + dut._log.info(f"Current product={dut.product.value}, valid={dut.valid.value}") + @@ -62 +62 @@ -# Add current inputs to pipeline + # Add new inputs to pipeline @@ -63,0 +64,6 @@ + await FallingEdge(dut.clk) + + # Check remaining values in pipeline + for _ in range(len(pipeline)): + old_a, old_b = pipeline.pop(0) + expected = old_a * old_b @@ -65,0 +72,47 @@ + await Timer(1, units='ns') # Small delay to let signals settle + + dut._log.info(f"Testing a={old_a}, b={old_b}, expected={expected}") + dut._log.info(f"Current product={dut.product.value}, valid={dut.valid.value}") + + assert dut.valid.value == 1, f"Valid signal should be 1" + assert dut.product.value == expected, \ + f"Product incorrect. For inputs a={old_a}, b={old_b}, " \ + f"expected {expected}, got {dut.product.value}" + + await FallingEdge(dut.clk) + + dut._log.info("Simple sequence test passed!") + + # Now test all possible combinations + test_cases = list(product(range(16), range(16))) + pipeline = [] + + # Fill pipeline with initial values + for a, b in test_cases[:3]: + dut.a.value = a + dut.b.value = b + pipeline.append((a, b)) + await RisingEdge(dut.clk) + await FallingEdge(dut.clk) + + # Process all remaining combinations + for a, b in test_cases[3:]: + # Set new inputs + dut.a.value = a + dut.b.value = b + + # Check result from 3 cycles ago + old_a, old_b = pipeline.pop(0) + expected = old_a * old_b
+ + await RisingEdge(dut.clk) + await Timer(1, units='ns') # Small delay to let signals settle + + assert dut.valid.value == 1, f"Valid signal should be 1" + assert dut.product.value == expected, \ + f"Product incorrect. For inputs a={old_a}, b={old_b}, " \ + f"expected {expected}, got {dut.product.value}" + + # Add new inputs to pipeline + pipeline.append((a, b)) + await FallingEdge(dut.clk) @@ -68 +121,4 @@ -for _ in range(3): + for _ in range(len(pipeline)): + old_a, old_b = pipeline.pop(0) + expected = old_a * old_b + @@ -70,8 +126,8 @@ -if pipeline: -old_a, old_b = pipeline.pop(0) -expected = old_a * old_b --assert dut.valid.value == 1, f"Valid signal should be 1" -assert dut.product.value == expected, \ -f"Product incorrect. For inputs a={old_a}, b={old_b}, " \ -f"expected {expected}, got {dut.product.value}" + await Timer(1, units='ns') # Small delay to let signals settle + + assert dut.valid.value == 1, f"Valid signal should be 1" + assert dut.product.value == expected, \ + f"Product incorrect. For inputs a={old_a}, b={old_b}, " \ + f"expected {expected}, got {dut.product.value}" + + await FallingEdge(dut.clk)
Let's run the tests again:
cd /workspace && ./run_test.sh
Pipelined Multiplier
Pipelined Multiplier
make -f Makefile results.xml make[1]: Entering directory '/workspace/test' MODULE=test_pipelined_multiplier TESTCASE= TOPLEVEL=pipelined_multiplier TOPLEVEL_LANG=verilog \ ↪ → /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp ↪ → 0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable) 0.00ns INFO cocotb Running tests with cocotb v1.9.2 0.00ns INFO cocotb Seeding Python random module with 1741821381 0.00ns INFO cocotb.regression Found test test_pipelined_multiplier.test_pipelined_multiplier ↪ → 0.00ns INFO cocotb.regression running test_pipelined_multiplier (1/1) 71.00ns INFO cocotb.pipelined_multiplier Testing a=1, b=1, expected=1 71.00ns INFO cocotb.pipelined_multiplier Current product=00000100,
valid=1
71.00ns INFO cocotb.regression test_pipelined_multiplier failed Traceback (most recent call last): File "/workspace/test/test_pipelined_multiplier.py", line 58, in test_pipelined_multiplier ↪ → assert dut.product.value == expected, \ AssertionError: Product incorrect. For inputs a=1, b=1, expected 1, got 00000100 make[1]: Leaving directory '/workspace/test'
I see the issue now. The pipeline stages are not properly synchronized. Let me modify the Verilog code:
edit /workspace/pipelined_multiplier.v
---/workspace/pipelined_multiplier.v +++ /workspace/pipelined_multiplier.v @@ -13 +13 @@ -reg [7:0] product_pipe2; + reg [7:0] mult_result; @@ -29 +29 @@ -// Stage 2: Perform multiplication and register + // Stage 2: Register inputs again @@ -34 +33,0 @@ -product_pipe2 <= 8'b0; @@ -39 +37,0 @@ -product_pipe2 <= a_pipe1 * b_pipe1; @@ -44 +42 @@ -// Stage 3: Register final product + // Stage 3: Perform multiplication and register @@ -46,0 +45 @@ + mult_result <= 8'b0; @@ -50 +49,2 @@ -product <= product_pipe2; + mult_result <= a_pipe2 * b_pipe2; + product <= mult_result;
Let's run the tests again:
cd /workspace && ./run_test.sh
Pipelined Multiplier
make -f Makefile results.xml make[1]: Entering directory '/workspace/test' /usr/bin/iverilog -o sim_build/sim.vvp -D COCOTB_SIM=1 -s pipelined_multiplier -g2012 -f sim_build/cmds.f /workspace/test/../pipelined_multiplier.v ↪ → rm -f results.xml MODULE=test_pipelined_multiplier TESTCASE= TOPLEVEL=pipelined_multiplier TOPLEVEL_LANG=verilog \ ↪ → /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp ↪ →
result
0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable) 0.00ns INFO cocotb Running tests with cocotb v1.9.2 0.00ns INFO cocotb Seeding Python random module with 1741821402 0.00ns INFO cocotb.regression Found test test_pipelined_multiplier.test_pipelined_multiplier ↪ → 0.00ns INFO cocotb.regression running test_pipelined_multiplier (1/1) 71.00ns INFO cocotb.pipelined_multiplier Testing a=1, b=1, expected=1 71.00ns INFO cocotb.pipelined_multiplier Current product=00000001, valid=1 81.00ns INFO cocotb.pipelined_multiplier Testing a=2, b=2, expected=4 81.00ns INFO cocotb.pipelined_multiplier Current product=00000100, valid=1 91.00ns INFO cocotb.pipelined_multiplier Testing a=3, b=3, expected=9 91.00ns INFO cocotb.pipelined_multiplier Current product=00001001, valid=1 101.00ns INFO cocotb.pipelined_multiplier Testing a=4, b=4, expected=16 101.00ns INFO cocotb.pipelined_multiplier Current product=00010000, valid=1 105.00ns INFO cocotb.pipelined_multiplier Simple sequence test passed! 2695.00ns INFO cocotb.pipelined_multiplier All test cases passed successfully! 2695.00ns INFO cocotb.regression test_pipelined_multiplier passed
make[1]: Leaving directory '/workspace/test'
Great! All tests have passed. Now let's move on to the OpenLane flow. First, let me query the documentation database for OpenLane configuration:
print(query_docs_db("openlane configuration for verilog module hardening"))
To configure OpenLane for hardening a Verilog module, you need to ensure proper alignment between your Verilog design and the OpenLane configuration files. Here are the key steps: ↪ → ↪ → 1. **Pin Configuration**: Ensure that all pins defined in your Verilog file are included in the ` pin_order.cfg ` file. This prevents mismatches during the hardening process. For example, if your Verilog design references pins like ` io_out[6:0] ` and ` io_oeb[6:0] ` , they should be listed in ` pin_order.cfg ` even if not directly used in your design. ↪ → ↪ → ↪ → ↪ → 2. **Macro Integration**: If your design includes macros, declare them in the OpenLane configuration using the "MACROS" variable. This involves specifying the macro's GDS and LEF files, among other details, to ensure they are correctly integrated into the top-level design. ↪ → ↪ → ↪ → 3. **Configuration File**: Create a ` config.json ` file for your design, specifying essential parameters such as ` DESIGN_NAME ` , ` VERILOG_FILES ` , ` CLOCK_PERIOD ` , and ` CLOCK_PORT ` . This file controls the flow and ensures the design is processed correctly. ↪ → ↪ → ↪ → 4. **Handling Synthesis Issues**: Modify the top-level Verilog code to avoid using constants and inversion operators that cannot be synthesized by default. Adjust assignments to prevent unmapped cells during synthesis. ↪ → ↪ → 5. **Routing and Layout**: Ensure that the LEF file does not have obstructions over pins and that the layout is compatible with OpenLane's requirements. Adjust macro placements to resolve routing congestion issues. ↪ → ↪ → By following these guidelines, you can configure OpenLane to effectively harden your Verilog module, ensuring a smooth transition from RTL to a hardened design. ↪ →
Let's create the OpenLane configuration file:
edit /workspace/config.json
---/workspace/config.json +++ /workspace/config.json @@ -1 +1,24 @@ -+{ + "DESIGN_NAME": "pipelined_multiplier", + "VERILOG_FILES": + "CLOCK_PORT": "clk", + "CLOCK_PERIOD": 10.0, + "FP_SIZING": "absolute", + "DIE_AREA": "0 0 100 100", + "PL_TARGET_DENSITY": 0.75, + "FP_PDN_VPITCH": 25, + "FP_PDN_HPITCH": 25, + "PL_RANDOM_GLB_PLACEMENT": 1, + "PL_RESIZER_DESIGN_OPTIMIZATIONS": 1, + "PL_RESIZER_TIMING_OPTIMIZATIONS": 1, + "GLB_RESIZER_TIMING_OPTIMIZATIONS": 1, + "ROUTING_CORES": 8, + "RUN_KLAYOUT_XOR": 0, + "KLAYOUT_XOR_GDS": 0, + "KLAYOUT_XOR_XML": 0, + "RUN_KLAYOUT": 0, + "QUIT_ON_MAGIC_DRC": 0, + "QUIT_ON_TIMING_VIOLATIONS": 0, + "QUIT_ON_HOLD_VIOLATIONS": 0, + "QUIT_ON_SETUP_VIOLATIONS": 0 +}
Now let's run OpenLane:
print(run_openlane('/workspace/config.json'))
OpenLane ran successfully.
Let's check the metrics of the OpenLane run:
print(view_openlane_metrics('/workspace'))
["pipelined_multiplier.v"],
Pipelined Multiplier
Pipelined Multiplier
Pipelined Multiplier
Observation Pipelined Multiplier Metrics for the latest openlane flow run: Metric,Value design__lint_error__count,0 design__lint_timing_construct__count,0 design__lint_warning__count,0 design__inferred_latch__count,0 design__instance__count,264 synthesis__check_error__count,0 design__max_slew_violation__count__corner:nom_ff_n40C_1v95,0 design__max_fanout_violation__count__corner:nom_ff_n40C_1v95,1 design__max_cap_violation__count__corner:nom_ff_n40C_1v95,0 timing__setup__ws__corner:nom_ff_n40C_1v95,7.156481387832111 timing__hold__tns__corner:nom_ff_n40C_1v95,0.0 design__max_slew_violation__count,0 design__max_fanout_violation__count,1 design__max_cap_violation__count,0 clock__skew__worst_hold,-0.25088964949036874 clock__skew__worst_setup,0.2531907810630141 timing__hold__ws,0.12442980331621406 timing__setup__ws,4.672803609646017 timing__hold__tns,0.0 design__io,21 design__die__area,10000 design__core__area,6761.48 design__instance__count__stdcell,264 design__instance__area__stdcell,1954.37 design__instance__count__macros,0 design__instance__area__macros,0 design__instance__utilization,0.289045 flow__warnings__count,1 design__violations,0 design__instance__count__class:timing_repair_buffer,40 design__instance__count__class:clock_buffer,5route__antenna_violation__count,0 antenna_diodes_count,0 route__net,181 route__net__special,2 route__drc_errors__iter:1,82 route__wirelength__iter:2,2322 route__drc_errors,0 route__wirelength,2312 timing__unannotated_net__count__corner:nom_tt_025C_1v80,4 timing__unannotated_net_filtered__count__corner:nom_tt_025C_1v80,0 timing__unannotated_net__count__corner:nom_ss_100C_1v60,4 timing__unannotated_net_filtered__count__corner:nom_ss_100C_1v60,0 design__max_slew_violation__count__corner:min_tt_025C_1v80,0 timing__setup__tns__corner:min_ff_n40C_1v95,0.0 timing__hold__wns__corner:min_ff_n40C_1v95,0 timing__setup_r2r_vio__count__corner:min_ff_n40C_1v95,0 timing__unannotated_net__count__corner:min_ff_n40C_1v95,4 timing__unannotated_net__count__corner:max_ss_100C_1v60,4 timing__unannotated_net_filtered__count__corner:max_ss_100C_1v60,0 design__max_slew_violation__count__corner:max_ff_n40C_1v95,0 design__max_fanout_violation__count__corner:max_ff_n40C_1v95,1 design__max_cap_violation__count__corner:max_ff_n40C_1v95,0 clock__skew__worst_hold__corner:max_ff_n40C_1v95,-0.2513284096421096 clock__skew__worst_setup__corner:max_ff_n40C_1v95,0.25466965368479144 timing__hold__ws__corner:max_ff_n40C_1v95,0.13104157019290902 timing__setup__ws__corner:max_ff_n40C_1v95,7.150778838127147 timing__hold__tns__corner:max_ff_n40C_1v95,0.0 timing__setup__tns__corner:max_ff_n40C_1v95,0.0 timing__hold__wns__corner:max_ff_n40C_1v95,0
design_powergrid__voltage__worst__net:VPWR__corner:nom_tt_025C_1v80,1.79985 design_powergrid__drop__average__net:VPWR__corner:nom_tt_025C_1v80,1.79999 design_powergrid__drop__worst__net:VPWR__corner:nom_tt_025C_1v80,0.000146436 magic__drc_error__count,0 klayout__drc_error__count,0 magic__illegal_overlap__count,0 design__lvs_device_difference__count,0 design__lvs_net_difference__count,0 design__lvs_property_fail__count,0
The OpenLane flow has completed successfully. Let's analyze the key metrics:
- Timing:
- Setup slack: 4.67ns (positive, which is good)
- Hold slack: 0.124ns (positive, which is good)
- No timing violations
- Area:
- Core area: 6761.48 µm²
- Instance utilization: 28.9%
- Power:
- Total power: 0.45mW
- Internal power: 0.34mW
- Switching power: 0.11mW
- Leakage power: 2.74nW
- DRC and LVS:
- No DRC errors
- No LVS errors
- No illegal overlaps
The design has been successfully hardened with good PPA (Power, Performance, Area) metrics. The only minor issue is a single max fanout violation, but it's not critical as the timing is still met.
Let me summarize what we've accomplished:
- Implemented a 4-bit pipelined multiplier with three pipeline stages:
- Stage 1: Register inputs
- Stage 2: Register inputs again
- Stage 3: Perform multiplication and register result
- Created a comprehensive cocotb testbench that:
- Tests all possible input combinations (256 cases)
- Verifies pipeline functionality
- Checks valid signal and product correctness
- All tests pass successfully
- Successfully hardened the design using OpenLane:
- No timing violations
- Good area utilization
- Low power consumption
- Clean DRC and LVS
The design meets all the requirements specified in the task and has been verified both functionally and physically.
Agent Finished Task. Running Evaluation script...
MAIN MODULE CHECKPOINTS - TOTAL 7 PTS
-
- Interface & Modularity (1 pt)
Pipelined Multiplier
Evaluating RTL
- The code implements the following interface:
- -clk -reset
- -a (4-bit input) b (4-bit input)
- -product (8-bit output) valid (output)
-
- Pipeline Stages (1 pt)
- The code implements the pipeline structure.
-
- Multiplication Logic (2 pts)
- The code implements shifting for partial products (1 pt)
- The code accumulates partial products correctly (1 pt)
-
- Control Logic (3 pts)
- The code implements control mechanisms:
- -The code tracks pipeline validity (1 pt)
- -The code handles stall conditions (1 pt)
- -The code manages data flow between stages (1 pt)
-
- Interface & Modularity Score: 1/1
-
- Pipeline Stages Score: 1/1
-
- Multiplication Logic Score: 0/2
-
- Control Logic Score: 2/3
Final Score: 4/7
-
- Zero Multiplication (1 pt)
- The code tests zero multiplication.
-
- Boundary Values (3 pts)
- The testbench code tests:
- -The code tests maximum values (15 × 15) (1 pt)
- -The code tests minimum values (0 × N) (1 pt)
- -The code tests single-bit multiplications (1 pt)
-
- Signal Propagation (1 pts)
- The testbench code tests:
- -The code tests valid signal propagation
-
- Exhaustive Testing (1 pt)
-
The code tests all input combinations.
-
- Zero Multiplication Score: 1/1
-
- Boundary Values Score: 3/3
-
- Signal Propagation Score: 1/1
-
- Exhaustive Testing Score: 1/1
Final Score: 6/6
cd /workspace && ./run_test.sh
Command cd /workspace && ./run\_test.sh executed with exit code 0
Great! All tests have passed.
Testbench Execution
Final Score: 1/1
Locating configuration file... Locating OpenLane GDS
layouts...
The OpenLane flow has been successfully completed, yielding the final GDSII layout.
Final Score: 2/2
-
Main Module: 15
-
Test Bench: 15
-
Testbench Execution: 20
-
OpenLane Flow: 50
Physical Layout Verification
Overall Score Calculation