How to Use cocotb for HDL Verification

How to Use cocotb for HDL Verification

Version: 1.0

Date: 2025-01-20 Status:

Draft

This guide explains how we use cocotb and pytest for HDL verification in this project.

1. Overview

What is cocotb?

cocotb (Coroutine-based Co-simulation Test Bench) is a Python library for testing HDL designs. Instead of writing testbenches in SystemVerilog or VHDL, you write them in Python.

Why Python? - Easier to write complex test scenarios - Rich ecosystem of libraries (numpy, PIL, etc.) - Familiar to software engineers - No compilation step for testbench changes

How it works:

<!-- image -->

cocotb connects to the simulator via VPI (Verilog Procedural Interface) and lets Python code drive signals and read values from your HDL design.

2. Key Concepts

pytest Basics

We use pytest as our test runner. pytest discovers and runs tests automatically.

Test discovery: pytest finds tests by looking for: - Files named test_*.py or _test.py - Functions named test_ inside those files

Running pytest:

# Run all tests pytest # Run specific test file pytest test/test_PanelInterface.py # Run specific test function pytest test/test_PanelInterface.py::test_panel_interface_icarus_runner

Further reading: pytest documentation

pytest.ini

The project includes a pytest.ini file at the repository root that configures pytest defaults:

[pytest] testpaths = test python_files = test_*.py python_classes = Test* python_functions = test_*

This means: No --rootdir needed - pytest automatically looks in test/ for tests Test files must be named test_.py - T est functions must be named test_

Fixtures

Fixtures provide test configuration and setup. They are functions that pytest injects into your tests.

Our project defines these fixtures in test/conftest.py :

| Fixture | Purpose | Command-line flag | |-----------|-------------------------------|---------------------| | waves | Enable waveform generation | --waves=True | | buildonly | Compile only, don't run tests | --build-only=True | | coverage | Enable coverage collection | --coverage=True |

How fixtures work:

# In conftest.py - fixture definition @pytest.fixture def waves(request): wave_option = request.config.getoption("--waves") return wave_option.lower() == "true" # In test file - fixture usage def test_my_runner(waves, buildonly): # pytest automatically passes fixture values

if waves: print("Waveforms enabled!")

Further reading: pytest fixtures

The cocotb Runner Pattern

cocotb tests use a two-layer pattern :

• cocotb tests ( @cocotb.test() ) - The actual stimulus and checking logic 1.
• pytest runners - Launch the simulator and run cocotb tests 2.

┌─────────────────────────────────────────────────────────────┐ │ Test Execution Flow │ │ │ │ pytest cocotb │ │ ────── ────── │ │ │ │ 1. test_xxx_runner() ──→ 2. runner.build() │ │ (pytest function) (compile HDL) │ │ │ │ ──→ 3. runner.test() │ │ (start simulator) │ │ │ │ ──→ 4. @cocotb.test() │ │ (run test logic) │ │ │ │ ←── 5. pass/fail result │ └─────────────────────────────────────────────────────────────┘

Example runner function:

@pytest.mark.iverilog # Use Icarus Verilog simulator @pytest.mark.parametrize("testcase", [ "test_basic_datapath", "test_8_lines", ]) def test_datapath_iverilog_runner(testcase, waves, buildonly): # Get simulator runner runner = get_runner("icarus") # Build (compile) HDL runner.build( sources=sources, hdl_toplevel="MyModule", build_dir=build_dir, ) # Run cocotb test (unless build-only mode) if not buildonly: runner.test( hdl_toplevel="MyModule", test_module="test_my_module",

testcase=testcase, )

Further reading: cocotb runner documentation

3. Architecture

Project Test Structure

test/ ├── conftest.py # Fixtures and runner base classes ├── devices/ # Device models (behavioral) │ ├── panel.py # DetectorPanel model │ ├── roic.py # ROIC (AFE2256) model │ ├── gate_driver.py # Gate driver model │ └── io_expander.py # I2C IO expander model ├── unit/ # Unit tests (single module) │ ├── test_AsyncFifo.py │ ├── test_PanelBitAlignment.py │ └── ... ├── test_PanelInterface.py # Integration tests ├── test_DetectorPrototypeFPGATop.py └── ...

Device Models

Device models are behavioral Python models of external hardware. They respond to signals from the DUT just like real hardware would.

<!-- image -->

4. Test Inventory

Simulators

| Simulator | Marker | Use Case | |----------------|------------------------|-----------------------------------------| | Icarus Verilog | @pytest.mark.iverilog | Open-source, good SystemVerilog support | | Verilator | @pytest.mark.verilator | Fast, no Intel IP support | | Questa | @pytest.mark.questa | Full Intel IP support, commercial |

Integration Tests

PanelInterface ( test_PanelInterface.py )

DUT: PanelInterface.sv | Simulator: Icarus Verilog

The most comprehensive testbench - validates the complete detector panel system with all device models.

<!-- image -->

│ └─────────────────────────────────────────────────────────────────────┘ │

└─────────────────────────────────────────────────────────────────────────────┘

Device models used: -DetectorPanel - Orchestrates pixel delivery across all ROICs and gate drivers Roic (x12) - AFE2256 ROIC models with SPI slave, LVDS outputs, test patterns GateDriver (x6) - NT39522DH gate driver models for line scanning TCA9535 - I2C IO expander for ROIC chip select control

Tests:

| Test Name | Description | |--------------------------|-------------------------------------------------| | test_basic_timing | Basic MCLK, SYNC timing and gate driver signals | | test_spi_broadcast | SPI broadcast configuration to all 12 ROICs | | test_full_frame_capture | Complete frame capture with all device models | | test_axis_output | AXI Stream datapath output validation | | test_continuous_scanning | Multi-frame continuous operation | | test_alignment_vector | Bit alignment vector verification | | test_config_mode_0 | CONFIG_MODE=0 (LSB-first with CRC) | | test_config_mode_1 | CONFIG_MODE=1 (MSB-first fixed pattern) |

PanelInterfaceDatapath ( test_PanelInterfaceDatapath.py )

DUT: DatapathFifoTestbench.sv (wrapper) | Simulator: Icarus Verilog

Tests the datapath from FIFO write interface through AXI Stream output, bypassing deserializer/alignment.

<!-- image --> <!-- image -->

Tests:

| Test Name | Description | |---------------------|----------------------------------------------------------| | test_basic_datapath | Write 3 lines, validate interleave→sequential conversion | | test_8_lines | Extended test with 8 lines for line counter handling |

AlignmentVector ( test_alignment_vector.py )

DUT: roic_test_wrapper.sv | Simulator: Icarus Verilog

Verifies AFE2256 alignment vector functionality per Section 6.3.1.3.2 (T able 6-4).

<!-- image -->

Alignment Vector Formats:

| CONFIG_MODE | Format | Bits | |---------------|-------------------------|---------------------| | 1 (MSB-first) | [Y X 1 1 1 0 0 0] | Y=ch0, X=BankA/B | | 0 (LSB-first) | [Y X W 0 0 0 CRC1 CRC0] | W=TP_SEL, CRC=2-bit |

Tests:

| Test Name | Description | |------------------------------------------|----------------------------------| | test_config_mode_1_y_bit_channel_0 | Y bit = 1 for channel 0 | | test_config_mode_1_x_bit_bank_toggle | X bit toggles Bank A/B each scan | | test_config_mode_1_fixed_pattern | Pattern bits [5:0] = 0x38 | | test_config_mode_0_w_bit_tp_sel_latching | Wbit reflects latched TP_SEL | | test_config_mode_0_crc_calculation | CRC[1:0] over 96-bit window | | test_config_mode_0_zero_bits | Bits [4:2] always 0 |

DetectorPrototypeFPGATop ( test_DetectorPrototypeFPGATop.py )

DUT: DetectorPrototypeFPGATop.sv | Simulator: Questa (Intel IP)

Smoke test for the full FPGA top-level design with Intel IP cores.

<!-- image -->

Note:

Requires Questa simulator for Intel IP support.

Tests:

| Test Name | Description | |------------------------------|-------------------------------------------| | detector_fpga_top_simple | Basic smoke test - verify simulation runs | | detector_fpga_top_clock_test | Clock generation verification |

DDR FF Ordering ( test_ddr_ff_ordering.py )

DUT: ddr_ff_test_wrapper.sv | Simulator:

Questa (Intel IP)

Investigates Intel DDR FF IP bit ordering (posedge vs negedge sampling).

<!-- image -->

Tests:

| Test Name | Description | |-----------------------|---------------------------------------------| | test_ddr_bit_ordering | Compares Intel vs behavioral DDR FF outputs |

SimpleTop ( test_SimpleTop.py )

DUT: SimpleTop.sv | Simulator: Questa (Intel IP)

Simple adder with Intel IP - used as a reference for Questa integration.

<!-- image -->

Tests:

<!-- image -->

| Test Name | Description | |-------------------------|------------------------------------------| | my_test_simple | Basic signal access test | | my_test_simple_testfail | Intentional failure test (commented out) |

SimpleTopNoIP ( test_SimpleTopNoIP.py )

DUT: SimpleTopNoIP.sv | Simulator: Verilator

Pipelined adder without Intel IP - primary test for Verilator/CI.

<!-- image --> <!-- image -->

Tests:

| Test Name | Description | |-------------------------|--------------------------------------| | my_test_simple | Basic signal access | | my_test_simple_testfail | Intentional failure (commented out) | | adder_random_test | 10,000 random input combinations | | adder_edge_cases_test | Edge cases: 0+0, 255+255, boundaries |

ROIC ADC Output ( test_roic_adc_output.py )

DUT: (requires wrapper) | Simulator: Skipped (needs HDL wrapper)

Tests continuous ADC output behavior - verifies scanning continues without repeated SYNC pulses.

<!-- image -->

Tests:

| Test Name | Description | |------------------------------------|------------------------------------| | test_adc_output_continuous_no_sync | Verify scans continue without SYNC |

Status: Currently skipped - requires minimal HDL wrapper.

ROIC FCLK Timing ( test_roic_fclk_timing.py )

DUT: roic_fclk_testbench.sv | Simulator: Icarus Verilog (skipped)

Tests FCLK timing behavior - verifies 50% duty cycle clock with STR-based frequency scaling.

<!-- image -->

Tests:

| Test Name | Description | |----------------------------------------|---------------------------| | test_fclk_equals_mclk_when_str0 | FCLK = MCLK when STR=0 | | test_fclk_rising_edge_aligns_with_dclk | FCLK/DCLK phase alignment | | test_fclk_frequency_scales_with_str | FCLK = MCLK / 2^STR |

Status: Currently skipped - FCLK scaling causes regressions.

Unit Tests

Located in test/unit/ . These test individual modules in isolation.

| Test File | DUT | Description | |----------------------------|-----------------------|----------------------| | test_AsyncFifo.py | AsyncFifo.sv | Gray-code CDC FIFO | | test_PanelBitAlignment.py | PanelBitAlignment.sv | Bit alignment logic | | test_PanelWordAlignment.py | PanelWordAlignment.sv | Word alignment logic | | test_PanelFifoMux.py | PanelFifoMux.sv | FIFO multiplexer | | test_PanelAxiCrossbar.py | PanelAxiCrossbar.sv | AXI crossbar |

5. Running Tests

Basic Commands

# Run all tests pytest # Run specific test file pytest test/test_PanelInterface.py # Run with waveforms (generates .fst files) pytest --waves=True # Build only (compile HDL, don't run simulation) pytest --build-only=True # Verbose output pytest -v # Show print statements pytest -s # Combined: verbose with print output pytest -sv

Running a Single Test

To run a specific cocotb test, you need both the runner function AND the -k filter:

# Pattern: pytest <file>::<runner_function> -k <testcase_name> pytest test/test_PanelInterface.py::test_panel_interface_icarus_runner -k basic_capture # More examples:

pytest test/test_SimpleTopNoIP.py::test_simple_top_no_ip_verilator_runner -k adder_random_test

pytest test/test_alignment_vector.py::test_alignment_vector_iverilog_runner -k test_config_mode_1

Why both? The runner function ( test_panel_interface_icarus_runner ) is parametrized with multiple testcases. Without -k , pytest runs ALL parametrized variants. The -k filter selects which testcase to run.

How Tests are Registered

Each cocotb test ( @cocotb.test() ) must be added to the runner's testcases list:

@pytest.mark.iverilog @pytest.mark.parametrize("testcase", [ "basic_startup", # Always runs "register_read_write_test", # Always runs pytest.param("full_capture", marks=pytest.mark.exclude_from_ci), # Excluded from CI ]) def test_panel_interface_icarus_runner(testcase, waves, buildonly): # ... runner code ... runner.test( hdl_toplevel=top_module, test_module="test_PanelInterface", testcase=testcase, # <-- This runs the specific cocotb test )

To add a new test: 1. Write the @cocotb.test() function 2. Add its name to the testcases list in @pytest.mark.parametrize

The exclude_from_ci Marker

Long-running tests can be excluded from CI using pytest.param() with the exclude_from_ci mark:

@pytest.mark.parametrize("testcase", [ "quick_test", # Runs in CI (~30s) pytest.param("slow_test", marks=pytest.mark.exclude_from_ci), # Skipped in CI (~10min) ])

Running tests:

# Run all tests EXCEPT those marked exclude_from_ci (used in CI) pytest -m "not exclude_from_ci" # Run ONLY excluded tests (for local development) pytest -m exclude_from_ci # Run everything including excluded tests pytest

Run by Simulator

# Run only Icarus Verilog tests pytest -m iverilog # Run only Verilator tests pytest -m verilator # Run only Questa tests pytest -m questa # Run only unit tests pytest -m unit # Combine markers: Icarus tests excluding CI-excluded pytest -m "iverilog and not exclude_from_ci"

Viewing Waveforms

After running with --waves=True , waveform files are in the build directory:

# Find waveform files find test/ -name "*.fst" -o -name "*.vcd" # Open with GTKWave gtkwave test/sim_build_xxx/dump.fst

6. Adding New Tests

This section shows how to add a new test to an existing runner. We'll use test_PanelInterfaceDatapath.py as an example.

Step 1: Understand the File Structure

A test file has two parts:

test_PanelInterfaceDatapath.py ├── TB class # Testbench helper (clocks, reset, stimulus methods) ├── @cocotb.test() # Actual test functions (async) │ ├── test_basic_datapath │ └── test_8_lines └── @pytest runner # Launches simulator with testcase parameter └── testcases list # <-- Add your test name here

Step 2: Write the cocotb Test Function

Add your test function before the pytest runner:

# ============================================================================= # Cocotb Tests

# ============================================================================= @cocotb.test() async def test_basic_datapath(dut): """Existing test.""" tb = TB(dut) await tb.reset() # ... test logic ... @cocotb.test() async def test_8_lines(dut): """Existing test.""" tb = TB(dut) await tb.reset() # ... test logic ... @cocotb.test() async def test_stress_100_lines(dut): # <-- NEW TEST """Stress test: Write 100 lines to verify sustained operation. Verifies: - No FIFO overflow under sustained load - No data corruption over long runs - Memory/resource stability """ tb = TB(dut) await tb.reset() NUM_LINES = 100 tb.log.info(f"Starting stress test with {NUM_LINES} lines") validation_task = cocotb.start_soon(tb.validate_output(NUM_LINES)) for line in range(NUM_LINES): await tb.write_line(line) await Timer(500, units='us') errors = await validation_task assert len(errors) == 0, f"Stress test failed with {len(errors)} errors" tb.log.info("test_stress_100_lines PASSED")

Step 3: Register in the Testcases List

Add your test name to @pytest.mark.parametrize :

# ============================================================================= # Pytest Runner # ============================================================================= @pytest.mark.iverilog @pytest.mark.parametrize("testcase", [ "test_basic_datapath",

"test_8_lines", "test_stress_100_lines", # <-- ADD HERE ]) def test_datapath_iverilog_runner(testcase, waves, buildonly): """Parametrized runner for PanelInterfaceDatapath tests.""" # ... runner code (no changes needed) ...

For long-running tests, use pytest.param with exclude_from_ci :

@pytest.mark.parametrize("testcase", [ "test_basic_datapath", "test_8_lines", pytest.param("test_stress_100_lines", marks=pytest.mark.exclude_from_ci), # <-- EXCLUDED FRO ])

Step 4: Run Your Test

# Run only your new test pytest test/test_PanelInterfaceDatapath.py::test_datapath_iverilog_runner -k test_stress_100_line # Run with waveforms for debugging pytest test/test_PanelInterfaceDatapath.py::test_datapath_iverilog_runner -k test_stress_100_line # Run all tests in the file pytest test/test_PanelInterfaceDatapath.py -v

Test Function Guidelines

@cocotb.test() async def test_descriptive_name(dut): """One-line summary. Verifies: - First thing being tested - Second thing being tested - Expected behavior or edge case """ # 1. Setup testbench tb = TB(dut) await tb.reset() # 2. Apply stimulus await tb.write_data(test_pattern) # 3. Wait for processing await Timer(100, units='us') # 4. Check results errors = await tb.validate() assert len(errors) == 0, f"Test failed: {errors}" # 5. Log success tb.log.info("test_descriptive_name PASSED")

7. Controlling Test Execution

Skip a Test

Use @pytest.mark.skip to skip a test entirely:

@pytest.mark.skip(reason="Not implemented yet") @pytest.mark.iverilog def test_my_runner(waves, buildonly): ...

Skip Conditionally

Use @pytest.mark.skipif to skip based on a condition:

import sys @pytest.mark.skipif(sys.platform == "win32", reason="Linux only") def test_my_runner(waves, buildonly): ...

Exclude from CI

Use @pytest.mark.exclude_from_ci for tests that should not run in CI:

@pytest.mark.exclude_from_ci @pytest.mark.iverilog def test_long_running_simulation(waves, buildonly): # This test takes too long for CI ...

To run excluding CI-excluded tests:

pytest -m "not exclude_from_ci"``` ### Expected Failures Use `@pytest.mark.xfail` for tests that are expected to fail: ```python @pytest.mark.xfail(reason="Known bug in DUT, fix in progress") @pytest.mark.iverilog def test_known_issue(waves, buildonly): ...

Summary of Markers

| Marker | Effect | Use Case | |-----------------------------------------|------------------------|---------------------| | @pytest.mark.skip(reason="...") | Always skip | Broken or not ready | | @pytest.mark.skipif(cond, reason="...") | Skip if condition true | Platform-specific | | @pytest.mark.xfail(reason="...") | Expected to fail | Known bugs | | @pytest.mark.exclude_from_ci | Skip in CI runs | Long-running tests |

Further reading: pytest markers

8. Quick Reference

# Run all tests pytest # Run specific file pytest test/test_MyModule.py # Run single cocotb test (requires -k filter) pytest test/test_PanelInterface.py::test_panel_interface_icarus_runner -k basic_capture # Run with waveforms pytest --waves=True # Build only (no simulation) pytest --build-only=True # Run by simulator pytest -m iverilog pytest -m verilator pytest -m questa # Run unit tests only pytest -m unit # Exclude CI-excluded tests (used in CI) pytest -m "not exclude_from_ci" # Run ONLY CI-excluded tests (local development) pytest -m exclude_from_ci # Combine markers pytest -m "iverilog and not exclude_from_ci"

# Verbose with print output pytest -sv

9. Further Reading

Official Documentation

• cocotb documentation ·
• cocotb triggers ·
• cocotb runner ·
• pytest documentation ·
• pytest fixtures ·
• pytest markers ·

cocotb Extensions Used

• cocotbext-axi - AXI, AXI-Lite, AXI-Stream ·
• cocotbext-spi - SPI master/slave ·

Project-Specific

• pytest.ini - pytest configuration (test paths, naming conventions) ·
• test/conftest.py - Fixtures and runner base classes ·
• test/devices/ - Device models (Panel, ROIC, Gate Driver) ·
• docs/oxos_coding_style_standard.md - HDL coding standards ·

Sources

• cocotb GitHub ·
• pytest documentation ·
• Icarus Verilog ·
• Verilator ·