Three engineers gathered around a laptop at a lab bench with equipment around them
Technology Nov 10, 2026 • 14 min read

Open Silicon, Hands-On, Part 4: Co-Simulation, or Trust but Diff

A self-checking Verilator testbench that diffs every register after every instruction against the model, then gets attacked with sixteen deliberate bugs.

Share:
Lee Foropoulos

Lee Foropoulos

14 min read

Continue where you left off?
Text size:

Contents

Open Silicon, Hands-On | Part 3: RTL That Synthesizes → Part 4: Co-Simulation → Part 5: Bus and Formal

Part 3 ended with RTL that lints clean, which tells you nothing about whether it's right. The next question is the one that matters: does the hardware do what the model says?

The approach is a diff, not a test case. The Python model writes, for each of 501 programs, the expected state after every instruction. A C++ testbench built by Verilator loads each program into the RTL, runs it, and compares the program counter, accumulator, flag and halt state at every retired instruction, stopping at the first difference.

Then we'll do the thing most tutorials skip: grade the testbench by breaking the design on purpose.

A woman wearing clear safety goggles looking off to the side
Goggles on. Everything after the first section is about trying to break the teaching design in ways a testbench ought to notice.

The architecture: a diff, not a test case

Here's the whole setup in one picture.

text
1model/acc8_model.py ──► tb/cases.txt ──► tb_cosim (C++ from Verilator) ──► acc8_cpu (the RTL)
2  expected state          one file         loads each program, runs it,       retire pulse plus
3  after every step                         compares at every retire           pc, acc, zf, halted

The retire output is a one-clock pulse that arrives the clock after an instruction completes, while the architectural outputs show the post-instruction state. Real cores expose similar retire interfaces for exactly this job. The testbench enforces six rules:

  1. After reset, every architectural register equals the model's reset state, whatever the flops held before.
  2. Programs load through the host port while the core is idle, with no back doors.
  3. After each retire pulse, PC, ACC, ZF and the halt state equal the model's values.
  4. The first retire comes exactly five clocks after start, and each later one exactly four clocks after the last.
  5. A halted core stays quiet, and memory read back through the host port matches the model's final memory.
  6. Every seventh case is aborted mid-run by a reset, and the machine must return to its reset state.
Check state after every step. A testbench that only checks the final answer finds bugs late and explains them badly.
A hand-drawn line chart on a notebook page next to a ruler and a pen
Sketch the handshake before you code it. Ours is five clocks to the first retire, then four per instruction. The testbench is that sketch, made executable.

The testbench, in two excerpts

Verilator is cycle-based: you set inputs, call eval(), flip the clock, and call eval() again. The small Tb struct below wraps that into tick() (one full clock period) and gives the test three verbs: reset, load a program through the host port, and peek at a memory byte.

cpp
1struct Tb {
2  VerilatedContext* ctx;
3  std::unique_ptr<Vacc8_cpu> top;
4  VerilatedVcdC* tfp = nullptr;
5  uint64_t cycles = 0, t = 0;
6
7  explicit Tb(VerilatedContext* c) : ctx(c), top(new Vacc8_cpu(c)) {
8    top->clk = 0; top->rst = 0; top->start = 0;
9    top->host_we = 0; top->host_addr = 0; top->host_wdata = 0;
10    top->eval();                                    // let the garbage show before reset
11  }
12  void tick() {                                     // one full clock period
13    top->clk = 0; top->eval(); if (tfp) tfp->dump(t); t += 5;
14    top->clk = 1; top->eval(); if (tfp) tfp->dump(t); t += 5;
15    cycles++;
16  }
17  void reset(int n) { top->rst = 1; for (int i = 0; i < n; i++) tick(); top->rst = 0; }
18  void load(const uint8_t* mem) {
19    for (int i = 0; i < 16; i++) { top->host_we = 1; top->host_addr = i; top->host_wdata = mem[i]; tick(); }
20    top->host_we = 0;
21  }
22  uint8_t peek(int addr) { top->host_addr = addr; tick(); return top->host_rdata; }
23};

The heart of the check is the loop that runs after a program is loaded. since counts clocks since the start edge or the last retire, so a timing deviation fails exactly like a wrong value does.

cpp
1    // 2. load the program through the host port, start, and compare after EVERY instruction
2    tb.load(c.mem);
3    tb.top->start = 1; tb.tick(); tb.top->start = 0;
4    uint64_t since = 1;                             // clocks since the start edge (or the last retire)
5    for (size_t i = 0; i < c.steps.size(); ) {
6      if (++since > 5) fail(c, i + 1, "timeout: no retire within 5 clocks", 0, 1, tb.cycles);
7      tb.tick();
8      if (!tb.top->retire) continue;
9      const Step& m = c.steps[i];
10      const size_t n = i + 1;
11      if (since != (i == 0 ? 5u : 4u)) fail(c, n, "clocks per instruction", (unsigned)since, i == 0 ? 5 : 4, tb.cycles);
12      if (tb.top->pc     != m.pc)     fail(c, n, "pc",     tb.top->pc,     m.pc,     tb.cycles);
13      if (tb.top->acc    != m.acc)    fail(c, n, "acc",    tb.top->acc,    m.acc,    tb.cycles);
14      if (tb.top->zf     != m.zf)     fail(c, n, "zf",     tb.top->zf,     m.zf,     tb.cycles);
15      if (tb.top->halted != m.halted) fail(c, n, "halted", tb.top->halted, m.halted, tb.cycles);
16      since = 0; i++; total_steps++;
17    }

The first mismatch stops everything and reports the case, step, field and both values, so no waveform is needed to know where to look. Building it is one command, wrapped in a script.

bash
1verilator --cc --exe --build -Wall $VFLAGS --trace --x-initial unique \
2  --Mdir "$OBJ" --top-module acc8_cpu \
3  rtl/acc8_isa_pkg.sv rtl/acc8_mem.sv "$CORE" rtl/acc8_cpu.sv \
4  tb/tb_cosim.cpp -o tb_cosim

And here's the real run: 501 programs, every retired instruction compared.

text
1+ obj_cosim/tb_cosim tb/cases.txt +verilator+rand+reset+2 +verilator+seed+1
2cosim: 501 cases, 10188 instructions compared, 62472 clocks, 382 final memories checked, 72 mid-run resets, random init ON, 0 mismatches
3PASS
4exit=0
10,188
retired instructions compared against the model, register by register, across 501 programs, in 12 milliseconds of wall time

Start every flop in a random state

Verilator's default is to start every flop at zero. That default is quietly the most dangerous setting in the tool, because a missing reset is invisible when everything starts at zero. Build the model with --x-initial unique, run it with +verilator+rand+reset+2, and every flop starts random instead. Add a seed with +verilator+seed+N and the run stays reproducible.

One subtlety: Verilator applies the seed when a model is constructed, so 501 models give you 501 identical garbage states. The fix is a one-line reseed before every case, ctx->randSeed(base_seed + c.id + 1). Now each program starts from different garbage, as the run log shows.

text
1case 0 powers up as: pc=2 acc=54 zf=1 halted=0 busy=1 retire=1
2case 1 powers up as: pc=A acc=4F zf=0 halted=0 busy=0 retire=1
3case 2 powers up as: pc=D acc=F6 zf=0 halted=0 busy=0 retire=0

Remember the latch bug from Part 3? Here's the same testbench run against that broken core, first with flops at zero and then random.

text
1+ tb_cosim tb/cases.txt +nomid                                  (flops start at zero)
2cosim: 501 cases, 10188 instructions compared, 59439 clocks, 382 final memories checked, 0 mid-run resets, random init off, 0 mismatches
3PASS
4+ tb_cosim tb/cases.txt +verilator+rand+reset+2 +verilator+seed+1   (flops start random)
5case 1 powers up as: pc=A acc=4F zf=0 halted=0 busy=0 retire=1
6FAIL case 1 step 1: acc dut=0xF0 model=0x00 (after 23 cycles)

The broken design passes 10,188 instructions with zeroed flops. With random initial state it fails on case 1, whose first instruction is a JZ that never writes the accumulator: the latched control bit powers up as 1 and quietly loads garbage into it.

Zero init hides missing resets

If your simulator defaults to zero or to a fixed pattern, "passes in simulation" can mean "the bug and the initial state happen to cancel out." Randomize every flop at power-up, reseed per test, log the seed, and treat a reset that only matters at power-up as an untested reset.

A missing reset is invisible when everything starts at zero. Randomize the start, or pay for it in silicon.

Who tests the testbench?

A passing testbench only says the design and the testbench agree. To learn whether that agreement means anything you have to try to fool it, and the discipline for that is mutation testing: make a tiny deliberate change to the RTL, rerun the testbench, and demand that it fails. If it doesn't, you've found a hole in the testbench, not a feature of the design.

I wrote sixteen mutants as one-line textual edits of the core: missing resets, a subtract that forgets its +1, a jump with an inverted condition, a store to the wrong address, a retire pulse a clock early, and so on. Each runs against two testbenches. The naive one starts flops at zero and resets once. The random one starts them random. The full testbench adds mid-run resets, and its results matched the random column for all sixteen.

Mutant and what it breaksNaiveRandom initFirst failure, full testbench
zf_no_reset: ZF not resetsurvivescaughtcase 0 step 0: zf after reset
acc_no_reset: ACC not resetsurvivescaughtcase 0 step 0: acc after reset
state_no_reset: FSM state not resetsurvivescaughtcase 0 step 0: busy after reset
retire_no_reset: retire flag not resetsurvivescaughtcase 0 step 0: retire after reset
sub_no_carry_in: SUB forgets the +1caughtcaughtcase 0 step 7: acc 0x03, model 0x04
jz_inverted: JZ jumps when ZF is clearcaughtcaughtcase 0 step 9: pc 0x0A, model 0x09
sta_sets_zf: STA also updates ZFcaughtcaughtcase 0 step 2: zf 1, model 0
zf_stale: ZF from the old accumulatorcaughtcaughtcase 0 step 1: zf 1, model 0
reserved_is_nop: reserved opcodes do nothingcaughtcaughtcase 3 step 1: halted 0, model 1
sta_wrong_addr: STA writes at the PCcaughtcaughtcase 0 step 3: acc 0x0E, model 0x00
ldi_sign_extends: LDI sign-extendscaughtcaughtcase 1 step 2: acc 0xFC, model 0x0C
jmp_off_by_one: JMP lands one byte latecaughtcaughtcase 0 step 10: pc 0x03, model 0x02
halt_not_sticky: HALT returns to IDLEcaughtcaughtcase 0: not quiet after halt
retire_early: retire a clock earlycaughtcaughtcase 0 step 1: clocks per instruction 4, expected 5
ir_captured_late: instruction register loads a clock latecaughtcaughtcase 0 step 1: acc 0x0F, model 0x05
dead_state_default: no recovery from unused FSM codessurvivessurvivesnone: 0 mismatches

The naive testbench caught 11 of 16, and the four it missed are exactly the four missing resets. Random initial state caught 15 of 16, and 13 of those died on the very first program. A directed program as ordinary as the sum loop caught the subtract bug at step 7.

11
of 16 deliberate bugs caught by the naive testbench with zero-initialized flops; random initial state raised it to 15
A scientist in a white coat working at a lab bench with glassware
Mutation testing is a controlled experiment: change one variable, then check whether the instrument notices.

A survivor is information

The one survivor, dead_state_default, removes the recovery path from the two unused FSM encodings. No stimulus through the ports can reach those states, so no testbench can ever tell the difference. Part 5 proves it formally: dead code by construction, which is a decision to make, not a bug to hunt.

A testbench you haven't tried to fool is a testbench you haven't tested.

"Passing tests tell you the testbench and the design agree. Mutation tests tell you whether that agreement means anything."

Waveforms without leaving the terminal

When the diff says where, a waveform tells you why. Verilator writes VCD with --trace, a plain-text standard every viewer reads, and the compact FST format with --trace-fst, which I did not run here. GTKWave is the classic viewer and Surfer a newer one; neither is installed on this machine.

So I wrote a 79-line script that turns a VCD into one table row per clock. It's a poor man's viewer that works over SSH. Here are the first thirteen clocks of the sum program, starting from the cycle where start is high.

text
1cyc     start   state_q  mem_addr mem_rdata      ir_q        pc       acc        zf    retire
2  0         1         0         2        9A        42         0         0         0         0
3  1         0         1         0         0        42         0         0         0         0
4  2         0         2         2        45        42         0         0         0         0
5  3         0         3         5         F        45         1         0         0         0
6  4         0         4         5         E        45         1         0         0         0
7  5         0         1         1         E        45         1         5         0         1
8  6         0         2         5        3E        45         1         5         0         0
9  7         0         3         E         E        3E         2         5         0         0
10  8         0         4         E         0        3E         2         5         0         0
11  9         0         1         2         0        3E         2         5         0         1
12 10         0         2         E         F        3E         2         5         0         0
13 11         0         3         F         5         F         3         5         0         0
14 12         0         4         F         0         F         3         5         0         0
15...

Read it down the state_q column: 1 is FETCH, 2 is LATCH, 3 is OPERAND, 4 is EXEC. The first instruction is LDI 5 (byte 0x45), visible in mem_rdata during the LATCH clock. In row 5 the accumulator becomes 5 and retire pulses, five rows after start. The next two retires land in rows 9 and 13, exactly four apart. Row 0 shows random power-up state at work, too: ir_q has no reset, so it still holds garbage (0x42), and mem_rdata holds 0x9A, the old random contents of the last cell the loader wrote.

5
clocks from start to the first retire, then exactly 4 between instructions, a timing contract the testbench checks on every one of the 10,188 steps
A laptop screen showing a dense line chart over a grid
A waveform is a graph with a story. The story is when each signal moves relative to the clock edge.

The Python alternative: cocotb

If C++ isn't your language, cocotb lets you write the same testbench as Python coroutines on top of a simulator. Not run here: cocotb isn't installed on this machine and its API shifts a little between versions, so treat the sketch as a map, not tested code.

python
1# not run here: the same check, sketched with cocotb
2import cocotb
3from cocotb.clock import Clock
4from cocotb.triggers import RisingEdge
5
6@cocotb.test()
7async def sum5_matches_model(dut):
8    cocotb.start_soon(Clock(dut.clk, 10, "ns").start())
9    steps = load_expected("cases.txt", case=0)      # your parser for the model's file
10    # ... reset, load the program through the host port, pulse start ...
11    for n, want in enumerate(steps, 1):
12        await RisingEdge(dut.clk)
13        while int(dut.retire.value) != 1:
14            await RisingEdge(dut.clk)
15        assert int(dut.acc.value) == want.acc, f"acc mismatch at step {n}"

The trade is simple. The C++ testbench runs 501 programs in a few hundredths of a second because nothing crosses a language boundary per signal. cocotb is nicer to write and extend, and it can import the Python model directly instead of reading a file.

Pick the testbench by the bug you expect

For a regression of ten thousand instructions that must finish in a blink, write C++ against Verilator. For protocol-heavy tests, constrained-random stimulus, or a model you'd rather import than serialize, write cocotb. Nothing stops you doing both.

Before you move on

You now have the loop that makes everything else cheaper: change the RTL, rerun a 12-millisecond regression, read a one-line verdict. Next, a standard bus goes in front of the CPU, along with a kind of checking that doesn't depend on picking the right stimulus at all.

Part 4 Checklist 0/6

In Part 5 the CPU becomes a Wishbone slave that software drives like any other peripheral. A formal tool then finds a protocol bug that a polite testbench never tripped, and a solver hands back the shortest possible way to raise the interrupt.

How was this article?

Share

Link copied to clipboard!

You Might Also Like

Lee Foropoulos

Lee Foropoulos

Business Development Lead at Lookatmedia, fractional executive, and founder of gotHABITS.

🔔

Never Miss a Post

Get notified when new articles are published. No email required.

You will see a banner on the site when a new post is published, plus a browser notification if you allow it.

Browser notifications only. No spam, no email.

0 / 0