Open Silicon, Hands-On | Part 3: RTL That Synthesizes → Part 4: Co-Simulation → Part 5: Bus and Formal
Part 3 ended with RTL that lints clean, which tells you nothing about whether it's right. The next question is the one that matters: does the hardware do what the model says?
The approach is a diff, not a test case. The Python model writes, for each of 501 programs, the expected state after every instruction. A C++ testbench built by Verilator loads each program into the RTL, runs it, and compares the program counter, accumulator, flag and halt state at every retired instruction, stopping at the first difference.
Then we'll do the thing most tutorials skip: grade the testbench by breaking the design on purpose.
The architecture: a diff, not a test case
Here's the whole setup in one picture.
1model/acc8_model.py ──► tb/cases.txt ──► tb_cosim (C++ from Verilator) ──► acc8_cpu (the RTL)
2 expected state one file loads each program, runs it, retire pulse plus
3 after every step compares at every retire pc, acc, zf, haltedThe retire output is a one-clock pulse that arrives the clock after an instruction completes, while the architectural outputs show the post-instruction state. Real cores expose similar retire interfaces for exactly this job. The testbench enforces six rules:
- After reset, every architectural register equals the model's reset state, whatever the flops held before.
- Programs load through the host port while the core is idle, with no back doors.
- After each retire pulse, PC, ACC, ZF and the halt state equal the model's values.
- The first retire comes exactly five clocks after
start, and each later one exactly four clocks after the last. - A halted core stays quiet, and memory read back through the host port matches the model's final memory.
- Every seventh case is aborted mid-run by a reset, and the machine must return to its reset state.
The testbench, in two excerpts
Verilator is cycle-based: you set inputs, call eval(), flip the clock, and call eval() again. The small Tb struct below wraps that into tick() (one full clock period) and gives the test three verbs: reset, load a program through the host port, and peek at a memory byte.
1struct Tb {
2 VerilatedContext* ctx;
3 std::unique_ptr<Vacc8_cpu> top;
4 VerilatedVcdC* tfp = nullptr;
5 uint64_t cycles = 0, t = 0;
6
7 explicit Tb(VerilatedContext* c) : ctx(c), top(new Vacc8_cpu(c)) {
8 top->clk = 0; top->rst = 0; top->start = 0;
9 top->host_we = 0; top->host_addr = 0; top->host_wdata = 0;
10 top->eval(); // let the garbage show before reset
11 }
12 void tick() { // one full clock period
13 top->clk = 0; top->eval(); if (tfp) tfp->dump(t); t += 5;
14 top->clk = 1; top->eval(); if (tfp) tfp->dump(t); t += 5;
15 cycles++;
16 }
17 void reset(int n) { top->rst = 1; for (int i = 0; i < n; i++) tick(); top->rst = 0; }
18 void load(const uint8_t* mem) {
19 for (int i = 0; i < 16; i++) { top->host_we = 1; top->host_addr = i; top->host_wdata = mem[i]; tick(); }
20 top->host_we = 0;
21 }
22 uint8_t peek(int addr) { top->host_addr = addr; tick(); return top->host_rdata; }
23};The heart of the check is the loop that runs after a program is loaded. since counts clocks since the start edge or the last retire, so a timing deviation fails exactly like a wrong value does.
1 // 2. load the program through the host port, start, and compare after EVERY instruction
2 tb.load(c.mem);
3 tb.top->start = 1; tb.tick(); tb.top->start = 0;
4 uint64_t since = 1; // clocks since the start edge (or the last retire)
5 for (size_t i = 0; i < c.steps.size(); ) {
6 if (++since > 5) fail(c, i + 1, "timeout: no retire within 5 clocks", 0, 1, tb.cycles);
7 tb.tick();
8 if (!tb.top->retire) continue;
9 const Step& m = c.steps[i];
10 const size_t n = i + 1;
11 if (since != (i == 0 ? 5u : 4u)) fail(c, n, "clocks per instruction", (unsigned)since, i == 0 ? 5 : 4, tb.cycles);
12 if (tb.top->pc != m.pc) fail(c, n, "pc", tb.top->pc, m.pc, tb.cycles);
13 if (tb.top->acc != m.acc) fail(c, n, "acc", tb.top->acc, m.acc, tb.cycles);
14 if (tb.top->zf != m.zf) fail(c, n, "zf", tb.top->zf, m.zf, tb.cycles);
15 if (tb.top->halted != m.halted) fail(c, n, "halted", tb.top->halted, m.halted, tb.cycles);
16 since = 0; i++; total_steps++;
17 }The first mismatch stops everything and reports the case, step, field and both values, so no waveform is needed to know where to look. Building it is one command, wrapped in a script.
1verilator --cc --exe --build -Wall $VFLAGS --trace --x-initial unique \
2 --Mdir "$OBJ" --top-module acc8_cpu \
3 rtl/acc8_isa_pkg.sv rtl/acc8_mem.sv "$CORE" rtl/acc8_cpu.sv \
4 tb/tb_cosim.cpp -o tb_cosimAnd here's the real run: 501 programs, every retired instruction compared.
1+ obj_cosim/tb_cosim tb/cases.txt +verilator+rand+reset+2 +verilator+seed+1
2cosim: 501 cases, 10188 instructions compared, 62472 clocks, 382 final memories checked, 72 mid-run resets, random init ON, 0 mismatches
3PASS
4exit=0Start every flop in a random state
Verilator's default is to start every flop at zero. That default is quietly the most dangerous setting in the tool, because a missing reset is invisible when everything starts at zero. Build the model with --x-initial unique, run it with +verilator+rand+reset+2, and every flop starts random instead. Add a seed with +verilator+seed+N and the run stays reproducible.
One subtlety: Verilator applies the seed when a model is constructed, so 501 models give you 501 identical garbage states. The fix is a one-line reseed before every case, ctx->randSeed(base_seed + c.id + 1). Now each program starts from different garbage, as the run log shows.
1case 0 powers up as: pc=2 acc=54 zf=1 halted=0 busy=1 retire=1
2case 1 powers up as: pc=A acc=4F zf=0 halted=0 busy=0 retire=1
3case 2 powers up as: pc=D acc=F6 zf=0 halted=0 busy=0 retire=0Remember the latch bug from Part 3? Here's the same testbench run against that broken core, first with flops at zero and then random.
1+ tb_cosim tb/cases.txt +nomid (flops start at zero)
2cosim: 501 cases, 10188 instructions compared, 59439 clocks, 382 final memories checked, 0 mid-run resets, random init off, 0 mismatches
3PASS
4+ tb_cosim tb/cases.txt +verilator+rand+reset+2 +verilator+seed+1 (flops start random)
5case 1 powers up as: pc=A acc=4F zf=0 halted=0 busy=0 retire=1
6FAIL case 1 step 1: acc dut=0xF0 model=0x00 (after 23 cycles)The broken design passes 10,188 instructions with zeroed flops. With random initial state it fails on case 1, whose first instruction is a JZ that never writes the accumulator: the latched control bit powers up as 1 and quietly loads garbage into it.
Zero init hides missing resets
If your simulator defaults to zero or to a fixed pattern, "passes in simulation" can mean "the bug and the initial state happen to cancel out." Randomize every flop at power-up, reseed per test, log the seed, and treat a reset that only matters at power-up as an untested reset.
Who tests the testbench?
A passing testbench only says the design and the testbench agree. To learn whether that agreement means anything you have to try to fool it, and the discipline for that is mutation testing: make a tiny deliberate change to the RTL, rerun the testbench, and demand that it fails. If it doesn't, you've found a hole in the testbench, not a feature of the design.
I wrote sixteen mutants as one-line textual edits of the core: missing resets, a subtract that forgets its +1, a jump with an inverted condition, a store to the wrong address, a retire pulse a clock early, and so on. Each runs against two testbenches. The naive one starts flops at zero and resets once. The random one starts them random. The full testbench adds mid-run resets, and its results matched the random column for all sixteen.
| Mutant and what it breaks | Naive | Random init | First failure, full testbench |
|---|---|---|---|
zf_no_reset: ZF not reset | survives | caught | case 0 step 0: zf after reset |
acc_no_reset: ACC not reset | survives | caught | case 0 step 0: acc after reset |
state_no_reset: FSM state not reset | survives | caught | case 0 step 0: busy after reset |
retire_no_reset: retire flag not reset | survives | caught | case 0 step 0: retire after reset |
sub_no_carry_in: SUB forgets the +1 | caught | caught | case 0 step 7: acc 0x03, model 0x04 |
jz_inverted: JZ jumps when ZF is clear | caught | caught | case 0 step 9: pc 0x0A, model 0x09 |
sta_sets_zf: STA also updates ZF | caught | caught | case 0 step 2: zf 1, model 0 |
zf_stale: ZF from the old accumulator | caught | caught | case 0 step 1: zf 1, model 0 |
reserved_is_nop: reserved opcodes do nothing | caught | caught | case 3 step 1: halted 0, model 1 |
sta_wrong_addr: STA writes at the PC | caught | caught | case 0 step 3: acc 0x0E, model 0x00 |
ldi_sign_extends: LDI sign-extends | caught | caught | case 1 step 2: acc 0xFC, model 0x0C |
jmp_off_by_one: JMP lands one byte late | caught | caught | case 0 step 10: pc 0x03, model 0x02 |
halt_not_sticky: HALT returns to IDLE | caught | caught | case 0: not quiet after halt |
retire_early: retire a clock early | caught | caught | case 0 step 1: clocks per instruction 4, expected 5 |
ir_captured_late: instruction register loads a clock late | caught | caught | case 0 step 1: acc 0x0F, model 0x05 |
dead_state_default: no recovery from unused FSM codes | survives | survives | none: 0 mismatches |
The naive testbench caught 11 of 16, and the four it missed are exactly the four missing resets. Random initial state caught 15 of 16, and 13 of those died on the very first program. A directed program as ordinary as the sum loop caught the subtract bug at step 7.
A survivor is information
The one survivor, dead_state_default, removes the recovery path from the two unused FSM encodings. No stimulus through the ports can reach those states, so no testbench can ever tell the difference. Part 5 proves it formally: dead code by construction, which is a decision to make, not a bug to hunt.
"Passing tests tell you the testbench and the design agree. Mutation tests tell you whether that agreement means anything."
Waveforms without leaving the terminal
When the diff says where, a waveform tells you why. Verilator writes VCD with --trace, a plain-text standard every viewer reads, and the compact FST format with --trace-fst, which I did not run here. GTKWave is the classic viewer and Surfer a newer one; neither is installed on this machine.
So I wrote a 79-line script that turns a VCD into one table row per clock. It's a poor man's viewer that works over SSH. Here are the first thirteen clocks of the sum program, starting from the cycle where start is high.
1cyc start state_q mem_addr mem_rdata ir_q pc acc zf retire
2 0 1 0 2 9A 42 0 0 0 0
3 1 0 1 0 0 42 0 0 0 0
4 2 0 2 2 45 42 0 0 0 0
5 3 0 3 5 F 45 1 0 0 0
6 4 0 4 5 E 45 1 0 0 0
7 5 0 1 1 E 45 1 5 0 1
8 6 0 2 5 3E 45 1 5 0 0
9 7 0 3 E E 3E 2 5 0 0
10 8 0 4 E 0 3E 2 5 0 0
11 9 0 1 2 0 3E 2 5 0 1
12 10 0 2 E F 3E 2 5 0 0
13 11 0 3 F 5 F 3 5 0 0
14 12 0 4 F 0 F 3 5 0 0
15...Read it down the state_q column: 1 is FETCH, 2 is LATCH, 3 is OPERAND, 4 is EXEC. The first instruction is LDI 5 (byte 0x45), visible in mem_rdata during the LATCH clock. In row 5 the accumulator becomes 5 and retire pulses, five rows after start. The next two retires land in rows 9 and 13, exactly four apart. Row 0 shows random power-up state at work, too: ir_q has no reset, so it still holds garbage (0x42), and mem_rdata holds 0x9A, the old random contents of the last cell the loader wrote.
The Python alternative: cocotb
If C++ isn't your language, cocotb lets you write the same testbench as Python coroutines on top of a simulator. Not run here: cocotb isn't installed on this machine and its API shifts a little between versions, so treat the sketch as a map, not tested code.
1# not run here: the same check, sketched with cocotb
2import cocotb
3from cocotb.clock import Clock
4from cocotb.triggers import RisingEdge
5
6@cocotb.test()
7async def sum5_matches_model(dut):
8 cocotb.start_soon(Clock(dut.clk, 10, "ns").start())
9 steps = load_expected("cases.txt", case=0) # your parser for the model's file
10 # ... reset, load the program through the host port, pulse start ...
11 for n, want in enumerate(steps, 1):
12 await RisingEdge(dut.clk)
13 while int(dut.retire.value) != 1:
14 await RisingEdge(dut.clk)
15 assert int(dut.acc.value) == want.acc, f"acc mismatch at step {n}"The trade is simple. The C++ testbench runs 501 programs in a few hundredths of a second because nothing crosses a language boundary per signal. cocotb is nicer to write and extend, and it can import the Python model directly instead of reading a file.
Pick the testbench by the bug you expect
For a regression of ten thousand instructions that must finish in a blink, write C++ against Verilator. For protocol-heavy tests, constrained-random stimulus, or a model you'd rather import than serialize, write cocotb. Nothing stops you doing both.
Before you move on
You now have the loop that makes everything else cheaper: change the RTL, rerun a 12-millisecond regression, read a one-line verdict. Next, a standard bus goes in front of the CPU, along with a kind of checking that doesn't depend on picking the right stimulus at all.
In Part 5 the CPU becomes a Wishbone slave that software drives like any other peripheral. A formal tool then finds a protocol bug that a polite testbench never tripped, and a solver hands back the shortest possible way to raise the interrupt.