Add waveform replay tool
When simulating with a --dpi-protect'ed module, the end user cannot see the internals and therefore a waveform capture of an issue may not be immediately useful to the IP vendor. We could create a tool which would extract the inputs from the waveform and play them back to a Verilated instance of the design. The IP vendor would then get the customer's waveform capture of the issue and then be able to create a waveform of what is happening inside of the --dpi-protect'ed module.
We could also check the captured outputs to compare them against the simulated outputs to verify that everything is working correctly with --dpi-protect.
#2 Updated by Wilson Snyder about 2 months ago
- Status changed from New to Feature
Would ideally like to see something that works on an arbitrary design without requiring --dpi-protect, and ideally usable on any simulator. This would greatly help debug of suspected Verilator or other simulator's internal bugs.
#5 Updated by Todd Strader 12 days ago
- Subject changed from Add waveform replay tool for --protect-lib to Add waveform replay tool
You are correct that this should not be tied to --protect-lib. I've updated the subject accordingly.
I have something kludged together which I need to spend some time cleaning up before I post it. However, I ended up picking up an external VCD parser to build this out: https://github.com/ben-marshall/verilog-vcd-parser
Before I go to much further down this path, I wanted to discuss if/how we should use this library in Verilator.
One option (which I'm partial to) is to add it as a submodule. It's under the MIT license and my IANAL understanding is that we can build GPL'ed Verilator against it and ship it in the Verilator tarball. Adding a submodule to Verilator may cause unintended consequences in the verilator_ext_tests head-to-head builds. But I can work around that. Or perhaps that's the behavior we want so we can keep an eye on divergence with external libraries?
Another option is to only build the Verilator option (I'm calling it --replay-tool) if this library is installed on the user's system. One issue here is that this library currently doesn't have install targets in its Makefile. This path would also make it less likely for users to have access to the feature because this library definitely isn't available via package managers so it's unlikely people could use --replay-tool unless they build Verilator themselves.
Of course, we could just build our own VCD parser from scratch. But that seems like a waste as this one seems to work pretty well out of the box.
Please let me know what you think.
#6 Updated by Wilson Snyder 12 days ago
https://github.com/ben-marshall/verilog-vcd-parser Before I go to much further down this path, I wanted to discuss if/how we should use this library in Verilator.
Interesting. Room for performance enhancements but probably good enough for us. Also could benefit from some friendly coding suggestions to remove all of the this's and other things.
I don't see support for extended VCD. We would probably want that as then have the info to eventually compare outputs. Probably easy thing to add. Maybe the author is willing to help/accept that.
BTW for VCD, my early 1990s Dinotrace parser, which is very fast, would be another but probably inferior starting point. This would need a lot of work to library-ize.
I'd suggest instead to look at using the gtkwave libraries. We already have the GTK writer library. I also think the reader should support FST reading; we want to encourage FST for the future, since VCD can't represent classes etc. I'm suspect Bybell will be happy to take patches if needed, as he has in the past.
Another option is to only build the Verilator option (I'm calling it --replay-tool)
I'm probably missing something, but does this need to be part of the verilator executable at all? Perhaps this flow?
# Read waveform and make verilog wrapper based on I/O in the VCD verilator_replay foo.vcd/foo.fst --wrap foo.v # Verilate wrapper, ideally would work on any simulator # (might need a foo.cpp also; maybe that Verilator is enhanced to make) verilator foo.v top.v # Run sim, this needs the file again to playback the data ./foo.v +replay+foo.fst
The reason for the extra step is ideally (eventually) the Verilog created should work on any simulator. This would massively help cross-simulator debug. The next step there would be to also have the runtime replay compare output state.
We should compare & discuss these options:
1. Part of Verilator binary
2. New binary tool in verilator repo
3. New tool in GTKwave repo
4. New tool standalone repo
Note another related project to consider, once the reader libraries are in place, would be to replace the relatively-abandoned vcddiff. (Which entertainly was written by someone who leveraged my Dinotrace VCD parser, which I only found out when poking in to fix a vcddiff bug.) This does seem like a good GTKwave feature (two traces in, diff trace or ascii report out.)
#7 Updated by Todd Strader 11 days ago
I'd suggest instead to look at using the gtkwave libraries.
Good point, I'll check out that codebase too. I'll try to combine gtkwave code with what I've got and then come back to discuss options 1-4.
I'm probably missing something, but does this need to be part of the verilator executable at all?
My starting point was to have Verilator create a C++ shell so it could make the replay executable from the verilated Makefile. But I can see the advantage of being simulator agnostic and starting from the trace file. How are you imagining feeding the trace data to the simulators in a generic fashion? PLI? DPI? Is that why your flow creates a top.v?
Right now I'm just manipulating the verilated class directly. I'd like to start there as I would like to use the replay tool as a way to benchmark native C++ Verilator against things such as cocotb + Verilator. I also plan to try to isolate the reading of the trace data from the simulation itself to aid in this (buffer all the trace data in Verilator-ready form?). Given this, even if we do make this simulator agnostic, I would still like to see bare-bones Verilator C++ support.
Just so you can see more what I'm talking about, here's what I've been messing with: https://github.com/toddstrader/verilator-dev/tree/replay-toolIt's a complete mess right now, but does replay a sim when I do something like this:
test_regress/t/t_case_reducer.pl --trace bin/verilator --cc --exe -CFLAGS "-I$PWD/obj_dir -I$PWD/../verilog-vcd-parser/build -I$PWD/include" -LDFLAGS "-L$PWD/../verilog-vcd-parser/build -lverilog-vcd-parser" +define+REPLAY_HACK --trace test_regress/t/t_case_reducer.v replay_main.cpp include/verilated_replay.cpp make -C obj_dir/ -f Vt_case_reducer.mk obj_dir/Vt_case_reducer test_regress/obj_vlt/t_case_reducer/simx.vcd top.t.test
The next step there would be to also have the runtime replay compare output state.
Also available in: Atom