Project

General

Profile

[logo] 
 
Home
About/Contact
Major Tools
  Dinotrace
  Verilator
  Verilog-mode
  Verilog-Perl
Other Tools
  IPC::Locker
  Parallel::Forker
  Voneline
General Info
  Papers

A way to support LXT2 file format natively

Added by Yu Sheng Lin almost 2 years ago

As everyone know, dumping VCD is very slow owing to its large file size and disk IO. LXT2 format, a dump format developed by another open software GTKWave, provides a much smaller size even when compared to FSDB format. While this discussion points out that it is possible to use mkfifo+vcd2lxt2 to enable LXT2 file dump, I think it could be very helpful to have native LXT2 support in Verilator.

GTKWave provides a LXT2 writer helper class, so the problem is how to wrap class VerilatedVcd inside verilated_vcd_c.cpp/h . I have made a proof of concept in my Github repo , and this toy example works smoothly, showing that supporting LXT2 with the helper class is very possible.

However, I think this example is far from usable, since I don't know the what every VerilatedVcd::* call exactly does clearly. Can anyone provide some instructions about how to do this correctly? Thanks.


Replies (14)

RE: A way to support LXT2 file format natively - Added by Wilson Snyder almost 2 years ago

Basically VerilatedVcd gets called by users to open/close the trace, and when opening by verilated code to declare the signals/modules.

Probably the right thing is to have verilated code call abstract base class which then does VerilatedVcd or new VerilatedLxt2.

Thanks for working towards a patch for this, I'll make some review comments as soon as I can get to it.

RE: A way to support LXT2 file format natively - Added by Wilson Snyder almost 2 years ago

Thanks for your work, the basic concepts look great. This would be very valuable for others and it would be great to get this upstreamed. Verilator 4.0 is scheduled to be released in a week or so, might it be possible to be done for then?

Here's comments on your patches.

Unfortunately it looks like you committed a modified version of verilator as your first commit, which makes it very difficult to see what needs to be changed. Can you please instead make your repo a clone of verilator with the changes needed on top? E.g. make a new repository, where the first step is to do this

git clone http://git.veripool.org/git/verilator # Only first time

(There's other ways to do it too if you're familiar with git/github.)

Then on top of that apply your changes.

GaloisLfsr.sv

This should be instead e.g. test_regress/t/t_trace_lxt2.v.  And add a new test_regress/t/t_trace_lxt2.cpp which has the top code. Then likewise make a new e.g. test_regress/t/t_trace_lxt2.pl file that runs the test and creates lxt2.  This would look something like this:
                scenarios(vlt_all => 1);

                top_filename("t/t_trace_lxt2.v");

                compile(
                    make_top_shell => 0,
                    make_main => 0,
                    v_flags2 => ["--trace --exe $Self->{t_dir}/$Self->{name}.cpp"],
                    );

                execute(
                    check_finished => 1,
                    );

                ok(files_identical("$Self->{obj_dir}/simx.lxt2",
                              "t/t_trace_lxt2.out"));

                ok(1);
                1;

README.md

Your comments should be moved to a new bin/verilator section describing how to use LXT2 traces e.g. "=item How do I generate waveforms (traces) in LXT2 format?"

Move lxt2_write.c/h to include/verilated/lxt2

lxt2/config.h should be eliminated this will make a mess of other people that run their own config processes. I guess rename it to include/verilated/lxt2/lxt2_config.h. It seems the lxt2 files only use HAVE_FSEEKO and HAVE_ALLOCA_H, at some point those could be elminated, but not your problem.

verilator/verilated_vcd_c.h/cpp move these to e.g. include/verilated_lxt2.c/h and change the class name to e.g. VerilatedLxt2.

Once you have all of this please file a veripool.org issue asking the patch get reviewed and merged, thanks!!

RE: A way to support LXT2 file format natively - Added by Yu Sheng Lin almost 2 years ago

Thanks for the reply. I have cloned it and applied the change to lxt2 branch of forked repo. The follows are the changes I made, and I do not modify or overwrite existing source codes.

  1. Put all of the dump writer codes to include/lxt2/wave_alloca.h,lxt2_config.h,lxt2_write.cpp/h. I slight modify the code from GTKWave by adding explicit type castings to malloc() such that the C++ compilers is happy with the code.
  2. Add the wrapper include/verilated_lxt2.cpp/h, and the CPP file include the lxt_write.cpp directly, eliminating the need for compiling extra files.
  3. Add description to bin/verilator.

Currently, to dump LXT2, users must manually change the include path to verilated_lxt2.h, and this is also required for the compiled codes. The XXX_classes.mk must be changed accordingly to use the verilated_lxt2.cpp. In the final version, these name and path should be simply configured by a switch such as --trace-lxt2 (in my opinion). The class name is still VerilatedVcd, which is a compatible API to the VCD version. This means it is impossible to support VCD and LXT2 in a single run, but this need should be rare?

However, a few questions must be clarified.
  1. What is the difference between VerilatedVcd::fullX and VerilatedVcd::chgX? Need I implement both to make everything work?
  2. When will the functions VerilatedVcd::declQuad/Array/TriBus... be used? They are not used even in a much larger module.
  3. How to run a single regression test? Specifically, how do I know whether my regression works or fails?
  4. Apparently it is impossible to produce exactly identical dump files in every future releases (dump order, file header), any recommendation?

TODO: Must add the SystemC version and timescale support later.

And again, thank you for taking time reviewing the codes.

RE: A way to support LXT2 file format natively - Added by Wilson Snyder almost 2 years ago

That repo looks much better, thanks!

IMO instead of reusing the VerilatedVcd classes you should use VerilatedLxt2. Then as you suggested add a --trace-lxt2 switch to verilator which turns on --trace and also changes traceClassBase() to VerilatedLxt2.

Note I just made some changes to verilated_vcd_* to cleanup spacing.

  1. What is the difference between VerilatedVcd::fullX and VerilatedVcd::chgX? Need I implement both to make everything work?

Full dumps a signal regardless of if the value has changed and is called at init time and is a cold function. The chg routines only enter a dump record if the value is different than last time it was dumped - these are hot functions. Both are needed, but perhaps share some code.

  1. When will the functions VerilatedVcd::declQuad/Array/TriBus... be used?

These declare the names of the signals.

  1. How to run a single regression test?

Run at the shell:

test_regress/t/t_EXAMPLE.pl

This should print "Passed 1".

Also see TESTING in internals.txt.

Apparently it is impossible to produce exactly identical dump files in every future releases (dump order, file header), any recommendation?

If a program change makes a different file then we'll update the golden file. If there's a datestamp inside the file, then we can make a custom "diff" tool to ignore the difference. Or ideally if there's already a lxt2diff program we can use that - for example the tests use vcddiff.

RE: A way to support LXT2 file format natively - Added by Yu Sheng Lin almost 2 years ago

I make another version based on the comments, also in the same repo . In this version, an extra --trace-lxt2 flag is added, and I add some functions that can return different strings according to the switch.

The regression is run by:

./test_regress/t/t_EXAMPLE.pl --vlt

The regression requires the lxt2vcd to work, which is a utility program from GTKWave. The output LXT2 file is converted to VCD and compared by vcddiff.

I have also create an issue and attach the patch.

RE: A way to support LXT2 file format natively - Added by Wilson Snyder almost 2 years ago

Thanks for the work, discussion continuing under bug1333.

RE: A way to support LXT2 file format natively - Added by Iztok Jeras almost 2 years ago

Hi,

What is the reasoning behind choosing LXT2, I assume it is file size.

I would prefer another GTKWave format to be supported, let me try to explain why FST might be a better choice.

My original concern was support for SystemVerilog objects, right now logic signals are traced into VCD as wire by Verilator. In a related discussion, GTKWave author describes LXT as an old unsupported format, and recommends FST.

The GTKWave User's Guide shows the set of icons for various SV hierarchical components on page 21.

Regards, Iztok Jeras

RE: A way to support LXT2 file format natively - Added by Wilson Snyder almost 2 years ago

As LXT2 patches were submitted, that's what was added, though if I had remembered FST was newer I would have recommended that instead. If you'd like to create patches to upgrade to FST, that would be great.

RE: A way to support LXT2 file format natively - Added by Yu Sheng Lin almost 2 years ago

I patched the LXT2 format since it conforms to current VCD-based API well, s.t. I can add LXT2 easily without modifying core functionalities. If I want to add SV signal dump, then much effort will be required.

RE: A way to support LXT2 file format natively - Added by David Stanford almost 2 years ago

From what I can tell, the file verilator/include/lxt2/wavealloca.h is a standard GPL2 or later licensed file, not an LGPL licensed file. Does that end up affecting the licensing of the rest of verilator? None of the rest of the files in that directory have the same license header.

RE: A way to support LXT2 file format natively - Added by Wilson Snyder almost 2 years ago

I contacted Tony Bybell, the author, and he changed the license in his upstream. This is copied to verilator and pushed to git towards 4.004.

RE: A way to support LXT2 file format natively - Added by Iztok Jeras almost 2 years ago

In 2013 I created a set of examples to be used to test waveform support for SystemVerilog datatypes.

https://github.com/jeras/waveform_examples

In the end I did not follow Icarus Verilog SV support, and I thought not much progress was made in GTKWave. Actually GTKWave release notes mentioned SV type support, but I did not know how to check myself.

I might start porting some of those examples to Icarus Verilog, so see how GTKWave behaves, and I could port them to Verilator. But I do not know what to use to compare the results against, since for example SV interfaces are not implemented in Icarus Verilog.

Regards, Iztok Jeras

RE: A way to support LXT2 file format natively - Added by Wilson Snyder almost 2 years ago

bug1356 discusses work towards adding FST format. If you have examples you'd be willing to port that would be excellent; looking at your repo perhaps it's easiest if all be put into one large test so you need only one set of golden files etc.

RE: A way to support LXT2 file format natively - Added by Sergi Granell almost 2 years ago

Hi! I've added very basic FST format support, check the bug Wilson Snyder referenced in the previous message.

    (1-14/14)