Memory usage for gate sim
Added by Guy Hutchison about 2 years ago
Hi,
I'm trying to use Verilator for a large gate simulation, and the memory consumption is too high for it to complete the compile step. I'm compiling on a machine with 8GB of memory and verilator rapidly uses up all memory on the machine and effectively brings it to a halt.
Is there a way to have Verilator break down its optimization so it only works on parts of the design at at time? I've tried output-split but it's not helping.
Thanks,
Guy
Replies (7)
RE: Memory usage for gate sim - Added by Wilson Snyder about 2 years ago
--output-split usually does it. There was one version of GCC (4.2.1) that seemed to have a bug where it would use infinite memory with -O2; the only solution was to use -O1 on a few files; you can try that.
RE: Memory usage for gate sim - Added by Guy Hutchison about 2 years ago
Hi,
Just reread my original post and I used the wrong terminology. It's not GCC that's running out of memory, it's the Verilator binary during the Verilog->C conversion process. It seems that some part of the optimization verilator does scales with design size, so if I feed it a very large number of instances, it can't complete the job.
Thanks,
Guy
RE: Memory usage for gate sim - Added by Wilson Snyder about 2 years ago
Yes, that's quite different. Verilator needs to flatten the whole design, so needs several times as much memory as the final simulation will. That said 8GB is larger than I've ever needed myself.
It may be that the design exposed a memory leak, causing it to run out of memory. Try using --debug --no-dump-tree while running "top" or "ps" in another window. See if the memory is all sucked up at once (likely bug) or dribbles (probably just need more memory). You can also crtl-c and "c" (continue) and "bt" backtrace in gdb to see where it is in the code while consuming memory.
RE: Memory usage for gate sim - Added by Guy Hutchison about 2 years ago
I reran with the above two options and memory usage seems to build slowly up to its maximum usage. There are several steps which cause large jumps in memory usage.
Initial memory usage grows rapidly during parsing to 2.8G Growth is slow through remaining steps until V3LinkDot:729, which jumps from 3.7G to 4.7G
Memory usage is stable through several more optimization steps until V3Inline.cpp:426 V3Tristate:696 increases from 4.7 to 4.9 V3Unknown:468 increases to 5.3 V3Inline:426 grows until it consumes all memory on the machine and it starts thrashing. At this point I need to kill the job because it has brought the machine to a near halt. Peak memory usage is 7.0G resident, 7.6G virtual.
BTW, the documentation mentions that Verilator moved away from straight translation of Verilog->SystemC a while ago. Is there a revisionary mode where it can go back to just doing basic translation?
Thanks,
Guy
RE: Memory usage for gate sim - Added by Wilson Snyder about 2 years ago
A 2.8GB parse is huge. Given the size of chip this must be, I suspect you have some larger servers somewhere :)
There isn't a straight translation any more, and it wouldn't be that much smaller anyhow as gate level models have so many instantiations. However you can use the -O{letter} options to disable some optimizations which may save memory.
Out of curiosity, why use Verilator for gate sims? Usually gate sims are on a signoff simulator where you also care about Xs and possibly back annotated timing.
RE: Memory usage for gate sim - Added by Guy Hutchison about 2 years ago
Actually, this isn't a particularly large chip. The block we are working on is about 6mm^2 in 65nm, which comes out to roughly 3-4M gates. Part of the problem is that we don't do signoff sims any more -- signoff is basic on static timing, and we don't necessarily have gate sim that is clean. We almost certainly have servers within the company that have more than 8G memory, but I would have to go find them. :)
In particular, this block has difficulty coming out of reset because of X-feedback, and we're trying to simulate an ECO change that only occurs after a fairly large amount of traffic. Since I would like the simulation to complete in my lifetime, doing a cycle-based 2-state simulation of this seemed like a good idea.
RE: Memory usage for gate sim - Added by Wilson Snyder about 2 years ago
I'm surprised it's that size, as larger designs have finished in 4GB, but maybe it's something to do with the cell library.
BTW if your problem is Xs, check your simulator manual; several have added randomize-X's flags recently. If you don't have that, I have a PLI script to get rid of them. Let me know and I'll dig it up; it'll take a dayish as I need to insure the code is releasable.
(1-7/7)
![[logo]](/img/veripool_small.png)