Project

General

Profile

[logo] 
 
Home
News
Activity
About/Contact
Major Tools
  Dinotrace
  Verilator
  Verilog-mode
  Verilog-Perl
Other Tools
  BugVise
  CovVise
  Force-Gate-Sim
  Gspice
  IPC::Locker
  Rsvn
  SVN::S4
  Voneline
  WFH
General Info
  Papers

Issue #750

Verilator crashes throwing std::bad_alloc exception

Added by Igor Lesik over 5 years ago. Updated over 5 years ago.

Status:
NoFixNeeded
Priority:
High
Assignee:
Category:
Configure/Make/Compiling
% Done:

0%


Description

I am trying to compile a big project with Verilator on 4 cores 4G ram machine. Seems that Verilator slowly eats all available memory and then crashes when "new" can't allocate more memory.

History

#1 Updated by Igor Lesik over 5 years ago

- V3Unroll.cpp:453:   unrollGen: 
- V3Unroll.cpp:453:   unrollGen:
 ********************************** MANY LINES LIKE THAT *****************************************
- V3Unroll.cpp:453:   unrollGen: 
- V3Unroll.cpp:453:   unrollGen: 
- V3Unroll.cpp:453:   unrollGen: 
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Program received signal SIGABRT, Aborted.
0xb7fdd424 in __kernel_vsyscall ()
#0  0xb7fdd424 in __kernel_vsyscall ()
#1  0xb7d1b1df in raise () from /lib/i386-linux-gnu/libc.so.6
#2  0xb7d1e825 in abort () from /lib/i386-linux-gnu/libc.so.6
#3  0xb7f8f13d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#4  0xb7f8ced3 in ?? () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#5  0xb7f8cf0f in std::terminate() () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#6  0xb7f8d05e in __cxa_throw () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#7  0xb7f8d67f in operator new(unsigned int) () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#8  0x080727b7 in AstSelPlus::clone (this=0x6c6ab520) at ../V3AstNodes.h:766
#9  0x080c5ef7 in AstNode::cloneTreeIter (this=0x6c6ab520) at ../V3Ast.cpp:625
#10 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c6ab520) at ../V3Ast.cpp:642
#11 0x080c5f08 in AstNode::cloneTreeIter (this=0x6c6abb80) at ../V3Ast.cpp:626
#12 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c6abb80) at ../V3Ast.cpp:642
#13 0x080c5f08 in AstNode::cloneTreeIter (this=0x6c6774e8) at ../V3Ast.cpp:626
#14 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c6774e8) at ../V3Ast.cpp:642
#15 0x080c5f25 in AstNode::cloneTreeIter (this=0x6c67f3b0) at ../V3Ast.cpp:627
#16 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c67f3b0) at ../V3Ast.cpp:642
#17 0x080c5f08 in AstNode::cloneTreeIter (this=0x6c67eea0) at ../V3Ast.cpp:626
#18 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c67eea0) at ../V3Ast.cpp:642
#19 0x080c5f08 in AstNode::cloneTreeIter (this=0x6c6ac390) at ../V3Ast.cpp:626
#20 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c6ac390) at ../V3Ast.cpp:642
#21 0x080c5f08 in AstNode::cloneTreeIter (this=0x6c6ad678) at ../V3Ast.cpp:626
#22 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c6ad678) at ../V3Ast.cpp:642
#23 0x080c5f08 in AstNode::cloneTreeIter (this=0x6c6ae960) at ../V3Ast.cpp:626
#24 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c6ae960) at ../V3Ast.cpp:642
#25 0x080c5f08 in AstNode::cloneTreeIter (this=0x6c6aeb68) at ../V3Ast.cpp:626
#26 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c6aeb68) at ../V3Ast.cpp:642
#27 0x080c5f08 in AstNode::cloneTreeIter (this=0x6c6aebd8) at ../V3Ast.cpp:626
#28 0x080c5fd9 in AstNode::cloneTreeIterList (this=0x6c6a75a0) at ../V3Ast.cpp:642
#29 0x080c609b in AstNode::cloneTree (this=0x6c6a75a0, cloneNextLink=true) at ../V3Ast.cpp:660
#30 0x0825e92a in UnrollVisitor::forUnroller (this=0xbfffeb1c, nodep=0x6c6eebf0, initp=0x6c69d0c8, precondsp=0x0, condp=0xbfd93868, incp=0x6c69d5c8, bodysp=0x6c6a75a0, numInit=..., cmpInstrp=0xbfd93868, numStop=..., incInstrp=0x6c69d558, numInc=...) at ../V3Unroll.cpp:278
#31 0x0825e2cd in UnrollVisitor::forUnrollCheck (this=0xbfffeb1c, nodep=0x6c6eebf0, initp=0x6c69d0c8, precondsp=0x0, condp=0xbfd93868, incp=0x6c69d5c8, bodysp=0x6c6a75a0) at ../V3Unroll.cpp:223
#32 0x0825f35f in UnrollVisitor::visit (this=0xbfffeb1c, nodep=0x6c6eebf0) at ../V3Unroll.cpp:372
#33 0x080790b8 in AstGenFor::accept (this=0x6c6eebf0, v=..., vup=0x0) at ../V3AstNodes.h:2464
#34 0x0825f731 in UnrollVisitor::UnrollVisitor (this=0xbfffeb1c, nodep=0x6c6eebf0, generate=true, beginName=...) at ../V3Unroll.cpp:436
#35 0x0825ca4b in V3Unroll::unrollGen (nodep=0x6c6eebf0, beginName=...) at ../V3Unroll.cpp:454
#36 0x0820efe1 in ParamVisitor::visit (this=0xbfffee64, nodep=0x6c6eeb78) at ../V3Param.cpp:312
#37 0x080d6daa in AstBegin::accept (this=0x6c6eeb78, v=..., vup=0x0) at ../V3AstNodes.h:2695
#38 0x080c667d in AstNode::iterateAndNext (this=0x6c6eeb78, v=..., vup=0x0) at ../V3Ast.cpp:788
#39 0x080c6434 in AstNode::iterateChildren (this=0x6c6eec60, v=..., vup=0x0) at ../V3Ast.cpp:750
#40 0x0820ea97 in ParamVisitor::visit (this=0xbfffee64, nodep=0x6c6eec60) at ../V3Param.cpp:254
#41 0x080746f0 in AstGenerate::accept (this=0x6c6eec60, v=..., vup=0x0) at ../V3AstNodes.h:1546
#42 0x080c667d in AstNode::iterateAndNext (this=0x6c600908, v=..., vup=0x0) at ../V3Ast.cpp:788
#43 0x080c645a in AstNode::iterateChildren (this=0x6c67e158, v=..., vup=0x0) at ../V3Ast.cpp:751
#44 0x0820e352 in ParamVisitor::visitModules (this=0xbfffee64) at ../V3Param.cpp:187
#45 0x0820e5ce in ParamVisitor::visit (this=0xbfffee64, nodep=0x842b458) at ../V3Param.cpp:216
#46 0x080b74e8 in AstNVisitor::visit (this=0xbfffee64, nodep=0x842b458, vup=0x0) at ./V3Ast__gen_visitor.h:129
#47 0x08073792 in AstModule::accept (this=0x842b458, v=..., vup=0x0) at ../V3AstNodes.h:1332
#48 0x080c667d in AstNode::iterateAndNext (this=0x842b458, v=..., vup=0x0) at ../V3Ast.cpp:788
#49 0x080c6434 in AstNode::iterateChildren (this=0x8402810, v=..., vup=0x0) at ../V3Ast.cpp:750
#50 0x0820e4a2 in ParamVisitor::visit (this=0xbfffee64, nodep=0x8402810) at ../V3Param.cpp:208
#51 0x080b5206 in AstNetlist::accept (this=0x8402810, v=..., vup=0x0) at ../V3AstNodes.h:4795
#52 0x0820f642 in ParamVisitor::ParamVisitor (this=0xbfffee64, nodep=0x8402810) at ../V3Param.cpp:386
#53 0x0820d96c in V3Param::param (rootp=0x8402810) at ../V3Param.cpp:570
#54 0x080b03a6 in process () at ../Verilator.cpp:181
#55 0x080b3bf2 in main (argc=17, argv=0xbffff264, env=0xbffff2ac) at ../Verilator.cpp:671

#2 Updated by Wilson Snyder over 5 years ago

  • Status changed from New to AskedReporter

It's unrolling some loop. Maybe it needs more than 4GB. Or maybe the unroll count is set too high, or there's a bug, but the code will be needed to say.

Run with --debug and in the .tree file search for the "nodep" value that is printed in the backtrace. That'll show {f###} where f is a letter corresponding to the filename and ### is the line number.

#3 Updated by Igor Lesik over 5 years ago

Do you mean cells.tree? nodep=0x6c6eebf0 is mentioned there but I do not see filename or line number.

grep 6c6eebf0 obj_dir/VDve_002_cells.tree
    1:2:1:2: GENFOR 0x6c6eebf0 <e23313300#> {bbk144}

#4 Updated by Wilson Snyder over 5 years ago

You didn't say if running with more memory fixed it. That's the first thing to do.

Otherwise, see the "debuging" chapter of internals.txt.

It's line 144 of file bkk which is many files into your design. It looks like it doesn't print the filenumbers unless it doesn't crash, so to get that, run with --gdb, then at the prompt do:

p nodep->dumpGdb()

#5 Updated by Igor Lesik over 5 years ago

I think I figure out what file corresponds to "bkk". Here is code around line 144:

142  genvar e;
143  generate
144    for (e=0; e<64; e=e+1) begin : my_e

I do not have machine with more than 4G at the momement.

Is it known that Verilator is hungry for memory? What memory is required to build OpenRISC?

#6 Updated by Wilson Snyder over 5 years ago

Many designs take more than 4GB; generally the assumption is there are huge memory machines available for synthesis etc anyways so it's not a big deal. It's a little surprising it's out of memory relatively early in the process, but given it is a generate loop it is a lot less likely a bug and there's also not much you can do to get around it as generates have to be unrolled. You could try changing the loop bound to just 2 and make sure it makes additional progress.

I have no idea if OpenRISC has been done before, but other groups at Sun have used it.

#7 Updated by Igor Lesik over 5 years ago

I will try to build verilator on my 16G workstation and see how it goes. But I hoped that at least Verilator compiler is not that crazy about memory; that could save some resources when many tests are compiled and run on a grid.

#8 Updated by Igor Lesik over 5 years ago

Did not re-test yet. But going to close ticket so it does not hang around.

#9 Updated by Wilson Snyder over 5 years ago

  • Status changed from AskedReporter to NoFixNeeded

Also available in: Atom