Make verilator_ext_tests head-to-head #1483
Comments
|
Original Redmine Comment I agree this is a good approach. Note Verilator, along with SwerRV is part of Chips Alliance, so if you want you can ask for commit privileges to it, and/or we can move the ext tests to the CHIPS Alliance repo if that helps in some way (e.g. they might pay for more Travis services). |
|
Original Redmine Comment #� captures the failures due to new WIDTH warnings. |
|
Original Redmine Comment There's more to do here, but I'm focused on the DPI protected modules for now. But I believe this should be land-able: If we turn on daily cron jobs, we'll get a head-to-head check of all these tests and Verilator every day. Also, the build has gone red since I looked at this last: This looks to be a legitimate SweRV issue this time. I suggest we add a requirement to the README that external projects have their own CI using whatever their fixed version of Verilator is. We can grandfather in the current two projects and politely ask them to set up their own CI. |
|
Original Redmine Comment These changes look fine. Personally I'd like to be able to run the tests pointing at a separate verilator tree (I am developing at that moment) without picking up the external for verilator. Do you know if there's a way we can do that? Or maybe I just live with the external fetch and set VERILATOR_ROOT to my local area? |
|
Original Redmine Comment That's what I've been doing. I just added ci/update_submodules.sh which does the remote submodule update for all submodules but excludes the verilator submodule if $VERILATOR_ROOT is set to something other than the in-tree copy. Does this work for your workflow? |
|
Original Redmine Comment Ah, driver error. Feel free to squash and push. If you didn't already please contact the SweRV team to report their bug. |
|
Original Redmine Comment Done and done. |
veripoolbot
added
area: tests
Author Name: Todd Strader (@toddstrader)
Original Redmine Issue: 1483 from https://www.veripool.org
Original Assignee: Todd Strader (@toddstrader)
This is still a WIP, but I think it warrants some discussion before I trudge ahead much further:
https://github.com/toddstrader/verilator_ext_tests/tree/head-to-head
I'm proposing that we have Travis always pull the latest master from every submodule (including Verilator, which I added). This way we don't have to keep bumping these submodules by hand. Right now, I'm just running Travis on my branch manually, or by pushing verilator_ext_tests. If we go with head-to-head, we can initially just turn on daily cron builds.
There's also some fanciness in Travis about public/private keys, encrypted YAML and protected environment variables. This supposedly allows you to "safely" kick off Travis builds via webhooks. I'm wondering if we could grant these projects we're building with these privileges so that they could kick off a verilator_ext_tests build when they run their own CI. This would just grant them the ability to control Travis runs and not permissions to GitHub. All of this is to say, I would need to do more research on this bit.
You'll notice that Travis is currently failing the SweRV test:
https://travis-ci.com/toddstrader/verilator_ext_tests/builds/120918474
This is actually a fortunate coincidence because the SweRV test passes if you run with the Verilator rev that is committed via the submodule (just don't update with --remote). But if you do update to HEAD of verilator, it breaks. So . . . heads up SweRV people (I'll post there).
I also learned how to cache the full Verilator build with Travis build stages. This has potential benefits for the Verilator proper CI. I'll have to circle back there. However, the solution I've landed on is a bit cumbersome. Essentially, I'd like to install Verilator with installation option #4 (from README.pod). However, this involves unsetting VERILATOR_ROOT which breaks driver.pl which these tests depend on. I'm wondering if VERILATOR_ROOT can be deprecated, but that's a topic for another thread.
What are your thoughts on this approach? It will obviously break more often. But I think that's what we want. If we have to manually bump submodule versions, I'm not sure how much more beneficial verilator_ext_tests is than these projects' own CI environments.
The text was updated successfully, but these errors were encountered: