Forums

RFC: VHDL testbench enhancements

Started by Jim Lewis April 3, 2007
Hi,
Let me try this again.

The VHDL standards community has been considering whether
to enhance VHDL to add advanced testbench features.

If you are a VHDL user,
   Do you want these features added to VHDL?
   Would you rather adopt a verification language that
   already supports these (SystemVerilog, SystemC, E, Vera).

I think VHDL needs these features to stay competitive.

The current plan in the VHDL working group is to make
these features similar to other verification languages
while at the same time keeping the nature of VHDL.

We need your to voice your support.  You can post here,
send your reply to me (let me know if I can use either
your name and/or company name when I tally the results
for the Accellera VHDL TSC), or join the Accellera VHDL
TSC (which you can do as a non-Accellera member by
registering) and post your reply there.

More on the politics of the situation are below.

Thanks,
Jim Lewis
VHDL Evangelist
SynthWorks VHDL Training



P.S.
One of the simulator vendors has indicated that will
only implement new features if the user community
demands them.  Business wise, this makes sense - you
don't build something unless someone will buy it.

If you fail to voice your support, they may be able to
successfully block these proposals from going forward.

If we let this opportunity to add these features to the
standard pass, I do not think we will have to opportunity
to add them later - hence you will be stuck with using
some other language for advanced verification features.

This work is work in progress and below is the current status.
Keep in mind too that your interest/support of this work will
help raise the focus and inspiration of those doing the work.

   Classes / OO:
     Classes are useful for creating verification data structures,
     transaction communication, and grouping for transaction based
     randomization (building relationships between separate data
     items).  Many of the data structures (such as scoreboards,
     memories, and linked structures) can already be created, however,
     classes give you the ability to hide all pointer manipulations.
     For example, using a memory would require a declaration with
     initialization of the memory data structure, then MemWrite and
     MemRead would allow a user to store and retrieve items from the
     data structure.  Pointers and allocation of the sparse data
     structure are handled by MemWrite and MemRead and the
     user would not need to be aware of it.
     Status:
     Peter Ashenden submitted a class proposal last year and
     provided updates to it this year at DVCon.  Currently
     he plans on finishing an updated draft soon.

   Randomization:
     Randomization is useful for designs that have numerous configurable
     features.  Testing features individually in an isolated manner is
     typically straightforward. However, testing how these features
     interact can be a large verification space � one that may not be able
     to be simulated completely. It is also may be difficult to predict
     all of the corner cases.  Randomization has been used to sequence a
     test in a non-deterministic way to get reasonably good coverage of
     this verification space.
     While I do not share the thought that randomization should be
     adapted to work for all verification problems, I do believe it
     to be a valuable technique for some problems.

     I wrote a draft of the randomization proposal and it is ready
     for review.

   Functional Coverage:
     Tool/structural coverage can tell you that you did a FIFO
     read or that the the FIFO went empty, but it can't tell
     you that you did a read while the FIFO was empty.

     Functional coverage constructs allow you to track this.
     Some functional coverage capability will come from assertions
     (since PSL has been integrated into VHDL).  Additional constructs
     will be added to allow data binning (coverage groups) and
     correlation between different coverage items (cross coverage).

     I have started working on this - anyone else who is interested
     is welcome to contribute as much as they would like.


With a focused effort, like the one to finish the Accellera 3.0
draft of the standard, I think we can be done with these by
September.

Although some have expressed doubt, it is clear that vendors
will do what their user community asks them to do - otherwise,
someone else will and, as a result, will earn your business.


Jim

On Apr 3, 11:10 am, Jim Lewis <j...@synthworks.com> wrote:
> Hi, > Let me try this again. > > The VHDL standards community has been considering whether > to enhance VHDL to add advanced testbench features. > > If you are a VHDL user, > Do you want these features added to VHDL? > Would you rather adopt a verification language that > already supports these (SystemVerilog, SystemC, E, Vera). > > I think VHDL needs these features to stay competitive. > > The current plan in the VHDL working group is to make > these features similar to other verification languages > while at the same time keeping the nature of VHDL. > > We need your to voice your support. You can post here, > send your reply to me (let me know if I can use either > your name and/or company name when I tally the results > for the Accellera VHDL TSC), or join the Accellera VHDL > TSC (which you can do as a non-Accellera member by > registering) and post your reply there. > > More on the politics of the situation are below. > > Thanks, > Jim Lewis > VHDL Evangelist > SynthWorks VHDL Training > > P.S. > One of the simulator vendors has indicated that will > only implement new features if the user community > demands them. Business wise, this makes sense - you > don't build something unless someone will buy it. > > If you fail to voice your support, they may be able to > successfully block these proposals from going forward. > > If we let this opportunity to add these features to the > standard pass, I do not think we will have to opportunity > to add them later - hence you will be stuck with using > some other language for advanced verification features. > > This work is work in progress and below is the current status. > Keep in mind too that your interest/support of this work will > help raise the focus and inspiration of those doing the work. > > Classes / OO: > Classes are useful for creating verification data structures, > transaction communication, and grouping for transaction based > randomization (building relationships between separate data > items). Many of the data structures (such as scoreboards, > memories, and linked structures) can already be created, however, > classes give you the ability to hide all pointer manipulations. > For example, using a memory would require a declaration with > initialization of the memory data structure, then MemWrite and > MemRead would allow a user to store and retrieve items from the > data structure. Pointers and allocation of the sparse data > structure are handled by MemWrite and MemRead and the > user would not need to be aware of it. > Status: > Peter Ashenden submitted a class proposal last year and > provided updates to it this year at DVCon. Currently > he plans on finishing an updated draft soon. > > Randomization: > Randomization is useful for designs that have numerous configurable > features. Testing features individually in an isolated manner is > typically straightforward. However, testing how these features > interact can be a large verification space - one that may not be able > to be simulated completely. It is also may be difficult to predict > all of the corner cases. Randomization has been used to sequence a > test in a non-deterministic way to get reasonably good coverage of > this verification space. > While I do not share the thought that randomization should be > adapted to work for all verification problems, I do believe it > to be a valuable technique for some problems. > > I wrote a draft of the randomization proposal and it is ready > for review. > > Functional Coverage: > Tool/structural coverage can tell you that you did a FIFO > read or that the the FIFO went empty, but it can't tell > you that you did a read while the FIFO was empty. > > Functional coverage constructs allow you to track this. > Some functional coverage capability will come from assertions > (since PSL has been integrated into VHDL). Additional constructs > will be added to allow data binning (coverage groups) and > correlation between different coverage items (cross coverage). > > I have started working on this - anyone else who is interested > is welcome to contribute as much as they would like. > > With a focused effort, like the one to finish the Accellera 3.0 > draft of the standard, I think we can be done with these by > September. > > Although some have expressed doubt, it is clear that vendors > will do what their user community asks them to do - otherwise, > someone else will and, as a result, will earn your business. > > Jim
Yes! I whole-heartedly support expanding the "testbench" features of VHDL. The main problem with using separate languages to create the testbench are two-fold. 1) multi-language simulators are more expensive, and 2) there are always some gotcha's at the interface when it comes to connecting the testbench to the UUT, especially if you use other than SL or SLV port types for your UUT. That being said, I would caution against creating too much variance in VHDL between what can be done in a simulation only environment, vs a synthesis environment. OO principles come to mind. The eventual application of OO to synthesis should be carefully considered when developing the OO structure and syntax, even though it is now just for simulation. Also, there are plenty of areas in synthesizable vhdl that need work too, like user definable modes for record ports, expanded integer synthesis (boolean operator definition, expanded range), as well as fixed point representations along the lines of ada. Andy
Andy,
Thanks for your post.  We are in the position where we need to
hear all users speak up (privately to me is ok), even when they
agree with what has already been stated.

> Yes! I whole-heartedly support expanding the "testbench" features of > VHDL. The main problem with using separate languages to create the > testbench are two-fold. 1) multi-language simulators are more > expensive, and 2) there are always some gotcha's at the interface when > it comes to connecting the testbench to the UUT, especially if you use > other than SL or SLV port types for your UUT. > > That being said, I would caution against creating too much variance in > VHDL between what can be done in a simulation only environment, vs a > synthesis environment. OO principles come to mind. The eventual > application of OO to synthesis should be carefully considered when > developing the OO structure and syntax, even though it is now just for > simulation. > > Also, there are plenty of areas in synthesizable vhdl that need work > too, like user definable modes for record ports, expanded integer > synthesis (boolean operator definition, expanded range), as well as > fixed point representations along the lines of ada.
Record ports is high on my list too. Have to finish the verification features first. The other items you mention also sound interesting. Cheers, Jim
On Apr 3, 2:25 pm, Jim Lewis <j...@synthworks.com> wrote:
> Andy, > Thanks for your post. We are in the position where we need to > hear all users speak up (privately to me is ok), even when they > agree with what has already been stated. > > > > > Yes! I whole-heartedly support expanding the "testbench" features of > > VHDL. The main problem with using separate languages to create the > > testbench are two-fold. 1) multi-language simulators are more > > expensive, and 2) there are always some gotcha's at the interface when > > it comes to connecting the testbench to the UUT, especially if you use > > other than SL or SLV port types for your UUT. > > > That being said, I would caution against creating too much variance in > > VHDL between what can be done in a simulation only environment, vs a > > synthesis environment. OO principles come to mind. The eventual > > application of OO to synthesis should be carefully considered when > > developing the OO structure and syntax, even though it is now just for > > simulation. > > > Also, there are plenty of areas in synthesizable vhdl that need work > > too, like user definable modes for record ports, expanded integer > > synthesis (boolean operator definition, expanded range), as well as > > fixed point representations along the lines of ada. > > Record ports is high on my list too. Have to finish the verification > features first. The other items you mention also sound interesting. > > Cheers, > Jim
These new features are greatly needed and appreciated. -- Amal
On Apr 3, 6:10 pm, Jim Lewis <j...@synthworks.com> wrote:
> Hi, > Let me try this again. > > The VHDL standards community has been considering whether > to enhance VHDL to add advanced testbench features. > > If you are a VHDL user, > Do you want these features added to VHDL? > Would you rather adopt a verification language that > already supports these (SystemVerilog, SystemC, E, Vera). > > I think VHDL needs these features to stay competitive. > > The current plan in the VHDL working group is to make > these features similar to other verification languages > while at the same time keeping the nature of VHDL. >
This would be excellent, and I will be happy to use such features. VHDL already supports a large (compared, say, to Verilog) non synthesizable subset for the sake of testbenches. Then why not make it more useful.
Hi Jim,

Jim Lewis <jim@synthworks.com> writes:

> Classes / OO: > Classes are useful for creating verification data structures, > transaction communication, and grouping for transaction based > randomization (building relationships between separate data > items). Many of the data structures (such as scoreboards, > memories, and linked structures) can already be created, however, > classes give you the ability to hide all pointer manipulations. > For example, using a memory would require a declaration with > initialization of the memory data structure, then MemWrite and > MemRead would allow a user to store and retrieve items from the > data structure. Pointers and allocation of the sparse data > structure are handled by MemWrite and MemRead and the > user would not need to be aware of it. > Status: > Peter Ashenden submitted a class proposal last year and > provided updates to it this year at DVCon. Currently > he plans on finishing an updated draft soon. >
Is there anyway us non-members can see this? I'm on the reflector, and see the links to files, but of course, I can't see them :-( I'd like to see more OO, and I agree with Andy, I'd like synthesis of these extensions to be taken very seriously early on.
> Randomization: > Randomization is useful for designs that have numerous configurable > features. Testing features individually in an isolated manner is > typically straightforward. However, testing how these features > interact can be a large verification space &#2013266070; one that may not be able > to be simulated completely. It is also may be difficult to predict > all of the corner cases. Randomization has been used to sequence a > test in a non-deterministic way to get reasonably good coverage of > this verification space. > While I do not share the thought that randomization should be > adapted to work for all verification problems, I do believe it > to be a valuable technique for some problems. > > I wrote a draft of the randomization proposal and it is ready > for review. >
I have to admit, I've never had a need for random testing... yet. However, I can see where it would be useful!
> Functional Coverage: > Tool/structural coverage can tell you that you did a FIFO > read or that the the FIFO went empty, but it can't tell > you that you did a read while the FIFO was empty. > > Functional coverage constructs allow you to track this. > Some functional coverage capability will come from assertions > (since PSL has been integrated into VHDL). Additional constructs > will be added to allow data binning (coverage groups) and > correlation between different coverage items (cross coverage). > > I have started working on this - anyone else who is interested > is welcome to contribute as much as they would like. >
Yes, this would be great! Thanks for all your efforts! Cheers, Martin -- martin.j.thompson@trw.com TRW Conekt - Consultancy in Engineering, Knowledge and Technology http://www.conekt.net/electronics.html