Non deterministic behaviour in quartus II ?

Today I have observed again a kind of unpredicatable behaviour in
quartus II that I had already experienced in another project.

In each case, the situation was:
- A specific signal on a design does not work as expected.
- To debug, an output pin is added to monitor a signal around the
problematic part of the circuit. No other changes are made.
- As a consequence of adding this pin, the original signal works ok.
- The debugging pin is removed and the design works perfectly.

It seems as if the compilation process is not 100% deterministic but
relies on previous runs. The same kind of problem has (had) been
experienced in quartus 4.2 and 5.1. Is there any option to force
quartus II to start each compilation from scratch?

Of course, there is the possibility that I am missing some point. Any
hint?

Thanks for your time

Pere

"oopere" <oopere@netscape.net> wrote in message 
news:1163007119.654717.219530@m73g2000cwd.googlegroups.com...
>
> Of course, there is the possibility that I am missing some point. Any
> hint?

Do you have any idea *why* your cicuit doesn't work when it doesn't?  That 
would be a useful clue.

I've had a design break after a Quartus upgrade, but it was only because it 
was a bad design and a subtle change in timing caused it to break. 
(Inadequate use of timing constraints on the design concealed this.)

I've also had one which wouldn't run when the FPGA was cold - I had to ship 
the equipment to a factory in the Pacific North West to find that one.  That 
was an unsynchronised state-machine input - again, just subtle timing 
changes in an already-broken design.

There are lots of things which make FPGA designs flakey, and despite what 
we'd like to believe, many of us are at least as unreliable as the 
toolchain...   You only have to read this group for a couple of weeks to see 
how often people get the 'async input' stuff wrong, for example.

Unless you're using 'incremental compilation' in Quartus, I think it always 
does start from scratch at the earliest phase necessary (I'm assuming 'smart 
compilation').  If you're really suspicious then delete the 'db' directory 
below your design.

Will

Will Dean wrote:
> "oopere" <oopere@netscape.net> wrote in message
> news:1163007119.654717.219530@m73g2000cwd.googlegroups.com...
> >
> > Of course, there is the possibility that I am missing some point. Any
> > hint?
>
> Do you have any idea *why* your cicuit doesn't work when it doesn't?  That
> would be a useful clue.
>

No idea why it did not work:

Part of the original design:

signal[14..0]--|-----
               | XOR >----DFF------out[14..0]
PN[14..0]------|-----

Even though "PN" changed at 15Mbit/s, "out" changed only at 1Mbit/s
(signal data rate)

I added a single xor for the MSBs of each:


signal[14]--|-----
            | XOR >-----dummy_out
PN[14]------|-----

and routed dummy_out outside of the block and to an output pin. Then
everything is ok: "out" changes at 15Mbit/s.

The funny thing is that after removing the dummy_out, everything runs
fine now.

And, yes, I am pretty sure that I did not change anything else.

Pere

oopere wrote:

> Of course, there is the possibility that I am missing some point. Any
> hint?

Did it meet timing?
Were your timing constraints actually correct?
Is it a synchronous design?
Are you using gated clocks?

Could be a lot of things before you point the finger at Quartus...

Did you delete your 'db' directory and compile from scratch?

Regards,

-- 
Mark McDougall, Engineer
Virtual Logic Pty Ltd, <http://www.vl.com.au>
21-25 King St, Rockdale, 2216
Ph: +612-9599-3255 Fax: +612-9599-3266

oopere wrote:
> Today I have observed again a kind of unpredicatable behaviour in
> quartus II that I had already experienced in another project.
>
> In each case, the situation was:
> - A specific signal on a design does not work as expected.
> - To debug, an output pin is added to monitor a signal around the
> problematic part of the circuit. No other changes are made.
> - As a consequence of adding this pin, the original signal works ok.
> - The debugging pin is removed and the design works perfectly.
>
> It seems as if the compilation process is not 100% deterministic but
> relies on previous runs. The same kind of problem has (had) been
> experienced in quartus 4.2 and 5.1. Is there any option to force
> quartus II to start each compilation from scratch?
>
> Of course, there is the possibility that I am missing some point. Any
> hint?
> 
> Thanks for your time
> 
> Pere

Mark McDougall wrote:
> oopere wrote:
>
> > Of course, there is the possibility that I am missing some point. Any
> > hint?
>
> Did it meet timing?
> Were your timing constraints actually correct?
> Is it a synchronous design?
> Are you using gated clocks?
>
> Could be a lot of things before you point the finger at Quartus...

Sure, there may be problems in my design. However, the main point of my
post is that the circuit behaviour is different in the following cases:

- State a). Original design
- State b). Originial design + output pin added + compiled + output pin
removed + compiled

And I have observed this kind of behavior happening twice, in different
designs. One aprox. one year ago and the other yesterday.

If I am the only one who has experienced this then obviously I am doing
something wrong, somewhere. But where?

Mark McDougall wrote:
> oopere wrote:
>
> > Of course, there is the possibility that I am missing some point. Any
> > hint?
>
> Did it meet timing?
> Were your timing constraints actually correct?
> Is it a synchronous design?
> Are you using gated clocks?
>
> Could be a lot of things before you point the finger at Quartus...

Sure, there may be problems in my design. However, the main point of my
post is that the circuit behaviour is different in the following cases:

- State a). Original design
- State b). Originial design + output pin added + compiled + output pin
removed + compiled

And I have observed this kind of behavior happening twice, in different
designs. One aprox. one year ago and the other yesterday.

If I am the only one who has experienced this then obviously I am doing
something wrong, somewhere. But where?

"oopere" <oopere@netscape.net> wrote in message 
news:1163062587.507437.291270@h48g2000cwc.googlegroups.com...
>
> If I am the only one who has experienced this then obviously I am doing
> something wrong, somewhere. But where?

If it's a timing problem which is affected by the temperature of the 
circuit, then getting it working by one method might mean that the whole 
thing then warms up to the point where it then continues to work in the old 
configuration.

I'm afraid I didn't really understand your description of the design 
yesterday, but I expect that you make sure you're rock solid on setup times 
(async inputs into multiple FF's) and timing constraints, then it will all 
work much more consistently.  If you have signals in multiple clock domains, 
which I suspect you do, then you probably need to treat them a lot more 
carefully than just XORing them together.

You're certainly not the only one to experience bizarrely inconsistent 
problems which are apparently related to something they later turn-out not 
to be related to.   If it's only working by luck in the first place, I 
wouldn't waste time by trying to second-guess precisely what causes that 
luck to run out.

Will

> - State a). Original design
> - State b). Originial design + output pin added + compiled + output pin
> removed + compiled

This is not the expected behaviour of Quartus.  Imagine how difficult
our job would be if we could not reproduce outputs for a given set of
inputs...

Quartus is deterministic.  The same input source and project settings
run on the same version of Quartus on the same OS, you should get the
same answer.

Try diff'ing the output files of Quartus for your two cases.  At what
point in the compile do the report and log files differ?  That may give
a clue as to what is causing this difference in observed behaviuor.
Are you absolutely positive that (b) gives the same inputs as (a)?  For
example, if you nuke your db/ directory in (a), and do the same in (b),
do you then get the same results?

If you can show that this is non-deterministic, please email me so that
we can arrange to get your test case.  We do occasionally find
non-determinism in the software due to safe but bad programming
practices -- for example, hashing on non-deterministic data.  But we
run determinism tests on the software, so these are usually flagged
in-house before the software ever reaches our users.

Regards,

Paul Leventis
Altera Corp.

Paul Leventis wrote:

> > - State a). Original design
> > - State b). Originial design + output pin added + compiled + output pin
> > removed + compiled
>
> This is not the expected behaviour of Quartus.  Imagine how difficult
> our job would be if we could not reproduce outputs for a given set of
> inputs...
>
> Quartus is deterministic.  The same input source and project settings
> run on the same version of Quartus on the same OS, you should get the
> same answer.
>
> Try diff'ing the output files of Quartus for your two cases.  At what
> point in the compile do the report and log files differ?  That may give
> a clue as to what is causing this difference in observed behaviuor.
> Are you absolutely positive that (b) gives the same inputs as (a)?  For
> example, if you nuke your db/ directory in (a), and do the same in (b),
> do you then get the same results?
>
> If you can show that this is non-deterministic, please email me so that
> we can arrange to get your test case.  We do occasionally find
> non-determinism in the software due to safe but bad programming
> practices -- for example, hashing on non-deterministic data.  But we
> run determinism tests on the software, so these are usually flagged
> in-house before the software ever reaches our users.
>
> Regards,
>
> Paul Leventis
> Altera Corp.

Thanks for your response.

By this time I have fixed the problem that caused the original circuit
not to work as supposed. There was some critical timing issue that
caused a shift register a) not to load b) to load only certain
flip-flops or c) to load correctly depending on how fast the "load" pin
was deasserted after the clock edge. My fault.

Coming to the "non-deterministic" question, I am pretty sure that
Quartus does a correct job. To find an explanation, I wonder if it is
possible that the same schematic ,i.e. screen drawing, may be given
different internal representations. I explain with an example: Many
years ago I wrote a circuit simulator where I assigned each circuit
node a number which was internally generated. If the user addded and
removed stuff, the internal node numbers changed but the screen
appearance was the same. In this case, this could not affect the
circuit response.

However, in this case I can imagine that the fitter tool, if all
settings are the same, starts to place things perhaps by ascending
order of an internal variable which may be different even if the screen
"appearence" is the same -perhaps due to repeated adding or deleting??

Could it be that, if there are no restrictions acting, the decision on
where to start placing may be based on some internal ordering, which
could cause those minor timing differences -which in the reported case
caused big behavior diferences due do a bad design?

(Sorry that I can't send you the original state, because, as said, it
worked after editing and this was done on the same project)

Pere Pal=E0