Real examples of metastability causing bugs

"John_H" <newsgroup@johnhandwork.com> wrote in message 
news:209b6b6e-a2eb-4ece-b4ac-3e5488271f4f@v67g2000hse.googlegroups.com...

> While XST may use the inputs in implementing the logic, it uses the
> synchronous set/reset rather than the asynchronous clear/preset
> equivalent.  Any synchronous implementation should be covered by the
> timing analysis.

OK, of course. Thanks, John_H! 

John_H wrote:
(snip)

> A real life situation for a missing synchronizer would be the "duh" 
> moment I had in one of my first processor interfaced designs.  I wrote a 
> value to the FPGA but the register write wasn't related to the system 
> clock.  Occasionally some of my logic got one part of the word on one 
> cycle before all the logic got the full register value on the second 
> cycle.  This schism where values were *supposed* to change 
> simultaneously but didn't caused me problems.

Again, I wouldn't call this metastability.  It can be made worse
by metastability, but the problem is that the propagation delay
to the different parts of the register is (always) slightly
different, and if you get close enough to the clock edge some will
get one value, and some the other.  That will still be true even
for perfect FFs.

> Just adding a synchronizing flop DOES NOT get rid of metastability 
> troubles.  Just adding two consecutive synchronizing flops DOES NOT get 
> rid of metastability.  Luckily most of the time the path between the two 
> flops ends up being short but too often the designer DOES NOT properly 
> constrain the path between those two flops.

Synchronizing FFs don't get rid of it, but if the probability is low
enough that is good enough.  Two synchronizing FFs will square the
probability of metastability on each clock cycle (assuming statistical
independence).

> The effect of hitting the metastability window is that the logic takes a 
> moment longer to decide if it's high or low.  While the static timing 
> analyzer will guarantee your results if you meet setup and hold, the 
> synchronizing flop specifically violates the setup and hold in order to 
> rarely hit that metastability window.  In this case, the constraints 
> MUST be changed to guarantee the metastability related errors will be in 
> the 1k or 1M year kind of range.

First the logic must be designed to avoid the multiple register
clocking problem.  In the case of FIFOs this is done by using gray
code such that only one bit changes on any cycle.  You get one or
the other, where both are valid.

The metastability problem comes not while crossing clock domains,
but after crossing clock domains.  It is normal for the output of
a register to go through other logic before the next register.
The delay of that logic, in addition to the possible metastability
delay, causes the metastability problem.  Efficient logic design
maximizes the logic between register stages, and so gets closer
to failure due to metastability caused delay.  A synchronizing
register allows the maximum time for metastability to be
resolved before entering the next FF.

> The timing constraint from a synchronizing flop to the next flop in the 
> sequence MUST be constrained to a time that's shorter than the 
> prevailing system clock period.  If the signal takes up to 2 ns longer 
> to decide what signal level it is, the following logic (or second 
> synchronizing flop) must have that additional headroom built-in through 
> timing constraints.

Yes.

-- glen

-jg wrote:

(snip)

> Most discussions use a simple model ot a log-log plot,
> on a couple of data points -  fine as a 'there be dragons' 
 > type warning, but not what I'd call true engineering..

The assumption is that it is exponential.  I don't know if you
can prove that or not.  Also, the measurements are not easy
and the result will be very sensitive to the exact timing.

I had the idea once in a discussion here of building a metastability
locked loop.  That is, a PLL with a FF in the feedback loop such that
the phase adjustment goes toward the metastability point.  That would
maximize the number of metastability events.  Then you need to find
a way to measure the resolving time and graph it...

-- glen

On Jan 9, 10:48=A0am, glen herrmannsfeldt <g...@ugcs.caltech.edu> wrote:
> -jg wrote:
>
> (snip)
>
> > Most discussions use a simple model ot a log-log plot,
> > on a couple of data points - =A0fine as a 'there be dragons'
>
Glen,
I have been thinking about this for decades, but I now consider it
hopeless.
If you believe the results of my statistical measurements, then you
realize that the capture window for a metastable delay of more than 2
ns is not picoseconds, but a fraction of a femtosecond.
There is no way to keep the circuitry stable within such a narrow
timing window. And if you could, how do you derive any quantitative
data from it?
I remain convinced that the randomly asynchronous testing approach is
the only one that gives us reliable results.
Peter Alfke

> =A0> type warning, but not what I'd call true engineering..
>
> The assumption is that it is exponential. =A0I don't know if you
> can prove that or not. =A0Also, the measurements are not easy
> and the result will be very sensitive to the exact timing.
>
> I had the idea once in a discussion here of building a metastability
> locked loop. =A0That is, a PLL with a FF in the feedback loop such that
> the phase adjustment goes toward the metastability point. =A0That would
> maximize the number of metastability events. =A0Then you need to find
> a way to measure the resolving time and graph it...
>
> -- glen

On Jan 9, 10:35=A0am, glen herrmannsfeldt <g...@ugcs.caltech.edu> wrote:
<snip>
> ... Efficient logic design
> maximizes the logic between register stages, and so gets closer
> to failure due to metastability caused delay. =A0A synchronizing
> register allows the maximum time for metastability to be
> resolved before entering the next FF.

A single synchronizing register DOES NOT allow the maximum time for
metastability.  Two synchronizing registers are closer to "correct."

With only one synchronizing flop for one control signal - ignoring
vectors for the moment - the only way to guarantee logic will work
properly with this single flop on "this side" of the time domain is to
tighten the timing constraint such that any metastability delay is
acceptable in the system.  If the tightened constraint is too
difficult, a second single flop is needed to distribute the
synchronized signal.

- John_H

In response to my post:

 >>I had the idea once in a discussion here of building a metastability
 >>locked loop.  That is, a PLL with a FF in the feedback loop such that
 >>the phase adjustment goes toward the metastability point.  That would
 >>maximize the number of metastability events.  Then you need to find
 >>a way to measure the resolving time and graph it...

Peter Alfke wrote:

> I have been thinking about this for decades, but I now consider it
> hopeless.

Much longer that I have thought about it.

> If you believe the results of my statistical measurements, then you
> realize that the capture window for a metastable delay of more than 2
> ns is not picoseconds, but a fraction of a femtosecond.
> There is no way to keep the circuitry stable within such a narrow
> timing window. 

It did occur to me while writing that, that temperature variations
would shift the metastable point.  The goal of the MLL is to maximize
the rate of observations of metastability...

 > And if you could, how do you derive any quantitative
 > data from it?

The only one I have thought of so far is to add a variable
voltage sine to the MLL feedback loop.  Increasing the voltage
should decrease the probability of 2ns metastability events.

Not having the design for the feedback loop of the MLL, it seems
that there might be a gate with a signal that is high for the
amount of time of the metastability, plus or minus some propagation
delays.  Those delays would have to be measured, and then the
average metastability time could be measured by the average
voltage on that line.  The change in the average time with
the sine voltage disturbing the feedback loop, and the time
variation due to the feedback voltage, would allow one to
determine the probability vs. metastability time curve.

> I remain convinced that the randomly asynchronous testing
> approach is the only one that gives us reliable results.

It would seem to require simpler measurements than the MLL.

-- glen

The asynchronous test is a real beauty.
By adjusting one frequency, you can have the detected metstable events
come in at kilohertz speed, or at the much lower rate of one or two
during the lunch break or even overnight.
And it confirms the logarithmic relationship.
Real fun! (If the phenomenon itself weren't so ugly...)
Peter Alfke

Hi John,
Just to point out that when using two FFs to mitigate metastability, the 
constraints file should include something (e.g. MAXDELAY) to make sure the 
signal delay between the two FFs is somewhat less than the default, which is 
the period of the FFs' clock. The P&R tools may not do this otherwise.
Cheers, Syms. 

On Wed, 09 Jan 2008 14:22:05 GMT, "KJ" <kkjennings@sbcglobal.net>
wrote:

[snip]
>>
>> The other point of my post was that because everyone has heard of
>> metastability and that it's usually easy to deal with - just add flip
>> flops and some timing slack - it gets taken into account in designs
>> and doesn't actually cause a lot of bugs.
>I think you're overestimating new designers ability to properly add these 
>flops based on postings in this and other newsgroups even when the poster 
>seems to have knowledge of metastability.

I wasn't talking about new designers or Usenet posters, I was talking
about a large group of experienced designers at Agilent.  They
wouldn't have been employed there if they didn't have a basic grasp of
fundamentals such as designing for metastability.

I apologise for not making it clear I wasn't talking about noobs.

[snip]

Allan

Symon wrote:

> I thought from the OP's post that he means that this 'input' has already 
> been sampled in the system clock domain, although it's not clear. If that is 
> the case, assuming setup and hold are met, this is a metastability problem. 
> If metastable signals only ever go to one place it's not a problem. That's 
> how the input resampler works.

Yes. The synthesis tools assume input synchronization.
When synchronization fails all bets are off.
This thread points out the importance of being able
to distinguish structures intended to *be* synchronizers
from structures that assume such synchronization.
I beginning to think I should code synchronizers as
separate entities/modules. This would simplify constraints
and make it easier to check for register duplication.

> p.s. FYI 
> http://www-ee.eng.hawaii.edu/~msmith/ASICs/HTML/Problems/asp06/PDF/8175.pdf 
> this FF is immune. (Actually, of course it isn't, but it's interesting to 
> see why it doesn't work.) 

Funny. I'm wiping the coffee off my monitor ;)
These inventions always move the problem around
but never solve it.

        -- Mike Treseler