Designing the right clock tree for a multi-FPGA setup

Hi folks,

I am planning to design a PCB featuring three Virtex-4 FX60 FPGAs and some 
periphery. The three FPGAs are needed due to the data processing complexity 
and the amount of high-speed IOs (MGTs). What I am most concerned about 
right now is to find an appropriate clocking solution.

In my opinion, there are mainly three alternatives to design the clocking 
scheme:

a) connection of the clock in a star-like topology, feeding each of the 
three FPGAs with the same clock signal (which has to be possibly duplicated 
by a clock buffer to generate three out of one clock reference signal, 
thereby introducing additional jitter)

b) clock in daisy-chain, feeding each of the three FPGAs with the identical 
clock signal which is routed from one device to another (in terms of jitter 
this is also not an optimal solution)

c) each FPGA device is supplied with its own clock (which than can be 
optimally routed to the device in short distances), but synchronization is a 
major issue then

Does anyone have sufficient experience in designing clock trees and is 
willing to share his experience, comments, hints and suggestions with me?

Thanks in advance

Gero 

Hi - 

Can you tell us a bit more about your requirements?  In particular:

 - What clock frequency are you distributing?

 - What are your synchronization requirements?

 - Do you plan to have synchronous buses or signals running between
the FPGAs?  What frequency are they running at?

Bob Perlman
Cambrian Design Works
http://www.cambriandesign.com


On Thu, 12 Jul 2007 17:56:19 +0200, "Geronimo Stempovski"
<geronimo.stempovski@arcor.de> wrote:

>Hi folks,
>
>I am planning to design a PCB featuring three Virtex-4 FX60 FPGAs and some 
>periphery. The three FPGAs are needed due to the data processing complexity 
>and the amount of high-speed IOs (MGTs). What I am most concerned about 
>right now is to find an appropriate clocking solution.
>
>In my opinion, there are mainly three alternatives to design the clocking 
>scheme:
>
>a) connection of the clock in a star-like topology, feeding each of the 
>three FPGAs with the same clock signal (which has to be possibly duplicated 
>by a clock buffer to generate three out of one clock reference signal, 
>thereby introducing additional jitter)
>
>b) clock in daisy-chain, feeding each of the three FPGAs with the identical 
>clock signal which is routed from one device to another (in terms of jitter 
>this is also not an optimal solution)
>
>c) each FPGA device is supplied with its own clock (which than can be 
>optimally routed to the device in short distances), but synchronization is a 
>major issue then
>
>Does anyone have sufficient experience in designing clock trees and is 
>willing to share his experience, comments, hints and suggestions with me?
>
>Thanks in advance
>
>Gero 
>

Bob,

Good point: depending on his requirements, he may have to BOTH send a
high quality, low jitter clock to all FPGAs, AND forward clocks from
each FPGA to other ones for source synchronous transfer operation on his
communications between devices.

Austin

On Jul 12, 11:56 am, "Geronimo Stempovski"
<geronimo.stempov...@arcor.de> wrote:
> Hi folks,
>
> I am planning to design a PCB featuring three Virtex-4 FX60 FPGAs and some
> periphery. The three FPGAs are needed due to the data processing complexity
> and the amount of high-speed IOs (MGTs). What I am most concerned about
> right now is to find an appropriate clocking solution.
>
> In my opinion, there are mainly three alternatives to design the clocking
> scheme:
>
> a) connection of the clock in a star-like topology, feeding each of the
> three FPGAs with the same clock signal (which has to be possibly duplicated
> by a clock buffer to generate three out of one clock reference signal,
> thereby introducing additional jitter)
>
> b) clock in daisy-chain, feeding each of the three FPGAs with the identical
> clock signal which is routed from one device to another (in terms of jitter
> this is also not an optimal solution)
>
> c) each FPGA device is supplied with its own clock (which than can be
> optimally routed to the device in short distances), but synchronization is a
> major issue then
>
> Does anyone have sufficient experience in designing clock trees and is
> willing to share his experience, comments, hints and suggestions with me?
>
> Thanks in advance
>
> Gero

Don't want to be too salesy - but why not take a look at Lattice Clock
Manager devices - very high performance, very low jitter, and per pin
voltage, termination, and skew control - just the ticket for FPGAs
without PLLs ;-)

http://www.latticesemi.com/products/ispclock/index.cfm?source=topnav&jsessionid=ba30b6308a23$24$03$3

(of course, our ECP2M  FPGA family might also be of interest ;-) )

hope the design goes well -

Mike Thomas
lattice SFAE NY/NJ

Okay, let's be more precise: The clock frequencies I'd like to distribute 
are in the range of 180 - 300 MHz, i.e. it is a challenging task. The signal 
busses between the FPGAs should carry signals in that range, too. Data is 
exchanged synchronously, so there is not much room for synchronization I 
think...!?

Gero 

In alt.engineering.electrical Geronimo Stempovski <geronimo.stempovski@arcor.de> wrote:

| a) connection of the clock in a star-like topology, feeding each of the 
| three FPGAs with the same clock signal (which has to be possibly duplicated 
| by a clock buffer to generate three out of one clock reference signal, 
| thereby introducing additional jitter)

Do you really need it buffered for isolation?  If you have enough clock
power, could you not feed all three from the one clock with impedance
matched lines?  What is the frequency, anyway?

-- 
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org)  /  Do not send to the address below |
| first name lower case at ipal.net   /  spamtrap-2007-07-13-0740@ipal.net |
|------------------------------------/-------------------------------------|

On Thu, 12 Jul 2007 17:56:19 +0200, "Geronimo Stempovski"
<geronimo.stempovski@arcor.de> wrote:

>Hi folks,
>
>I am planning to design a PCB featuring three Virtex-4 FX60 FPGAs and some 
>periphery. The three FPGAs are needed due to the data processing complexity 
>and the amount of high-speed IOs (MGTs). What I am most concerned about 
>right now is to find an appropriate clocking solution.
>
>In my opinion, there are mainly three alternatives to design the clocking 
>scheme:
>
>a) connection of the clock in a star-like topology, feeding each of the 
>three FPGAs with the same clock signal (which has to be possibly duplicated 
>by a clock buffer to generate three out of one clock reference signal, 
>thereby introducing additional jitter)
>
>b) clock in daisy-chain, feeding each of the three FPGAs with the identical 
>clock signal which is routed from one device to another (in terms of jitter 
>this is also not an optimal solution)
>
>c) each FPGA device is supplied with its own clock (which than can be 
>optimally routed to the device in short distances), but synchronization is a 
>major issue then
>
>Does anyone have sufficient experience in designing clock trees and is 
>willing to share his experience, comments, hints and suggestions with me?
>
>Thanks in advance
>
>Gero 
>

The most conservative way to do this would be to use three LVDS
drivers at the oscillator (or one of those new fancy LVDS clock
driver/tweaker chips), go differential star-routed to each chip, and
terminate at each one. Most FPGAs accept differential clocks these
days. That said, it's still a good idea to route the diff clock traces
carefully, low-skew, good impedance control, and away from any
crosstalk aggressors.

We also like to use a fast, hard 3.3 volt single-ended driver, source
terminated, and put a tiny logic schmitt trigger at the other end,
right next to the fpga clock input. FPGA clock inputs tend to have
little or no edge-noise immunity, and are hyper-delicate, so the
schmitt really helps if you're not running super fast and can tolerate
the added delay.

Long single-trace daisy chains are the riskiest, unless you regenerate
the clock at every way-station.

John

"Geronimo Stempovski" <geronimo.stempovski@arcor.de> wrote in message 
news:46964f24$0$3841$9b4e6d93@newsspool4.arcor-online.net...
> Hi folks,
>
> I am planning to design a PCB featuring three Virtex-4 FX60 FPGAs and some 
> periphery. The three FPGAs are needed due to the data processing 
> complexity and the amount of high-speed IOs (MGTs). What I am most 
> concerned about right now is to find an appropriate clocking solution.
>
> In my opinion, there are mainly three alternatives to design the clocking 
> scheme:
>
> a) connection of the clock in a star-like topology, feeding each of the 
> three FPGAs with the same clock signal (which has to be possibly 
> duplicated by a clock buffer to generate three out of one clock reference 
> signal, thereby introducing additional jitter)
>
Hi Geronimo,
This alternative will work. You should use a clock buffer to duplicate your 
clock three times.. The amount of jitter is generates will be at least an 
order of magnitude less than the jitter introduced just getting on to the 
FPGAs' internal clock trees.
>
> b) clock in daisy-chain, feeding each of the three FPGAs with the 
> identical clock signal which is routed from one device to another (in 
> terms of jitter this is also not an optimal solution)
>
You might also consider source synchronous busses to get data between FPGAs 
in addition to the star topology.
>
> c) each FPGA device is supplied with its own clock (which than can be 
> optimally routed to the device in short distances), but synchronization is 
> a major issue then
>
I see no advantages in using this third method.
>
> Does anyone have sufficient experience in designing clock trees and is 
> willing to share his experience, comments, hints and suggestions with me?
>
> Thanks in advance
>
> Gero
>
It's hard to advise you without a specific set of requirements, but I'd say 
your design stands a good chance of success simply because you're thinking 
hard about this up front! :-)
HTH., Syms. 

Gero,

OK, then I am correct, you not only need a good system synchronous clock
to each FPGA, but you also must use clock forwarding to send/receive
data between devices.

Austin

Geronimo Stempovski wrote:
> Okay, let's be more precise: The clock frequencies I'd like to distribute 
> are in the range of 180 - 300 MHz, i.e. it is a challenging task. The signal 
> busses between the FPGAs should carry signals in that range, too. Data is 
> exchanged synchronously, so there is not much room for synchronization I 
> think...!?
> 
> Gero 
> 
> 

"Symon" <symon_brewer@hotmail.com> schrieb im Newsbeitrag 
news:f7805h$1sk$1@aioe.org...
>>
> You might also consider source synchronous busses to get data between 
> FPGAs in addition to the star topology.
>>

Sounds interesting. What do you mean by that? Could you please go a little 
bit more into details? I heard about "source synchronous busses" a while ago 
but unfortunately I don't know what it is... sorry!

Thanks.

Gero