Re: Xilinx GbE performance

On Tue, 2009-06-02 at 09:53 -0700, Antti wrote:
> Hi
> 
> does anybody have real and realistic performance figures for Xilinx
> GbE solution with XPS_TEMAC/MPMC ?
> 
> we need to get 60% of GbE wirespeed, UDP transmit only but it seems
> like real hard target to reach :(
> 
> MPMC has memory latency of 23 cycles (added to EACH memory access
> cycle) so the ethernet
> SDMA takes a lot of bandwith already, there is another DMA writing
> data at same speed, and the
> PPC itself uses the same memory too
> 
> Antti

With custom Ethernet core + MPMC we get data rates slightly above
100MBps, depending on MTU. The single memory is shared by MicroBlaze/PPC
for code and data access, at least one streaming data source (custom PIM
for NPI) and the custom Ethernet IP (MAC + some packet composers,
decoders, etc.) again connected to NPI.
We rejected to use XPS_TEMAC because its low performance. The problem is
I lost my benchmark results. Sorry.

Jan

On 2 June, 21:05, Jan Pech <inva...@void.domain> wrote:
> On Tue, 2009-06-02 at 09:53 -0700, Antti wrote:
> > Hi
>
> > does anybody have real and realistic performance figures for Xilinx
> > GbE solution with XPS_TEMAC/MPMC ?
>
> > we need to get 60% of GbE wirespeed, UDP transmit only but it seems
> > like real hard target to reach :(
>
> > MPMC has memory latency of 23 cycles (added to EACH memory access
> > cycle) so the ethernet
> > SDMA takes a lot of bandwith already, there is another DMA writing
> > data at same speed, and the
> > PPC itself uses the same memory too
>
> > Antti
>
> With custom Ethernet core + MPMC we get data rates slightly above
> 100MBps, depending on MTU. The single memory is shared by MicroBlaze/PPC
> for code and data access, at least one streaming data source (custom PIM
> for NPI) and the custom Ethernet IP (MAC + some packet composers,
> decoders, etc.) again connected to NPI.
> We rejected to use XPS_TEMAC because its low performance. The problem is
> I lost my benchmark results. Sorry.
>
> Jan

hum.. my current task is to optimize a XPS_TEMAC based system
(with 1 single DDR2 chip as main memory!)
to reach about 580MBps

:(

I have never said that to be possible, but i need money :(
and if the goal cant be reached there will be none...

over 100MBps is sure possible (with XPS_TEMAC too)
 but 580MBps is beyong doable i think for sure

Antti

<Antti.Lukats@googlemail.com> wrote in message 
news:45a07ecd-3a6c-4047-a640-cb5706d0b26b@k2g2000yql.googlegroups.com...
> On 2 June, 21:05, Jan Pech <inva...@void.domain> wrote:
>> On Tue, 2009-06-02 at 09:53 -0700, Antti wrote:
>> > Hi
>>
>> > does anybody have real and realistic performance figures for Xilinx
>> > GbE solution with XPS_TEMAC/MPMC ?
>>
>> > we need to get 60% of GbE wirespeed, UDP transmit only but it seems
>> > like real hard target to reach :(
>>
>> > MPMC has memory latency of 23 cycles (added to EACH memory access
>> > cycle) so the ethernet
>> > SDMA takes a lot of bandwith already, there is another DMA writing
>> > data at same speed, and the
>> > PPC itself uses the same memory too
>>
>> > Antti
>>
>> With custom Ethernet core + MPMC we get data rates slightly above
>> 100MBps, depending on MTU. The single memory is shared by MicroBlaze/PPC
>> for code and data access, at least one streaming data source (custom PIM
>> for NPI) and the custom Ethernet IP (MAC + some packet composers,
>> decoders, etc.) again connected to NPI.
>> We rejected to use XPS_TEMAC because its low performance. The problem is
>> I lost my benchmark results. Sorry.
>>
>> Jan
>
> hum.. my current task is to optimize a XPS_TEMAC based system
> (with 1 single DDR2 chip as main memory!)
> to reach about 580MBps
>
> :(
>
> I have never said that to be possible, but i need money :(
> and if the goal cant be reached there will be none...
>
> over 100MBps is sure possible (with XPS_TEMAC too)
> but 580MBps is beyong doable i think for sure
>
> Antti
>
>

>
 >

>over 100MBps is sure possible (with XPS_TEMAC too)

really? over GbE?  impossible!

I take it you mean over 100Mbps which is far more plausible.

Phil 

On Tue, 2009-06-02 at 11:32 -0700, Antti.Lukats@googlemail.com wrote:
> On 2 June, 21:05, Jan Pech <inva...@void.domain> wrote:
> > On Tue, 2009-06-02 at 09:53 -0700, Antti wrote:
> > > Hi
> >
> > > does anybody have real and realistic performance figures for Xilinx
> > > GbE solution with XPS_TEMAC/MPMC ?
> >
> > > we need to get 60% of GbE wirespeed, UDP transmit only but it seems
> > > like real hard target to reach :(
> >
> > > MPMC has memory latency of 23 cycles (added to EACH memory access
> > > cycle) so the ethernet
> > > SDMA takes a lot of bandwith already, there is another DMA writing
> > > data at same speed, and the
> > > PPC itself uses the same memory too
> >
> > > Antti
> >
> > With custom Ethernet core + MPMC we get data rates slightly above
> > 100MBps, depending on MTU. The single memory is shared by MicroBlaze/PPC
> > for code and data access, at least one streaming data source (custom PIM
> > for NPI) and the custom Ethernet IP (MAC + some packet composers,
> > decoders, etc.) again connected to NPI.
> > We rejected to use XPS_TEMAC because its low performance. The problem is
> > I lost my benchmark results. Sorry.
> >
> > Jan
> 
> hum.. my current task is to optimize a XPS_TEMAC based system
> (with 1 single DDR2 chip as main memory!)
> to reach about 580MBps
> 
> :(
> 
> I have never said that to be possible, but i need money :(
> and if the goal cant be reached there will be none...
> 
> over 100MBps is sure possible (with XPS_TEMAC too)
>  but 580MBps is beyong doable i think for sure
> 
> Antti
> 

Just a simple calculation:
125000000 / 1024 / 1024 = 119.2MBps
It is without protocol overhead, FCS, IFGs. How do you want to exceed
limit of Gigabit Ethernet?

Jan

On Tue, 2009-06-02 at 20:47 +0200, Jan Pech wrote:
> On Tue, 2009-06-02 at 11:32 -0700, Antti.Lukats@googlemail.com wrote:
> > On 2 June, 21:05, Jan Pech <inva...@void.domain> wrote:
> > > On Tue, 2009-06-02 at 09:53 -0700, Antti wrote:
> > > > Hi
> > >
> > > > does anybody have real and realistic performance figures for Xilinx
> > > > GbE solution with XPS_TEMAC/MPMC ?
> > >
> > > > we need to get 60% of GbE wirespeed, UDP transmit only but it seems
> > > > like real hard target to reach :(
> > >
> > > > MPMC has memory latency of 23 cycles (added to EACH memory access
> > > > cycle) so the ethernet
> > > > SDMA takes a lot of bandwith already, there is another DMA writing
> > > > data at same speed, and the
> > > > PPC itself uses the same memory too
> > >
> > > > Antti
> > >
> > > With custom Ethernet core + MPMC we get data rates slightly above
> > > 100MBps, depending on MTU. The single memory is shared by MicroBlaze/PPC
> > > for code and data access, at least one streaming data source (custom PIM
> > > for NPI) and the custom Ethernet IP (MAC + some packet composers,
> > > decoders, etc.) again connected to NPI.
> > > We rejected to use XPS_TEMAC because its low performance. The problem is
> > > I lost my benchmark results. Sorry.
> > >
> > > Jan
> > 
> > hum.. my current task is to optimize a XPS_TEMAC based system
> > (with 1 single DDR2 chip as main memory!)
> > to reach about 580MBps
> > 
> > :(
> > 
> > I have never said that to be possible, but i need money :(
> > and if the goal cant be reached there will be none...
> > 
> > over 100MBps is sure possible (with XPS_TEMAC too)
> >  but 580MBps is beyong doable i think for sure
> > 
> > Antti
> > 
> 
> Just a simple calculation:
> 125000000 / 1024 / 1024 = 119.2MBps
> It is without protocol overhead, FCS, IFGs. How do you want to exceed
> limit of Gigabit Ethernet?
> 
> Jan
> 

Or did I get you wrong and you talk about Mbits per second? I was
talking about Mbytes per sec.
If it is so, your goal should be reachable using xps_ll_temac instead of
xps_temac.
Jan

On 2 June, 21:46, "Phil Jessop" <p...@noname.org> wrote:
> <Antti.Luk...@googlemail.com> wrote in message
>
> news:45a07ecd-3a6c-4047-a640-cb5706d0b26b@k2g2000yql.googlegroups.com...
>
>
>
> > On 2 June, 21:05, Jan Pech <inva...@void.domain> wrote:
> >> On Tue, 2009-06-02 at 09:53 -0700, Antti wrote:
> >> > Hi
>
> >> > does anybody have real and realistic performance figures for Xilinx
> >> > GbE solution with XPS_TEMAC/MPMC ?
>
> >> > we need to get 60% of GbE wirespeed, UDP transmit only but it seems
> >> > like real hard target to reach :(
>
> >> > MPMC has memory latency of 23 cycles (added to EACH memory access
> >> > cycle) so the ethernet
> >> > SDMA takes a lot of bandwith already, there is another DMA writing
> >> > data at same speed, and the
> >> > PPC itself uses the same memory too
>
> >> > Antti
>
> >> With custom Ethernet core + MPMC we get data rates slightly above
> >> 100MBps, depending on MTU. The single memory is shared by MicroBlaze/P=
PC
> >> for code and data access, at least one streaming data source (custom P=
IM
> >> for NPI) and the custom Ethernet IP (MAC + some packet composers,
> >> decoders, etc.) again connected to NPI.
> >> We rejected to use XPS_TEMAC because its low performance. The problem =
is
> >> I lost my benchmark results. Sorry.
>
> >> Jan
>
> > hum.. my current task is to optimize a XPS_TEMAC based system
> > (with 1 single DDR2 chip as main memory!)
> > to reach about 580MBps
>
> > :(
>
> > I have never said that to be possible, but i need money :(
> > and if the goal cant be reached there will be none...
>
> > over 100MBps is sure possible (with XPS_TEMAC too)
> > but 580MBps is beyong doable i think for sure
>
> > Antti
>
> =A0>
>
> >over 100MBps is sure possible (with XPS_TEMAC too)
>
> really? over GbE? =A0impossible!
>
> I take it you mean over 100Mbps which is far more plausible.
>
> Phil

yes, sorry, i did mean
XPS_TEMAC/MPMC, GbE (1000 base-X fiber)

> 100MBps is OK
580MBps -- hardly possible

Antti

On 2 June, 21:47, Jan Pech <inva...@void.domain> wrote:
> On Tue, 2009-06-02 at 11:32 -0700, Antti.Luk...@googlemail.com wrote:
> > On 2 June, 21:05, Jan Pech <inva...@void.domain> wrote:
> > > On Tue, 2009-06-02 at 09:53 -0700, Antti wrote:
> > > > Hi
>
> > > > does anybody have real and realistic performance figures for Xilinx
> > > > GbE solution with XPS_TEMAC/MPMC ?
>
> > > > we need to get 60% of GbE wirespeed, UDP transmit only but it seems
> > > > like real hard target to reach :(
>
> > > > MPMC has memory latency of 23 cycles (added to EACH memory access
> > > > cycle) so the ethernet
> > > > SDMA takes a lot of bandwith already, there is another DMA writing
> > > > data at same speed, and the
> > > > PPC itself uses the same memory too
>
> > > > Antti
>
> > > With custom Ethernet core + MPMC we get data rates slightly above
> > > 100MBps, depending on MTU. The single memory is shared by MicroBlaze/=
PPC
> > > for code and data access, at least one streaming data source (custom =
PIM
> > > for NPI) and the custom Ethernet IP (MAC + some packet composers,
> > > decoders, etc.) again connected to NPI.
> > > We rejected to use XPS_TEMAC because its low performance. The problem=
 is
> > > I lost my benchmark results. Sorry.
>
> > > Jan
>
> > hum.. my current task is to optimize a XPS_TEMAC based system
> > (with 1 single DDR2 chip as main memory!)
> > to reach about 580MBps
>
> > :(
>
> > I have never said that to be possible, but i need money :(
> > and if the goal cant be reached there will be none...
>
> > over 100MBps is sure possible (with XPS_TEMAC too)
> > =A0but 580MBps is beyong doable i think for sure
>
> > Antti
>
> Just a simple calculation:
> 125000000 / 1024 / 1024 =3D 119.2MBps
> It is without protocol overhead, FCS, IFGs. How do you want to exceed
> limit of Gigabit Ethernet?
>
> Jan

ups silly me :(
B is as byte

I wanted say we need 580Mbit/s

Antti

On 2 June, 21:50, Jan Pech <inva...@void.domain> wrote:
> On Tue, 2009-06-02 at 20:47 +0200, Jan Pech wrote:
> > On Tue, 2009-06-02 at 11:32 -0700, Antti.Luk...@googlemail.com wrote:
> > > On 2 June, 21:05, Jan Pech <inva...@void.domain> wrote:
> > > > On Tue, 2009-06-02 at 09:53 -0700, Antti wrote:
> > > > > Hi
>
> > > > > does anybody have real and realistic performance figures for Xili=
nx
> > > > > GbE solution with XPS_TEMAC/MPMC ?
>
> > > > > we need to get 60% of GbE wirespeed, UDP transmit only but it see=
ms
> > > > > like real hard target to reach :(
>
> > > > > MPMC has memory latency of 23 cycles (added to EACH memory access
> > > > > cycle) so the ethernet
> > > > > SDMA takes a lot of bandwith already, there is another DMA writin=
g
> > > > > data at same speed, and the
> > > > > PPC itself uses the same memory too
>
> > > > > Antti
>
> > > > With custom Ethernet core + MPMC we get data rates slightly above
> > > > 100MBps, depending on MTU. The single memory is shared by MicroBlaz=
e/PPC
> > > > for code and data access, at least one streaming data source (custo=
m PIM
> > > > for NPI) and the custom Ethernet IP (MAC + some packet composers,
> > > > decoders, etc.) again connected to NPI.
> > > > We rejected to use XPS_TEMAC because its low performance. The probl=
em is
> > > > I lost my benchmark results. Sorry.
>
> > > > Jan
>
> > > hum.. my current task is to optimize a XPS_TEMAC based system
> > > (with 1 single DDR2 chip as main memory!)
> > > to reach about 580MBps
>
> > > :(
>
> > > I have never said that to be possible, but i need money :(
> > > and if the goal cant be reached there will be none...
>
> > > over 100MBps is sure possible (with XPS_TEMAC too)
> > > =A0but 580MBps is beyong doable i think for sure
>
> > > Antti
>
> > Just a simple calculation:
> > 125000000 / 1024 / 1024 =3D 119.2MBps
> > It is without protocol overhead, FCS, IFGs. How do you want to exceed
> > limit of Gigabit Ethernet?
>
> > Jan
>
> Or did I get you wrong and you talk about Mbits per second? I was
> talking about Mbytes per sec.
> If it is so, your goal should be reachable using xps_ll_temac instead of
> xps_temac.
> Jan

oh, i am talking wrong today
yes Mbit/sec or Mbps
and sure XPS_LL_TEMAC with ALL hardware options tuned to maximum
and we do not copy buffers, and do not calc UDP checksum with PPC

but, even TRECKs marketing booklet promised only 355 Mbps for MTU1500
abd i need 580Mbps

Antti

<Antti.Lukats@googlemail.com> wrote in message news:244fac96-a937-48b4-949e-
>
> but, even TRECKs marketing booklet promised only 355 Mbps for MTU1500
> abd i need 580Mbps

Antti,

I think Treck numbers assume TCP/IP. I am actually in the middle of 
evaluating the same thing. I have a design similar to the one described in 
XAPP1041 running on a custom V4FX60 board and I seem to be getting the 
numbers you are looking for (raw Ethernet Tx traffic) although it is early 
for me to say whether they are "real", i.e. I haven't yet analyzed properly 
what the Xilinx perf_app software does.


/Mikhail 

<snip>
> > > > hum.. my current task is to optimize a XPS_TEMAC based system
> > > > (with 1 single DDR2 chip as main memory!)
> > > > to reach about 580MBps
>
> > > > :(
>
> > > > I have never said that to be possible, but i need money :(
> > > > and if the goal cant be reached there will be none...
>
> > > > over 100MBps is sure possible (with XPS_TEMAC too)
> > > > =A0but 580MBps is beyong doable i think for sure
>
> > > > Antti
>
> > > Just a simple calculation:
> > > 125000000 / 1024 / 1024 =3D 119.2MBps
> > > It is without protocol overhead, FCS, IFGs. How do you want to exceed
> > > limit of Gigabit Ethernet?
>
> > > Jan
>
> > Or did I get you wrong and you talk about Mbits per second? I was
> > talking about Mbytes per sec.
> > If it is so, your goal should be reachable using xps_ll_temac instead o=
f
> > xps_temac.
> > Jan
>
> oh, i am talking wrong today
> yes Mbit/sec or Mbps
> and sure XPS_LL_TEMAC with ALL hardware options tuned to maximum
> and we do not copy buffers, and do not calc UDP checksum with PPC
>
> but, even TRECKs marketing booklet promised only 355 Mbps for MTU1500
> abd i need 580Mbps
>
> Antti

Antti,
The USRP2 (http://en.wikipedia.org/wiki/
Universal_Software_Radio_Peripheral) is a software-defined radio that
uses a Spartan III + gigE PHY chip to reach 800 Mbits/sec sustained.
I believe the MAC in their FPGA has a few limitations (only supports
1000 Base-T) but was originally based on the opencores tri-mode MAC
(though significant modifications were needed to make it reliable,
IIRC).  The other caveat here is that the USRP2 guys push raw ethernet
frames into a PC...i.e., they don't use TCP or UDP.  I believe their
analysis showed that they needed a custom network layer to support the
sustained high data rates.

So I wouldn't give up hope on making something work here at 580 Mbits/
sec.  All of the USRP2 code (software + HDL) is open-sourced, and
should be available through their subversion repositories.

Good Luck,
John