PCI Express development board

On Nov 6, 1:10=A0pm, Michael S <already5cho...@yahoo.com> wrote:
> On Nov 4, 1:15=A0pm, "zsolt.garamvolgyi" <zsolt.garamvol...@gmail.com>
> wrote:
>
>
>
>
>
>
>
>
>
> > Hi,
>
> > I'm looking for an FPGA-based PCI Express development board which is
> > capable of transmitting data at about 1.4 GByte/sec to the host
> > computer (PCIe Gen1 x8 or Gen2 x4/x8).
>
> > Further considerations:
> > 1. included IDE license
> > 2. included PCIe IP core license, which is also valid for other
> > designs based on the same FPGA
> > 3. minimal HW complexity (i.e., the smaller the FPGA, the better)
> > 4. price
>
> > My current candidate is the Altera Arria II GX FPGA Development Kit
> > (although I'm not sure if the IP core license which comes with it is
> > not only for evaluation).http://www.altera.com/products/devkits/altera/=
kit-aiigx-pcie.html
>
> > I would appreciate some information about your development experience
> > with this board or other boards which satisfy the above requirements.
> > Thank you for your time!
>
> > Regards,
> > Zsolt
>
> In our experience Altera's hard IP PCIe core present in StratixIV GX
> and in ArriaII GX is broken rather badly.
> That is, you can find certain hosts where it appears to work most of
> the time, but that's exception rather than rule.
> Soft IP core in these devices works relatively better, but still badly
> violates power up timing specifications defined in the PCIe standard,
> so we generally prefer to plug it into slots that support hot plug,
> since such slots are typically more tolerant to this sort of timing
> violations. Unfortunately for you, you want x8 slot. x8 slots with
> support for hot plug are significantly rarer than x4/x1 slots.
>

This is really interesting. Can you tell me, exactly which hosts (if
any) did you manage to get the hard IP core work with correctly?


> Overall, if you decided to go with Altera, the most robust combination
> is old StatixII GX + soft PCIe core + Quartus 9.1 Sp1 or Sp2.
>

I would prefer using an Altera device as I have more experience with
their design tools.
As the Stratix II GX board is obsolete, I think I could use the Arria
II GX board with the soft IP core, too, with the drawbacks of
additional license cost and FPGA resource usage. The Arria II GX +
hard IP combination is quite compelling (at least on paper).

Do you have experience with other vendors' PCIe boards/IP cores? Is
there a more robust solution available?


> BTW, even on x8 Gen1, hitting 1.4 GB/s =A0in the read direction will be
> very very hard =A0(but it sounds like you don't need it). Hitting 1.4 GB/
> s in =A0in write direction is significantly easier, but still non-
> trivial, esp. if you want to work with default 256B packets size.

Yes, I'm aware of these limitations, but
1. downstream data transfer is not a concern,
2. 1.4 GB/s is an absolute worst case data rate estimation, and most
probably will be relaxed in the final specification. It's not
impossible that even a PCIe x4 board will fit the requirements.

Regards,
Zsolt

I still have that Stratix II GX PCI-e development kit, but the FPGA doesn't 
have hard PCI-e block, even if there are 4 lanes of the bus, so the core is 
instantiated as software mode PCI-e. If the author is interested in Stratix 
II GX, I am ready for negotiations :)

> Can you tell me which Quartus version did you use for your project?
> I've been working on another PCI Express project with an Altera
> Stratix II GX eval. board using Quartus II 9.0 and setting up the
> example design was relatively painless. Unfortunately, that kit is no
> longer available.

>I still have that Stratix II GX PCI-e development kit, but the FPGA doesn't
> have hard PCI-e block, even if there are 4 lanes of the bus, so the core 
> is instantiated as software mode PCI-e. If the author is interested in 
> Stratix II GX, I am ready for negotiations :)

Oh my bad, it's x8, not x4. The board I have is this one: 
http://www.altera.com/products/devkits/altera/kit-pciexpress_s2gx.html

On Nov 7, 12:43=A0pm, "zsolt.garamvolgyi" <zsolt.garamvol...@gmail.com>
wrote:
> On Nov 6, 1:10=A0pm, Michael S <already5cho...@yahoo.com> wrote:
>
>
>
> > On Nov 4, 1:15=A0pm, "zsolt.garamvolgyi" <zsolt.garamvol...@gmail.com>
> > wrote:
>
> > > Hi,
>
> > > I'm looking for an FPGA-based PCI Express development board which is
> > > capable of transmitting data at about 1.4 GByte/sec to the host
> > > computer (PCIe Gen1 x8 or Gen2 x4/x8).
>
> > > Further considerations:
> > > 1. included IDE license
> > > 2. included PCIe IP core license, which is also valid for other
> > > designs based on the same FPGA
> > > 3. minimal HW complexity (i.e., the smaller the FPGA, the better)
> > > 4. price
>
> > > My current candidate is the Altera Arria II GX FPGA Development Kit
> > > (although I'm not sure if the IP core license which comes with it is
> > > not only for evaluation).http://www.altera.com/products/devkits/alter=
a/kit-aiigx-pcie.html
>
> > > I would appreciate some information about your development experience
> > > with this board or other boards which satisfy the above requirements.
> > > Thank you for your time!
>
> > > Regards,
> > > Zsolt
>
> > In our experience Altera's hard IP PCIe core present in StratixIV GX
> > and in ArriaII GX is broken rather badly.
> > That is, you can find certain hosts where it appears to work most of
> > the time, but that's exception rather than rule.
> > Soft IP core in these devices works relatively better, but still badly
> > violates power up timing specifications defined in the PCIe standard,
> > so we generally prefer to plug it into slots that support hot plug,
> > since such slots are typically more tolerant to this sort of timing
> > violations. Unfortunately for you, you want x8 slot. x8 slots with
> > support for hot plug are significantly rarer than x4/x1 slots.
>
> This is really interesting. Can you tell me, exactly which hosts (if
> any) did you manage to get the hard IP core work with correctly?
>

I recollect that we had better luck with hard IP on Altera's own
Stratix IV GX FPGA Development Board (EP4SGX230KF40 device) in x8 Gen2
slot of Intel S3420GP motherboard. Still not robust enough to ship it
to client, but it was handy during development (faster compilation
time than soft IP core and no need for license). However combination
of the same board with the same Altera device and the Hard IP core did
not work at all on Terrasic DE4.
The same Terrasic DE4 works pretty well with soft IP core.

As to hard IP on ArriaII-GX, I didn't try it myself. The colleges
reported zero success rate.


> > Overall, if you decided to go with Altera, the most robust combination
> > is old StatixII GX + soft PCIe core + Quartus 9.1 Sp1 or Sp2.
>
> I would prefer using an Altera device as I have more experience with
> their design tools.
> As the Stratix II GX board is obsolete, I think I could use the Arria
> II GX board with the soft IP core, too, with the drawbacks of
> additional license cost and FPGA resource usage. The Arria II GX +
> hard IP combination is quite compelling (at least on paper).
>

In our experience paper and silicon are quite different.
Now, soft IP PCIe cores obviously work both on StratixII-GX and on
ArriaII-GX. However on StratixII-GX they work better. If all you need
is 1 or 2 boards, is it so hard to find Stratix II GX dev boards?

> Do you have experience with other vendors' PCIe boards/IP cores?

No, I don't.

> Is there a more robust solution available?
>

For x4 Gen1 the Gennum GN4124 is probably the most robust solution,
but I am not aware of ready-to-buy development boards with GN4124 +
Altera FPGA. On the other hand, I didn't look for them.

> > BTW, even on x8 Gen1, hitting 1.4 GB/s =A0in the read direction will be
> > very very hard =A0(but it sounds like you don't need it). Hitting 1.4 G=
B/
> > s in =A0in write direction is significantly easier, but still non-
> > trivial, esp. if you want to work with default 256B packets size.
>
> Yes, I'm aware of these limitations, but
> 1. downstream data transfer is not a concern,
> 2. 1.4 GB/s is an absolute worst case data rate estimation, and most
> probably will be relaxed in the final specification. It's not
> impossible that even a PCIe x4 board will fit the requirements.
>
> Regards,
> Zsolt

Good for you.

Michael S wrote:
> On Nov 7, 12:43=A0pm, "zsolt.garamvolgyi" <zsolt.garamvol...@gmail.com>
> wrote:
> > On Nov 6, 1:10=A0pm, Michael S <already5cho...@yahoo.com> wrote:
> > > On Nov 4, 1:15=A0pm, "zsolt.garamvolgyi" <zsolt.garamvol...@gmail.com=
>
> > > wrote:
> >
> > > > Hi,
> >
> > > > I'm looking for an FPGA-based PCI Express development board which i=
s
> > > > capable of transmitting data at about 1.4 GByte/sec to the host
> > > > computer (PCIe Gen1 x8 or Gen2 x4/x8).
>>>>> [snip]
> >
> > > In our experience Altera's hard IP PCIe core present in StratixIV GX
> > > and in ArriaII GX is broken rather badly.
> > > That is, you can find certain hosts where it appears to work most of
> > > the time, but that's exception rather than rule.
> > > Soft IP core in these devices works relatively better, but still badl=
y
> > > violates power up timing specifications defined in the PCIe standard,
> > > so we generally prefer to plug it into slots that support hot plug,
> > > since such slots are typically more tolerant to this sort of timing
> > > violations. Unfortunately for you, you want x8 slot. x8 slots with
> > > support for hot plug are significantly rarer than x4/x1 slots.
> >
> > This is really interesting. Can you tell me, exactly which hosts (if
> > any) did you manage to get the hard IP core work with correctly?
> >
>

I did a little more research.
The situation with Altera Hard IP is not as hopeless as I presented in
my previous posts.
In Quartus 10.1 (and hopefully in 11, but I didn't test) Altera seems
to implement a workarounds for majority of the Startix IV and Arria II
hard IP silicon problems. Powerup timing is still non standard, and
I'd guess, not much could be done about it, but the rest works o.k. At
least so far we saw no problems with Terrasic DE4 on Intel X58-based
PC.

Our problems with Hard IP on Arria2-GX could be traced back to
combination of bug in Quartus 9.1. and lack of attention on part of
our board developer. Hard IP in Arria2 GX requires certain fixed
assignment of the PCIe lanes to FPGA GX transceivers. However,
Quartus 9.1 does not properly detects incorrect assignments. We were
just unlucky to fall into this pitfall. You can be 100% sure that
nothing like that would happen if you buy development board from
Altera or Terrasic.

Back to recommended host platform.
I think, among single-CPU platforms, Intel X58 (the platform used with
Core-i7 9xx series of CPUs) is still the safest. Of course, it is not
as popular or easy-to-find as platforms based on Intel LGA 1156 or LGA
1155 processors, but it does feature hot plug on all PCIe slots. As a
consolation for higher price and lower price/performance, X58 gives
you the highest memory bandwidth and the highest memory capacity.
For comparison, typical LGA 1156- based boards have just one hot-plug
x4 Gen1 slot. All x8/x16 and/or Gen2 slots have no support for hot
plug, so it's far more likely that non-standard powerup timing of
Altera cores (both soft and hard IP variants) will cause problems,
even in "cold plug" scenarios.
I have no 1st hand experience with Altera PCIe solutions on the newest
Intel platform, i.e. LGA 1155, a.k.a. i7 Gen2, a.k.a. Sandy Bridge. I
am afraid that it is similar to LGA 1156 and is similarly unforgiving
to timing violations.
But test it yourself - this one is easiest to get and even if you find
that it does not satisfy you as PCIe host, at least you will enjoy ass-
kicking development machine ;-)

As to dual-CPU Intel platforms, IIRC, on dual-CPU chipsets all PCIe
slots support hot plug, so, in theory everything should work. However,
except for old and obsolete Blackford (5000 series), I didn't try it
myself.

As to dual-CPU Intel platforms, IIRC, on dual-CPU chipsets all PCIe
slots support hot plug, so, in theory everything should work. However,
except for old and obsolete Blackford (5000 series), I didn't try it
myself.


Hello,
maybe anyone can confirm that? What's the scenario of fpga restart then? 
Unload the module, reflash the binary and load module again? What about 
development on Windows then?

Regards,
Tomas D. 

On Nov 14, 10:42=A0am, "scrts" <hid...@email.com> wrote:
>> As to dual-CPU Intel platforms, IIRC, on dual-CPU chipsets all PCIe
>> slots support hot plug, so, in theory everything should work. However,
>> except for old and obsolete Blackford (5000 series), I didn't try it
>> myself.
>
> Hello,
> maybe anyone can confirm that?

Confirm what?

>
> What's the scenario of fpga restart then?
> Unload the module, reflash the binary and load module again? What about
> development on Windows then?
>
> Regards,
> Tomas D.

On Windows it is the same.
Assuming you have plug&play driver (which, for PCIe, is the only
reasonable option anyway) and assuming that your device is not a boot
device and not a main display card:
1) disable your device in Device Manager;
2) reload FPGA;
3) re-enable device in Device Manager.
Or, if PCI Vendor/Device ID of your new design differs from the
previous one, you instruct device manager to scan for new devices
between steps 2 and 3.

"Michael S" <already5chosen@yahoo.com> wrote in message 
news:f6ea35a5-8957-4b6e-98e4-fb70d273a3bb@v5g2000yqn.googlegroups.com...
On Nov 14, 10:42 am, "scrts" <hid...@email.com> wrote:
>> As to dual-CPU Intel platforms, IIRC, on dual-CPU chipsets all PCIe
>> slots support hot plug, so, in theory everything should work. However,
>> except for old and obsolete Blackford (5000 series), I didn't try it
>> myself.
>
> Hello,
> maybe anyone can confirm that?

>Confirm what?

Dual-CPU chipsets really support PCI-e hotplug? In this case, all the 
current chipset systems should support that feature.

The part about image reflash is clear now, thanks.


Regards,
Tomas D.