FPGARelated.com
Forums

Configure FPGA via PCIe

Started by Allan Herriman February 24, 2009
Hi,

I always seem to come up against systems that have some sort of cpu (I'm 
talking about a real cpu here, not a core in an FPGA) that has to 
configure some sort of FPGA.

Solutions in the past have involved ugliness with CPLDs, Flash memories, 
parallel local buses, etc.

The newer embedded processors seem to be favouring PCIe over the other 
types of interfaces.  E.g. (just an example; I'm not actually going to 
use one of these any time soon) the Intel Atom (or its support chip) has 
2 x PCIe, several x USB, I2C, but no simple local bus (just PATA).

It looks like my future designs will need to configure FPGAs through a 
PCIe bus.  The current generation of FPGAs might have PCIe support, but 
as yet, none offer any sort of configuration via PCIe.

So far, it seems the best way is (despite the FPGA on-board transceivers) 
is to use a PCIe to local bus bridge chip from PLX or Gennum.

I guess my questions are:

1.  Is my take on the future of PCIe right?  Will it become the only high 
speed interface in future mid- or high-level embedded CPUs?

2.  Will future generations FPGAs be able to be programmed via PCIe?  If 
so, when?

3.  Is there a better way of configuring a current generation (e.g. V5 or 
V6) FPGA than via a bridge chip?

Thanks,
Allan
"Allan Herriman" <allanherriman@hotmail.com> wrote in message 
news:Xns9BBCD0C44FABBallanherrimanhotmail@216.151.153.43...
> Hi, > > I guess my questions are: > > 2. Will future generations FPGAs be able to be programmed via PCIe? If > so, when? >
Now? ;) Just need to use some external components to tie it together...or use partial reconfiguration (PR) flow.
> 3. Is there a better way of configuring a current generation (e.g. V5 or > V6) FPGA than via a bridge chip?
In our setup we're using ml555 and V5 internal pcie endpoint in order to perform PR of the reconfigurable core connected to one of the BARs. We're still in the early development stage,so its's hard to judge if its any better, but it works. Regards, Krzysztof Kepa
"Krzysztof Kepa" <nospam_blondikk@poczta.fm> wrote in
news:go0pr0$6jj$1@aioe.org: 

> > "Allan Herriman" <allanherriman@hotmail.com> wrote in message > news:Xns9BBCD0C44FABBallanherrimanhotmail@216.151.153.43... >> Hi, >> >> I guess my questions are: >> >> 2. Will future generations FPGAs be able to be programmed via PCIe? >> If so, when? >> > Now? ;) Just need to use some external components to tie it > together...or use partial reconfiguration (PR) flow.
I meant "gluelessly". With PR, one must get the original configuration into the FPGA somehow, and (as I understand it) that cannot be done with PCIe. Thanks for your comments. Allan
"Allan Herriman" <allanherriman@hotmail.com> wrote in message 
news:Xns9BBD4BFB4EB0allanherrimanhotmail@216.151.153.43...
> "Krzysztof Kepa" <nospam_blondikk@poczta.fm> wrote in > news:go0pr0$6jj$1@aioe.org: > >> >> "Allan Herriman" <allanherriman@hotmail.com> wrote in message >> news:Xns9BBCD0C44FABBallanherrimanhotmail@216.151.153.43... >>> Hi, >>> >>> I guess my questions are: >>> >>> 2. Will future generations FPGAs be able to be programmed via PCIe? >>> If so, when? >>> >> Now? ;) Just need to use some external components to tie it >> together...or use partial reconfiguration (PR) flow. > > I meant "gluelessly". With PR, one must get the original configuration > into the FPGA somehow, and (as I understand it) that cannot be done with > PCIe. >
We're using PCIe link for both, loading partial bitstreams (which contain new accelerator design) and accessing accelerator functionality. However, in our case the original configuration (with PR accelerator blank) is loaded in the "normal way" from the external flash or CF during FPGA power up. Maybe you could also do some crude bit-banging on the PCIe bridge IO in order to configure the FPGA...or look for a PCIe counterpart of the Cypress FX2 chip used in USB designs (and FPGA programming cables :) Regards, Krzysztof
On Feb 24, 8:27=A0am, Allan Herriman <allanherri...@hotmail.com> wrote:
> "Krzysztof Kepa" <nospam_blond...@poczta.fm> wrote innews:go0pr0$6jj$1@ai=
oe.org:
> > > > > "Allan Herriman" <allanherri...@hotmail.com> wrote in message > >news:Xns9BBCD0C44FABBallanherrimanhotmail@216.151.153.43... > >> Hi, > > >> I guess my questions are: > > >> 2. =A0Will future generations FPGAs be able to be programmed via PCIe? > >> If so, when? > > > Now? ;) Just need to use some external =A0components to tie it > > together...or use partial reconfiguration (PR) flow. > > I meant "gluelessly". =A0With PR, one must get the original configuration > into the FPGA somehow, and (as I understand it) that cannot be done with > PCIe. > > Thanks for your comments. > > Allan
I haven't looked at processors like the Atom in a while, but I seem to recall that you get some number of general purpose I/O pins on most any processor. You only need four pins to completely control an FPGA configuration; PRGM, CCLK, DONE, INIT. They can be bit banged in software. The FPGA interface is not used very much when considered in the grand scheme of things. So I expect you will never see direct support for it on processors. I also doubt that you will see dedicated support for PCIe in FPGAs. This is a bit lofty for a configuration interface. PCIe is point to point (right?) and taking one of the two PCIe interfaces on an Atom for booting the FPGA would be a bit of overkill. There are few applications where a processor like the Atom is used and you can't wait the few milliseconds for the FPGA to boot serially. Rick
Allan Herriman wrote:
(snip, someone wrote)

>>>2. Will future generations FPGAs be able to be programmed via PCIe? >>>If so, when?
> I meant "gluelessly". With PR, one must get the original configuration > into the FPGA somehow, and (as I understand it) that cannot be done with > PCIe.
If you have an FPGA that can do partial reconfiguration: Load from serial EEPROM enough configuration to get it onto the PCIe bus. Then the external data source supplies data through the PCIe bus as reconfiguration data for another part of the FPGA. Or, again start with the FPGA configured through serial EEPROM. Load the actual configuration data through the FPGA into static RAM. Then switch the configuration source and configure from the RAM. -- glen
rickman <gnuarm@gmail.com> wrote in
news:3774b465-11f4-4daa-aee7-d44f72fce9a6@n20g2000vba.googlegroups.com: 

> On Feb 24, 8:27&#4294967295;am, Allan Herriman <allanherri...@hotmail.com> wrote: >> "Krzysztof Kepa" <nospam_blond...@poczta.fm> wrote >> innews:go0pr0$6jj$1@ai > oe.org: >> >> >> >> > "Allan Herriman" <allanherri...@hotmail.com> wrote in message >> >news:Xns9BBCD0C44FABBallanherrimanhotmail@216.151.153.43... >> >> Hi, >> >> >> I guess my questions are: >> >> >> 2. &#4294967295;Will future generations FPGAs be able to be programmed via >> >> PCIe? If so, when? >> >> > Now? ;) Just need to use some external &#4294967295;components to tie it >> > together...or use partial reconfiguration (PR) flow. >> >> I meant "gluelessly". &#4294967295;With PR, one must get the original >> configuration into the FPGA somehow, and (as I understand it) that >> cannot be done with PCIe. >> >> Thanks for your comments. >> >> Allan > > I haven't looked at processors like the Atom in a while, but I seem to > recall that you get some number of general purpose I/O pins on most > any processor. You only need four pins to completely control an FPGA > configuration; PRGM, CCLK, DONE, INIT. They can be bit banged in > software. The FPGA interface is not used very much when considered in > the grand scheme of things. So I expect you will never see direct > support for it on processors. > > I also doubt that you will see dedicated support for PCIe in FPGAs. > This is a bit lofty for a configuration interface. PCIe is point to > point (right?) and taking one of the two PCIe interfaces on an Atom > for booting the FPGA would be a bit of overkill. There are few > applications where a processor like the Atom is used and you can't > wait the few milliseconds for the FPGA to boot serially.
That PCIe isn't wasted; it is needed for FPGA <-> cpu comms after configuration. I was actually thinking of applications without the chance of using GPIO lines, e.g. on a PCIe plugin card. It still seems that the PCIe to local bus bridge is the best solution for my hypothetical problem. The other suggestions involved Flash memories (with an implied extra step during manufacturing to program the dang things, not to mention the extra complexity involved with in-the-field upgrades of that image) or a non-trivial amount of glue logic. I thought Glen's suggestion of loading the image into SRAM first was neat, but probably not suitable for me. Thanks, Allan
On Feb 25, 4:49 am, Allan Herriman <allanherri...@hotmail.com> wrote:
> rickman <gnu...@gmail.com> wrote innews:3774b465-11f4-4daa-aee7-d44f72fce9a6@n20g2000vba.googlegroups.com: > > > > > On Feb 24, 8:27 am, Allan Herriman <allanherri...@hotmail.com> wrote: > >> "Krzysztof Kepa" <nospam_blond...@poczta.fm> wrote > >> innews:go0pr0$6jj$1@ai > > oe.org: > > >> > "Allan Herriman" <allanherri...@hotmail.com> wrote in message > >> >news:Xns9BBCD0C44FABBallanherrimanhotmail@216.151.153.43... > >> >> Hi, > > >> >> I guess my questions are: > > >> >> 2. Will future generations FPGAs be able to be programmed via > >> >> PCIe? If so, when? > > >> > Now? ;) Just need to use some external components to tie it > >> > together...or use partial reconfiguration (PR) flow. > > >> I meant "gluelessly". With PR, one must get the original > >> configuration into the FPGA somehow, and (as I understand it) that > >> cannot be done with PCIe. > > >> Thanks for your comments. > > >> Allan > > > I haven't looked at processors like the Atom in a while, but I seem to > > recall that you get some number of general purpose I/O pins on most > > any processor. You only need four pins to completely control an FPGA > > configuration; PRGM, CCLK, DONE, INIT. They can be bit banged in > > software. The FPGA interface is not used very much when considered in > > the grand scheme of things. So I expect you will never see direct > > support for it on processors. > > > I also doubt that you will see dedicated support for PCIe in FPGAs. > > This is a bit lofty for a configuration interface. PCIe is point to > > point (right?) and taking one of the two PCIe interfaces on an Atom > > for booting the FPGA would be a bit of overkill. There are few > > applications where a processor like the Atom is used and you can't > > wait the few milliseconds for the FPGA to boot serially. > > That PCIe isn't wasted; it is needed for FPGA <-> cpu comms after > configuration.
In your case that may be useful. My point is that in the general case this is not something that would be widely useful since PCIe ports are a very "expensive" way to configure a part when all you really need is four GPIO pins on the host.
> I was actually thinking of applications without the chance of using GPIO > lines, e.g. on a PCIe plugin card.
I don't know PCIe much, but I have worked with PCI before and I don't think it is practical to use an FPGA to provide the PCI interface. This issue may have been resolved at some point, but I want to say that an FPGA does not boot fast enough to respond to the initial enumeration comms on the bus. Has this issue been addressed so that an FPGA can boot from an on board PROM and provide a PCIe interface?
> It still seems that the PCIe to local bus bridge is the best solution for > my hypothetical problem. The other suggestions involved Flash memories > (with an implied extra step during manufacturing to program the dang > things, not to mention the extra complexity involved with in-the-field > upgrades of that image) or a non-trivial amount of glue logic.
I think the idea is that the image required to boot the FPGA enough to make it bootable over the PCIe interface would be pretty stable. So the concerns of needing to fix that would be a bit overblown. In fact, it is likely that you could design the interface to the boot Flash so that it can be programmed over the PCIe bus. Then, as long as the reprogramming did not go bust in the middle, you could update the Flash just like you load the rest of the image. Of course, the local bus interface still requires some extra logic to boot the FPGA, no? Or does the bridge chip provide signals that can be used directly?
> I thought Glen's suggestion of loading the image into SRAM first was > neat, but probably not suitable for me.
You don't really need to load the FPGA fresh on **every** boot do you? You could provide the Flash to boot from and only update it when you have changes. You could use a protected boot block to load the initial image. If an update is installed in the rest of the Flash, the initial image would then reboot the FPGA from that updated image. This gives you a guaranteed level of boot so that you never loose the ability to boot and can still support updates. The selection of whether to reboot from the secondary image could be put under software control. I've been working with a flash based FPGA which uses the Flash as a shadow of the configuration RAM. The part can be booted from the Flash or the RAM can be directly configured without disturbing the Flash for temporary boots. I think this provides some interesting boot and update possibilities when controlled by a host. Rick
>That PCIe isn't wasted; it is needed for FPGA <-> cpu comms after >configuration.
>I was actually thinking of applications without the chance of using GPIO >lines, e.g. on a PCIe plugin card.
This problem has been around for a long time. It also happens with PCI (no e). The fundamental problem is that you need a place to stand while you are programming the FPGA so it's tricky to get the FPGA to program itself before it has been programmed. In order to program a FPGA from a CPU, you need something like GPIO pins. Most modern compute CPUs (as compared to embedded CPUs) don't have any "extra" pins like that. The best approach I know of is to load the FPGA from the normal serial PROM and setup a few extra pins on the FPGA so that it can reprogram the PROM. It may require some glue. As long as you don't shoot yourself in the foot you can do that over and over. You probably die if you lose power in the middle of re-programming. I forget the vendor name, but somebody makes a set of PCI to xxx chips. They are often used to talk to FPGAs. They have a bus to transfer large clumps of data, and also several GPIO pins. I expect they do/will make a PCIe version. If that fits with your design, you could use one of them between your FPGA and the PCIe bus. You could wire up a few of the GPIO pins to program the FPGA. This is ugly in the sense that it requires another big expensive chip, but it might let you use a smaller FPGA since the bus interface can be simpler. -- These are my opinions, not necessarily my employer's. I hate spam.
rickman <gnuarm@gmail.com> wrote in
news:9e5e3729-4da1-41b4-9ec2-9c8f5dfe24d6@r36g2000vbp.googlegroups.com: 

> On Feb 25, 4:49 am, Allan Herriman <allanherri...@hotmail.com> wrote: >> rickman <gnu...@gmail.com> wrote >> innews:3774b465-11f4-4daa-aee7-d44f72fce9a6@n20g2000vba.googlegroups.c >> om: >> >> >> >> > On Feb 24, 8:27 am, Allan Herriman <allanherri...@hotmail.com> >> > wrote: >> >> "Krzysztof Kepa" <nospam_blond...@poczta.fm> wrote >> >> innews:go0pr0$6jj$1@ai >> > oe.org: >> >> >> > "Allan Herriman" <allanherri...@hotmail.com> wrote in message >> >> >news:Xns9BBCD0C44FABBallanherrimanhotmail@216.151.153.43... >> >> >> Hi, >> >> >> >> I guess my questions are: >> >> >> >> 2. Will future generations FPGAs be able to be programmed via >> >> >> PCIe? If so, when? >> >> >> > Now? ;) Just need to use some external components to tie it >> >> > together...or use partial reconfiguration (PR) flow. >> >> >> I meant "gluelessly". With PR, one must get the original >> >> configuration into the FPGA somehow, and (as I understand it) that >> >> cannot be done with PCIe. >> >> >> Thanks for your comments. >> >> >> Allan >> >> > I haven't looked at processors like the Atom in a while, but I seem >> > to recall that you get some number of general purpose I/O pins on >> > most any processor. You only need four pins to completely control >> > an FPGA configuration; PRGM, CCLK, DONE, INIT. They can be bit >> > banged in software. The FPGA interface is not used very much when >> > considered in the grand scheme of things. So I expect you will >> > never see direct support for it on processors. >> >> > I also doubt that you will see dedicated support for PCIe in FPGAs. >> > This is a bit lofty for a configuration interface. PCIe is point >> > to point (right?) and taking one of the two PCIe interfaces on an >> > Atom for booting the FPGA would be a bit of overkill. There are >> > few applications where a processor like the Atom is used and you >> > can't wait the few milliseconds for the FPGA to boot serially. >> >> That PCIe isn't wasted; it is needed for FPGA <-> cpu comms after >> configuration. > > In your case that may be useful. My point is that in the general case > this is not something that would be widely useful since PCIe ports are > a very "expensive" way to configure a part when all you really need is > four GPIO pins on the host. > > >> I was actually thinking of applications without the chance of using >> GPIO lines, e.g. on a PCIe plugin card. > > I don't know PCIe much, but I have worked with PCI before and I don't > think it is practical to use an FPGA to provide the PCI interface. > This issue may have been resolved at some point, but I want to say > that an FPGA does not boot fast enough to respond to the initial > enumeration comms on the bus. Has this issue been addressed so that > an FPGA can boot from an on board PROM and provide a PCIe interface? > > >> It still seems that the PCIe to local bus bridge is the best solution >> for my hypothetical problem. The other suggestions involved Flash >> memories (with an implied extra step during manufacturing to program >> the dang things, not to mention the extra complexity involved with >> in-the-field upgrades of that image) or a non-trivial amount of glue >> logic. > > I think the idea is that the image required to boot the FPGA enough to > make it bootable over the PCIe interface would be pretty stable. So > the concerns of needing to fix that would be a bit overblown. In > fact, it is likely that you could design the interface to the boot > Flash so that it can be programmed over the PCIe bus. Then, as long > as the reprogramming did not go bust in the middle, you could update > the Flash just like you load the rest of the image. > > Of course, the local bus interface still requires some extra logic to > boot the FPGA, no? Or does the bridge chip provide signals that can > be used directly?
They usually have a few GPIO lines as well as a parallel data bus. This allows some sort of slave parallel mode to be used without glue logic. Just in case you are interested, here are some typical parts: 1 Lane: http://www.plxtech.com/products/expresslane/pex8311.asp 4 Lanes with dedicated FPGA loading support: http://www.gennum.com/video/products/gn4124 Here's the article that inspired this thread: http://www.pldesignline.com/howto/210300269 Regards, Allan