PCI newbie problems

Hi everyone,

I'm trying to set up a toy design on a Xilinx Virtex 5 (Avnet PCI
Express Development Kit board) and I'm having big problems to
understand how PCI works. I downloaded the PCI endpoint core bitstream
for the Programmed IO on the board and plugged it into a pc,
everything seems to work fine as a matter of fact PCI tree succeeds in
recognizing the board. My aim now is really simple, I would like to
write a byte on the memory space of the board (with PCI tree) and to
display the byte value on a seven-segment display. What I can't
understand is "where and how" I have to take the byte (from the FPGA
side), that is which signals of the PIO design I have to handle in my
module.
I apologize if my question is not very clear, but I'm a total newbie
in PCI, any hint or advice to understand it better would be really
appreciated.

Thanks in advance.

Massi wrote:

> I apologize if my question is not very clear, but I'm a total newbie
> in PCI, any hint or advice to understand it better would be really
> appreciated.

Learn vhdl or verilog hardware description
to collect the byte from the interface
and drive the display.

On Dec 24, 11:40=A0am, Massi <massi_...@msn.com> wrote:
> Hi everyone,
>
> I'm trying to set up a toy design on a Xilinx Virtex 5 (Avnet PCI
> Express Development Kit board) and I'm having big problems to
> understand how PCI works. I downloaded the PCI endpoint core bitstream
> for the Programmed IO on the board and plugged it into a pc,
> everything seems to work fine as a matter of fact PCI tree succeeds in
> recognizing the board. My aim now is really simple, I would like to
> write a byte on the memory space of the board (with PCI tree) and to
> display the byte value on a seven-segment display. What I can't
> understand is "where and how" I have to take the byte (from the FPGA
> side), that is which signals of the PIO design I have to handle in my
> module.
> I apologize if my question is not very clear, but I'm a total newbie
> in PCI, any hint or advice to understand it better would be really
> appreciated.
>
> Thanks in advance.


You may want to watch Implementing PCI Express v2.0 Compliant Designs
with Xilinx Virtex-5 FPGAs Tech Seminar

http://www.techonline.com/learning/webinar/211800334

Are you referring to the "LogiCore Endpoint for PCIexpress"? If so,
there is a manual on the Xilinx web site that I would look at first.
Essentially there are two 'packet ports' one for incoming packets and
one for outgoing (response) packets. Here you will have to attach some
form of state machine to generate/handle your data. I am sure there are
some examples available from Xilinx. PCIexpress is a packet-based protocol.

To be honest, I think you will find it extremely hard work if you don't
familiarise yourself with PCIexpress first.  A good starting point is
the Mindshare book by Anderson/Budruk/Shanley. I think there is also an
electronic version available. It's a big book but the first few chapters
are enough to get you started. For your test case, an understanding of
the following points is most important.
1) How to decode a PCIexpress packet. (Essentially a sequence of 32-bit
words)
2) When and how to generate a response packet (i.e. for all accesses
except Memory-Write/Message packets).

As regards writing to memory using PCItree. I am not so sure whether
that will work at all. If I remember correctly, PCItree really aims at
accessing configuration space. You will probably not be able to access
device memory space, either memory or I/O (to be more accurate, you
shouldn't be able to) if your device hasn't been enabled by the
operating system (Bit 0 in Command register in configuration space set
to '1'). For this you will need a device driver, since an OS typically
does not enable a PCI device for which it cannot find a driver. If you
are using Windows I'd recommend the KMDF framework (available from the
MS website). If you are working under Linux, take a good look at the
kernel sources and also Rubini's book which is available at O Reilly or
a number of online sources. In your case for just writing to a seven
segment display, under Linux you might get away without a driver since
many parts of the PCI API can be accessed from user mode.
If you do this with memory writes you don't even have to worry about
generating response packets in you hardware. But I expect you still have
to somehow set Bit 0 in Configuration-space.Command-reg first.


Massi schrieb:
> Hi everyone,
> 
> I'm trying to set up a toy design on a Xilinx Virtex 5 (Avnet PCI
> Express Development Kit board) and I'm having big problems to
> understand how PCI works. I downloaded the PCI endpoint core bitstream
> for the Programmed IO on the board and plugged it into a pc,
> everything seems to work fine as a matter of fact PCI tree succeeds in
> recognizing the board. My aim now is really simple, I would like to
> write a byte on the memory space of the board (with PCI tree) and to
> display the byte value on a seven-segment display. What I can't
> understand is "where and how" I have to take the byte (from the FPGA
> side), that is which signals of the PIO design I have to handle in my
> module.
> I apologize if my question is not very clear, but I'm a total newbie
> in PCI, any hint or advice to understand it better would be really
> appreciated.
> 
> Thanks in advance.

On 26 Dic, 12:37, Charles Gardiner <inva...@invalid.invalid> wrote:
> Are you referring to the "LogiCore Endpoint for PCIexpress"? If so,
> there is a manual on the Xilinx web site that I would look at first.
> Essentially there are two 'packet ports' one for incoming packets and
> one for outgoing (response) packets. Here you will have to attach some
> form of state machine to generate/handle your data. I am sure there are
> some examples available from Xilinx. PCIexpress is a packet-based protoco=
l.
>
> To be honest, I think you will find it extremely hard work if you don't
> familiarise yourself with PCIexpress first. =A0A good starting point is
> the Mindshare book by Anderson/Budruk/Shanley. I think there is also an
> electronic version available. It's a big book but the first few chapters
> are enough to get you started. For your test case, an understanding of
> the following points is most important.
> 1) How to decode a PCIexpress packet. (Essentially a sequence of 32-bit
> words)
> 2) When and how to generate a response packet (i.e. for all accesses
> except Memory-Write/Message packets).
>
> As regards writing to memory using PCItree. I am not so sure whether
> that will work at all. If I remember correctly, PCItree really aims at
> accessing configuration space. You will probably not be able to access
> device memory space, either memory or I/O (to be more accurate, you
> shouldn't be able to) if your device hasn't been enabled by the
> operating system (Bit 0 in Command register in configuration space set
> to '1'). For this you will need a device driver, since an OS typically
> does not enable a PCI device for which it cannot find a driver. If you
> are using Windows I'd recommend the KMDF framework (available from the
> MS website). If you are working under Linux, take a good look at the
> kernel sources and also Rubini's book which is available at O Reilly or
> a number of online sources. In your case for just writing to a seven
> segment display, under Linux you might get away without a driver since
> many parts of the PCI API can be accessed from user mode.
> If you do this with memory writes you don't even have to worry about
> generating response packets in you hardware. But I expect you still have
> to somehow set Bit 0 in Configuration-space.Command-reg first.

Thank you really much for your help, I have already purchased the book
you suggested and started to learn something more about PCIe. Though I
still have some doubts about how transactions happen and the memory
spaces defined by PCIe. In particular:

1. Where do the three PCIe spaces physically reside in? I mean, when I
generate my enpoint block IP core I create of course a configuration
space and (in my case) a memory space of 64K; Have I to think to them
as ram memories instantiated in the endpoint block?

2. Which is the different between I/O spaces and the memory space?

3. When a memory write transaction happens, a packet (header+payload)
is transferred (for example) from the system memory to the FPGA. As
far as I see, the whole packet is written in the memory space and then
processed by the endpoint block which present it to the user logic
(for esample a PIO module), is this right or does the transaction
happen in a different way?

Again, I'm really grateful for your help.

Massi schrieb:
> On 26 Dic, 12:37, Charles Gardiner <inva...@invalid.invalid> wrote:
>> Are you referring to the "LogiCore Endpoint for PCIexpress"? If so,
>> there is a manual on the Xilinx web site that I would look at first.
>> Essentially there are two 'packet ports' one for incoming packets and
>> one for outgoing (response) packets. Here you will have to attach some
>> form of state machine to generate/handle your data. I am sure there are
>> some examples available from Xilinx. PCIexpress is a packet-based protocol.
>>
>> To be honest, I think you will find it extremely hard work if you don't
>> familiarise yourself with PCIexpress first.  A good starting point is
>> the Mindshare book by Anderson/Budruk/Shanley. I think there is also an
>> electronic version available. It's a big book but the first few chapters
>> are enough to get you started. For your test case, an understanding of
>> the following points is most important.
>> 1) How to decode a PCIexpress packet. (Essentially a sequence of 32-bit
>> words)
>> 2) When and how to generate a response packet (i.e. for all accesses
>> except Memory-Write/Message packets).
>>
>> As regards writing to memory using PCItree. I am not so sure whether
>> that will work at all. If I remember correctly, PCItree really aims at
>> accessing configuration space. You will probably not be able to access
>> device memory space, either memory or I/O (to be more accurate, you
>> shouldn't be able to) if your device hasn't been enabled by the
>> operating system (Bit 0 in Command register in configuration space set
>> to '1'). For this you will need a device driver, since an OS typically
>> does not enable a PCI device for which it cannot find a driver. If you
>> are using Windows I'd recommend the KMDF framework (available from the
>> MS website). If you are working under Linux, take a good look at the
>> kernel sources and also Rubini's book which is available at O Reilly or
>> a number of online sources. In your case for just writing to a seven
>> segment display, under Linux you might get away without a driver since
>> many parts of the PCI API can be accessed from user mode.
>> If you do this with memory writes you don't even have to worry about
>> generating response packets in you hardware. But I expect you still have
>> to somehow set Bit 0 in Configuration-space.Command-reg first.
> 
> Thank you really much for your help, I have already purchased the book
> you suggested and started to learn something more about PCIe. Though I
> still have some doubts about how transactions happen and the memory
> spaces defined by PCIe. In particular:
> 
> 1. Where do the three PCIe spaces physically reside in? I mean, when I
> generate my enpoint block IP core I create of course a configuration
> space and (in my case) a memory space of 64K; Have I to think to them
> as ram memories instantiated in the endpoint block?

Taking an FPGA-centric view of your task, configuration space is
generally included in the PCIexpress IP-Core. Configuration space, as
the name suggests, is an address space that is accessible as soon as the
device has been turned on and before configuration takes place.
Assigning the resources (interrupts, memory base address, memory windows
etc.) is made through configuration space.
Memory Space and I/O space are the parts you must add yourself and are
external to the IP-Core. These are typically control registers, memory
areas etc. and represent the resources on your add-in card which can be
accessed by other PCI/PCIexpress devices or the PC during normal
operation. Note that you do not have to implement control registers in
I/O space (see below) it is better to map them into memory space. So the
64K memory space you plan on using is fine.

> 
> 2. Which is the different between I/O spaces and the memory space?

I/O Space is really a legacy feature from the early Intel architectures.
Control registers in the peripherals were in I/O space, bulk memory in
memory space. Most of the other architectures never made a distinction.
Control registers were just mapped into selected areas of memory space
which allowed uniform access mechanisms. Unless you are trying to
emulate an older device, PCIexpress advises not to use I/O space at all
any more.

> 
> 3. When a memory write transaction happens, a packet (header+payload)
> is transferred (for example) from the system memory to the FPGA. As
> far as I see, the whole packet is written in the memory space and then
> processed by the endpoint block which present it to the user logic
> (for esample a PIO module), is this right or does the transaction
> happen in a different way?

This is where the IP-Cores from the FPGA manufacturers follow slightly
different philosophies. Personally, I am most familiar &#7811;ith the Lattice
Core but from a quick look at the Xilinx data sheet it seems to follow
the same strategy. The IP-core provides the user with two 'packet
ports'. One receives packets the other is where the user logic transmits
packets. The incoming packets (e.g. a memory write with your device as
target) appear as a stream of double-words (32 Bits) initiated by some
start signal. When your logic recognises the start signal, it begins
collecting the incoming packet. You collect the target address from the
header and write the payload (which arrives immediately after the
header) to the corresponding location (register or memory) in your
application logic. If your device has been located below 4GByte in
memory space the address will be the last header double-word. If your
device has been located at or above 4GByte, the address will be the last
two header double-words. The first double word in the header contains a
field indicating whether you must expect a 32-bit or 64-bit address. But
that is all well described in the Mindshare book.

With your reference to the PIO module and your earlier mail, I am
assuming that your Xilinx bitstream is some example application which
already contains the PCIexpress IP-Core plus some additional registers
that Xilinx have already connected to the 'packet port' described above.
 If this is so, you probably just have to bring out the PIO ports to
FPGA pins and can use them to directly control your seven-segment
display. You just have to know at which address(es) this/these PIO
registers are accessible. The remaining point is as I said in my
previous posting, a well behaved PCIexpress IP-core will not pass any
memory write request through to the application logic until bit 0 in the
PCI Command register (in configuration space) has been set to '1'.
Normally, the operating system does this after it recognises and
configures your device. If the operating system doesn't do it (which is
most likely in your case, since you will be presenting an unknown
device) you ideally need a device driver to take care of it. For a
start, under windows, you probably could set this bit using the PCItree
tool but you will still need some way of writing to the PIO register.
Under windows you cannot do this directly from a user-space program.

> 
> Again, I'm really grateful for your help.
>