JOP as SOPC component

"Martin Schoeberl" <mschoebe@mail.tuwien.ac.at> wrote in message 
news:44dfa33e$0$8024$3b214f66@tunews.univie.ac.at...
> >
>>> You almost never want to have a fixed number of wait states but want to 
>>> simply have the Avalon slave provide a wait request output and tell 
>>> Avalon that by specifying that in the PTF file.
>>
>> Completely agree. When not writing and reading too many posts
>> I'm working on that version of the SRAM interface. It was just
>> a quick start as shown in the Quartus manual.
>
> BTW (to KJ): Do you have this type of Avalon slave
> for an SRAM? Would save some time and errors for me ;-)
>
No, over the past several years my use of async SRAMs has gone to 0 even 
though I used to use them quite heavily.  They've been replaced by internal 
FPGA memory in the Stratix, Stratix II, Cyclone II parts.  Any external 
memory has tended to need to be much larger than async SRAM could affordably 
provide so DDR has been used.

I'm assuming that you've checked and that Altera didn't toss one in as a 
MegaCore?  Too bad.

Oh well, I'll stop posting and let you get back to work.
KJ 

>> To avoid this little issue and the additional cycle I do usually (with
>> my SimpCon SRAM controller) clock the nwr with the inverted clock to
>> shift it after address setup.
> But now what about the trailing edge or write?  The address could start changing and the write signal will still be active.

The nwe on the negative clock is set to '1' before address change.
That's the reason for using the negative clock - get nwr '1' again
without an additional cycle.

>> Not for the read (in my case) as I'm waiting for the read data in the
>> processor. In some cases I can hide the latency by execution of additional
>> code. However, in this case I need the data registerd in the slave. which
>> is again not possible....
>
> It is if you write some code for your component and use the 'readdatavalid' Avalon signal it will work.  Once you have the address 
> and command safely

Still not for my case. The slave data is valid only the single cycle
when readdatavaild is set. And that one is controled by the slave.
I cannot force the slave to hold the read data for the master valid
for several cycles.

> assert wait request at all.  Since wait request is not asserted then the master device is free to go off and start up another 
> transaction with any device (i.e. it has not been stalled).

My master, a processor, cannot issue just any other transaction when
a read is issued. I need a.) low latency on read and b.) pipelined
read for an efficient cache fill. That's it. No write transaction
during an outstanding read.

> Unfortunately if a second read is started (even if it not to the SRAM, even if it is to a device that has 0 wait state reads) that 
> read will be greeted by a wait request because Avalon needs to insure that read data is supplied back in the order in which the 
> master requested it.  In order to do this it

Yes, for a more flexible system we would need out-of-order completion.
However, this is a completely different story.

Martin 

can't resist to answer ;-)

>>>> You can do it when your template 'controls' the master logic but not
>>>> the other way round.
>>>>
>>> Not sure what you mean by 'not the other way around'.  This template is only for the master side control logic.
>>
>> Yes, but your trigger of the trasnaction 'within' your Avalon master
>> template. However, for me the Avalon interface is just an
>> interface. It has to react on the request from the CPU. And
>> the CPU requests the transaction from 'outside' of the
>> template/interface.
>>
> OK, lost again I think.  Now it sounds like the CPU even though embedded within the FPGA doesn't have a native Avalon interface 
> and you're talking about a bridge to get you from the CPU interface over to Avalon.  Such a bridge though would typically not be 
> terribly application specific but instead is tailored to the signals on the CPU and Avalon.  Just like you can make a bridge 
> between Wishbone and Avalon.  If the CPU design is your homebrew though a simpler approach is to simply make it have an Avalon 
> compatible interface.  When you get to writing that code is where my template would be placed.

It is a general case - not about my homebrew CPU:

You have a component that (deeply inside not knowing it is connected
to Avalon) triggers a read request. With your template this trigger
gets registered when waitrequest is '0'. And this registering of
the read request adds one cycle latency.

Another point: If the waitrequest condition gets deeply embedded
in the component it would

BTW: for my CPU design - as Avalon is Altera specific I would
never make Avalon the native interface. JOP runns quite well on
Xilinx devices ;-)

Martin 

I can only afford a short reply, but ...

Martin Schoeberl wrote:
>>> Another point is, in my opinion, the wrong role who has to hold data
>>> for more than one cycle. This is true for several busses (e.g. also
>>> Wishbone). For these busses the master has to hold address and write
>>> data till the slave is ready. This is a result from the backplane
>>> bus thinking. In an SoC the slave can easily register those signals
>>> when needed longer and the master can continue.
>> When happens then when you issue another request to a slave which hasn't finished processing the first? Any queue will be finite 
>> and eventually you'd have to deal with stalling anyway. Any issue is that there are generally many more slaves than masters so it 
>> makes sense to move the complication to the master.
> 
> I disagree ;-)
> How hard is it for a slave to hold the read data more than one cycle?
> Until the next read data is requested and available? That comes almost
> for free. It's a single register, trivial logic. Ok, is a little overhead
> for an on-chip peripheral. However, you need usually a MUX in the
> peripheral for select the IO registers (now using register with a different
> meaning). Making this MUX registered is almost for free.

Focusing on the overhead for one slave supporting one outstanding 
command is missing the point.

Non-trivial slaves can support multiple simultaneous outstanding 
requests (say N), so they would need at least a queue N deep.  Not a 
problem.  Now, I have multiple slaves and multiple masters on the 
interconnect.  Each master must be able to have at least M outstanding 
requests. Any one slave can only accept one request pr cycle so the 
interconnect (the arbitration) needs buffer the requests in lots of 
FIFOs and _they_ add significant latency, logic, and complication (pick 
two).

I actually love decoupled interfaces like these (and they are not a new 
invention) as it removes the handshaking from the critical paths, but as 
a general purpose interconnect fabric it just doesn't scale.


I'll need to study SimpCon more to understand what you mean by it's 
support for multiple outstanding requests. Just to clarify, I'm talking 
about completely independent requests, not bursts. Different masters may 
issue multiple of these (up to some limit) while previously issued 
requests are still not complete. I do insist the requests complete in 
the order they were issued, mostly to simplify things (such as the 
arbitration). Really just a subset of Avalon.

Tommy

"Martin Schoeberl" <mschoebe@mail.tuwien.ac.at> wrote in message 
news:44dfb517$0$12384$3b214f66@tunews.univie.ac.at...
> can't resist to answer ;-)
>
>>>
>> OK, lost again I think.  Now it sounds like the CPU even though embedded 
>> within the FPGA doesn't have a native Avalon interface and you're talking 
>> about a bridge to get you from the CPU interface over to Avalon.  Such a 
>> bridge though would typically not be terribly application specific but 
>> instead is tailored to the signals on the CPU and Avalon.  Just like you 
>> can make a bridge between Wishbone and Avalon.  If the CPU design is your 
>> homebrew though a simpler approach is to simply make it have an Avalon 
>> compatible interface.  When you get to writing that code is where my 
>> template would be placed.
>
> It is a general case - not about my homebrew CPU:
>
> You have a component that (deeply inside not knowing it is connected
> to Avalon) triggers a read request. With your template this trigger
> gets registered when waitrequest is '0'. And this registering of
> the read request adds one cycle latency.
>
> Another point: If the waitrequest condition gets deeply embedded
> in the component it would
>
OK, for the homebrew (or anything where you own the 'master' side code) if 
you want to bring out read/write and address combinatorially, the template 
would then be

process(Put your signals here in place of clock)
begin
    --  Not wanted for combinatorial version if rising_edge(Clock) then
        if (Reset = '1') then
            Read <= '0';
            Write <= '0';
            -- Address, Writedata, Next_State initializations go here also
            -- Note:  For the synchronous version of the template, address
            -- and Writedata inits are not required since there is no Avalon
            -- requirement for such.  For the combinatorial version they
            -- are needed either here or (more safely) outside the entire
            -- if statement to provide a default to avoid latches.
        elsif (WaitRequest = '0') then
            -- Put your code here for whenever it is you want to read/write
            -- When writing you would also set WriteData here
            -- Also set Address of course
            -- Next state of state machines (if any) would be put here also
        else
            -- Read, Write, Address and Writedata get set to hold their
            -- current state.  The registers for this would be in the
            -- 'second' process of the two process approach.
            -- If any state machines are in here than they would also have
            -- the equivalent of 'Next_State <= Current_State here.
        end if;
    -- end if; Not wanted since this is no longer combinatorial
end process;

Besides the obligatory differences in the sensitivity list and 'if 
rising_edge' all I've added is the 'else' branch to hold the current state 
at whatever it happens to be at.  This same code could also have been added 
to the synchronous template version but would not be necessary.  With the 
combinatorial version it would be required of course to avoid latches.

Again using either template one can
- Be assured of meeting Avalon compatibility since it will be painfully 
obvious that the Avalon signals will not change state on a clock cycle when 
wait request has been set.
- The guts of the state machine (i.e. the section between 'elsif 
(WaitRequest = '0') then' and the 'else' will not be littered with checks 
about wait request with the potential that one check will be forgotten 
leading to a possible Avalon incompatibility that may be difficult to debug 
down to.
- The code for the guts of the state machine can concentrate on doing what 
it needs to be doing which is reading/writing the Avalon bus upon requests 
from the CPU side.

Again, these templates would be for master side code where the native 
interface is Avalon.

> BTW: for my CPU design - as Avalon is Altera specific I would
> never make Avalon the native interface. JOP runns quite well on
> Xilinx devices ;-)

That being the case then the Altera specific code is performing the function 
of a bridge between the CPU bus and the Avalon bus and would be segregated 
as such.  The ease/difficulty of that bridge design would then be a function 
of how close/different those two busses are.

In any case, you certainly know your design far better than I and obviously 
have been able to master Avalon enough to put together a working JOP design.

KJ 

>>>> You almost never want to have a fixed number of wait states but want to simply have the Avalon slave provide a wait request 
>>>> output and tell Avalon that by specifying that in the PTF file.
>>>
>>> Completely agree. When not writing and reading too many posts
>>> I'm working on that version of the SRAM interface. It was just
>>> a quick start as shown in the Quartus manual.
>>
>> BTW (to KJ): Do you have this type of Avalon slave
>> for an SRAM? Would save some time and errors for me ;-)
>>
> No, over the past several years my use of async SRAMs has gone to 0 even though I used to use them quite heavily.  They've been 
> replaced by internal
> I'm assuming that you've checked and that Altera didn't toss one in as a MegaCore?  Too bad.

Their core for the (older) NIOS boards just uses the tri-state
bridge .PTF approach - no VHDL

>
> Oh well, I'll stop posting and let you get back to work.
> KJ

Ok, I have now two versions of the SRAM interface: the plain
PTF version and the VHDL version (with I/O registers at
the FPGA pins to access a 15ns SRAM in two cycles at 100 MHz,
and the nwr with the neg. clock to save one cycle on write).

Here are some performance numbers of this JOP/SRAM
interface on an embedded benchmark. It measures iterations/s
and therefore higher numbers are better. All versions are clocked
at 100 MHz, 4 KB instruction cache and 512 Byte stack cache.
FPGA is Cyclone EP1C6-6, Memory is 32-bit SRAM 15ns. The only
difference is the memory interface.

SimpCon: 16,633
Avalon (PTF version): 14,015
Avalon (VHDL version): 13,920

So for me, the additional latency cycle(s) and not having
the early ack information for the CPU pipeline degrades
JOP's performance. Perhaps some Avalon specialist can do better.

However, it is compensated by the many peripherals that are
now just a mous click away ;-)

Martin

> Here are some performance numbers of this JOP/SRAM
> interface on an embedded benchmark. It measures iterations/s
> and therefore higher numbers are better. All versions are clocked
> at 100 MHz, 4 KB instruction cache and 512 Byte stack cache.
> FPGA is Cyclone EP1C6-6, Memory is 32-bit SRAM 15ns. The only
> difference is the memory interface.
>
> SimpCon: 16,633
> Avalon (PTF version): 14,015
> Avalon (VHDL version): 13,920
>
some additional numbers from the Altera DE2 board with
Cyclone II at 100 MHz with SDRAM and using on-chip
memory (the EP2C35 is big enough to run the benchmark
in on-chip memory).

Avalon SDRAM: 7,288
Avalon on-chip memory: 15,769

The performance issue with the SDRAM is clear. Just needs
some more caching to get a big (8 MB) memory with
acceptable performance ;-)
However, even the fast on-chip memory Avalon solution is
slightly slower than the two cycle SRAM connected via
SimpCon.

Martin 

"Martin Schoeberl" <mschoebe@mail.tuwien.ac.at> wrote in message 
news:44e0b13c$0$11352
> However, even the fast on-chip memory Avalon solution is
> slightly slower than the two cycle SRAM connected via
> SimpCon.
>

Well now THAT is incredibly surprising since the on-chip memory should be 
giving you 0 wait state, 0 latency performance (i.e. WaitRequest should 
always be low when accessing memory).  That would seem to point to either 
some issue that comes up every now and then in your 'CPU to Avalon' bridge 
master interface logic or something equally odd inside the Avalon fabric 
itself connecting the CPU to the memory.

I'd be interested to hear what you find.

KJ 

> Well now THAT is incredibly surprising since the on-chip memory should be giving you 0 wait state, 0 latency performance (i.e. 
> WaitRequest should

That's not right anymore. You have at minimum one cycle latency
as addresses are registered in current on-chip RAMs. Probably also
the output is registered. However, I don't know - would have to
look into the VHDL code.

> always be low when accessing memory).  That would seem to point to either

waitrequest always low would only be possible with pipelining using
datavalid. That helps on cach fill, but not on an ordinary read.

Perhaps I should try to connect the on-chip RAM to my
'native' SimpCon interface and check the performance.
That should be better than the 2 cycle SRAM. However,
this is a more theoretical experiment as Java programs
usually will not fit into on-chip RAMs ;-)
C programs with NIOS are more code efficient.

> some issue that comes up every now and then in your 'CPU to Avalon' bridge master interface logic or something equally odd inside 
> the Avalon fabric itself connecting the CPU to the memory.

One issue is that my CPU takes advantage from this 'counting down
ready' signal (the bsy_cnt in SimpCon). I can't do this with the
Avalon spec. Therefore, there is a preformance penalty - Inherent
due to the design.

> I'd be interested to hear what you find.

The CPU/Avalon bridge is probably sub-optimal. Will
try to check this out (First I have to get the Altera ModelSim
version running - would make it easier - still havn't compiled
the missing SOPC libraries for ModelSim).

Martin 

>> Here are some performance numbers of this JOP/SRAM
>> interface on an embedded benchmark. It measures iterations/s
>> and therefore higher numbers are better. All versions are clocked
>> at 100 MHz, 4 KB instruction cache and 512 Byte stack cache.
>> FPGA is Cyclone EP1C6-6, Memory is 32-bit SRAM 15ns. The only
>> difference is the memory interface.
>>
>> SimpCon: 16,633
>> Avalon (PTF version): 14,015
>> Avalon (VHDL version): 13,920
>>
> some additional numbers from the Altera DE2 board with
> Cyclone II at 100 MHz with SDRAM and using on-chip
> memory (the EP2C35 is big enough to run the benchmark
> in on-chip memory).
>
> Avalon SDRAM: 7,288
> Avalon on-chip memory: 15,769
>
and some more:

JOP at 100MHz on the Altera DE2 using the 16-bit SRAM:

Avalon: 11,322
SimpCon: 14,760

So for the SRAM interface SimpCon is a clear winner ;-)
The 16-bit SRAM SimpCon solution is even faster than
the 32-bit SRAM Avalon solution.

BTW: the embedded benchmark is a control application
which is does not need a high memory bandwith. For a
different benchmark (a UDP/IP application with IP
processing - lot of buffer access) the difference is
larger. With the 16-bit SRAM:

Avalon: 4,302
Simpcon: 5,716

again - higher number is better

Martin