Best FPGA for floating point performance

Good posts,

Proabably the best thread I've read through in a long while.

Serious stuff, with real numbers.

Thanks for all who posted.

I still claim that by instantiating more than one core in an FPGA, they 
still win oven even the best uP, however.

Massive parallelism is of course nearly impossible to use effectively, 
as there is no language that preserves if operations could be done in 
parallel.

We are stuck with a "paper tape" programming mentality in a flash memory 
age.

Austin

gantlord wrote:

> Oops, forgot to say where I'm getting this from:
> 
> Keith Underwood from Sandia Labs has been looking at this sort of
> stuff. I'm just parroting it out here for you guys (I wouldn't anyone
> thinking I'd been guilty of an original thought).
> 
> R
> 

well you can parallel ifs if you have enough gates... just do every possible
combination and pick the correct one...
Isn't that how they cracked DES ?

I would have to agree with Austin about the rest... being both a programmer
and a hardware engineer... I know what's good for what.. you can't beat
hardware for shear grunt... it doesn't matter how fast the processor is..
clock for clock the dedicated hardware will beat it.. sometime even not
clock for clock... the hardware can be dedicated to a single task which is
extremely efficient,  Processors by nature aren't... Processors may be able
to do 50 million FLOPS, but they also need to be able to get the
instructions and data from memory.  and then they have loops and if your
running Linux or windows... they are trying to do so much work that they get
bogged down.

In saying this, there are FPU's up to the grade even if they are specialized
... your average 3D graphics processor will leave an x86 for dead FPU
wise... they are by far the most impressive floating point units in mass
production today (The X800 does 200 billion FLOPS).


Simon


"austin" <austin@xilinx.com> wrote in message
news:derheh$b5t2@cliff.xsj.xilinx.com...
> Good posts,
>
> Proabably the best thread I've read through in a long while.
>
> Serious stuff, with real numbers.
>
> Thanks for all who posted.
>
> I still claim that by instantiating more than one core in an FPGA, they
> still win oven even the best uP, however.
>
> Massive parallelism is of course nearly impossible to use effectively,
> as there is no language that preserves if operations could be done in
> parallel.
>
> We are stuck with a "paper tape" programming mentality in a flash memory
> age.
>
> Austin
>
> gantlord wrote:
>
> > Oops, forgot to say where I'm getting this from:
> >
> > Keith Underwood from Sandia Labs has been looking at this sort of
> > stuff. I'm just parroting it out here for you guys (I wouldn't anyone
> > thinking I'd been guilty of an original thought).
> >
> > R
> >

robin.bruce@gmail.com wrote:
> Regarding  x86 vs FPGA for double-precision floating-point arithmetic:
>
> Good points about the relative performances, but you need to look at
> more than the peak FLOPs/s to do a comparison between the
> architectures. Non-trivial double precision algorithms running on
> microprocessors can use anything between 5 and 90 % of their peak
> FLOPs/s. Floating-point algorithms are typically memory bound for
> microprocessors. FPGAs on the other hand are typically bound by peak
> FLOPs/s. FPGAs are able to significantly outperform modern
> microprocessors on memory bandwidth sensitive double precision
> operations. Microprocessors still beat FPGAs for non-memory bound
> operations because they have higher peak FLOPs/s, but this situation
> won't last. Peak FLOPs/s for FPGAs are set to exceed those of uP's in
> the not too distant future. To chase the increases in peak FLOPs/s on
> microprocessors you need ever more complex memory hierarchies, while in
> FPGAs it looks like it will be some time before such techniques will be
> needed to get the most out of the floating-point units.



Some thoughts

I think that both FPGAs and Multi cpus could go through some serious
evolution and end up much closer to the same spot.

FPGAs will obviously get some FPU capability sooner or later and be
able to cross over current single threaded cpus but maybe one day high
end cpus will get some FPGA capability too (inside not outside, but at
1/10th the clock or so).

Single cpus will give way to Multis, ever so slowly programmers are
going to learn to deal with concurrency, many will need a big wallop to
the head before they get a real clue. We have already been there
(transputers, occam ->HandelC there is a large body of knowledge on how
to do parallel SW well but most choose to ignore it).

Now if cpu guys really get moving away from x86 backwards
compatibility, we could see much more massive reduction in complexity
of cpus which immediately makes space for many more simple cores which
would put them far ahead again of FPGAs in FP. But the world is
addicted to x86 compatibility (for no good reason IMHO) except of
course in the embedded space where FPGAs are, so this will remain the
domain of Clearspeed and a few other massive cpu-count on a chip
vendors.

If two guys sharing an apartment, one at say I/A/I, one at say A/X had
a clean sheet to design new massively multi cpu on a chip and again for
FPGA with cpu components like FPU, we would end up in a more similar
place.  Both will be memory IO bound since either can use the same near
1000 pin packages. Cpus though have traditionally used el cheapo junk
DRAM with massive SRAM caches to make up for the low speed, turning
100ns into 1ns BUT only for highly localised codes. Use a semi random
no to index large array bigger than L2 and fastest cpu in the world
reduces to a few Mips. FPGA guys such as Ray and many others here are
far more familiar with space time trade off, some times less is really
more if you can go serial faster, and far easier to P/R.

This leads to use of exotic memories where needed, if you want more
memory IO to speed up the application, we have RLDRAM, FCDRAM, QDR SRAM
with orders more useful bandwidth than SDRAM with effective issue rates
closer to the cpu L2 SRAM caches (ie 2.5ns RLDRAM). I think if X or A
were seriously going to put significant nos of FPUs in the fabric, also
put in ready to use memory controllers that can match the speed of
these memories. Right now RLDRAM moves at 400MHz but FPGAs can only
drive them at 300MHz and next year Micron promises 533MHz sub 2ns issue
rates. Rambus too has amazing IO rates but useless latencies. It would
be neat to see an FPGA actually drive the fastest memories out there
while watching cpus miss their TLBs and pass through the OS and DRAM to
get data.

If you combine these memories with ready to use highest speed mem
interface and FPUs in fabric in quantity to make up for clock speed,
then things get interesting. An FPGA will limit density of FPUs  to
something like 1 every 4 BlockRams or 4 18.18 muls. On the cpu side, a
cluster of cpus will be limited to the relative cost of FPU v whole
superscaler+cache+TLB+ stuff. If cpus continue to use the current model
of shared cache models they will lose the race eventually and
deservedly so even with a 10x clock speed advantage.

If cpu designers also look more deeply at using high issue rate DRAMs
(<3ns) with multi threading to hide the relatively small left over
latency (20ns and falling), then cpus will likely get the upper hand,
but I don't see such a radical change there. I will be presenting such
a processor (Transputer) design though at cpa2005 in a few weeks.

If I could influence FPU development, I might suggest a more humble
approach, build a std double capable IEEE unit around a single 18b mul
and do it serially, clocks don't matter, wires do. Most of the cost is
in the mul array, it can be used for both int or real use. The
controller logic to do this in FPGA fabric eats up resources no end but
a slow but steady serial FP controller might not use so much logic.
Anyway as hard logic its no doubt 20x smaller and can be disabled out.
Now if every 18.18 mul is 20% more expensive and can be enabled as a
serial real math unit thats very valuable on a fine grain and if there
were upto 500 of them they can be ganged more practically in the time
line than space area. I would hate to have to deal with P/R for a real
64 bit engine that would be available in the wrong place.

just my 2 bits

johnjakson at usa ...
transputer2 at yahoo ...

On 28 Aug 2005 08:54:43 -0700, "JJ" <johnjakson@yahoo.com> wrote:

>Now if cpu guys really get moving away from x86 backwards
>compatibility, we could see much more massive reduction in complexity
>of cpus which immediately makes space for many more simple cores which
>would put them far ahead again of FPGAs in FP. But the world is
>addicted to x86 compatibility (for no good reason IMHO) except of
>course in the embedded space where FPGAs are, so this will remain the
>domain of Clearspeed and a few other massive cpu-count on a chip
>vendors.

Actually I don't think this is an issue anymore. These days the ISA
management is really a small portion of a CPU implementation. Take a
look at the following picture
http://www.chip-architect.com/news/Opteron_Instr_Cache.jpg and decide
for yourself how much area you would save if you replaced the x64
decoder with a mips 64 style decoder. The rest of the die is mostly
cache and integer and fp alus which you'd need anywhere.

mk wrote:
> On 28 Aug 2005 08:54:43 -0700, "JJ" <johnjakson@yahoo.com> wrote:
>
> >Now if cpu guys really get moving away from x86 backwards
> >compatibility, we could see much more massive reduction in complexity
> >of cpus which immediately makes space for many more simple cores which
> >would put them far ahead again of FPGAs in FP. But the world is
> >addicted to x86 compatibility (for no good reason IMHO) except of
> >course in the embedded space where FPGAs are, so this will remain the
> >domain of Clearspeed and a few other massive cpu-count on a chip
> >vendors.
>
> Actually I don't think this is an issue anymore. These days the ISA
> management is really a small portion of a CPU implementation. Take a
> look at the following picture
> http://www.chip-architect.com/news/Opteron_Instr_Cache.jpg and decide
> for yourself how much area you would save if you replaced the x64
> decoder with a mips 64 style decoder. The rest of the die is mostly
> cache and integer and fp alus which you'd need anywhere.

While the significance of any ISA implementation starts to shrink in
comparison to the caches, that is exactly the point I was making. The
caches are the way to make huge slow memory seem faster through ever
more complexity in the cpu design though every comp arch trick in the
book. In the end you only get 1 or 2 cpus for now (at 2-3GHz though)
hanging on to the caches coat tails. Notice the Itanium is also almost
entirely cache too and thats why it is also lower cost than most would
expect since the SRAM is repairable while cpus are not.

Again if you build single threaded architectures (any ISA will do), you
end up with something like an Opteron/G5 cache hierarchy. If you go to
highly threaded cpus and matching highly interleaved memory
architectures, the threading on both side allows the latencies to more
or less match for very much simpler cpu designs. In effect DRAM can
replace cache as long as the issue rate compares with L2 access rates
and it now does with RLDRAM and as long as the latency hiding is in the
same ballpark as the reduced DRAM latency. In return you can use lots
of small latency hiding cpus but one must now deal with lots of threads
something that some people know what to do with.

johnjakson at usa ..

On 28 Aug 2005 16:49:15 -0700, "JJ" <johnjakson@yahoo.com> wrote:

>Again if you build single threaded architectures (any ISA will do), you
>end up with something like an Opteron/G5 cache hierarchy. If you go to
>highly threaded cpus and matching highly interleaved memory
>architectures, the threading on both side allows the latencies to more
>or less match for very much simpler cpu designs. In effect DRAM can
>replace cache as long as the issue rate compares with L2 access rates
>and it now does with RLDRAM and as long as the latency hiding is in the
>same ballpark as the reduced DRAM latency. In return you can use lots
>of small latency hiding cpus but one must now deal with lots of threads
>something that some people know what to do with.
>
the problem with highly threaded cpus is that they are not very good
at running wordprocessors, spreadsheets and fpga p&r tools and that's
where most cpus are used so the two cpu developers put most of their
money into developing slightly threaded architectures with full
multi-cores instead of smt. if you noticed the new multi-core i86
implementations don't support ht anymore. another reason this idea is
not very easy to implement is that regardless what's happening with
power on 90nm and lower processes, speed still counts and embedding
dram on a high speed logic process is a big problem. if and when a new
memory structure comes out which can be embedded in logic process and
as inexpensive as 1t+1c dram, i am sure isa architects will look at
highly threaded cpus again but probably not before then. also keep in
mind that developing cpus is a very expensive endevour and anyone who
is not developing an x86 compatible one seems to be giving up
including intel. i expect sun will drop sparc pretty soon and intel
will drop itanium too; moving their itanium developers to xeon
projects doesn't bode too well which is for the better as we won't
have to deal with a completely proprietary, fully moated with patents
isa.

mk wrote:
> the problem with highly threaded cpus is that they are not very good
> at running wordprocessors, spreadsheets and fpga p&r tools and that's
> where most cpus are used

Actually, that's rather to generous to us fpga p&r users.  If instead 
you do take a minute to look at the kind of benchmarks that are always 
put forward to test the latest iteration of x86, you'd find 
predominately parallel friendly workloads (eg. anything "multimedia" and 
to a moderately degree games).  Spreadsheets are inherently not parallel 
hostile and performance of wordprocessors are not really an issue.

I believe John's point was that the x86, RISC, and all the rest of the 
inheriently sequential architectures will never scale and we should 
abandon them.  It's an old war cry that we've been hearing on and off 
for decades, but recent Hotchips 17, not least David Kirk's keynote, 
really felt like the wind of change.   Power efficiency has become a 
mainstream concept and it will slowly but surely require us to change 
our ways.

Back in the real world, John, people have sizable investments in 
existing (x86) software which aren't going away anytime soon, but the 
change is coming: on-chip multi cores, Cell, xbox-360, Niagria.

Eh, isn't this the "Not enough parallelism in programming" on comp.arch? 
  What has this got to do with "Best FPGA for floating point 
performance" :-)

Tommy

well its an interesting follow on... the more cores.. the better the
floating point.. I think that's the aim here... but I'm surprised no one has
mentioned the cell... it has 9 processors on board.

Simon

"Tommy Thorn" <foobar@nowhere.void> wrote in message
news:AvxQe.10949$p%3.42685@typhoon.sonic.net...
> mk wrote:
> > the problem with highly threaded cpus is that they are not very good
> > at running wordprocessors, spreadsheets and fpga p&r tools and that's
> > where most cpus are used
>
> Actually, that's rather to generous to us fpga p&r users.  If instead
> you do take a minute to look at the kind of benchmarks that are always
> put forward to test the latest iteration of x86, you'd find
> predominately parallel friendly workloads (eg. anything "multimedia" and
> to a moderately degree games).  Spreadsheets are inherently not parallel
> hostile and performance of wordprocessors are not really an issue.
>
> I believe John's point was that the x86, RISC, and all the rest of the
> inheriently sequential architectures will never scale and we should
> abandon them.  It's an old war cry that we've been hearing on and off
> for decades, but recent Hotchips 17, not least David Kirk's keynote,
> really felt like the wind of change.   Power efficiency has become a
> mainstream concept and it will slowly but surely require us to change
> our ways.
>
> Back in the real world, John, people have sizable investments in
> existing (x86) software which aren't going away anytime soon, but the
> change is coming: on-chip multi cores, Cell, xbox-360, Niagria.
>
> Eh, isn't this the "Not enough parallelism in programming" on comp.arch?
>   What has this got to do with "Best FPGA for floating point
> performance" :-)
>
> Tommy
>

Tommy Thorn wrote:
> mk wrote:
> > the problem with highly threaded cpus is that they are not very good
> > at running wordprocessors, spreadsheets and fpga p&r tools and that's
> > where most cpus are used
>
> Actually, that's rather to generous to us fpga p&r users.  If instead
> you do take a minute to look at the kind of benchmarks that are always
> put forward to test the latest iteration of x86, you'd find
> predominately parallel friendly workloads (eg. anything "multimedia" and
> to a moderately degree games).  Spreadsheets are inherently not parallel
> hostile and performance of wordprocessors are not really an issue.
>

dittos

> I believe John's point was that the x86, RISC, and all the rest of the
> inheriently sequential architectures will never scale and we should
> abandon them.  It's an old war cry that we've been hearing on and off
> for decades, but recent Hotchips 17, not least David Kirk's keynote,
> really felt like the wind of change.   Power efficiency has become a
> mainstream concept and it will slowly but surely require us to change
> our ways.
>

Yes I was around the 1st time (Inmos) in early 80s, toooo early to cry
wolf..

Yes, SRAM at mini DRAM sizes is a really stupid idea esp if its only a
2 or 3 times faster (than RLDRAM that is), interleaving across banks is
the way to go forward but even more is needed....

SRAM power is many orders higher than DRAM cell leakage but both can
have the same effective throughputs see QDR SRAM v RLDRAM if threading
is used. SRAMs real value lies in having lots of teeny weeny blocks
(BlockRams), its the total parallel bandwidth that FPGAs have that
shines. Right now the cpu vendors are looking ever more stupid by
serializing all their SRAM accesses through one MMU pipeline even if it
runs at 1 or 2GHz to L1 (1ns?) and less for L2 (4ns?).

> Back in the real world, John, people have sizable investments in
> existing (x86) software which aren't going away anytime soon, but the
> change is coming: on-chip multi cores, Cell, xbox-360, Niagria.
>
> Eh, isn't this the "Not enough parallelism in programming" on comp.arch?
>   What has this got to do with "Best FPGA for floating point
> performance" :-)
>
> Tommy

Yes precisely, precisely, dittos...

I will have to look up the Kirk keynote.

Back in the real world, of course x86 is not going anywhere different
for some time. It doesn't matter one iota that MTA architecture isn't
any good at office SW although I would actually argue thats not even
true. I have written EDA and text editers and can see plenty of
concurrency possibilities there too but not practical in C though, but
perfectly practical in Occam with C. Serializing software has always
been about being lazy and not having the right language. If people were
familiar with BeOS/Haiku they would know what a multithreaded OS looks
like too, it wallops any MS or Linux lookylike in responsiveness, but
it never had many apps. It used C++ but thats not the right language
for threaded apps at a fine grain, only course grain. Next generation
MS Vista is demanding a 3GHz Pentium in order to be useable yet all its
going to deliver is crome and shiny effect (both of which are highly
parallelizable hehe). Most of the simple user improvements in W2K I
want to see will never come. I can see that shinyness in PC GUIs is
just another form of multimedia graphics.


The reason the post is on this side is that FPGAs are the only
practical way to demonstrate other comp arch ideas with out access to
$100M VLSI/ASIC model.

The upside of FPGAs is that a larger FPGA can demonstrate lots of
smaller cpus but only upto the 300MHz clock wall and only for simpler
RISC designs. After demonstration, there is no reason why FPGA
processor can't be ASICed at about 3x the speed, the limit being in my
case a dual port BlockRam equiv std cell which Samsung has at 1GHz.

FPGAs can freely use any available memory technology, esp these RLDRAMs
with high issue rates that are better than L2 SRAM access times. Most
all DSP and Networking people fully understand why hardware is threaded
at all levels, its starts with EE101, but in CS101 its 90* sequential,
the comp arch + user world people don't quite get it yet.

The downside of FPGA for the HPC crowd is the FPU limit, it makes it
impossible for any FPGA processor to match some of the spectactular FP
chips out there but then I never much used FP in software either.

Now if an 18.18 mul had a freeish FPU serial controller around it, it
might be very useable for a somewhat serial MTA processor in spades.

johnjakson at usa ...
transputer2 at yahoo..

Simon Peacock wrote:
> well its an interesting follow on... the more cores.. the better the
> floating point.. I think that's the aim here... but I'm surprised no one has
> mentioned the cell... it has 9 processors on board.
>
> Simon
>


Well I will see the Cell architects at CPA2005. They will describe the
Cell communications architecture to a group of people most interested
in parallelism and some might be esp interested in FP performance too.

My own view is the Cell is one way forward but not the way I would
prefer. Its really one magnificent core surrounded by 8 lesser slaves
(as much as I know). The gaming world may have a much harder time
exploiting those slaves than IBM might wish, but then that would be
true for almost any multi processor, I still remember driving (game)
cars into and out of walls, their physics isn't all that good.

In the end, I think the memory model comes 1st, the FPU and the rest
follows on.

John