compressing Xilinx bitstreams

Forgive me if this has been asked before, but does anybody have
comments or links to simple methods of compressing/decompressing
Xilinx configuration bitstreams? I've been perusing a few of my .rbt
files, and they have long bunches of 1s and 0s (interestingly,
different designs seem to have more 1s, others mostly 0s.) I'd think
that something very simple might achieve pretty serious (as, maybe
2:1-ish) compression without a lot of runtime complexity. We generally
run a uP from EPROM, with the uP code and the packed Xilinx config
stuff in the same eprom, with the uP bit-banging the Xilinx FPGA at
powerup time. So a simple decompressor would be nice.

I did google for this... haven't found much.

Thanks,

John

John Larkin wrote:
> Forgive me if this has been asked before, but does anybody have
> comments or links to simple methods of compressing/decompressing
> Xilinx configuration bitstreams? I've been perusing a few of my .rbt
> files, and they have long bunches of 1s and 0s (interestingly,
> different designs seem to have more 1s, others mostly 0s.) I'd think
> that something very simple might achieve pretty serious (as, maybe
> 2:1-ish) compression without a lot of runtime complexity. We generally
> run a uP from EPROM, with the uP code and the packed Xilinx config
> stuff in the same eprom, with the uP bit-banging the Xilinx FPGA at
> powerup time. So a simple decompressor would be nice.
> 
> I did google for this... haven't found much.

  There was discussion on this some months ago.
Might show up here ? http://www.fpga-faq.com/

  You can run std ZIP tools on the files, to get a quick
'practical limit' indication.
  We did some work with Run length compression, which is
very simple (simple enough to code into CPLD), but has medium
compression gains. ISTR about half the gains of ZIP ?

  It could be improved with a RLC-Compiler/optimiser that looked
for the best pattern/lengths for that chip, or even bitstream,
as you could store the RLC params as a 'header', but we did not
go that far.

  Be a good project for Xilinx to do as an app note :)

  It would make sense to target to the pattern-repeat sizes on
devices like FPGAs.

-jg

The bit generation tool has an option to compress the .bit file. I use this
when I'm loading over JTAG to save time. I assume Xilinx has info on in
system programming with a compressed .bit file.

However, I've observed the same phenomenon as you: when I zip a .bit file it
is usually less than 50% of the original size. My guess is even a trivial
run length encoding compression would be helpful. There are plenty of
resources for Lempel Ziv compression on the web:

see http://www.dogma.net/markn/articles/lzw/lzw.htm

If you get it working please post/send the result.


"John Larkin" <jjlarkin@highSNIPlandTHIStechPLEASEnology.com> wrote in
message news:hh34d0tud78se5vqejirkc2bvufbd8io3d@4ax.com...
>
> Forgive me if this has been asked before, but does anybody have
> comments or links to simple methods of compressing/decompressing
> Xilinx configuration bitstreams? I've been perusing a few of my .rbt
> files, and they have long bunches of 1s and 0s (interestingly,
> different designs seem to have more 1s, others mostly 0s.) I'd think
> that something very simple might achieve pretty serious (as, maybe
> 2:1-ish) compression without a lot of runtime complexity. We generally
> run a uP from EPROM, with the uP code and the packed Xilinx config
> stuff in the same eprom, with the uP bit-banging the Xilinx FPGA at
> powerup time. So a simple decompressor would be nice.
>
> I did google for this... haven't found much.
>
> Thanks,
>
> John
>

First, please be aware that the ACSII .rbt file is 8x the simple .bin file
size.  Check the bitgen options and you'll find the ability to generate the
straight binary file - 1s and 0s at the bit level, not the ASCII character
level.  Compression beyond that may be what you're looking for, but please -
start with the binary file.

"John Larkin" <jjlarkin@highSNIPlandTHIStechPLEASEnology.com> wrote in
message news:hh34d0tud78se5vqejirkc2bvufbd8io3d@4ax.com...
>
> Forgive me if this has been asked before, but does anybody have
> comments or links to simple methods of compressing/decompressing
> Xilinx configuration bitstreams? I've been perusing a few of my .rbt
> files, and they have long bunches of 1s and 0s (interestingly,
> different designs seem to have more 1s, others mostly 0s.) I'd think
> that something very simple might achieve pretty serious (as, maybe
> 2:1-ish) compression without a lot of runtime complexity. We generally
> run a uP from EPROM, with the uP code and the packed Xilinx config
> stuff in the same eprom, with the uP bit-banging the Xilinx FPGA at
> powerup time. So a simple decompressor would be nice.
>
> I did google for this... haven't found much.
>
> Thanks,
>
> John
>

John,

I think that I had heard that zipping, and unzipping bit files led to 
the most compression (2:1 or better). (classic unix or windows zip/unzip)

I think that a zip/unzip routine would be a great example of something a 
  uP could do without an unreasonable amount of memory (ROM+RAM) support.

Austin


John Larkin wrote:
> Forgive me if this has been asked before, but does anybody have
> comments or links to simple methods of compressing/decompressing
> Xilinx configuration bitstreams? I've been perusing a few of my .rbt
> files, and they have long bunches of 1s and 0s (interestingly,
> different designs seem to have more 1s, others mostly 0s.) I'd think
> that something very simple might achieve pretty serious (as, maybe
> 2:1-ish) compression without a lot of runtime complexity. We generally
> run a uP from EPROM, with the uP code and the packed Xilinx config
> stuff in the same eprom, with the uP bit-banging the Xilinx FPGA at
> powerup time. So a simple decompressor would be nice.
> 
> I did google for this... haven't found much.
> 
> Thanks,
> 
> John
> 

Austin Lesea wrote:
> John,
> 
> I think that I had heard that zipping, and unzipping bit files led to 
> the most compression (2:1 or better). (classic unix or windows zip/unzip)

  Yes, but a compress targeted to FPGA content should be more efficent, 
and use less resource than a generic compress.

> I think that a zip/unzip routine would be a great example of something a 
>  uP could do without an unreasonable amount of memory (ROM+RAM) support.

One engineer's reasonable is anothers excessive :)

There are two main classes of uC loader:
* Ones that store the compressed stream on-chip, and so can expect
   to have good random access, for things like decompress tables.
   Large-code uC also tend to have larger RAM
* Ones that store the compressed stream in low-cost serial flash.
   In this class, table handling is not as easy.
   uC used here could be as miniscule as the PIC10F in SOT23
   PIC10F starts at 16 bytes RAM and 256 Words Code space...
CPLDs are also used for loaders, and they can do simple decompress.

=jg

"John Larkin" <jjlarkin@highSNIPlandTHIStechPLEASEnology.com> wrote in
message news:hh34d0tud78se5vqejirkc2bvufbd8io3d@4ax.com...
>
> Forgive me if this has been asked before, but does anybody have
> comments or links to simple methods of compressing/decompressing
> Xilinx configuration bitstreams? I've been perusing a few of my .rbt
> files, and they have long bunches of 1s and 0s (interestingly,
> different designs seem to have more 1s, others mostly 0s.) I'd think
> that something very simple might achieve pretty serious (as, maybe
> 2:1-ish) compression without a lot of runtime complexity. We generally
> run a uP from EPROM, with the uP code and the packed Xilinx config
> stuff in the same eprom, with the uP bit-banging the Xilinx FPGA at
> powerup time. So a simple decompressor would be nice.
>
> I did google for this... haven't found much.
>
> Thanks,
>
> John

VCC did a package called HOTMan that does compression. It takes the bit file
and turns it into a compressed file that looks like...

int testArray[2669]=\
{
0xddedda78,0xe55c8c5f,0xefe1c079.... }

We get at least 4 to 1 and small designs in big chip can get 50 to 1. The
above format allows you to compile the design into a C/C++ program.

Steve

John Larkin wrote:

> Forgive me if this has been asked before, but does anybody have
> comments or links to simple methods of compressing/decompressing
> Xilinx configuration bitstreams? I've been perusing a few of my .rbt
> files, and they have long bunches of 1s and 0s (interestingly,
> different designs seem to have more 1s, others mostly 0s.) I'd think
> that something very simple might achieve pretty serious (as, maybe
> 2:1-ish) compression without a lot of runtime complexity. We generally
> run a uP from EPROM, with the uP code and the packed Xilinx config
> stuff in the same eprom, with the uP bit-banging the Xilinx FPGA at
> powerup time. So a simple decompressor would be nice.
> 
> I did google for this... haven't found much.
> 
> Thanks,
> 
> John
> 

No links, but have you considered simple run-length limiting?  I can 
think of at least one scheme that would be guaranteed sub-optimal from a 
compression standpoint but that wouldn't take much code -- just encode 
any string of 0xff or 0x00 bytes as that byte followed by a count -- so 
that 0x00 0x00 0x00 0x00 becomes 0x00 0x04, for instance.  You have the 
overhead that 0x00 becomes 0x00 0x01, and you also can't encode anything 
that spans bytes -- but you may be happy with it none the less.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

On Thu, 17 Jun 2004 22:14:15 GMT, "John_H" <johnhandwork@mail.com>
wrote:

>First, please be aware that the ACSII .rbt file is 8x the simple .bin file
>size.  Check the bitgen options and you'll find the ability to generate the
>straight binary file - 1s and 0s at the bit level, not the ASCII character
>level.  Compression beyond that may be what you're looking for, but please -
>start with the binary file.
>

Of course. We have a little utility, vaguely like a linker, that
gobbles up Motorola .s28 files and Xilinx .rbt files and builds a rom
image, all properly squashed into bits. It's cute... it even saves the
beginning of the rbt ASCII header in the rom image for FPGA version
verification. My observation was that the bits themselves include long
runs of 1s or 0s. 

I'd like to design a board using a 28-pin eprom (space is at a premium
here) but plan hooks for using a bigger Xilinx chip some day, and then
I'd run out of rom space to store the config bits. So having a
compression scheme would give us the margin to use the small eprom.

Suppose the compressed data were an array of bytes. If the MS bit of a
byte were 0, the remaining 7 bits are to be loaded verbatum; if the MS
bit is a 1, the other 7 bits specify a run of up to 63 1's or 0's.

Something like that; the exact numbers may need tuning. Very easy to
unpack, not hard to encode. I'd have to test some actual config files
to see how good something like this could compress.

John

On Thu, 17 Jun 2004 14:41:01 -0700, John Larkin
<jjlarkin@highSNIPlandTHIStechPLEASEnology.com> wrote:

>
>Forgive me if this has been asked before, but does anybody have
>comments or links to simple methods of compressing/decompressing
>Xilinx configuration bitstreams? I've been perusing a few of my .rbt
>files, and they have long bunches of 1s and 0s (interestingly,
>different designs seem to have more 1s, others mostly 0s.) I'd think
>that something very simple might achieve pretty serious (as, maybe
>2:1-ish) compression without a lot of runtime complexity. We generally
>run a uP from EPROM, with the uP code and the packed Xilinx config
>stuff in the same eprom, with the uP bit-banging the Xilinx FPGA at
>powerup time. So a simple decompressor would be nice.
>
>I did google for this... haven't found much.
>
>Thanks,
>
>John

See:

www.ee.washington.edu/people/faculty/hauck/publications/runlength.PDF
www.ee.washington.edu/people/faculty/hauck/publications/runlengthTR.PDF
www.ee.washington.edu/people/faculty/hauck/publications/runlengthJ.pdf

It should be straightforward to generate some RLL compression and
decompression code.  You might want to test the algorithms on a PC to
make sure that the decompressed output ends up the same as the
uncompressed input.  A garbled bitstream can have the same effect as
the MC6800 HCF opcode...



================================

Greg Neff
VP Engineering
*Microsym* Computers Inc.
greg@guesswhichwordgoeshere.com