CMOS camera w/ USB2 -- crazy?

Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise.  My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0.  His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.).  Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
 micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!

GB wrote:
> 
> Hi,
> 
> I'm a firmware guy pulled into a project well out of my area of
> expertise.  My boss wants to build (essentially) a digital camera
> using an image sensor chip (1600x1200) and output it's data
> "as fast as possible" using USB2.0.  His initial concept, being
> that I'm a firmware guy, was to use a "really fast micro."
> Normally the company is involved with small 8-bitter micro
> projects, so you can see I'm well out of my normal bounds.
> 
> Now this seems like a pretty big stretch to me... and I don't see
> how it can be done without the speed of hardware (the image
> chips all seem to have clock speeds in the tens of MHz and the
> USB2 transfer rate is 480Mbps (theor.).  Do aspects of this
> project require an FPGA to keep the data "as fast as possible?"
> If we use and FPGA for camera-to-RAM and then use a
>  micro for the USB2 part, what kind of fast micros can
> move data at that rate?

I don't think the micro should be in the data path.  A micro can be used
for control and management, but the data path should be pure hardware
for maximum throughput.  If you look around, you will find that there
are USB chips with integrated micros (many are 8051s).  But the USB 2.0
chips typically have a DMA engine for the data path.  

> Also, this is something that I am sure we will have to contract
> out, so if you have any past experience with this, please let
> me know your thoughts (and if you are available).

We have some background in this area.  If you are interested, please
contact me at the email address below or at sales (at) arius.com.  

-- 

Rick "rickman" Collins

rick.collins@XYarius.com
Ignore the reply address. To email me use the above address with the XY
removed.

Arius - A Signal Processing Solutions Company
Specializing in DSP and FPGA design      URL http://www.arius.com
4 King Ave                               301-682-7772 Voice
Frederick, MD 21701-3110                 301-682-7666 FAX

What image sensor chip are you looking at ???

GB wrote:

> Hi,
> 
> I'm a firmware guy pulled into a project well out of my area of
> expertise.  My boss wants to build (essentially) a digital camera
> using an image sensor chip (1600x1200) and output it's data
> "as fast as possible" using USB2.0.  His initial concept, being
> that I'm a firmware guy, was to use a "really fast micro."
> Normally the company is involved with small 8-bitter micro
> projects, so you can see I'm well out of my normal bounds.
> 
> Now this seems like a pretty big stretch to me... and I don't see
> how it can be done without the speed of hardware (the image
> chips all seem to have clock speeds in the tens of MHz and the
> USB2 transfer rate is 480Mbps (theor.).  Do aspects of this
> project require an FPGA to keep the data "as fast as possible?"
> If we use and FPGA for camera-to-RAM and then use a
>  micro for the USB2 part, what kind of fast micros can
> move data at that rate?
> 
> Also, this is something that I am sure we will have to contract
> out, so if you have any past experience with this, please let
> me know your thoughts (and if you are available).
> 
> Thanks!
> 
> 

GB wrote:

> I'm a firmware guy pulled into a project well out of my area of
> expertise.  My boss wants to build (essentially) a digital camera
> using an image sensor chip (1600x1200) and output it's data
> "as fast as possible" using USB2.0.  His initial concept, being
> that I'm a firmware guy, was to use a "really fast micro."
> Normally the company is involved with small 8-bitter micro
> projects, so you can see I'm well out of my normal bounds.
> 
> Now this seems like a pretty big stretch to me... and I don't see
> how it can be done without the speed of hardware (the image
> chips all seem to have clock speeds in the tens of MHz and the
> USB2 transfer rate is 480Mbps (theor.).  Do aspects of this
> project require an FPGA to keep the data "as fast as possible?"
> If we use and FPGA for camera-to-RAM and then use a
>  micro for the USB2 part, what kind of fast micros can
> move data at that rate?
> 
> Also, this is something that I am sure we will have to contract
> out, so if you have any past experience with this, please let
> me know your thoughts (and if you are available).

Sounds like a good project for an FPGA.  Micros don't keep up with this kind of data rates, standard parts do what everyone else is doing (and this doesn't sound standard), and ASICS can do this, if you have the volume to make it worthwhile, but it would be wise to prototype the design in a FPGA first.

I've done several FPGA designs to solve similar problems.  I'm not available, but I'm fairly sure I know of someone fairly good that is available, and if you send mail to (attbi.com phil-hays) I'll get you in contact with him.


-- 
Phil Hays

"hamilton" <hamilton@deminsional.com> wrote in message
news:3f5b561c_3@omega.dimensional.com...

> What image sensor chip are you looking at ???
>

That's another undecided at this time, but CMOS most likely.

GB

"rickman" <spamgoeshere4@yahoo.com> wrote in message
news:3F5B4FAC.41D84F7B@yahoo.com...

> I don't think the micro should be in the data path.  A micro can be used
> for control and management, but the data path should be pure hardware
> for maximum throughput.  If you look around, you will find that there
> are USB chips with integrated micros (many are 8051s).  But the USB 2.0
> chips typically have a DMA engine for the data path.

From what I understand, the image size is 1-2 MB (only need 8-bit depth)
large and the USB packets are quite small in comparison.  So there needs to
be some smarts for setting up what part of the data gets DMA'd to the
USB endpoint buffers for any given transfer?  Or does the FPGA
implementation just point to the image data stored in RAM (assuming image
frame captured to some local RAM buffer) and never move the data from
image_buffer -> to_endpoint -> to_USB (essentially skipping the middle
man by just using a pointer of sorts)?

It's just a still picture (not a movie).  Doesn't this seem like a somewhat
common function -- meaning wouldn't there be standard chips or
implementations that do this?

Thanks again,
GB

GB wrote:

> It's just a still picture (not a movie).  

Aha, that's an important fact as it greatly reduces the requirements.
Depending on what "as fast as possible" (in terms of latency) means, 
a DSP or a better RISC microprocessor might be a better solution 
for you. At least you won't have to introduce yourself to the new 
(for you) area of FPGAs.

I don't know how the interface of the CCD chip you are planning to 
use is working, but I can imagine that it uses some standard interface
that can be found on higher-class processors.

So I suggest you to check some DSPs first (Analog Devices, Texas
Instruments, Motorola, ...). Although you might not need the computing
power they provide, perhaps you can use their hardware interfaces.
DSPs are often used for image processing and therefore have a close
relationship to CCDs.

Perhaps you could also ask for this issue in comp.dsp.

Regards,
Mario   

-- 
----------------------------------------------------------------------
Digital Force / Mario Trams      Mario.Trams@informatik.tu-chemnitz.de
                                      Mario.Trams@wooden-technology.de
Chemnitz University of Technology       http://www.tu-chemnitz.de/~mtr
Dept. of Computer Science                     Tel.: (+49) 371 531 1660
Chair of Computer Architecture                Fax.: (+49) 371 531 1818
----------------------------------------------------------------------

GB wrote:
> 
> "rickman" <spamgoeshere4@yahoo.com> wrote in message
> news:3F5B4FAC.41D84F7B@yahoo.com...
> 
> > I don't think the micro should be in the data path.  A micro can be used
> > for control and management, but the data path should be pure hardware
> > for maximum throughput.  If you look around, you will find that there
> > are USB chips with integrated micros (many are 8051s).  But the USB 2.0
> > chips typically have a DMA engine for the data path.
> 
> From what I understand, the image size is 1-2 MB (only need 8-bit depth)
> large and the USB packets are quite small in comparison.  So there needs to
> be some smarts for setting up what part of the data gets DMA'd to the
> USB endpoint buffers for any given transfer?  Or does the FPGA
> implementation just point to the image data stored in RAM (assuming image
> frame captured to some local RAM buffer) and never move the data from
> image_buffer -> to_endpoint -> to_USB (essentially skipping the middle
> man by just using a pointer of sorts)?
> 
> It's just a still picture (not a movie).  Doesn't this seem like a somewhat
> common function -- meaning wouldn't there be standard chips or
> implementations that do this?

 Yes, Data-> USB 2.0 is common, and a good example of a std chip
is the Cypress chip someone has already mentioned.

 You may need a CPLD or small FPGA to run the CMOS sensor
clocks, and help the transfer, but the idea is to do as much in
the standard  uC.USB2.0 package as you can.

 I would also nail down just what "as fast as possible" really
means. Tell your boss there is a speed/price/development time
tradeoff, and find what speed the product really NEEDS.

 Also find out, early on, if any form of compression is 
envisioned, or if a simple 'frame dump' is fine.

-jg

On Sun, 07 Sep 2003 15:03:39 GMT, "GB" <donotspam_grantbt@jps.net>
wrote:

>Hi,
>
>I'm a firmware guy pulled into a project well out of my area of
>expertise.  My boss wants to build (essentially) a digital camera
>using an image sensor chip (1600x1200) and output it's data
>"as fast as possible" using USB2.0.  His initial concept, being
>that I'm a firmware guy, was to use a "really fast micro."
>Normally the company is involved with small 8-bitter micro
>projects, so you can see I'm well out of my normal bounds.
>
>Now this seems like a pretty big stretch to me... and I don't see
>how it can be done without the speed of hardware (the image
>chips all seem to have clock speeds in the tens of MHz and the
>USB2 transfer rate is 480Mbps (theor.).  Do aspects of this
>project require an FPGA to keep the data "as fast as possible?"
>If we use and FPGA for camera-to-RAM and then use a
> micro for the USB2 part, what kind of fast micros can
>move data at that rate?
>
>Also, this is something that I am sure we will have to contract
>out, so if you have any past experience with this, please let
>me know your thoughts (and if you are available).
>
>Thanks!
>
>

Hello :

The first question that comes to mind is why are you designing this ?
( other than your boss told you to )

One answer may be high volume production where an absolutely cost
optimized solution is the correct answer.

Another answer may be special requirements for your application. If
this is so then you have not provided enough information above to tell
us what these special requirements are to suggest an architecture.

I ask this question simply because there are numerous small oem camera
modules on the market with hi res cmos sensors and USB2 output that
can get you to market rapidly and become cost effective as volumes
increase.   An example may be found at
http://www.lumenera.com/oem.htm  

If we proceed along the path that you need to design such a module :

Several companies are coming out with single chip imaging controllers
( national ,etc ) but I am not aware of an available part to do
exactly what you are looking for ... but I would do a very careful
search as this is the most streamlined approach.

An FPGA may be a good solution to interconnect the sensor module ,
capture memory and the USB controller.  The controller ( Cypress )
will normally setup the USB endpoints etc. and then get out of the
way. The FPGA can handle all of the high speed timing and perhaps a
SDRAM controller. 

Another most intriguing solution (hey this is the comp.arch.fpga
group) , however the most engineering intensive ($) ,  would be to use
an FPGA with softcore processor and perhaps a USB core ... so the
entire solution except sensor, memory and USB physical drive chip
could be in the FPGA. The softcore processor could be programmed in C
and code changes simply bootloaded in to a flash memory by the
softcore processor without an FPGA design change ( nice when you have
contracted the FPGA design out of house yet the in house programmers
can write C code which runs on the processor in the FPGA ). ( I am
currently involved with a video design utilizing  Altera Nios softcore
processor / Cyclone FPGA designs )

Good Luck, check out the link below, we have significant video and
imaging experience and may be able to assist your design efforts.

Khim Bittle
khimbittle@cliftonREMOVEsystems.com   
(remove REMOVE from email address to respond)

Clifton Systems Inc.
http://www.cliftonsystems.com/design

"GB" <donotspam_grantbt@jps.net> wrote in message news:<GFJ6b.2610$PE6.1782@newsread3.news.pas.earthlink.net>...
> "hamilton" <hamilton@deminsional.com> wrote in message
> news:3f5b561c_3@omega.dimensional.com...
> 
> > What image sensor chip are you looking at ???
> >
> 
> That's another undecided at this time, but CMOS most likely.
> 
> GB

Until recently I was under the impression that CMOS sensors were junk
and were of low resolution & quality typically 320.240 used in $0-50
cameras. I have an old Connectix webcam device thats is all green that
demonstrates that.

Recently though a Micron engineer persuaded me that CMOS is looking
very much up and showed me some upcoming sensors for both high
resolution & higher quality still not quite as good as CCD but far
cheaper to make & potentially far easy to integrate with externl
interfaces. If the quality is there, I would expect USB2/FW to end up
on chip.

I will wait and see if they show up in the mid range cameras coming
soon.

But check out their imaging division for specs if you care.