Slightly unmatched UART frequencies

Joel Kolstad wrote:
> 
> Hal Murray <hmurray@suespammers.org> wrote:
> >> You bright up a good subject, and you're absolutely correct that if you
> >> continuously send data from one serial port at 9600.01bps to a receiver
> >> at 9600, sooner or later there must be a buffer overflow. ...
> >
> > I think you are missing a key idea.  The receiver has to make
> > sure that it will tolerate early start bits.  That is the receiver
> > has to start looking for a new start-of-start-bit right after
> > it has checked the middle of the stop bit rather than wait unitl
> > the end of the stop bit to start looking.
> 
> Unless your (slightly slower) transmitter also has the capability of
> producing shortened start (or stop) bits, how those this approach 'fix' the
> problem?  If the date rates are, say, 9601 received BPS and 9600 transmitted
> BPS, detecting early start bits just buys you one extra bit interval before
> your overrun your buffers, doesn't it?

I think I just understood what is wrong with my thinking.  I was only
looking at one direction.  The problem the OP has is that he is looping
back the received data and retransmitting it at the same slow(er) rate
the receiver is using.  Since the transmitter must use a full 10 bits,
then the buffer between the receiver and transmitter will overflow at
some point.  

This is a problem that is found in communications systems.  All units
must run at the same rate, but may use a different reference.  So if
they receive data at 8.0000 kHz and try to DAC it at 7.9999 kHz, about
once per ten seconds there will be a byte too many and a sample will
have to be dropped.  This will cause a noticable click or other
distortion of the voice.  

Some systems try to buffer this out, but that only postpones the
problem.  Most systems use a common reference that is very accurate and
stable.  Then the ADC and DAC clocks must be sync'd to the reference.  I
worked on a system that used a bendable VCXO to establish the 8 kHz
rate.  As the buffer data count varied from half full, the VCXO rate was
increased or decreased by small amounts to keep the average rates
matched.  

In the case of the OP, the first channel can use two stop bits and the
echo channel can use one stop bit.  This may not be pretty, but it will
work.  

-- 

Rick "rickman" Collins

rick.collins@XYarius.com
Ignore the reply address. To email me use the above address with the XY
removed.

Arius - A Signal Processing Solutions Company
Specializing in DSP and FPGA design      URL http://www.arius.com
4 King Ave                               301-682-7772 Voice
Frederick, MD 21701-3110                 301-682-7666 FAX

Hi Rick et al.,

rickman <spamgoeshere4@yahoo.com> wrote:
> Since the transmitter must use a full 10 bits,
> then the buffer between the receiver and transmitter will overflow at
> some point.

I'm glad someone else agrees with me!  I think we'd all agree now, then,
that either you need an adjustable output clock or the part to transmit with
a shortened stop (or start) bit, which is effectively the same thing as
adjusting your output clock rate anyway.

> This is a problem that is found in communications systems.  All units
> must run at the same rate, but may use a different reference.

> Some systems try to buffer this out, but that only postpones the
> problem.  Most systems use a common reference that is very accurate and
> stable.

...or they force the data to be encoded with its clock so that a PLL can
extract the exact frequency reference and work with that.

> In the case of the OP, the first channel can use two stop bits and the
> echo channel can use one stop bit.  This may not be pretty, but it will
> work.

It certainly will but the price is a reduction of ~10% of the average
throughput!  Of course, that may be negligible depending on the system.

---Joel

On Tue, 25 Nov 2003 07:20:58 -0800, "juergen Sauermann" <juergen.sauermann@t-online.de> wrote:
>Philip, 
>after thinking about the problem once more, I hate to admit that, yes, you are right. 

Sorry.

>I still do not believe, though, that inserting idle time one way or the other
>(including cutting the transmitter's stop bit) is a solution. Consider the following: 
>
>Left side: Slow (9600 Baud) 
>Right side: Fast (9700 Baud) 
>
>Both sides use e.g. 8N2 for Tx and 8N1 for Rx. 

The extra stop bit gives you about 9% margin, the difference between
9600 and 9700 is about 1%

>At some point in time, Left see's its buffer filling up and hence skips
>a few stop bits here and there (using 8N1) in order to compensate this.
>Left is now faster that Right, despite of the clock rates.

I agree. More exactly, Left's RX is faster than Right's TX.

The examples I gave in my prior post help with unidirectional
messages between a faster transmitter and a slower receiver, and
assume the system at the slower receiver can process the received
character in the local clock domain in 10 bit times. But if you
retransmit the character with 2 stop bits in the slower clock
domain, that takes 11 bit times, and the system will fail. So
retransmitting with 11 bits throws away the advantage of the
RX using 1 stop bit and the far end TX using 2 stop bits.

If you knew which system had the slower clock, you could set its
transmitter for 1 stop bit and then the system would work.
Unfortunately this is not normally possible.


>As a consequence, Right sees its buffer filling up and skips
>stop bits (using 8N1) as well.
>This continues until both sides transmit with 8N1 all the time;
>at this time Left will loose data. 

This is not what I intended. I am assuming that the number of
stop bits is fixed, and is dependent on which end has the faster
clock.

Try this:

Left side: Slow (9600 Baud)
           RX: 1 + 8 + 1
           TX: 1 + 8 + 1

Right side: Fast (9700 Baud) 
           RX: 1 + 8 + 1
           TX: 1 + 8 + 2

I believe you can send stuff continuously all day this way without
over-run, in both directions. Only problem is that you need to know
which end has the faster clock.

You could figur this out by running both TX at 1+8+1 and see which
RX has over-run errors first, then adjust the other end's TX. Not
pretty, but could be made to work. Maybe OK for a 1 off project, but
not for production.


>Thus, there must be some kind of understanding between Left and Right,
>which of the two is the "clock master", that ultimately controls the
>transmission speed. Unfortunately this is sometimes not possible, for
>instance in symmetric configurations.

I agree. hence the need for flow control.

>/// Juergen

Philip



Philip Freidin
Fliptronics

Hi all,



I've posted some code to do a UART last week on the fpga4fun site...

http://www.fpga4fun.com/SerialInterface.html



For the receiver, the bits are sampled in the middle - with the sampling
point configurable.

Also, the receiver starts checking for a "start" as soon as the "stop"
mid-point bit is detected, so that should be consistent with the
recommendations posted earlier.

Jean



"valentin tihomirov" <valentinNOMORESPAM@abelectron.com> wrote in message
news:bpum8f$1srj32$1@ID-212430.news.uni-berlin.de...
> UART is used to transfer a byte in serial form bit-by-bit. I know that 10%
> deriviations in frequencies of transmitter and receiver are permissible. I
> was learnt that UARTs synchronyze at the falling edge (1to0) of start bit;
> hence, there should allow for transfer of a stream of bytes of arbitrary
> length.
>
> I have developed a simple UART. It's receiver and transimtter run at 9600
> bps with 16x oversampling. Both receiver and transmitter have 1-byte
buffer.
> To test the design I've created an echo device; it merely mirrors all the
> bytes sent to it back to the sender. It works fine with one of COM ports
on
> my PC. Another COM port has its crystal running at a bit faster
fundamental
> frequency. This causes a problem when it sends a long stream of bytes to
my
> UART. In fact, sender and recepient cannot synchronize at falling edge of
> start bit because one of them is slower and is processing a previous byte
> wrile sender proceeds to next byte transmitting start bit. Despite of the
> fact, my receiver still works fine beacuse it is ready to receive next
byte
> right after a first half of stop bit is received. Just to clarify,
receiver
> acquares values from serial input at the middle of each data bit slice; it
> reports BYTE_READY in the middle of stop bit and from this moment is ready
> to accept next byte, i.e. ready fror synchronization. Therefore, if data
is
> coming slightly faster and falling edge of start bit is located within
stop
> bit (according to my UART's clock) receiver is still capable not to
overlook
> the data.
> On the other hand, transmitter should transmit all 10 bits (start + 8 data
+
> stop) @ 9600 bps. Consider for instance an UART forwarder or an echo
device.
> If data is coming faster than I forward it I get a buffer overrun
> ultimately. That is, receiver is ready with a byte in its buffer to be
> copied into transmitter to forward but slow transmitter is still shifting
> data out and its buffer is blocked.
> I have a "fast" solution for my UART echo device; if transmitter has
> transmitted > half of stop bit and sences that there is a next byte
received
> it stops sending current stop bit and starts transmitting a start bit for
> next byte. Untimely ceasing transmission is not good solution because
> transmitter may be connected to a good matched or slightly slower running
> UART. Design may be not a forwarder thus data provider may differ from
9600
> bps receiver. In this case, starting early transmission of next byte while
> remote peer is still receiving stop bit causes stop bit error.
>
> What is interesting in this situation is the fact I can build a good echo
> device from any industrial manufactured UART (I've used standalone 16c750
> and ones built into i8051). They never have a buffer overrun issue despite
> sending port is slightly faster than receiving (like sending data from my
> fast COM port to slow one). Note, no flow control is used, buffers are
> always 1-byte long. Which trick do they use? Again, 10% frequency
> dereviations between sender and receiver are considered permittable and no
> flow control is not required since sender and receiver both run at formal
> 9600bps.
>
> I feel this should be a well-known problem (solution) and just wonder why
I
> did not encounter this consideration before.
>
> Thanks.
>
>
>

Jean Nicolle wrote:

> I've posted some code to do a UART last week on the fpga4fun site...

> http://www.fpga4fun.com/SerialInterface.html

> For the receiver, the bits are sampled in the middle - with the sampling
> point configurable.

> Also, the receiver starts checking for a "start" as soon as the "stop"
> mid-point bit is detected, so that should be consistent with the
> recommendations posted earlier.

I finally found from another post that the OP was trying to build a loop 
back device that would send back everything it received.  Since it is 
possible to receive a character in (about) 9.5 bit times, but it takes 
10 to send it, (8N1), an unbuffered device could quickly lose data.

It seems that this is not one of the cases the protocol was designed to 
handle.    As far as I know, the popular full duplex with remote echo 
would not have been considered when the protocols were defined.

Also, as far as I know both the popular XON/XOFF and RTS/CTS flow 
control are relatively recent inventions.  RTS/CTS is from the real half 
duplex days, where only one side could transmit at a time.

Part of the asynchronous protocol is that both ends have their own clock.

For T1 and related protocols, one end synchronizes its clock to the 
other, so that there is no speed matching problem.

For ethernet, the receiver synchronizes to the transmitter from clock 
information in the data stream.

It should be possible to build a device with a UART and PLL to 
synchronize the transmit clock to the incoming data rate.  There should 
probably be some buffer to last until the PLL locks.

-- glen

In article <bpum8f$1srj32$1@ID-212430.news.uni-berlin.de>, 
valentinNOMORESPAM@abelectron.com mentioned...
> UART is used to transfer a byte in serial form bit-by-bit. I know that 10%
> deriviations in frequencies of transmitter and receiver are permissible. I
> was learnt that UARTs synchronyze at the falling edge (1to0) of start bit;
> hence, there should allow for transfer of a stream of bytes of arbitrary
> length.
> 
> I have developed a simple UART. It's receiver and transimtter run at 9600
> bps with 16x oversampling. Both receiver and transmitter have 1-byte buffer.
> To test the design I've created an echo device; it merely mirrors all the
> bytes sent to it back to the sender. It works fine with one of COM ports on
> my PC. Another COM port has its crystal running at a bit faster fundamental
> frequency. This causes a problem when it sends a long stream of bytes to my
> UART. In fact, sender and recepient cannot synchronize at falling edge of
> start bit because one of them is slower and is processing a previous byte
> wrile sender proceeds to next byte transmitting start bit. Despite of the
> fact, my receiver still works fine beacuse it is ready to receive next byte
> right after a first half of stop bit is received. Just to clarify, receiver
> acquares values from serial input at the middle of each data bit slice; it
> reports BYTE_READY in the middle of stop bit and from this moment is ready
> to accept next byte, i.e. ready fror synchronization. Therefore, if data is
> coming slightly faster and falling edge of start bit is located within stop
> bit (according to my UART's clock) receiver is still capable not to overlook
> the data.
> On the other hand, transmitter should transmit all 10 bits (start + 8 data +
> stop) @ 9600 bps. Consider for instance an UART forwarder or an echo device.
> If data is coming faster than I forward it I get a buffer overrun
> ultimately. That is, receiver is ready with a byte in its buffer to be
> copied into transmitter to forward but slow transmitter is still shifting
> data out and its buffer is blocked.
> I have a "fast" solution for my UART echo device; if transmitter has
> transmitted > half of stop bit and sences that there is a next byte received
> it stops sending current stop bit and starts transmitting a start bit for
> next byte. Untimely ceasing transmission is not good solution because
> transmitter may be connected to a good matched or slightly slower running
> UART. Design may be not a forwarder thus data provider may differ from 9600
> bps receiver. In this case, starting early transmission of next byte while
> remote peer is still receiving stop bit causes stop bit error.
> 
> What is interesting in this situation is the fact I can build a good echo
> device from any industrial manufactured UART (I've used standalone 16c750
> and ones built into i8051). They never have a buffer overrun issue despite
> sending port is slightly faster than receiving (like sending data from my
> fast COM port to slow one). Note, no flow control is used, buffers are
> always 1-byte long. Which trick do they use? Again, 10% frequency
> dereviations between sender and receiver are considered permittable and no
> flow control is not required since sender and receiver both run at formal
> 9600bps.
> 
> I feel this should be a well-known problem (solution) and just wonder why I
> did not encounter this consideration before.
> 
> Thanks.

UARTs are almost always crystal controlled, so the tolerance is less 
than 1 percent, and there shouldn't be a problem with this.  When a 
serial data stream is run through a multiplexer, where the sampling 
rate is less than 10x per bit, then you might see such large 
deviations ("dereviations").


-- 
@@F@r@o@m@@O@r@a@n@g@e@@C@o@u@n@t@y@,@@C@a@l@,@@w@h@e@r@e@@
###Got a Question about ELECTRONICS?   Check HERE First:###
http://users.pandora.be/educypedia/electronics/databank.htm
My email address is whitelisted.  *All* email sent to it 
goes directly to the trash unless you add NOSPAM in the 
Subject: line with other stuff.  alondra101 <at> hotmail.com
Don't be ripped off by the big book dealers.  Go to the URL
that will give you a choice and save you money(up to half).
http://www.everybookstore.com  You'll be glad you did!
Just when you thought you had all this figured out, the gov't
changed it: http://physics.nist.gov/cuu/Units/binary.html
@@t@h@e@@a@f@f@l@u@e@n@t@@m@e@e@t@@t@h@e@@E@f@f@l@u@e@n@t@@

Error accumulates. 1% means that receiver accepts 9.9bit long farmes while
transmitter transmits 10.0 bit-long frames. After re-transmitting 100 bytes
(frames) we get 10-bit error. 10 bit is one byte, so 1-byte buffer is not
enaught. The longer the stream the lager the buffer required. Infinite
buffer is required for continous stream. As masters stated, UART was not
designed for this purpose.

"valentin tihomirov"
> Error accumulates. 1% means that receiver accepts 9.9bit long farmes while
> transmitter transmits 10.0 bit-long frames. After re-transmitting 100
bytes
> (frames) we get 10-bit error. 10 bit is one byte, so 1-byte buffer is not
> enaught. The longer the stream the lager the buffer required. Infinite
> buffer is required for continous stream. As masters stated, UART was not
> designed for this purpose.

 Some UARTs are designed for this purpose
- look at the data for Philips 26C94 families, and esp. the
area of Sending fractional stop bits.
-jg

you guys seem to miss the point.. Async RS232 ALWAYS has mismatched clocks
... ALL async UARTs MUST be capable of handling this.. that's why there's
flow control.

You will find that 99.9% of them start looking for the start bit 1/2 way
thru the stop bit.  If you want to build a repeater then you will need a
fractional stop bit generator OR you can over spec the crystal by 1.5%  (or
the baud rate generator) OR you add RTS / CTS flow control like everybody
else.

Simon

"valentin tihomirov" <valentinNOSPAM@abelectron.com> wrote in message
news:bqd493$1ter3o$1@ID-212430.news.uni-berlin.de...
> Error accumulates. 1% means that receiver accepts 9.9bit long farmes while
> transmitter transmits 10.0 bit-long frames. After re-transmitting 100
bytes
> (frames) we get 10-bit error. 10 bit is one byte, so 1-byte buffer is not
> enaught. The longer the stream the lager the buffer required. Infinite
> buffer is required for continous stream. As masters stated, UART was not
> designed for this purpose.
>
>
>

Simon Peacock <nowhere@to.be.found> wrote:
> you guys seem to miss the point.. Async RS232 ALWAYS has mismatched clocks
> ... ALL async UARTs MUST be capable of handling this.. that's why there's
> flow control.

There are a lot of systems in the world that don't use flow control and
simply rely on a 'mutual agreement' between the interconnected devices that
large streams of unbroken serial data isn't produced.  This is especially
the case of, e.g., sensors that produce real-time RS-232 output data with no
buffering of their own -- in such a case flow control would just throw data
away, so it's often not implemented to begin with.  (I have a motto
something to the effect of... if detecting an error does no more good than
just ignoring it, you might as well ignore it... Although unfortunately this
idea is often perverted into, 'if detecting an error takes effort, even if
ignoring it causes the machine to rip off someone's arm, you might as well
ignore it because otherwise the customer will immediately be able to pin the
blame on you rather than having to guess whose fault it is.' :-( )

> You will find that 99.9% of them start looking for the start bit 1/2 way
> thru the stop bit.  If you want to build a repeater then you will need a
> fractional stop bit generator OR you can over spec the crystal by 1.5%
> (or the baud rate generator) OR you add RTS / CTS flow control like
> everybody else.

1 -- The fractional stop bit generator is a good idea, and is apparently
implemented in -- some -- commercial devices.  This was news to me, but it
does seem to be a very good solution.
2 -- Overclocking by 1.5% strikes me as a poor idea because the 'solution'
is inherently non-scalable -- I can't connect together a dozen of these
repeaters unless each one is progressively 1.5% faster, which isn't viable.
3 -- I'd agree that flow control is the 'proper' or 'textbook' solution,
just that the real world doesn't always play by those rules. :-)

---Joel Kolstad