GroupStudy.com GroupStudy.com - A virtual community of network engineers
 Home  BookStore  StudyNotes  Links  Archives  StudyRooms  HelpWanted  Discounts  Login
RE: SMTP Time outs and re-transmissions, multiple [7:64617] posted 03/07/2003
[Chronological Index] [Thread Index] [Top] [Date Prev][Date Next] [Thread Prev][Thread Next]


Symon Thurlow wrote:
> 
> HI Priscilla,
>  
> Thanks for your response. This problem is driving me nuts.
>  
> Looking at sh int, sh frame pvc etc I can't see any significant
> errors or problems.
>  
> We have been using auth for some time. No significant changes,
> other than a power down on Friday, problem notice don Monday. I
> have gone over the router configs and nothing looks wrong. No
> routes flapping, all times on routes are consistent. There are
> no firewalls or access lists in this equation.
>  
> From the client side, this is the packet flow:
>  
> 3 way handshake ok
> Final ack from the client (WINXP) declares a 62420 byte window
> size.
> SMTP starts, ehlo, auth, mail from, etc all ok. 
> After "send data..."  from the server, client sends an ack,
> with Windows size 63795 (I guess the client is thinking at this
> point that it has a good fast connection, and the server is
> emptying it's buffers quickly?).
>  
> I then see 5 packets of SMTP data at 
>  
> ARRRGHHH!!!! It's 8.15pm, i'm still at the clients site and my
> laptop just blue screened (it has the captures...). yay.
>  
> Right, I then see 5 packets of SMTP data (from client to
> server) at 1514 bytes a piece. I then see  one packet with smtp
> data from client to server at 946 bytes. not sure why.>  
> Then I see 3 acks from the server to the client, ack'ing the
> 3rd, 5th, and get this the NEXT SMTP data packet. 

TCP uses a forward acknowledgement, so that may be normal. Please remember
also that TCP ACKs bytes, not packets. The sequence number identifies the
byte number in the stream of bytes. It identifies the byte number of the
first byte in the segment being sent. The ACK number indentifies the next
expected byte.

> It looks like
> the packet and ack were recieved at the same time. THe acks all
> have a window size of 8760. This tells me that either the
> recieve buffers are full or set too small on the server?

Usually servers have larger receive windows than that these days, but not
necessarily. I've seen a lot that have a similar size, and it shouldn't
really matter with SMTP I don't think.

To tune TCP's window size you would have to tune the TCP stack in the
registry probably. You mentioned tuning the buffers on the NIC. I don't
think that helps TCP. I don't think TCP knows about the NIC's buffers.

That's not to say that tuning the NIC buffers won't help the problem. It
might cause the NIC to drop fewer frames, which would certainly help.

>  
> The above 5 packets and 3 acks happens another 4 times (the
> email is about 400 Kbytes), then the re-transmissions start.
>  
> I get 6 retransmissions of the 4th to last SMTP data packet
> from the client to the server, then the whole sequence starts
> all over again from 3 way handshake.

Sounds like the server stopped ACKing or left a hole in what it has ACKed?
You'll have to study the ACK numbers carefully. It sure sounds like the
server has some problems and is losing packets somewhere, either hardware or
software. Maybe you need to replace its NIC? You said you tuned the buffers
which might help too.


? No rst or fin or anything.
>  
> On the mail server I can see it logging that it is waiting for
> the message body, and times out waiting.
>  
> Perhaps the buffer fills up on the server and it can;t empty it
> anymore? In this case I would expect to see some sort of
> notification from the server to stop sending (a reduced window
> size perhpas?)

Yes, that's what you should expect if TCP gets behind. More evidence that
the problem is lower down, with the NIC perhaps??

>  
> I'm going to train it home (with my nose in TCP/IP illustrated)
> then study the captures some more. 

Sounds like a good idea. TCP/IP Illustrated is great. 

> If anyone is interested, I
> can send you the captures, they are from Sniffer Pro.

If you get stuck, I'll take a look. I predict you'll figure it out yourself,
though. Good luck.

Priscilla


>  
> Cheers,
>  
> Symon
>  
>  
>  
> ________________________________
> 
> From:	 Priscilla Oppenheimer [mailto:nobody@xxxxxxxxxxxxxx]	
> Sent:	 Thu 06/03/2003 6:32 PM	
> To:	 cisco@xxxxxxxxxxxxxx	
> Subject:	 RE: SMTP Time outs and re-transmissions, multiple TCP
> [7:64617]
>  	
> 
> Symon Thurlow wrote:
> >
> > Hi all,
> >
> > I am trying to problem solve and issue where I have multiple
> > frame sites
> > (1750's) all connecting to a site (3640) to send and receive
> > email (POP3
> > SMTP).
> >
> > They are having problems sending SMTP. Looking at a packet
> > trace from a
> > site trying to send SMTP, I see lots of re-transmissions.
> >
> > I have checked with the Frame provider and they assure me that
> > no
> > packets are being dropped, even though there are a lot getting
> > marked
> > DE.
> >
> > I have pretty much excluded the server from being an issue.
> >
> > IN the packet sniff I also see multiple acks with the same
> > sequence
> > number, one after the other. This does not look right to me. I
> > am about
> > to delve in to TCP/IP illustrated to find out about it, but
> > does this
> > behaviour trigger anything in your minds?
> 
> Multiple ACKs with the same sequence number are probably just
> TCP
> keepalives. The original TCP RFC didn't mention a keepalive
> process but the
> Host Requirements RFC does. That's how it works. After some
> time of silence
> a host just sends the last ACK over and over again for a couple
> hours
> (typical timeout).
> 
> Is the client side sending these multiple ACKs?
> 
> If the server were sending them, and you're seeing them on the
> client side,
> then you would have some proof that the server can at least get
> through, but
> I bet it's the client side sending them?
> 
> If you're sniffing on the client side and you see it send
> retransmissions
> and multiple ACKs, but you don't see much from the server, I
> would suspect
> that the server's packets are getting lost. But what would
> cause SMTP to get
> lost but not POP3? A firewall misconfiguration? An MTU issue?
> 
> What is the last packet exchange you see for SMTP?
> 
> What error do the users see?
> 
> For SMTP, did you recently start using authentication or any
> other new
> features? We recently started requiring our users to use
> authentication and
> had lots of strange problems due to the mix of operating
> systems. Some of
> them didn't behave correctly at all, although their misbehavior
> wasn't
> exactly what you're describing.
> 
> Can the users still receive mail? I think you implied that they
> can. POP3 is
> based on TCP too. Does it still work? If it still works, but
> SMTP doesn't,
> that's a huge clue that TCP and the internetwork are fine.
> 
> If POP has problems too, start bugging the Frame provider more
> and do some
> checking yourself with "show int serial?" Are there lots of
> errors?
> 
> Actually, before you blame the Frame provider, though, if
> possible do some
> sniffing on the server side and make sure the server is really
> still sending
> packets and they are making it through local routers and
> firewalls. Make
> sure it is sending packets with data, not just ACKs. It might
> be hard to
> sniff on the outside side of the firewall, but if you can that
> would a good
> spot.
> 
> Speaking of firewalls, are there any in the picture that could
> be messing
> with TCP and causing problems or timing out? It scares me how
> much you tell
> a firewall to mess with TCP.
> 
> What else has changed recently, if anything?
> 
> Feel free to send us more info. Thanks,
> 
> Priscilla
> 
> 
> >
> > Cheers,
> >
> > Symon
> =============================================
> 
>  This email has been content filtered and
>  subject to spam filtering. If you consider
>  this email is unsolicited please forward
>  the email to postmaster@xxxxxxxxxxx and
>  request that the sender's domain be
>  blocked from sending any further emails.
> 
> =============================================
> 
> 




Message Posted at:
http://www.groupstudy.com/form/read.php?f=7&i=64751&t=64617
--------------------------------------------------
FAQ, list archives, and subscription info: http://www.groupstudy.com/list/cisco.html
Report misconduct and Nondisclosure violations to abuse@xxxxxxxxxxxxxx