[SASAG] NFS Timeouts any Tips?

Ross Wong rosswon at yahoo.com
Thu Jun 5 14:45:20 PDT 2008

have you try jumbo frame, if you runing database or with large block size files ?

----- Original Message ----
From: Skylar Thompson <skylar at cs.earlham.edu>
To: Nick Webb <webbn at acm.org>
Cc: members at lists.sasag.org
Sent: Thursday, June 5, 2008 2:21:56 PM
Subject: Re: [SASAG] NFS Timeouts any Tips?

A couple questions:

1. Do you have a bad NIC? Check netstat or ifconfig on the server and 
client to see if there's an errors.

2. Are you using TCP or UDP for your NFS transport? If you don't specify 
it explicitly, it'll default to whatever is specified nfs(5). This 
should be TCP, but you can make sure by looking in /proc/mounts. TCP is 
likely what you want to be using due to the flow-control.

3. If you're using TCP, are you running out of buffer space? Do a 
"netstat -a" on the client and server, and look in the Recv-Q and Send-Q 
columns for the NFS connection. If you're running close to the values 
set in the sysctl's net.core.rmem_max net.core.wmem_max, increase those 
values. I've seen recommendations for GigE of between 256kB-512kB, which 
is considerably higher than the 100kB default.

4. Is your server running out of IOps? Mail servers are likely going to 
be doing lots of little IOs, particularly if you're using Maildir. This 
puts a lot of strain on disks, particularly IDE and SATA disks. Do an 
"iostat -x 5" and look in the util column to see if your disks are pegged.

5. If you use maildir on ext3, do you have directory indexing enabled? 
Do a "tune2fs -l /dev/<some-device>" on the device with your mail spool 
filesystem, and look in the "Filesystem features" column for the 
dir_index value. You can kind of tell if you need it if your NFS server 
is spending a lot of time in system/kernel mode. You can enable this 
with "tune2fs -O dir_index /dev/<some-device>". You then have to unmount 
the filesystem and fsck it with "e2fsck -D /dev/<some-device>". I've 
found this improves small-file performance with ext3 by at least an 
order of magnitude.

-- Skylar Thompson (skylar at cs.earlham.edu)
-- http://www.cs.earlham.edu/~skylar/


More information about the Members mailing list