[SASAG] NFS Timeouts any Tips?
rosswon at yahoo.com
Thu Jun 5 14:45:20 PDT 2008
have you try jumbo frame, if you runing database or with large block size files ?
----- Original Message ----
From: Skylar Thompson <skylar at cs.earlham.edu>
To: Nick Webb <webbn at acm.org>
Cc: members at lists.sasag.org
Sent: Thursday, June 5, 2008 2:21:56 PM
Subject: Re: [SASAG] NFS Timeouts any Tips?
A couple questions:
1. Do you have a bad NIC? Check netstat or ifconfig on the server and
client to see if there's an errors.
2. Are you using TCP or UDP for your NFS transport? If you don't specify
it explicitly, it'll default to whatever is specified nfs(5). This
should be TCP, but you can make sure by looking in /proc/mounts. TCP is
likely what you want to be using due to the flow-control.
3. If you're using TCP, are you running out of buffer space? Do a
"netstat -a" on the client and server, and look in the Recv-Q and Send-Q
columns for the NFS connection. If you're running close to the values
set in the sysctl's net.core.rmem_max net.core.wmem_max, increase those
values. I've seen recommendations for GigE of between 256kB-512kB, which
is considerably higher than the 100kB default.
4. Is your server running out of IOps? Mail servers are likely going to
be doing lots of little IOs, particularly if you're using Maildir. This
puts a lot of strain on disks, particularly IDE and SATA disks. Do an
"iostat -x 5" and look in the util column to see if your disks are pegged.
5. If you use maildir on ext3, do you have directory indexing enabled?
Do a "tune2fs -l /dev/<some-device>" on the device with your mail spool
filesystem, and look in the "Filesystem features" column for the
dir_index value. You can kind of tell if you need it if your NFS server
is spending a lot of time in system/kernel mode. You can enable this
with "tune2fs -O dir_index /dev/<some-device>". You then have to unmount
the filesystem and fsck it with "e2fsck -D /dev/<some-device>". I've
found this improves small-file performance with ext3 by at least an
order of magnitude.
-- Skylar Thompson (skylar at cs.earlham.edu)
More information about the Members