BXadmin:AFS Notes

sysnames

• OS X 10.6 x86 = x86_darwin_100
• OS X 10.5 ppc = ppc_darwin_90

building openafs from SRPM

On 32-bit, some extra magic is necessary:

$ unset ARCH
$ rpmbuild -bb --define 'build_modules 1' --define 'build_userspace 1' --target i686 openafs.spec

To build just the kmod-openafs for a specific kernel on 32-bit:

$ unset ARCH
$ rpmbuild -bb --define 'build_userspace 0' --define 'build_modules 1' --define 'kernvers 2.6.18-238.9.1.el5' --target i686 openafs.spec

xstat

xstat_fs_test and xstat_cm_test aren't included in the RPMs by default. Easiest way to get them is pull down the source, ./configure && make, and grab them out of src/xstat/

Run xstat_fs_test on a fileserver:

$ ./xstat_fs_test  -fsname fs4 -collID 3 -onceonly

Starting up the xstat_fs service, no debugging, one-shot operation

------------------------------------------------------------
           1143193 DeleteFiles
          34118245 DeleteCallBacks
           6000838 BreakCallBacks
         101283213 AddCallBack
               746 GotSomeSpaces
              1462 DeleteAllCallBacks
              3164 nFEs
              3290 nCBs
            100000 nblks
            261656 CBsTimedOut
                 0 nbreakers
              1122 GSS1
               561 GSS2
               185 GSS3
                 0 GSS4
               561 GSS5

The GSS values show you how many times GetSomeSpaces was called; basically the number of times the fileserver had to free up existing callbacks to make room for more. To alleviate this condition (as it can hurt performance if GetSomeSpaces is called a lot), increase -cb. The default for this is 64,000. The fileserver above is a home directory fileserver running with -cb 100000, which clearly is not enough.

fs/dafs tuning

In general, these are the settings we're using. The only variable we tweak depending on the underlying hardware is -p. For most machines, -p 128. For smaller machines, -p 64 to take some of the load off the machine (I've seen load avg in the double digits on 4-core 2.8GHz Xeon with -p 128).

fs

fileserver -p 128 -busyat 600 -rxpck 4096 -s 10000 -l 1200 -cb 1000000 -b 240 -vc 1200 -abortthreshold 0 -syslog -d 1 -udpsize 1048576
volserver -p 16 -udpsize 1048576 -d 1
salvager -parallel all8

dafs

dafileserver -p 128 -busyat 600 -rxpck 4096 -s 10000 -l 1200 -cb 1000000 -b 240 -vc 1200 -vattachpar 128 -vlruthresh 1440 -vlrumax 8 -abortthreshold 0 -syslog -d 1 -udpsize 1048576
davolserver -p 64 -udpsize 1048576 -d 1
salvageserver
dasalvager -parallel -all8

• -nojumbo should default to on in at least 1.4.12
• -abortthreshold 0 can be dangerous if we have misbehaving clients, but more often than not it far too easily DOS's a client when someone on that client hits their quota, or repeatedly runs into ACL issues (like trying to copy lots of files to someplace they don't have permission).