AFS: Andrew File System

History

Development of AFS started in 1983 at Carnegie Mellon University with the Andrew Project. Most parts of this project are replaced by newer implementations. Only the file system survives nowadays even though it had an eventful past:

in 1988 Transarc was founded by Carnegie Mellon employees, who commercially supported server and client code for several different operating systems.
To create an open version of the AFS client, the Arla project started in 1993 at the Royal Institute of Technology in Stockholm (Sweden).
In 1994 Transarc was sold to IBM and became the IBM Pittsburgh Lab in 1999.
The next year (in 2000), IBM open sourced the AFS code as OpenAFS under the IBM Public License.
Nowadays OpenAFS is actively developed, and some maintenance work is done on Arla.

The goal of the Andrew File System is to provide a scalable and global file system that allows storage and sharing of files independent of location and operating system. This has been achieved with a server side consisting of a namespace database, volume based fileservers and clients for almost every desktop operating system available.

AFS and Kerberos

Part of the AFS server suite is the Authentication Server which provided mutual authentication and was implemented using a set of algorithms developed during Massachusetts Institute of Technology's Project Athena; hence these were known as Kerberos. This original authentication server is still available in the OpenAFS code, but has been superseded by newer implementations: Kerberos5 from both MIT and Heimdal. The current AFS servers are compatible with Kerberos5 even though the kerberos ticket on the user side still has to be converted to an AFS token. In the example below this is done by a additional command, but some implementations of kinit can do this nativly.

#>kinit <username>@<REALM>
Please enter the password for <username>@<REALM>: ********
#>aklog <cell>
#>tokens
Tokens held by the Cache Manager:
Tokens for afs@<cell> [Expires Mar  2 07:40]
   --End of list--

For the conversion command, several implementations are available and may be called aklog (OpenAFS), afslog (Arla) or afs5log (something RedHat has come up with).

AFS design

One of the great things about AFS is it being a global file system: it has a namespace through which all available AFS cells (an administrative unit) can be reached. The root of the namespace tends to be the /afs directory. When doing a ls in this root directory, arla contacts the vlserver of its parent cell to query the available cells and displays them:

#>ls /afs
.e.kth.se               besserwisser.org        iastate.edu             ncsa.uiuc.edu           stacken.kth.se
.kth.se                 cern.ch                 ies.auc.dk              northstar.dartmouth.edu su.se
...

If you go a step further into the filesystem and query e.g. the cell stacken.kth.se, the AFS client will first contact the vlserver of this cell to point to the root volume of the cell. Next the client contacts the fileserver containing this volume to provide the actual data:

#>ls /afs/stacken.kth.se
admin           hp_ux102        i386_fbsd_51    i386_obsd29     pkg             sparc_nbsd13    sun4c_open_21   usr
alpha_dux40d    i386_fbsd22     i386_linux24    i386_obsd30     pkg.old         sparc_nbsd14    sun4m_413       var
...

Something similar happens when contacting the cell pitts.edu:

#>ls /afs/pitt.edu
backup  pittnet usr11   usr19   usr26   usr33   usr40   usr48   usr55   usr62   usr7    usr77   usr84   usr91   usr99
class   public  usr12   usr2    usr27   usr34   usr41   usr49   usr56   usr63   usr70   usr78   usr85   usr92   web
...

The example above shows two great features: AFS allows a user to browse through cells all over the world, as simple as moving through a local filesystem. Secondly it doesn't require any knowledge of servers or shares within a cell, the namespace will take care of providing a view, and pointing to the correct fileservers.

Have a look here to find more info on the server side.

AFS usage

anonymous and authenticated access

The above ls examples can be done by anyone, since the you are by definition part of the system:anyuser group. This group normally has list and read rights for the root of a AFS cel.

#> fs listacl /afs/meiland.nl/
Access list for /afs/meiland.nl/ is
Normal rights:
  system:administrators rlidwka
  system:anyuser rl

When descending into a directory to which you don't have rights, you will be stopped:

#> fs la /afs/meiland.nl/users/hugo/
fs: You don't have the required access rights on '/afs/meiland.nl/users/hugo/'

After becoming a authenticated user:

#> kinit hugo
hugo@MEILAND.NL's Password: 
kinit: NOTICE: ticket renewable lifetime is 1 week
#> aklog meiland.nl
#> tokens

Tokens held by the Cache Manager:

Tokens for afs@meiland.nl [Expires Mar  2 20:41]
   --End of list--

#> fs la /afs/meiland.nl/users/hugo
Access list for /afs/meiland.nl/users/hugo is
Normal rights:
  system:administrators rlidwka
  hugo rlidwka

#> echo "hello afs" > /afs/meiland.nl/users/hugo/hello.txt
#> cat /afs/meiland.nl/users/hugo/hello.txt 
hello afs

There are 7 levels of access to a directory or files in a directory:

r: read (files)
l: list (directory)
i: insert (directory)
d: delete (directory)
w: write (files)
k: lock (files)
a: administer (directory)

These acl's can only be set on directories, not at file level.

ls
volume properties
quota
directory acl

OpenAFS support and FreeBSD

There are two main OpenAFS major versions at present, the 1.4 series and the 1.6 series. The 1.4 series has largely been untouched since FreeBSD 5.x, and is not expected to work as a client on modern FreeBSD versions. As of mid-may, OpenAFS 1.6.0 is nearing release; currently 1.6.0pre5 is available and has received enough attention so as to be usable on current FreeBSD versions. The client is not quite usable on FreeBSD 7.x, but for 8.x and 9-CURRENT the client is useful under moderate load. (There are known bugs which are expected to trigger under high load.) Unfortunately, due the the chunk-based caching architecture of the client, the VFS locking is very hard to get right when using a disk-based cache, so only a memory cache is usable.

OpenAFS installation through ports

#>cd /usr/ports/net
#>fetch http://web.mit.edu/freebsd/openafs/openafs.shar
#>sh openafs.shar
#>cd openafs
#>make install

(tested on FreeBSD 8.1, 8.2 and a 9-CURRENT snapshot ca. May 2011)

OpenAFS has several configuration files; defaults are provided in many cases but are not reasonable for all configurations. For client functionality, afsd_enable="YES" must be set in rc.conf. The configuration file /usr/local/etc/openafs/ThisCell sets the "home cell" for the client machine, and /usr/local/etc/openafs/cacheinfo tells the client where to mount AFS, where the cache partition is (irrelevant for memory cache), and the size of the cache (in kB). In order for the client to run, the mountpoint /afs must exist, and the "cache partition" (or directory) which defaults to /usr/vice/cache must exist as well. The initscript will not start if any of these conditions are not met. Though the cache partition is irrelevant for a memory cache client, the cache manager still checks for its existence at startup. When a disk-based cache is used, though, it is EXTREMELY IMPORTANT to ensure that the filesystem on which the disk cache lives has at least as much space available as claimed in the cacheinfo file; if the cache partition fills, bad things will occur. For this reason, it is preferred to have the disk cache live on a dedicated slice or partition.

#>/usr/local/etc/rc.d/afsd start
#>ls /afs
1ts.org                         kloe.infn.it
acm-csuf.org                    kth.se
acm.uiuc.edu                    laroia.net
adrake.org                      lcp.nrl.navy.mil
ams.cern.ch                     le.infn.it
andrew.cmu.edu                  lees.mit.edu
anl.gov                         lnf.infn.it
asu.edu                         lngs.infn.it
athena.mit.edu                  lrz-muenchen.de
atlas.umich.edu                 mars.asu.edu
atlass01.physik.uni-bonn.de     math.cornell.edu
ba.infn.it                      math.unifi.it
bazquux.org                     mathi.uni-heidelberg.de
biocenter.helsinki.fi           mcc.ac.gb
bme.hu                          md.kth.se
caspur.it                       mech.kth.se
cats.ucsc.edu                   membrain.com
cede.psu.edu                    meteo.uni-koeln.de
cern.ch                         mpe.mpg.de
cgv.tugraz.at                   mrow.org
chem.cmu.edu                    mrph.org
ciemat.es                       msc.cornell.edu
citi.umich.edu                  mstacm.org
clarkson.edu                    msu.edu
club.cc.cmu.edu                 mw.andrew.cmu.edu
cmf.nrl.navy.mil                nada.kth.se
cms.hu-berlin.de                ncsa.uiuc.edu
cnf.cornell.edu                 nd.edu
coed.org                        nersc.gov
combi.tfh-wildau.de             net.mit.edu
crc.nd.edu                      nikhef.nl
cs.cmu.edu                      nimlabs.org
cs.hm.edu                       nomh.org
cs.pitt.edu                     northstar.dartmouth.edu
cs.rose-hulman.edu              numenor.mit.edu
cs.stanford.edu                 oc7.org
cs.uwm.edu                      p-ng.si
cs.wisc.edu                     pdc.kth.se
dapnia.saclay.cea.fr            pfriedma.org
dbic.dartmouth.edu              phy.bris.ac.uk
dementia.org                    physics.ucsb.edu
desy.de                         physics.unc.edu
dev.mit.edu                     physics.wisc.edu
dia.uniroma3.it                 physik.uni-freiburg.de
doe.atomki.hu                   physik.uni-mainz.de
dsrw.org                        physik.uni-wuppertal.de
e18.ph.tum.de                   physnet.uni-hamburg.de
ece.cmu.edu                     physto.se
eecs.berkeley.edu               pi.infn.it
eecs.harvard.edu                pitt.edu
enea.it                         psc.edu
eng.utah.edu                    psi.ch
engr.wisc.edu                   psm.it
epfl.ch                         qatar.cmu.edu
epitech.net                     rhic.bnl.gov
es.net                          riscpkg.org
ethz.ch                         rl.ac.uk
extundo.com                     roma3.infn.it
f9.ijs.si                       rose-hulman.edu
fnal.gov                        rpi.edu
freedaemon.com                  rrz.uni-koeln.de
fusione.it                      ruk.cuni.cz
glue.umd.edu                    rz.uni-jena.de
gppc.de                         s-et.aau.dk
grand.central.org               sanchin.se
hackish.org                     sbp.ri.cmu.edu
hep-ex.physics.metu.edu.tr      scoobydoo.psc.edu
hep.caltech.edu                 scotch.ece.cmu.edu
hep.man.ac.uk                   setfilepointer.com
hep.sc.edu                      sinenomine.net
hep.wisc.edu                    sipb.mit.edu
hephy.at                        slac.stanford.edu
i1.informatik.rwth-aachen.de    slackers.net
iastate.edu                     soap.mit.edu
ic-afs.arc.nasa.gov             sodre.cx
ic.ac.uk                        sph.umich.edu
icemb.it                        stacken.kth.se
ics.muni.cz                     su.se
ictp.it                         sums.math.mcgill.ca
idahofuturetruck.org            syd.kth.se
ies.auc.dk                      tgrid.it
ifca.unican.es                  tproa.net
ifh.de                          tu-bs.de
ific.uv.es                      tu-chemnitz.de
illigal.uiuc.edu                ugcs.caltech.edu
impetus.uni-koeln.de            umbc.edu
in2p3.fr                        umich.edu
inf.ed.ac.uk                    uncc.edu
infn.it                         uni-freiburg.de
ing.uniroma1.it                 uni-hohenheim.de
interdose.net                   uni-mannheim.de
ipp-garching.mpg.de             uni-paderborn.de
ir.stanford.edu                 urz.uni-heidelberg.de
isis.unc.edu                    usatlas.bnl.gov
isk.kth.se                      vn.uniroma3.it
it.kth.se                       wam.umd.edu
italia                          wu-wien.ac.at
itp.tugraz.at                   zcu.cz
jpl.nasa.gov                    ziti.uni-heidelberg.de
kfki.hu                         zone.mit.edu

Note that the above list of cells in /afs will vary based on the version of CellServDB and/or the default cell configured in the client. New versions of CellServDB can be downloaded from grand.central.org; the ports packaging will be updated for future CellServDB changes.