• PortingHAMMERFS

On porting the HAMMER file system from DragonflyBSD to FreeBSD

A discussion with Matt Dillon:

There are several major differences:

   * All VOP operations related to the namecache... those using named
     path elements like lookup, rename, etc... those are completely
     different.

     The biggest one is that our VOP_NRESOLVE() API is completely different
     from the old VOP_LOOKUP() API.  The old VOP_LOOKUP() API had to handle
     numerous side effects and I don't recall the FreeBSD namecache
     subsystem doing any namespace locking (beyond locking vnodes, that
     is).

     Also our VOP_NREMOVE() is fairly radically different... it's an all
     in one call whereas the older VOP_REMOVE was paired with a VOP_LOOKUP
     that stored side-effects (not sure if FreeBSD has cleaned that up yet).
     i.e. old was LOOKUP(save side effects) + REMOVE(use side effects).
     And DragonFly's is just NREMOVE().

   * HAMMER uses a far more sophisticated buffer cache call-back interface
     which allows HAMMER to veto write requests from the buf_daemon's (to
     enforce certain write ordering requirements).

   * DragonFly uses a locking model that I don't think FreeBSD has an API
     for (our token locking model which releases held locks within the
     model when a thread switches out and reacquires them when the thread
     switches back in).  If FreeBSD also has an API for this model this
     would be easy to port (if not for all of the above issues).

   * DragonFly uses a slightly different stacked BUF/BIO mechanic for I/O.
     This would be easy to port (if not for all of the above issues).

   * DragonFly's BUF/BIO allows for different buffer sizes within the
     file buffers (not just the device buffers).  Generally speaking
     there can be significant mixing of buffer sizes and the cluster code
     has to be able to deal with it given appropriate argumetns.  I don't
     think FreeBSD's cluster code could handle it.

   Personally I think it might be too much for a GSOC project.

                                       -Matt

:Do you think that the this API could be emulated, however inefficient,
:with the FreeBSD's API?

   Perhaps, but doing LOOKUP/RENAME/REMOVE wouldn't be trivial.  My
   (several years old) understanding is that FreeBSD relies on locking
   the directory vnode in order to lock the namespace, then issues a LOOKUP
   to obtain an index into the directory... information which is stored
   out of band.  Then RENAME or REMOVE uses that information to actually
   delete/create the entries.  For UFS.

   DragonFly relies on the namecache to lock the namespace at a fine-grain.
   In DragonFly the actual filename in the namespace is what is locked
   and not the governing directory vnode.  It's a major differentiation.

   While DragonFly does have a wrapper for filesystems which use the
   old API (basically for UFS), the wrapper only implements the bare
   minimum features used by higher level kernel code and would not be
   useful in a FreeBSD context because the higher level kernel code in
   FreeBSD uses a larger feature set for LOOKUP.

   I don't know how difficult it would be to make it all work under FreeBSD.
   An experienced filesystem coder would not have too much trouble but an
   inexperienced one would get lost very quickly.

::... allows HAMMER to veto write requests from the buf_daemon's (to
::certain write ordering requirements).
:
:How complicated would this be to implement, and would its
:implementation require the modification of other file systems, or
:could it be relatively self-contained to the new file system?

   The issue here is that callback might cause a deadlock to occur under
   FreeBSD.  HAMMER is fairly careful about deadlocks but relies a lot
   on its token lock API.

   Otherwise it isn't a big problem, the callbacks only occur in a few
   places and can be sussed out with simple searches of the DragonFly kernel
   code.  The bigger problem would be getting the FreeBSD meisters to
   accept the changes... highly unlikely in my view.

:FreeBSD uses conventional mutex & rwlock models. From what you wrote
:here it looks like the HAMMER code requires much extensive support
:from the OS to ensure locking, i.e. that the models are not
:equivalent?

   The models are not equivalent, no.  I vaguely recall some discussion
   on the FreeBSD lists about having a lock API which automatically
   unlocks when a thread blocks voluntarily and relocks when it resumes,
   I don't know if such an API was ever implemented though.  If it was
   then you would be in good shape.  If it wasn't then you would have a
   problem because this sort of mechanic has to be implemented in the
   scheduler core and that might not be acceptable to the FreeBSD meisters.

                                       -Matt