FreeBSD GEOM Storage Virtualisation Layer (''gvirstor'')
Author: Ivan Voras (see page IvanVoras for contact information)
Mentor: pjd
Synopsis
A FreeBSD kernel GEOM class providing storage virtualisation facility, regardless of type of underlying devices or its usage. To steal a description from des@, it is a kind of "virtual memory" for disks - you pre-allocate address space (i.e. make a large virtual drive) and worry later about where to find physical space to back it.
Status
gvirstor will be committed to -CURRENT "real soon now". It needs testing on non-i386 architectures!
Here's a screenshot of a simple virstor device, virtual size 40GB, on physical providers totalling 2GB: gvirstor-status-fsck.png. It shows a virtual device being created, newfs'ed (with UFS1, which immediately initializes inodes), and fsck'ed.
READ THE README FILE for instructions on how to use gvirstor.
2007-08-13 Fourth release candidate available gvirstor-rc4.tbz
- Fixed a bug with data corruption on big (1 TB+) virtual devices.
2007-04-24 Third release candidate available gvirstor-beta5.tbz
- Implemented BIO delay in ENOSPC case (IO is unconditionally delayed until physical space is added).
2007-02-03 Second release candidate available gvirstor-beta4.tbz
2006-08-25 Beta3 available - release candidate gvirstor-beta3a.tbz
Features
Here is a short list of features gvirstor provides:
- Creates storage devices in "overcommit" mode, larger than physically available storage
- Can create devices of arbitrary size, upto 2^63 bytes, with arbitrary chunk (extent) size
- Notifies via kernel-syslog mechanism when used-up storage approaches available storage (watermark values for these warnings are controllable with sysctls)
- Allows adding new storage devices (components) at runtime, when virstor device is "hot"
- Allows removing storage devices (components) when they are unused and at the end of list of added components
- Updates virstor allocation table (i.e. metadata) synchronously with write requests (all considerations with drive caches still remain, but if the drive doesn't lie, the data is safe)
Project Description
When doing server consolidation a popular step is storage virtualisation, the goal of which is to provide storage space on demand. The purpose of gvirstor module is to provide this facility by offering the ability to create a virtual storage device of arbitrarily large size (typically several terabytes) which consists of an arbitrary number of physical storage devices (actually any lower-level GEOM providers, including RAID devices) of arbitrary size (typically 50 GB - 400 GB hard drives). Storage space from these components is carved into small chunks (for example 4 MB) and allocated (committed) to the virtual device on as-needed basis. For example, two writes to a freshly created device: one at the beggining of virtual storage device and one at the end will commit two chunks from physical device components, chosen at the convenience of the GEOM module. When all available physical storage is exhaused, no new chunks can be allocated until more physical components are added. In this way, storage space can be managed with flexibility and forethought for future needs.
This work will be licensed under the BSD license.
Technical Details
Basically, gvirstor would be an important part of a logical volume manager. Existing GEOM classes in FreeBSD offer RAID functionality, but up to now there is no native GEOM storage virtualisation class. The facility will be implemented as a GEOM class in a kernel-loadable module, with accompanying userland utility that controls it (using facilities provided by the GEOM framework). Special consideration is to be given to monitoring and notification of available physical storage so the administration staff can react and add additional devices/components on time. The virtual storage device can be used for arbitrary purposes, including hosting file systems.
More information on the GEOM framework can be found here: http://www.freebsd.org/cgi/man.cgi?query=geom&sektion=4
Benefits for FreeBSD
Storage virtualisation is an important component in new enterprise data centres, and several expensive commercial products have emerged that offer this functionality on various storage systems (for example Fibre Channel). Virtual storage module included with FreeBSD will allow usage of inexpensive direct attached storage devices (a.k.a. disk drives) in virtual storage configurations, or even creation of dedicated or embedded devices for this purpose based on a inexpensive Open-source operating system with a business-friendly license.
About me
My main wiki page is IvanVoras.
I'm a student at the final year at Faculty of Electrical Engineering and Computing (FER) in Zagreb, Croatia. I'm a full-time FreeBSD user and developer and also administrate several FreeBSD servers and a few desktop systems. My homepage is http://ivoras.sharanet.org/.
Misc
A version of the original proposal is gvirstor-proposal.txt.