200808 DevSummit Cambridge
The 4 and a bit steps of vimage integration
General comments
- All commits go through vimage-commit2 branch step by step. Julian and Marko bringing in the basic set and reviewers are going to finalize and produce the patch.
RobertWatson, BrooksDavis, KrisKennaway volunteered to also review patches.
BjoernZeeb volunteered to keep an eye on the schedule, progress, ...
After step 2 we are going to provide a clean p4 branch people can use to branch their own working area off to virtualize other subsystems like vimage.sysvipc. If you want to work on anything before this use the 'vimage' p4 branch (//depot/projects/vimage/...).
Timeline
- Step 1.5 and step 2 are planned to take about 2 weeks each with step 3 being done at latest before November 1st preferably before EuroBSDCon2008. This would give us about the following deadlines:
- Step 1.5: Sep 1st
- Step 2: Sep 15th
- Step 3: ideally Sep 30th, in case thing go bad and move, Oct 13, worst Halloween.
We are going to reflect on this at the upcoming EuroBSDCon2008 DevSummit.
- REALITY* Step 2 was committed DECEMBER 10
The 4.5 steps
1. [DONE] Apply the V_ macro prefix:
- NOP (non functional) change.
- Invalidates most of the patches out there.
1.5. [DONE] Further style changes (NOP) bringing in more infrastructure:
- [DONE] Resolve file local statics with conflicting names.
- [DONE] Import INIT_VNET_* as NOPs along with net/vnet.h, netinet/vinet.h, netinet6/vinet6.h, netgraph/vnetgraph.h, ...,
- move #defins from vimage.h to those header files, make the macros NOPs in vimage.h. Put things into present header files.
- [DONE]Import "V_SYSCTLs", make them behave SYSCTLs w/o vimage for the moment. update man pages.
2.[DONE] Move the globals into a struct (g.v):
- Make sure the macros from step 1.5 are no longer NOPs.
- Bring in per module constructor functions.
- Move globals to structs.
- Make it possible to revert to globals for the commit and remove that shortly afterwards.
3. Apply virtualization (g->v):
- Add destructor functions.
- Add iterator macros.
- Per vimage pointers.
- INIT_VNET_INET would start doing something.
4. Finish more subsystems
- That is where your hands are needed and when you need to know what to do!
- Reorder variables in structs for cache locality.
- James Gritton's management framework.
- User space application will be called jail(8). There will be a switch to flip the behavior from classical to new style.
Syscalls will be called jail_*.
- In kernel structure will be called whatever they will be called. He who implements decides.
- We want an explicit opt-in for any recursive changes.
Progress report Jan 31 09
I (julian) have been testing going backwards and forwards using the VIMAGE_GLOBALS option and have see no significant performance changes between the original code and the same code compiled with a set of global structures. Looking at the generated code I can see that the compiler is resolving the locations of each field in the structures and the resulting instructions are almost identical with only the addesses being differnet. This gives me confidence that when VIMAGE is compiled out the performance numbers are in practice, as we had postulated, identical to teh original numbers. This makes VIMAGE a much more attractive feature if those who do need the performance and don't need multiple contexts can be confident we will not be hurting them.
HOWTO identify file static global variables with colliding names
You can use nm(1) on your kernel/object files. Use something like:
nm *.o | awk '/ [bd] / { print $3 }' | sort | uniq -c | awk '{ if ($1 > 1) { print $0 } }'
HOWTO verify that the pure style changes are all right, aka. The MD5 Dance
- Turn off INVARIANTS/INVARIANT_SUPPORT (since this records line numbers everywhere).
cd /sys/<arch>/conf make LINTGenerate a VIMAGE_LINT (or an equivalent for GENERIC):include LINT ident VIMAGE-LINT nooptions INVARIANTS nooptions INVARIANT_SUPPORT
Bump __FreeBSD_version in unpatched tree (to match the patched tree) if needed.
__LINE__ appears in some .o files via macros, so whitespace needs to be added to unpatched files that are modified by e.g. addition of #include <sys/vimage.h>.
- Prepending blank lines to the top of the file should be good enough at least for most of them. If it is not, for example at a later stage, you need to prepare a 'counterpatch' that adds the same whitespace to the plain kernel as the vimage diff adds new lines to its tree. You may also need to add some temporary whitespace to the vimage tree in case you remove lines and do not add new ones back. To avoid to much empty line shuffling you may want to change the lock line number to a static like:
--- sys/sys/lock.h.orig 2008-09-25 22:21:06.000000000 +0000 +++ sys/sys/lock.h 2008-09-25 22:21:49.000000000 +0000 @@ -132,4 +132,4 @@ struct lock_class { #if LOCK_DEBUG > 0 #define LOCK_FILE __FILE__ -#define LOCK_LINE __LINE__ +#define LOCK_LINE 0xc0defeed #else
- Prepending blank lines to the top of the file should be good enough at least for most of them. If it is not, for example at a later stage, you need to prepare a 'counterpatch' that adds the same whitespace to the plain kernel as the vimage diff adds new lines to its tree. You may also need to add some temporary whitespace to the vimage tree in case you remove lines and do not add new ones back. To avoid to much empty line shuffling you may want to change the lock line number to a static like:
- Build kernel once with the whitespace and once with the real patch.
config VIMAGE_LINT cd ../compile/VIMAGE_LINT make -s cleandepend __MAKE_CONF=/dev/null SRCCONF=/dev/null make -s depend __MAKE_CONF=/dev/null SRCCONF=/dev/null make -s __MAKE_CONF=/dev/null SRCCONF=/dev/nullor run following from src/make buildkernel KERNCONF=VIMAGE_LINT __MAKE_CONF=/dev/null SRCCONF=/dev/null
Strip the *.o files in sys/<arch>/compile/VIMAGE_LINT or obj/sys/VIMAGE_LINT (subsitute the . argument to find in that case):
find . -type f -name "*.o" -print | xargs -n 1 strip
- Run md5 on each .o file and save the output depending on the tree.
find . -type f -name "*.o" -print | xargs md5 | sort > /tmp/VIMAGE_LINT.{plain,vimage}
- Comparing the md5 output of both runs, checksums of *.o should only differ in vers.o, which records the timestamp of the kernel build.
diff -u /tmp/VIMAGE_LINT.plain /tmp/VIMAGE_LINT.vimage
Unvirtualized (or not finished) parts of the network stack
- ...
- ...
- ...
General virtual talk on Sunday
- Jail is useful for name for virtualization for "branding"
- You have jail safe file systems, but you should be able to only allow some file systems to be mounted in a jail
- Perhaps only allow jexec instead of jattach since jattach is "ugly" and leave things from outside
- You want jails without processes hanging around e.g. if "jails" which just does forwarding
- Perhaps keep jail(8) for compat, and new jcreate(8) for advanced fancy jails. jail(8) is just a jcreate(8) wrapper