-Bsuildbot linux rc builder for OpenAFS
+Buildbot Linux RC builder for OpenAFS
=====================================
-Changes to the mainline Linux source code frequently requires changes to the
-OpenAFS kernel module code. In order to catch build errors as the Linux and
-OpenAFS code changes, a nightly builder on the OpenAFS buildbot has been
-created to automatically build the various git branches of OpenAFS on the most
-recent Linux kernel release candidate every day. The builder spins up a
-virtual machine guest, installs the most recent Linux release candidate image
-and headers, reboots the guest, then attempts to build and run OpenAFS on the
-release candidate.
+Most releases of Linux require OpenAFS code changes, since OpenAFS is out of
+tree due to licensing incompatibilities between GPL and the IBM public license.
+In order to catch build errors early, a nightly builder on the OpenAFS buildbot
+has been created to automatically build the various git branches of OpenAFS on
+the most recent Linux kernel release candidate every night. The builder spins
+up a virtual machine guest, installs the most recent Linux release candidate
+image and headers, reboots the guest to run the new Linux release candidate,
+then attempts to build and run OpenAFS on the release candidate.
-Linux KVM virtualization and `libvirt` commands is used for virtualization.
-The builder downloads pre-build Linux release candidates published by the
-Ubuntu Kernel Team.
+Linux KVM and `libvirt` commands are used for virtualization. The builder
+downloads pre-build Linux release candidates published by the Ubuntu Kernel
+Team.
The components of this system are:
Linux KVM host system
---------------------
-todo
+The host system is currently configured to use Linux KVM to create ephemeral
+virtual machines to run the build and tests. The host system must have cpu
+virtualization extensions (Intel VT-x or AMD-V) to run KVM.
+
+See your distribution specific documentation for KVM and libvirt installation
+instructions.
Ubuntu base image
-----------------
-todo
+Create a base image to be cloned by the buildbot worker. The base image can be
+created with the libvirt-manager GUI program. The OpenAFS buildbot is currently
+using an Ubuntu 18.04 LTS release image.
+
+Install the `afsutil` package from python pip. This command is used to install
+the built binaries and create a test cell on ephemeral guests.
+
+ sudo pip install afsutil
-`kernel-ppa`
-------------
+Install the build tool chain needed to build OpenAFS from source using `sudo
+afsutil getdeps` or by running apt-get install of the packages needed. At this
+point it is best to verify the OpenAFS binaries can be built from a git
+checkout on a running base image.
-todo
+The `kernel-ppa` script is used to find and install the most recent Linux
+kernel image and headers from the Ubuntu Kernel Team's site. Download
+`kernel-ppa` from github and copy to a location in the path on the base image.
-`virt-run`
-----------
+ git clone https://github.com/meffie/virt-utils.git
+ cd virt-utils
+ git checkout virt-run
+ sudo cp kernel-ppa /usr/local/bin/
-todo
+Allow the buildbot user on the guest to run `kernel-ppa` and `afsutil` as sudo,
+without a password, in order to change the kernel on ephemeral guests.
+
+ $ cat /etc/sudoers.d/buildbot
+ buildbot ALL=(ALL) NOPASSWD: /usr/local/bin/afsutil
+ buildbot ALL=(ALL) NOPASSWD: /usr/local/bin/kernel-ppa
+
+Keep the base image guest running until the buildbot worker's ssh public key is
+installed (see the next section). After which, the base image guest should be
+shutdown, otherwise libvirt will not be able to clone the base image when a
+build is scheduled.
Buildbot worker
---------------
-todo
+The buildbot worker should be running on the host system or in a long-running
+virtual machine.
-Buildbot master configuration
------------------------------
+Create a 'buildbot' service account on the worker system and create an ssh
+key-pair for the communication from the buildbot worker to the ephemeral
+virtual machines. The ssh-key should be password-less for the automation.
+Copy the public ssh key to the buildbot user's authorized keys file on the
+running base image guest. After the key is in place and verified to work,
+shutdown the base image guest so it can be cloned.
+
+Configure the account to have access to `libvirt` commands by setting the
+environment variable `LIBVIRT_DEFAULT_URI`. See the KVM and libvirt
+documentation.
+
+Install the `virt-run` command from the 'virt-utils' project on github.
-todo
+ git clone https://github.com/meffie/virt-utils.git
+ cd virt-utils
+ git checkout virt-run
+ sudo make install
+Verify the `virt-run` command is able to clone the base-image to spin up an
+ephemeral guest and run commands on it. See the `examples` directory in
+`virt-utils` for some example usage.
+Finally, install the buildbot worker from your distibution packages or from
+python pip and configure the slave to contact the buildbot master.
+
+Buildbot master configuration
+-----------------------------
-====
+Changes to the buildbot master can be deployed by one of the OpenAFS buildbot
+admins. The buildbot master configuration is maintained in the OpenAFS-Contrib
+project on [github][1].
-https://github.com/meffie/virt-utils.git
+Currently, the buildbot master is configured to `virt-run` to spin up a clone,
+install the current linux release candidate, reboot the clone, and then build
+OpenAFS.
+[1]: https://github.com/openafs-contrib/afsbotcfg.git
git://git.openafs.org / openafs-wiki.git / blobdiff