-*How to set up a BuildBot Slave for OpenAFS*
+# How to set up a BuildBot Worker for OpenAFS
-_Rationale_
+## Rationale
-Why would I want to? OpenAFS supports multiple platforms, but the strength of support for
-each of those individual platforms is directly proportional to the amount of testing done thereon.
-Although there are many different types of testing that can be done, one of the most basic is
-that of changes to the code base. A BuildBot slave allows for automated testing of each gerrit
-commit and is therefore a great resource to developers, most of whom do not have the time or
-resources to test their code on every platform prior to submission. Therefore, we encourage you
-to contribute a system as a build slave for a platform not already covered! You know you want to!
+Why would I want to? OpenAFS supports multiple platforms, but the strength of
+support for each of those individual platforms is directly proportional to the
+amount of testing done thereon. Although there are many different types of
+testing that can be done, one of the most basic is that of changes to the code
+base. A BuildBot worker allows for automated testing of each gerrit commit and
+is therefore a great resource to developers, most of whom do not have the time
+or resources to test their code on every platform prior to submission.
+Therefore, we encourage you to contribute a system as a build worker for a
+platform not already covered! You know you want to!
-_Approach_
+## Approach
-This is a consumer-oriented, high-level "how-to" to (hopefully) show how simple it is to set up
-your own build slave (and thereby contribute to OpenAFS!). This document is not intended to be
-platform-specific -- there are just too many different ways of doing things across
-Linux/Windows/SunOS/IRIX/AIX/BSD etc. There are additional notes elsewhere on this Wiki
-(see: [[BuildbotMasterNotes]]) about setting up the BuildBot Master, and some of this material
-is covered there as well -- however, this page is an attempt to make the process more understandable
-(specifically the buildslave setup).
+This is a consumer-oriented, high-level how-to to show how simple it is to set
+up your own build worker. This document is not intended to be
+platform-specific -- there are just too many different ways of doing things
+across different platforms.
-_Overview_
+## Overview
A quick overview of the process:
-1. Acquire suitable system
-1. Install required software
-1. Compile OpenAFS from git
-1. Create buildbot account and home directory
-1. Contact OpenAFS BuildBot admin for configuration details
-1. Edit configuration files
-1. Start build slave
-1. Set up startup scripts
-1. Optional configuration
+1. Acquire suitable system which will be continuously available
+1. Install the required software to build OpenAFS on the worker
+1. Verify you are able to compile OpenAFS from a git checkout on the worker
+1. Contact OpenAFS BuildBot admin for worker name and password
+1. Install the buildbot worker and verify it starts automatically
-*Of note:* since BuildBot slaves pull all their information from the master (rather than having it pushed
-to them by the master), there is no need to open an incoming port on your firewall, etc. As long as the
-slave can contact the master on the port you're given (which will be explained later), you're good to go!
+Since BuildBot workers pull all their information from the master, rather than
+having it pushed to them by the master, there is no need to open an incoming
+port on your firewall. As long as the worker can contact the master on the
+port you are given, you are good to go!
-_Hardware Requirements_
+## Hardware Requirements
-* Any system which OpenAFS currently supports (the greater diversity of build slaves, the better)
+* Any system which OpenAFS currently supports
* Enough free disk space to build OpenAFS tree (a couple gigabytes should do)
-The machine does not necessarily need to be dedicated entirely to being a buildslave; a build slave
-could run on any lightly-loaded, non-critical system (subject to your security requirements, etc.,
-of course). Some of the OpenAFS buildslaves are run by volunteers at their own homes, for instance.
+The machine does not necessarily need to be dedicated entirely to being a build
+worker; a build worker could run on any lightly-loaded, non-critical system
+subject to your security requirements, etc., of course. Some of the OpenAFS
+build workers are run by volunteers at their own homes, for instance.
-If you happen to have an older system with multiple processors, you can run more than one build slave
-instance on the same machine. The workload will be distributed across them by the master: successive
-gerrit commit builds are started on the next free instance, thus reducing the overall time to
-test multiple commits.
+## Software Requirements
-_Software Requirements_
+The software to build OpenAFS is platform dependent. See
+[[HowToBuildOpenAFSFromSource]] to install the tools to build OpenAFS.
-BuildBot has the following software dependencies:
+You platform may already include a buildbot worker package included, otherwise
+use python `pip` to install the buildbot worker software and required
+dependencies.
-* Python: [[http://www.python.org/]]
-* Twisted: [[http://twistedmatrix.com/]]
-* ZopeInterface: [[http://www.zope.org/Products/ZopeInterface]]
-* Git: [[http://git-scm.com/]]
+ pip install buildbot-worker
-Links to the above software websites are provided for reference only -- you may very well have a more
-convenient method for installing software on your platform than downloading the source, building,
-and installing by hand. Check public software repositories for your platform first to save yourself some
-effort. BuildBot is not particularly sensitive to dependency versioning, so you may not need the
-absolute latest version of everything installed. (For instance, BuildBot 0.7.12 is known to work with Python 2.5.2,
-Git 1.7.9, ZopeInterface 3.3.0, Twisted 9.0.0.)
+If you cannot or do not wish to install the buildbot into a site-wide location,
+you can also install it into the account's home directory or any other location
+using a tool like `virtualenv`.
-Git version 1.7.9 or above is required. The "git clean -e" option must be present and the
-"git clean -X -e excluded-file" behavior must be correct.
+## Compile OpenAFS from git
-And, of course, you'll need to install BuildBot itself: [[http://buildbot.net/trac]]
+Use git to clone the openafs repo, then make sure the software will compile
+successfully. For Unix platforms, the typical build process is:
+ git clone git://git.openafs.org/openafs.git
+ cd openafs
+ sh regen.sh
+ ./configure
+ make
-_Compile OpenAFS from git_
+## Create buildbot account and home directory
-Use git to clone the openafs repo, then make sure the software will compile successfully. For Unix platforms, the typical build process is
-"sh regen.sh && ./configure && make && make dest"
-The actual buildbot builds will configure using the --enable-checking argument
-to configure (as well as several others); this activates more compiler errors
-and you will probably want to do manual builds with checking enabled, to
-fix any build errors encountered before the buildslave enters regular use.
+It is recommended that you use a special-purpose non-privileged user account
+to run the buildbot worker. The name 'buildbot' is commonly used as the
+account name to run the buildbot worker.
-_Create buildbot account and home directory_
+The build master does not need to know the account's password -- in fact, the
+account does not necessarily need remote login privileges and may be locked or
+otherwise secured as long as your platform still allows you to run commands as
+that user.
-It is recommended that you use a special-purpose non-priviledged user account to run the build slave
-(i.e., "buildbot"). The build master does not need to know the account's password -- in fact, the
-account does not necessarily need remote login priviledges and may be locked or otherwise secured
-as long as your platform still allows you to run commands as that user.
+## Contact OpenAFS BuildBot Master admin for configuration details
-Test that you can run the buildbot command successfully (without error messages that might indicate
-installation issues): /path/to/buildbot
+The buildbot admin can be contacted by making a request on the
+openafs-devel@openafs.org mailing list.
-_Contact OpenAFS BuildBot Master admin for configuration details_
+The admin will provide the following settings:
-Contact jason at rampaginggeek.com regarding the settings you'll
-need to add your machine as an OpenAFS build slave. These settings will include:
+ * A buildbot worker name
+ * A buildbot worker password. Keep this secret.
-* buildmaster_host: this is the OpenAFS build master
-* port: this is the port on the build master which your slave will talk to
-* slavename: a unique descriptive name for your new build slave, usually representing your operating system and architecture (and not necessarily anything to do with your systems' hostname)
-* passwd: a password assigned to your slave for use in connecting with the master
+## Create your buildbot worker
-_Create your build slave configuration_
+Log in as the `buildbot` user. Create a sub directory called
+`/home/buildbot/openafs`. Buildbot will not touch anything outside of this
+directory.
-As your buildbot user:
+As the `buildbot` user, create the buildbot worker configuration with the name
+and password provided by the buildbot admin:
-Inside the user's home directory (say, /home/buildbot), create a subdirectory which will contain all of the buildslave files (say, /home/buildbot/slave1).
-Buildbot will not touch anything outside of this directory.
+ buildbot-worker create-worker /home/buildbot/openafs buildbot.openafs.org <NAME> <PASSWORD>
-Substituting the appropriate values from above, the execute:
+Edit file `openafs/info/admin` to set your name and email
+address. This is the worker admin address, and will be visible from the build
+status page, so you may wish to munge it a bit if address-harvesting spambots
+are a concern.
-/path/to/buildbot create-slave /home/buildbot/slave1 buildmaster_host:port slavename passwd
+Edit the file `openafs/info/host` to set a brief description of the host, such
+as OS, version, memory size, CPU speed, versions of relevant libraries
+installed.
-Starting with buildbot version 0.8, the command-line tool was split into
-buildbot and buildslave, so the buildslave utility should be used with such
-newer buildbot versions.
-Check the configuration file /home/buildbot/slave1/buildbot.tac to make sure the settings you entered are reflected in it.
-Make sure that "usepty" is set to "0". "usepty = 1" it is known to fail on at least one architecture.
+## Start build worker
-*Optional, but recommended:* Edit the /home/buildbot/slave1/info/{admin,host} files to describe the system (in host) and send
-yourself a shout-out (tell who you are in admin). These show up on the web if you drill down to the build slave information
-on the OpenAFS buildbot web portal, so you may want to obfuscate your contact information.
+As the `buildbot` user, manually verify the buildbot worker starts and connects
+to the buildbot master:
-*Note:* if you are planning on running multiple OpenAFS buildslave instances, you simply repeat this and all later steps,
-specifying a different base directory (i.e., /home/buildbot/slave2, /home/buildbot/slave3, etc.)
+ buildbot-worker start openafs
-_Start build slave_
+Check the logs to make sure it started properly. Check the OpenAFS BuildBot
+web portal to see the build progress presented graphically.
-/path/to/buildbot start
+## Set up startup scripts
-Check the logs to make sure it started properly. Check the OpenAFS BuildBot web portal to see the build progress presented
-graphically.
+Make sure the buildbot worker starts automatically when the host is started.
+This, of course is platform dependent. If you installed the buildbot worker
+with your package manager, the init scripts systemd files should already be
+installed. If you installed the buildbot worker with pip, you will need to
+setup init scripts or systemd files to start the worker when the host is
+started.
-There may be some kinks to work out of the system with the first few builds, especially if you are contributing a slave system
-for a less-heavily-used platform whose support might suffer from bit rot between releases (hey, it happens!).
+Here is an example systemd unit file to start the buildbot worker on
+systemd based hosts. Save as `/etc/systemd/system/buildbot-worker.service`
-_Set up startup scripts_
+ [Unit]
+ Description=BuildBot worker service
+ After=network.target
+
+ [Service]
+ User=buildbot
+ Group=buildbot
+ WorkingDirectory=/home/buildbot/openafs
+ ExecStart=/home/buildbot/venv/bin/buildbot-worker start --nodaemon
+
+ [Install]
+ WantedBy=multi-user.target
-Self-explanatory. Make sure they run /path/to/buildbot start as your buildbot user.
+If the worker is running on a virtual machine, be sure the guest will be
+automatically restarted when the hypervisor is restarted.
-_Set up log rotation and various other useful settings_
+## Set up log rotation and various other useful settings
-The buildbot.tac configuration file can be used to optimize your build slave. For instance, you may want to change maxRotatedFiles
-so you don't end up with a bunch of logs littered about /home/buildbot/slave1/. Have a look at the BuildBot documentation if
-you wish, though you can expect to be able to take a fairly hands-off approach to administering your new OpenAFS build slave.
+The buildbot.tac configuration file can be used to optimize your buildbot
+worker. For instance, you may want to change maxRotatedFiles so you don't end
+up with a bunch of logs littered about. Have a look at the BuildBot
+documentation if you wish, though you can expect to be able to take a fairly
+hands-off approach to administering your new OpenAFS build slave.
Link to current BuildBot documentation: [[http://buildbot.net/buildbot/docs/current/]]
Thanks for your help!
-
git://git.openafs.org / openafs-wiki.git / blobdiff