-There are Jason's notes for setting up the buildbot master
+OpenAFS Buildbot Master Configuration
+=====================================
-The buildbot master was installed in a VM on debian lenny. The VM has the following configuration:
+Buildbot master installation
+----------------------------
-* 10GB disk (partitions: 3GB root, 500MB swap, the rest for /home
-* RAM 384MB
+The following instructions describe how to use `pip` to install the buildbot
+master in a Python virtual environment. With sudo/root access, install Python3
+and the development packages for it.
-When installing lenny, uncheck desktop system and check web server
+Ensure TCP ports 9989 and 8010 are open. Create a `buildbot` user on the
+system. The remaining steps to not require sudo access and should be run as
+the `buildbot` user.
-edit sources.list and comment out the cdrom source
+Optionally create a project level directory, for example:
- root# apt-get build-dep buildbot
- root# apt-get install git-core
- root# apt-get install openssh-server
+ $ mkdir openafs-buildbot
+ $ cd openafs-buildbot
- login as buildmaster user
- % git clone git://github.com/jaysoffian/buildbot.git buildbot-src
- % cd buildbot-src/
- % git checkout -b gerrit e4c0d458374b9a0ce34c
+Create a Python virtual environment:
- // fix a debugclient bug
- % git cherry-pick 9ab140a03378baa5ec98
+ $ python3 -m venv venv
- // allow for not clobbering the git tree each build
- % git cherry-pick 7fa5137bbc449144d76334bd366920d8b6a7c2c0
+Activate the virtual environment for the installation:
- // following
- http://buildbot.net/buildbot/docs/0.8.1/Installing-the-code.html#Installing-the-code
- // follow 8.1 docs
- % python setup.py build
- root# python setup.py install
- root# apt-get install python-json
- root# apt-get install python-jinja
+ $ source venv/bin/activate
- // cvs is required to make the buildbot test suite go all clear
- root# apt-get install cvs
+Install buildbot and it's dependencies:
- // run the unit tests
- % PYTHONPATH=. trial buildbot.test
+ $ pip install --upgrade pip
+ $ pip install 'buildbot[bundle]'
- % mkdir $HOME/buildbot/master/openafs
- // create the buildbot master
- % buildbot create-master $HOME/buildbot/master/openafs
+Create the buildbot master instance:
-edit config
+ $ buildbot create-master master
- ...
- repourl="git://git.openafs.org/openafs.git"
+The virtual environment can now be deactivated:
- from buildbot.process import factory
- from buildbot.steps.source import Git
- from buildbot.steps.shell import Compile
- from buildbot.steps.shell import Configure
+ $ deactivate
- f1 = factory.BuildFactory()
- f1.addStep(Git(repourl=repourl, mode="clobber"))
- f1.addStep(Configure(command=["sh","regen.sh"]))
- f1.addStep(Configure(command=["./configure","--enable-checking"]))
- f1.addStep(Compile(command=["make"]))
- f1.addStep(Compile(command=["make","dest"]))
+Master configuration
+--------------------
- b1 = {'name': "buildbot-full",
- 'slavename': "debian-i386",
- 'builddir': "full",
- 'factory': f1,
- }
- c['builders'] = [b1]
- ...
- =================================================
+Download the buildbot master configuration:
- // test config
- % buildbot checkconfig master.cfg
- // start the buildmaster
- % /usr/bin/buildbot start $HOME/buildbot/master/openafs
+ $ git clone https://github.com/openafs-contrib/afsbotcfg.git
+ $ cd afsbotcfg
-check http://localhost:8010 for web output
+Create the `Makefile` and deploy the buildbot `master.cfg` and
+sample `settings.ini` file:
- //install openafs builddeps
- root# apt-get build-dep openafs
- root# apt-get install automake1.9 linux-headers-2.6-686 flex bison dblatex docbook-xsl
+ $ python configure.py
+ $ make install
- root# useradd -d /home/buildslave -m buildslave
+Make a link to the makefile in the top level directory:
- buildslave% mkdir -p $HOME/buildbot/slave/openafs
- buildslave% buildbot create-slave $HOME/buildbot/slave/openafs localhost:9989 debian-i386 PASSWORD
+ $ cd ..
+ $ ln -s afsbotcfg/Makefile
-edit files in /home/buildslave/buildbot/slave/openafs/info
+Master settings
+---------------
- // start the buildslave
- buildslave% buildbot start $HOME/buildbot/slave/openafs
+Edit the `settings.ini` file in the `master` directory. This file stores
+information we do not track with git, such as the buildbot worker passwords.
+The `settings.ini` file is an INI-style file with the following sections:
+* local - settings specific to the local environment
+* admins - the list of user emails and passwords for authenticated access
+* email - emails for notifications
+* workers - the list of worker names and passwords
+Example:
- /// begin setting up gerrit stuff
- // create ssh key for buildbot
- // sign in to gerrit and add buildbot ssh key
- buildmaster% touch ~buildmaster/buildbot-src/contrib/__init__.py
+ $ cat master/settings.ini
+ [local]
+ buildbotURL = http://buildbot.openafs.org:8010/
+
+ [admins]
+ tycobb@yoyodyne.com = password
+
+ [workers]
+ example-worker-1 = secret1
+ example-worker-2 = secret2
-Here is a working config snippet (minus the slave passwords) for gerrit+buildbot:
+Gerrit account
+--------------
- import sys
- sys.path.append('/home/buildmaster/buildbot-src/contrib')
+The buildbot master needs an account on the [OpenAFS Gerrit][3] to listen for
+Gerrit events and to report verified changes on successful builds. The name of
+the account is `buildbot`. Place the ssh keys for the buildbot's Gerrit account
+in the `.ssh` directory under the home directory of the local account running
+the buildbot master. The key file name should match the ones defined in the
+`master.cfg` file.
- from gerrit import GerritChangeSource
- #c['change_source'] = GerritChangeSource()
+Starting the master
+-------------------
- ## configure the Schedulers
+Check the buildbot master configuration with the command:
- from buildbot.scheduler import Scheduler, Periodic
+ $ make check
- mybuilders = []
- mybuilders.append("debian-linux-i386-builder")
- mybuilders.append("win2008-builder")
+Start the buildbot master with the command:
- stable14_builders = []
- stable14_builders.append("debian-linux-i386-builder")
+ $ make start
+Stop the buildbot master with the command:
+ $ make stop
- c['schedulers'] = []
- c['schedulers'].append(Scheduler(name="all", branch=None,
- treeStableTimer=30,
- builderNames=mybuilders))
-
-
- c['schedulers'].append(Periodic("every_4_hours", ["irix-builder"], 4*60*60))
-
- from gerrit import GerritScheduler
- c['schedulers'].append(GerritScheduler("gerrit_scheduler", mybuilders, branches=["master","openafs-devel-1_5_x"]))
-
- c['schedulers'].append(GerritScheduler("gerrit_stable_scheduler", stable14_builders, branches=["openafs-stable-1_4_x"]))
-
- ####### BUILDERS
-
- repourl="git://git.openafs.org/openafs.git"
-
- from buildbot.process import factory
- from buildbot.steps.source import Git
- from buildbot.steps.shell import Compile
- from buildbot.steps.shell import Configure
- from buildbot.steps.shell import ShellCommand
-
- f1 = factory.BuildFactory()
- #f1.addStep(ShellCommand(command=["git","clean","-xdf"]))
- #f1.addStep(Git(repourl=repourl, mode="clobber"))
- f1.addStep(ShellCommand(command=["sleep","120"]))
- f1.addStep(Git(repourl=repourl, mode="update", ignore_ignores=True, retry=[120,60]))
- f1.addStep(ShellCommand(command=["sh","regen.sh"]))
- f1.addStep(Configure(command=["./configure","--enable-checking","--enable-supergroups","--enable-namei-fileserver"]))
- f1.addStep(Compile(command=["make"]))
- f1.addStep(ShellCommand(command=["make","dest"]))
-
- b1 = {'name': "debian-linux-i386-builder",
- 'slavename': "debian-i386",
- 'builddir': "full",
- 'factory': f1,
- }
-
- c['builders'] = [b1]
-
- irix_configure_env={'CC':"cc"}
-
- irixfactory = factory.BuildFactory()
- irixfactory.addStep(ShellCommand(command=["sleep","120"]))
- irixfactory.addStep(Git(repourl=repourl, mode="update", ignore_ignores=True, retry=[120,60]))
- irixfactory.addStep(ShellCommand(command=["sh","regen.sh"]))
- irixfactory.addStep(Configure(command=["./configure","--enable-checking","--enable-supergroups","--enable-namei-fileserver","--with-krb5=/usr/nekoware"],env=irix_configure_env))
- irixfactory.addStep(Compile(command=["gmake"]))
- irixfactory.addStep(ShellCommand(command=["make","dest"]))
-
-
- irixbuilder = {'name': "irix-builder",
- 'slavename': "sgi_irix_65-64bit",
- 'builddir': "irix-65-64bit",
- 'factory': irixfactory,
- }
-
- c['builders'].append(irixbuilder)
-
-
- ###### win_amd64 free builder
-
- win_amd64_free_factory = factory.BuildFactory()
-
- # wait for 2 minutes
- win_amd64_free_factory.addStep(ShellCommand(command=["ping","-n","120","localhost"]))
- win_amd64_free_factory.addStep(Git(repourl=repourl, mode="update", ignore_ignores=True, retry=[120,60]))
- win_amd64_free_factory.addStep(Compile(command=["build-openafs.cmd", "amd64", "free"]))
-
- win_amd64_free_builder = {'name': "win2008-builder",
- 'slavename': "win2008-amd64",
- 'builddir': "win-amd64-free",
- 'factory': win_amd64_free_factory,
- }
- c['builders'].append(win_amd64_free_builder)
-
- ####### STATUS TARGETS
-
- # 'status' is a list of Status Targets. The results of each build will be
- # pushed to these targets. buildbot/status/*.py has a variety to choose from,
- # including web pages, email senders, and IRC bots.
-
- c['status'] = []
-
- # Use allowForce=True (boolean, not a string. ie: not 'True') to allow
- # Forcing Builds in the Web User Interface. The default is False.
- from buildbot.status import html
- c['status'].append(html.WebStatus(http_port=8010,allowForce=True))
-
- from gerrit import GerritStatusPush
- c['status'].append(GerritStatusPush('gerrit-prod'))
-
-
- from buildbot.status import client
- c['status'].append(client.PBListener(9988))
-
-
- ####### PROJECT IDENTITY
- c['projectName'] = "OpenAFS"
- c['projectURL'] = "http://www.openafs.org/"
- c['buildbotURL'] = "http://buildbot.openafs.org:8010/"
+[1]: https://buildbot.openafs.org/
+[2]: https://openafs.org
+[3]: https://gerrit.openafs.org/
git://git.openafs.org / openafs-wiki.git / blobdiff