Chinaunix首页 | 论坛 | 博客
  • 博客访问: 1155087
  • 博文数量: 290
  • 博客积分: 3468
  • 博客等级: 中校
  • 技术积分: 3461
  • 用 户 组: 普通用户
  • 注册时间: 2010-12-28 22:21










2012-06-25 22:05:47


This documentation is written to explain how to setup a build daemon to recompile the whole Debian unstable archive.

Note: this howto has been written for etch - things may have changed since.

How it works

Buildd setup works with 3 packages:

  • wanna-build
  • sbuild
  • buildd

wanna-build is used to store informations about packages in a database. sbuild is used to actually build packages into a chroot. buildd is used to fetch packages to build via wanna-build and run sbuild to build them.

Setting up the box Install Debian

Install debian. Version installed here is etch.

Install packages

The following packages are needed:

build-essential devscripts bzip2 fakeroot patch subversion Fetch the code and install packages

Currently, everything is not packaged. That's a shame.

sbuild is in the archive, but wanna-build and buildd are not. You fetch the code via Subversion from

Here is how I build sbuild and wanna-build:

$ svn co $ cd wanna-build $ vi debian/control --> Remove versionned dependency on sbuild, because we can use the version from unstable $ debuild -uc -us

Then, install packages:

# aptitude install sbuild debootstrap cron-apt libmldbm-perl dupload quinn-diff # dpkg -i wanna-build_2006.01.08_sparc.deb buildd_2006.01.08_sparc.deb

You can use the sbuild version from sid or from the svn repository. Currently I use both.

Create system environment

You can use /usr/share/doc/buildd/examples/buildd-setup-system.gz directly (zcat | sh), but I prefer to do everything by myself.

# as root adduser --system --shell /bin/sh --uid 60000 --gecos 'Build Daemon' --group --disabled-password buildd mkdir -p /var/lib/debbuild/srcdep-lock chown -R buildd:buildd /var/lib/debbuild chmod -R 2775 /var/lib/debbuild cd ~buildd zcat /usr/share/doc/buildd/examples/buildd.conf.gz > buildd.conf zcat /usr/share/doc/sbuild/examples/sbuildrc.gz > .sbuildrc chown buildd:buildd buildd.conf .sbuildrc mkdir -p logs build old-logs upload bin stats/graphs mqueue echo "|/usr/bin/buildd-mail-wrapper" > .forward chown buildd:buildd logs build old-logs upload bin stats .forward mqueue Setup chroot

Now it's time to create our chroot! The buildd-make-chroot command is not available in the sbuild package from sid, so use the one you grabbed from svn, be smart!

# cd ~buildd # ~/wanna-build/buildd-make-chroot buildd etch build/chroot-etch

Note: actually, this command fails because of dependencies, with the following error:

dpkg: dependency problems prevent removal of makedev: gnupg depends on makedev (>= 2.3.1-13) | devfsd | hurd; however: Package makedev is to be removed. Package devfsd is not installed. Package hurd is not installed.

The solution is to keep makedev (which is removed by buildd-make-chroot). Apply the following patch_:

--- buildd-make-chroot.orig 2007-05-11 12:31:44.000000000 +0000 +++ buildd-make-chroot 2007-05-11 12:31:57.000000000 +0000 @@ -74,7 +74,7 @@ chmod -R 02775 $TARGET/var/debbuild echo include /etc/*.conf >> $TARGET/etc/ (cd $TARGET/dev ; ./MAKEDEV fd) -sudo chroot $TARGET dpkg -P debconf-i18n debconf liblocale-gettext-perl libtext-charwidth-perl libtext-iconv-perl libtext-wrapi18n-perl procps makedev +sudo chroot $TARGET dpkg -P debconf-i18n debconf liblocale-gettext-perl libtext-charwidth-perl libtext-iconv-perl libtext-wrapi18n-perl procps echo "Successfully setup chroot for a buildd" echo Possible commands to append to fstab: echo echo $SUITE-proc $TARGET/proc proc defaults 0 0 \>\> /etc/fstab

You can finish the install:

# as root mkdir -p ~buildd/build/chroot-etch/var/lib/sbuild/srcdep-lock chown -R buildd.buildd ~buildd/build/chroot-etch/var/lib/sbuild/ mkdir ~buildd/build/chroot-etch/home/buildd chown buildd.buildd ~buildd/build/chroot-etch/home/buildd cd ~buildd/build && ln -s chroot-etch chroot-stable

After, edit the /etc/apt/sources.list in your chroot:

# as root vi ~buildd/build/chroot-etch/etc/apt/sources.list chroot build/chroot-etch /bin/sh apt-get update exit

You should mount proc into your chroot otherwise some build will failed. Add the following line to your /etc/fstab:

etch-proc /home/buildd/build/chroot-etch/proc proc defaults 0 0 Give root to your buildd

sbuild should be able to use schroot, but, in fact, that's terribly b0rked right now. So we will use the old sudo method.

# visudo Add: Defaults:buildd env_keep+="APT_CONFIG DEBIAN_FRONTEND SHELL" buildd ALL= NOPASSWD: ALL Setup sbuild

Change your user to buildd (sudo -H -s -u buildd), and configure sbuild.

# vi .sbuildrc # egrep -v '^(#|$)' .sbuildrc $build_dir="$HOME/build"; $log_dir = "$HOME/logs"; $mailto = "pollux\"; $maintainer_name='Pierre Chifflier '; $purge_build_directory="always"; $stalled_pkg_timeout = 90; 1; Setup buildd

Change your user to buildd and edit ~buildd/buildd.conf:

$ egrep -v '^(#|$)' buildd.conf require "/etc/"; @take_from_dists = qw(stable testing unstable); @no_auto_build = qw(boot-floppies); @weak_no_auto_build = qw(); $autoclean_interval = 86400; $admin_mail = "buildd admin "; $build_log_keep = 1; $statistics_mail = 'pollux\'; $dupload_to = "anonymous-inl"; $dupload_to_security = "anonymous-inl"; $log_queued_messages = 0; $wanna_build_user = "buildd_arch-hostname"; $wanna_build_dbbase = "build-db"; $sshcmd = ""; $sshsocket = ""; 1; Configure dupload

The buildd user should have a working dupload installation (.dupload.conf):

package config; $default_host = 'anonymous-inl'; $cfg{'anonymous-inl'} = { fqdn => "", method => "scpb", login => "pollux", incoming => "/var/www/debian/mini-dinstall/incoming/", dinstall_runs => 1, }; Add cron jobs

Some commands should be added to the crontab, to ensure that buildd is running, and to upload packages:

$ crontab -l # m h dom mon dow command 30 0,6,12,18 * * * /usr/bin/buildd-uploader 20 0,6,12,18 * * * /usr/bin/buildd-watcher Setup wanna-build

Copy wanna-build local configuration:

# zcat /usr/share/doc/wanna-build/examples/wanna-build.conf.local.gz > /etc/wanna-build.conf.local # chown buildd /etc/wanna-build.conf.local
Then you can create an empty wanna-build database:
# # as root: # install -m 755 -d -obuildd -gbuildd /var/debbuild # # as user buildd: $ wanna-build -d stable --create-db --clean-db

Now you will need to fill up the database with all packages. That's easy to do with a little script (thanks to Julien Louis).

#!/bin/sh cd /tmp wget -q -O Sources.main.gz wget -q -O Sources.contrib.gz if [ -e Sources.main.gz ] && [ -e Sources.contrib.gz ]; then zcat Sources.main.gz Sources.contrib.gz > Sources fi touch Packages quinn-diff -A @dpkg-architecture -qDEB_HOST_ARCH@ 2>/dev/null | wanna-build --merge-quinn -d stable rm -f Sources Packages
You can run this script once a day to update the database. Building arch=all packages

The default configuration of buildd is not to build architecture-independant packages. To force buildd to build arch=all packages, apply the following patch to '/usr/bin/buildd':

--- buildd.OLD 2007-05-14 11:59:20.000000000 +0000 +++ buildd 2007-05-14 11:59:47.000000000 +0000 @@ -466,7 +466,8 @@ my @sbuild_args = ( 'nice', '-n', "$conf::nice_level", 'sbuild', '--batch', "--stats-dir=$main::HOME/stats", - "--dist=$dist" ); + "--dist=$dist", + "--arch-all" ); my $sbuild_gb = '--auto-give-back'; if ($conf::sshcmd) { $sbuild_gb .= "="; Useful commands List packages which needs to be build $ /usr/bin/wb-short/list-needs-build -d stable unknown/glpi_0.68.3-3 [unknown:uncompiled] Total 1 package(s) Give back a package for build (aka retry) $ /usr/bin/wb-short/give-back-build -d stable -U buildd_arch-hostname glpi_0.68.3-3 Remove package state from database $ /usr/bin/wb-short/forget-build -d stable -U buildd_arch ocsinventory-server_1.01-2 binNMU $ cd ~buildd/build $ sbuild --make-binNMU="Rebuild for erase unwelcomed dependency" --binNMU=1 -d unstable -v hello_2.1.1-4
  • "--make-binNMU" defines message to add changelog.
  • "--binNMU" defines minor revision. If you set 1, hello_2.1.1-4.0.1 will be built.
Mutt script for buildd auto-reply

When a build is complete (and successful), a mail is sent (see ~buildd/.sbuildrc). To complete the build and upload the package, the .changes file must be extracted from the build log, must be signed and sent back to the buildd. As this is quite boring, here is some scripts to make this part easier.


First, we will apply the script only to mails stored in the =buildd folder. So we use a folder-hook to associate this folder with a specific script:

folder-hook =buildd source ~/.mutt/defaults.buildd Mutt preferences and send-hook

This is the big part: we set some preferences for how to display the folder, and we set a send-hook to make an almost automatic reply, passing the build log to a shell script to extract the .changes file, and then signing it. Practically, all what you have to do is to press 'r' (reply) on a mail, then 'y' to send it (you will have to enter the passphrase if not in memory).

# -*- muttrc -*- # Use the global defaults first. source ~/.mutt/ # This is the index format for mailing lists. set index_format="%4C %Z %{%b %d} %-15.15n (%4l) %s" # Sort on threads set sort=threads # Give mailing lists a more "slrn" like feel. set pager_index_lines=9 # do not ask for 'to' and 'subject' on reply set fast_reply=yes # do not save a copy of the reply set record="" send-hook '~t buildd ~s success' "set signature=_ pgp_autosign=yes indent_string=_ edit_headers=yes editor='~/bin/' fast_reply=yes pgp_create_traditional=yes include=yes" send-hook '!(~t buildd ~s success)' "set pgp_autosign=no indent_string='> ' editor=vim fast_reply=no pgp_create_traditional=no include=yes"

'Note': please note that I am no mutt expert, so the line can be wrong, or there may be an easier way to do that. In both cases, I'm interested if you have any suggestion.

Extracting changes file from build log

We use a shell script, containing a bit of sed magic to suppress all lines before and after the .changes file:

#!/bin/bash +e cat $1 > ~/.buildd/orig # 11 is the number of lines of your headers in default reply egrep -v '^Fcc:' $1 | sed -e '11,/\.changes\:$/d' -e '/\/.*\.deb\:/,$d' > ~/.buildd/changes cat ~/.buildd/changes > $1 Related Links

# apt-get install wanna-build  buildd   sbuild  quinn-diff
# mkdir -p /root/upload-security
# mkdir -p /root/upload
# mkdir -p /root/build
# touch


  1. #!/bin/sh
  2. user=$LOGNAME
  3. dist=unstable
  4. # default to host architecture. you can also use i386 on amd64 or vice versa
  5. arch=$(dpkg --print-architecture)
  6. mirror=

  7. sbuild-adduser $user
  8. sbuild-update --keygen

  9. # optional, but recommended if not using a local mirror:
  10. echo /var/cache/apt/archives /var/cache/apt/archives none rw,bind 0 0 >>/etc/schroot/sbuild/fstab

  11. mkdir -p /srv/chroot
  12. sbuild-createchroot --keyring=/etc/apt/trusted.gpg --arch=$arch $dist /srv/chroot/$dist-$arch-sbuild $mirror
# sh -x

I: Configuring dpkg-dev...
I: Configuring build-essential...
I: Base system installed successfully.
I: Configured /etc/hosts:
   │ debian-loongson localhost
I: Configured /usr/sbin/policy-rc.d:
   ┌─────────────────────────────────────────────────────────────────────   ┌────────────────────────────────────────────────────────────────────────
   │description=Debian unstable/i386 autobuilder
I: Please rename and modify this file as required.
mkdir /etc/sbuild/chroot
I: sudo chroot configuration linked as /etc/sbuild/chroot/unstable-i386-sbuild.
I: Setting reference package list.
I: Updating chroot.
命中 unstable Release.gpg
获取:1 unstable/main Translation-en [4,008 kB]
获取:2 unstable/main Translation-zh [1,528 B]  
获取:3 unstable/main Translation-zh_CN [134 kB]
命中 unstable Release                                   
获取:4 unstable/main Sources [6,333 kB]                
获取:5 unstable/main i386 Packages/DiffIndex [7,876 B] 
下载 10.5 MB,耗时 2分 20秒 (74.8 kB/s)                                       
正在读取软件包列表... 完成
正在读取软件包列表... 完成
正在分析软件包的依赖关系树... 完成
正在对升级进行计算... 完成
升级了 0 个软件包,新安装了 0 个软件包,要卸载 0 个软件包,有 0 个软件包未被升级。
I: Successfully set up unstable chroot.
I: Run "sbuild-adduser" to add new sbuild users.



* wanna-build (

* buildd (
一个周期性的检查由wanna-build所维护数据库 的后台进程,并且调用sbuild来进行打包。buildd记录打包的成功或者失败的情况,当管理员对打包日志确认之后,它将上传打好的包。

* sbuild (

* quinn-diff (

* andrea


阅读(2775) | 评论(1) | 转发(1) |

fbwfbi2013-11-06 10:17:46