DEB build scripts: major update aimed at simplifying the use for

non-root users.

Signed-off-by: Michele Calgaro <michele.calgaro@yahoo.it>
pull/3/head
Michele Calgaro 7 years ago
parent 77398a607f
commit e324c2e7e4

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,6 +1,6 @@
#!/bin/bash
. ./_build_set_common.sh
. ./internals/_build_set_common.sh
#----------------------------
set_log_start

@ -1,7 +1,7 @@
#!/bin/bash
# Load common code and initialization
. ./_build_common.sh
. ./internals/_build_common.sh
init_common
_BUILDALL_TIMER=2

@ -13,33 +13,34 @@ A) Environment preparation
- this guide has been prepared based on a clean debian testing netinstall system without any other DE.
It should work for other debian/ubuntu distros as well, but eventually you may run into small differences
here and there.
- you can change folder names below, as long as you update the "build_config.sh" file accordingly.
- you can change folder names below, as long as you update the "_config.sh" file accordingly.
1) Install following packages: bc, cdbs, git, links2, pbuilder
1) Install following packages: bc, cdbs, git, links2, pbuilder, rsync and required dependencies.
NOTE: sudo should already be installed. If not, install sudo as well.
2) Create a base folder for TDE, hereafter referred to as TDE_SRC (for example $HOME/tde_src)
2) Create a base folder for TDE, hereafter referred to as TDE_DIR (for example $HOME/tde_src)
3) Create the following folders [ see NOTE at point 0) about folder names ]
- in TDE_SRC: 0_logs : contains log files for repo update and global builds (more on this later)
- in TDE_DIR: 0_logs : contains log files for repo update and global builds (more on this later)
1_git : contains the git repo clones and build hook scripts
2_build: folder used for build preparation and for local builds
3_repo : local repo for package installation (to be configured in /etc/apt/sources.list)
buildscripts: contains a local copy of the build scripts, which can be modified as required
- in TDE_SRC/1_git:
- in TDE_DIR/1_git:
edeps : contains extra dependency modules necessary to build TDE
hooks : contains build hook scripts to execute ad-hoc code before and after the building process.
Mainly used to apply patches automatically during the building process
4) Clone TDE git repositories and extra dependency modules:
- TDE main repo
cd "TDE_SRC/1_git"
cd "TDE_DIR/1_git"
git clone https://mirror.git.trinitydesktop.org/gitea/TDE/tde
cd tde
git submodule init -- scripts
git submodule update -- scripts
./scripts/switch_all_submodules_to_head_and_clean anonymous
- TDE packaging repo
cd "TDE_SRC/1_git"
cd "TDE_DIR/1_git"
git clone https://mirror.git.trinitydesktop.org/gitea/TDE/tde-packaging
- extra dependency packages
using a browser, download the source code for extra dependency modules from Slavek's PPA
@ -52,15 +53,16 @@ A) Environment preparation
the button to update. You should get a list of 10-15 modules, depending on the distro.
* Click on one module at a time, it will expand (or open another page) and show a list of .deb and other files.
* Save those files (exclude .deb files. Only .orig.tar.xz, .dsc, .debian.tar.xz are required) and store
them to TDE_SRC/1_git/edeps/<MODULE NAME> folder (one folder per module).
them to TDE_DIR/1_git/edeps/<MODULE NAME> folder (one folder per module).
5) Add your user to the sudoers group (not required if you are root)
5) Add your user to the sudo group (not required if you are root)
su -c "adduser <username> sudo"
Optionally, consider extending your sudo timeout interval to avoid having to type your password too often (this
could be quite painful especially during long builds, which would not be able to complete if unattended).
Logout and login again to make sure the setting is applied.
6) Copy the files from "TDE_SRC/1_git/tde-packaging/debian/_buildscripts/local/scripts/additional files" folder
6) Copy the files from "TDE_DIR/1_git/tde-packaging/debian/_buildscripts/local/buildscripts/additional files" folder
to the respectivily named folders.
- make sure the files in "/root/.pbuilder/hooks" are executable
7) Create pbuilder base package with the command:
sudo pbuilder --create
@ -68,11 +70,12 @@ A) Environment preparation
sudo DIST=<target distro> ARCH=<target architecture> pbuilder --create
8) Setup the build scripts locally:
- copy build scripts from "TDE_SRC/1_git/tde-packaging/debian/_buildscripts/local/scripts" to "TDE_SRC/buildscripts"
- cd "TDE_SRC/buildscripts"
- cp _build_config_template.sh build_config.sh
- edit "build_config.sh" to set your preferences and check that the various folder names match the structure on your disk.
Make sure to set the variable TDE_SRC to the correct path and the variables DISTRO, DISTRO_NAME and ARCHITECTURE to
- copy build scripts from "TDE_DIR/1_git/tde-packaging/debian/_buildscripts/local/buildscripts" to "TDE_DIR/buildscripts"
and make sure all shell scripts are executable
- cd "TDE_DIR/buildscripts"
- cp ./internals/_config_template.sh _config.sh
- edit "_config.sh" to set your preferences and check that the various folder names match the structure on your disk.
Make sure to set the variable TDE_DIR to the correct path and the variables DISTRO, DISTRO_NAME and ARCHITECTURE to
match the distro and architecture you want to build for.
@ -81,58 +84,52 @@ A) Environment preparation
B) Notes about scripts
----------------------
1) * Notes *
Modules are built using the build_module.sh script. After the build is completed, the installation .deb files are located in TDE_SRC/2_build/debs/<MODULE NAME>/ and the source code and build reports in TDE_SRC/2_build/debs/<MODULE NAME>/src/
Modules are built using the build_module.sh script. After the build is completed, the installation .deb files are located in TDE_DIR/2_build/debs/<MODULE NAME>/ and the source code and build reports in TDE_DIR/2_build/debs/<MODULE NAME>/src/
The source code can be either the git repo or a local copy in TDE_SRC/2_build/build/<MODULE NAME>.
The source code can be either the git repo or a local copy in TDE_DIR/2_build/build/<MODULE NAME>.
A module can be built in a clean chroot environment using pbuilder (default option) or locally using dpkg-buildpackage (useful for quick debugging/developing).
When using pbuilder, a hook can be used to invoke a shell when the build fails.
Build logs are automatically stored to files, but can also be displayed during the build process.
The default location of a module build log is TDE_SRC/2_build/debs/<MODULE NAME>/src/__build__.log
The default location of a module build log is TDE_DIR/2_build/debs/<MODULE NAME>/src/__build__.log
When building sets of modules or the whole TDE, a global build summary is automatically stored to TDE_SRC/0_logs/build_result.log to quickly check what built and what failed. It is recommended to delete that file before starting a new TDE build (if not, build results will be appended at the end of the file).
When building sets of modules or the whole TDE, a global build summary is automatically stored to TDE_DIR/0_logs/build_result.log to quickly check what built and what failed. It is recommended to delete that file before starting a new TDE build (if not, build results will be appended at the end of the file).
2) * Scripts description *
- scripts in 'internals' folder
Scripts used intenally by other scripts. No need for invoking these directly.
- update_git_repository.sh:
Scripts used to update the local clone of the git repositories.
Scripts used to update the local clone of the git repositories.
Usage:
update_git_repository.sh
- build_module.sh: build a single module
- build_module.sh
Build a single module.
Usage:
build_module.sh [options] module_name
Options:
-g (Git) : build from git repo sources. If missing, build from the local copy in build folder.
-l (Local) : build the module locally. If missing, build in a clean chroot environment
-sl (Show Log) : output the building logs to terminal while the build is ongoing
-lr (Log Result) : log (append) build result (OK, FAILED) to TDE_SRC/0_logs/build_result.log file
-lr (Log Result) : log (append) build result (OK, FAILED) to TDE_DIR/0_logs/build_result.log file
-sh (Shell Hook) : use a shell hook for failing builds, only valid if building using pbuilder (clean chroot environment)
-po (Prepare Only): only prepare the source folder but do not build the module. Useful to prepare the source code before
doing local changes/development. The module can then be built from the modified local folder
-d (Debug) : enable debug symbols if possible (debian/rules file must contain "RelWithDebInfo" for this to work)
-ip (Internal Pbuilder): build using internal pbuilder mode (experimental)
- _build_config_template.sh
Template file for building configuration. Will be copied into TDE_SRC/build_config.sh upon first execution, if not already
done when the environment was set up.
- _build_common.sh
Utility script containing common code. No need for invoking this directly.
- _build_set_common.sh
Utility script used to build sets of modules. No need for invoking this directly.
- <dd>_<set_name>.sh
A number of scripts used to build sets of modules. Each script builds an individual set.
Modules are built from the git sources and the build result is automatically appended to
TDE_SRC/0_logs/build_result.log.
A number of scripts used to build sets of modules. Each script builds an individual set.
Modules are built from the git sources and the build result is automatically appended to
TDE_DIR/0_logs/build_result.log.
Usage:
<dd>_<set_name>.sh [options] set_name
Options:
-s N: if specified, skip first N modules from the set of modules
The sets are logically grouped as "base system", "applications" and "others"
The sets are logically grouped as "base system", "applications" and "others"
00_extradeps.sh : extra dependencies modules
01_base_01.sh - 03_base_03.sh : TDE base system
04_application_01.sh - 08_application_05.sh : application modules
@ -141,19 +138,19 @@ The sets are logically grouped as "base system", "applications" and "others"
92_build_applications.sh : all TDE applications in 04_application_01.sh - 08_application_05.sh
93_build_others.sh : all modules in 09_others_01.sh - 10_others_02.sh
- build_TDE.sh
Script used to build the whole TDE at once. Calls the dd_setname.sh set scripts in order.
- 99_build_TDE.sh
Script used to build the whole TDE at once. Calls the dd_setname.sh set scripts in order.
Usage:
build_TDE.sh [options]
Options:
-s N: if specified, skip first N sets from the list of sets
- create_repo.sh
Creates a local repository from the .deb files currently stored in TDE_SRC/2_build/debs. This repo can be used as installation repository, just add the following lines to /etc/apt/sources.list
# --- LOCAL TDE REPOSITORY ---
deb [arch=<YOUR ARCH> trusted=yes] file:TDE_SRC/3_repo <YOUR DISTRO> main
For example:
deb [arch=amd64 trusted=yes] file:/home/tde_src/3_repo/ buster main
Creates a local repository from the .deb files currently stored in TDE_DIR/2_build/debs.
Usage:
create_repo.sh [options]
Options:
-b (Backup) : create a backup of the existing repository in TDE_DIR/CFG_REPO_DIR.backup
3) * Building hooks *
@ -161,7 +158,7 @@ Hooks are available to execute ad-hoc code before and after the build process. F
There are two type of hooks:
- pre_build : applied before switching the module to quilt format and build
- post_build: applied after the build (dpkg-buildpackage or pbuilder) has terminated
To use a hook, create an executable script (pre_build.sh and/or post_build.sh) in the TDE_SRC/1_git/hooks/<MODULE NAME> folder. The scripts are executed in the build_module.sh environment, so have access to all the variables defined in that file. See the files in the "hook examples" folder for real usage samples.
To use a hook, create an executable script (pre_build.sh and/or post_build.sh) in the TDE_DIR/1_git/hooks/<MODULE NAME> folder. The scripts are executed in the build_module.sh environment, so have access to all the variables defined in that file. See the files in the "hook examples" folder for real usage samples.
@ -169,36 +166,42 @@ To use a hook, create an executable script (pre_build.sh and/or post_build.sh) i
C) How to use the scripts
-------------------------
1) Follow the steps in section "A) Environment preparation" (only required the first time).
2) cd "TDE_SRC/buildscripts"
2) cd "TDE_DIR/buildscripts"
3) Update to latest git repository using
./update_git_repository.sh
Wait until "Update completed" is printed out. Check log in TDE_SRC/0_logs/ if you wish.
./update_repositories.sh
Wait until "Update completed" is printed out. Check log in TDE_DIR/0_logs/ if you wish.
4) Run "sudo pbuilder update" at the beginning of the day to update the base package to the latest version. This will speed up
the process when building several modules in sequence.
5) Build modules as per your needs.
- If you are not root, you need to use sudo in front of each command.
5) Build modules as per your needs. You don't need to use "sudo" directly since the scripts will do that automatically if
required. Just type your sudo password when prompted to do so.
6) Create a local repository from the packages you have just built, to be used as installation repository.
Add the following lines to /etc/apt/sources.list and follow the usual way to install TDE.
# --- LOCAL TDE REPOSITORY ---
deb [arch=<YOUR ARCH> trusted=yes] file:TDE_DIR/3_repo <YOUR DISTRO> main
For example:
deb [arch=amd64 trusted=yes] file:/home/tde_src/3_repo buster main
Examples of real usage:
4.1) build a single module
- [sudo] ./build_module.sh -g -sl "dependencies/libr" -> build libr package. This is a good test to check
whether everything is working fine
- [sudo] ./build_module.sh -g "tdelibs" -> build "tdelibs" from git sources in a clean chroot environment
- [sudo] ./build_module.sh -g -l -sl "applications/amarok" -> build "amarok" locally from git sources and
display building logs during building
- [sudo] ./build_module.sh -sh -lr "tdebase" -> build "tdebase" from the local sources (in TDE_SRC/2_build/build/tdebase)
in a clean chroot environment and launch a shell in case of building failure.
Append the build result (OK, FAILED) to TDE_SRC/scripts/logs/build_result.log
- [sudo] ./build_module.sh -g -po "tdelibs" -> prepare "tdelibs" for building from git sources. Source code will be available
in TDE_SRC/2_build/build/tdelibs. After you have made changes to the source and
want to build the modified package, run './build_module.sh "tdelibs"'
4.2) build a single set
(optional) delete the TDE_SRC/0_logs/build_result.log file
[sudo] ./01_base_01.sh : build this set.
[sudo] ./03_base_03.sh -s 3 : build this set but skip the first 3 modules of the set.
4.3) build all TDE
[sudo] ./build_TDE.sh : build all TDE
[sudo] ./build_TDE.sh -s 4 : build all TDE, but skip the first 4 sets
Examples of real usage:
1) build a single module
- ./build_module.sh -g -sl "dependencies/libr" -> build libr package. This is a good test to check
whether everything is working fine
- ./build_module.sh -g "tdelibs" -> build "tdelibs" from git sources in a clean chroot environment
- ./build_module.sh -g -l -sl "applications/amarok" -> build "amarok" locally from git sources and
display building logs during building
- ./build_module.sh -sh -lr "tdebase" -> build "tdebase" from the local sources (in TDE_DIR/2_build/build/tdebase)
in a clean chroot environment and launch a shell in case of building failure.
Append the build result (OK, FAILED) to TDE_DIR/scripts/logs/build_result.log
- ./build_module.sh -g -po "tdelibs" -> prepare "tdelibs" for building from git sources. Source code will be available
in TDE_DIR/2_build/build/tdelibs. After you have made changes to the source and
want to build the modified package, run './build_module.sh "tdelibs"'
2) build a single set
(optional) delete the TDE_DIR/0_logs/build_result.log file
./01_base_01.sh : build this set.
./03_base_03.sh -s 3 : build this set but skip the first 3 modules of the set.
3) build all TDE
./99_build_TDE.sh : build all TDE
./99_build_TDE.sh -s 4 : build all TDE, but skip the first 4 sets

@ -1,53 +1,179 @@
# this is your configuration file for pbuilder.
# the file in /usr/share/pbuilder/pbuilderrc is the default template.
# /etc/pbuilderrc is the one meant for editing.
#
# read pbuilderrc.5 document for notes on specific options.
BASETGZ=/var/cache/pbuilder/base.tgz
BUILDPLACE=/var/cache/pbuilder/build/
#!/bin/bash
# general options
BUILDPLACE="/var/cache/pbuilder/build/"
USEPROC=yes
USEDEVPTS=yes
USEDEVFS=no
BUILDRESULT=/var/cache/pbuilder/result/
# the username and ID used by pbuilder, inside chroot. Needs fakeroot, really
BUILDUSERID=1234
BUILDUSERNAME=pbuilder
# make debconf not interact with user
export DEBIAN_FRONTEND="noninteractive"
DEBEMAIL=""
# for pbuilder debuild (sudo -E keeps the environment as-is)
BUILDSOURCEROOTCMD="fakeroot"
PBUILDERROOTCMD="sudo -E"
# command to satisfy build-dependencies; the default is an internal shell
# implementation which is relatively slow; there are two alternate
# implementations, the "experimental" implementation,
# "pbuilder-satisfydepends-experimental", which might be useful to pull
# packages from experimental or from repositories with a low APT Pin Priority,
# and the "aptitude" implementation, which will resolve build-dependencies and
# build-conflicts with aptitude which helps dealing with complex cases but does
# not support unsigned APT repositories
PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends"
#Command-line option passed on to dpkg-buildpackage.
DEBBUILDOPTS=""
# this is necessary for running 'apt-ftparchive' in a hook script if required
EXTRAPACKAGES="$EXTRAPACKAGES apt-utils"
#APT configuration files directory
APTCONFDIR=""
# the username and ID used by pbuilder, inside chroot. Needs fakeroot, really
BUILDUSERID=1234
BUILDUSERNAME=pbuilder
# Set the debootstrap variant to 'buildd' type.
DEBOOTSTRAPOPTS[0]='--variant=buildd'
# Set the PATH I am going to use inside pbuilder: default is "/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin"
# Set the PATH to be used inside pbuilder
export PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin"
# SHELL variable is used inside pbuilder by commands like 'su'; and they need sane values
# Shell to be used inside pbuilder by commands like 'su'
export SHELL=/bin/bash
# The name of debootstrap command.
DEBOOTSTRAP="debootstrap"
# default PKGNAME_LOGFILE
PKGNAME_LOGFILE="__build__.log"
#-- get desired distribution and architecture
if [ "$DIST" = "" ]; then
DIST=$(lsb_release -cs)
fi
DISTRIBUTION=$DIST
if [ "$ARCH" = "" ]; then
ARCH=$(dpkg --print-architecture)
fi
ARCHITECTURE=$ARCH
if [ -n "$T" ]; then
if [ "${T##*-}" = "vm" ]; then
BUILD_VM="qemu"
T=${T%-vm}
fi
if [ "${T#*@}" = "$T" ]; then
DISTRIBUTION=${T%-*}
ARCHITECTURE=${T##*-}
else
DISTRIBUTION=${T%@*}
ARCHITECTURE=${T##*@}
fi
if [ "${DISTRIBUTION##*-}" = "backports" ]; then
BACKPORTS="yes"
DISTRIBUTION=${DISTRIBUTION%-backports}
fi
fi
BASETGZ="/var/cache/pbuilder/base-$DIST@$ARCH.tgz"
#-- select apt components
case "$DISTRIBUTION" in
sid|buster|stretch|jessie|wheezy|squeeze|lenny|etch|sarge)
COMPONENTS="main contrib non-free"
;;
ceres|beowulf|ascii)
COMPONENTS="main"
DEBOOTSTRAPOPTS=(--variant=buildd --no-check-gpg)
EXTRAPACKAGES="$EXTRAPACKAGES devuan-keyring"
;;
bionic|artful|zesty|yakkety|xenial|wily|vivid|utopic|trusty|saucy|raring|quantal|precise|oneiric|natty|maverick|lucid)
COMPONENTS="main restricted universe multiverse"
DEBOOTSTRAPOPTS=(--variant=buildd)
;;
raspbian-jessie|raspbian-wheezy)
COMPONENTS="main contrib non-free"
DEBOOTSTRAPOPTS=(--variant=buildd --no-check-gpg)
;;
esac
EXTRAPACKAGES="$EXTRAPACKAGES fakeroot apt-transport-https ca-certificates"
#-- select base apt sources
case "$DISTRIBUTION" in
sid|buster|stretch|jessie|wheezy)
MIRRORSITE=http://deb.debian.org/debian
;;
squeeze|lenny|etch|sarge)
MIRRORSITE=http://archive.debian.org/debian
OTHERMIRROR="#deb http://archive.debian.org/backports.org $DISTRIBUTION-backports main contrib non-free"
;;
ceres|beowulf|ascii)
MIRRORSITE=http://auto.mirror.devuan.org/merged
;;
bionic|artful|zesty|yakkety|wily|vivid|trusty|precise)
if [ "$ARCHITECTURE" = "amd64" ] || [ "$ARCHITECTURE" = "i386" ]; then
MIRRORSITE=http://archive.ubuntu.com/ubuntu
else
MIRRORSITE=http://ports.ubuntu.com
fi
;;
xenial)
if [ "$ARCHITECTURE" = "amd64" ] || [ "$ARCHITECTURE" = "i386" ]; then
MIRRORSITE=http://archive.ubuntu.com/ubuntu
else
MIRRORSITE=http://ports.ubuntu.com
fi
OTHERMIRROR="deb $MIRRORSITE $DISTRIBUTION-updates $COMPONENTS"
;;
utopic|saucy|raring|quantal|oneiric|natty|maverick|lucid)
MIRRORSITE=http://old-releases.ubuntu.com/ubuntu
;;
raspbian-jessie|raspbian-wheezy)
MIRRORSITE=http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian
;;
esac
if [ -n "${ARCH}" ]; then
NAME="$NAME-$ARCH"
DEBOOTSTRAPOPTS=("--arch" "$ARCH" "${DEBOOTSTRAPOPTS[@]}")
fi
BUILDRESULT="/var/cache/pbuilder/$DISTRIBUTION/result/"
APTCACHE="/var/cache/pbuilder/aptcache/$DISTRIBUTION/"
if [ -n "$APTCACHE" ] && [ ! -d "$APTCACHE" ]; then
mkdir $APTCACHE
fi
if [ -z "$(stat -L --print "%d\n" $APTCACHE/. /var/cache/pbuilder/build/. | uniq -d)" ]; then
# apt cache for build on tmpfs is managed by hook scripts
BINDMOUNTS="${BINDMOUNTS} ${APTCACHE}"
APTCACHEHARDLINK=no
APTCACHE=""
fi
# default PKGNAME_LOGFILE
PKGNAME_LOGFILE="__build__.log"
#-- extra options
DEBBUILDOPTS="-B"
if [ "$ARCHITECTURE" == "amd64" ]; then
DEBBUILDOPTS="-b"
fi
if [ "${DISTRIBUTION#raspbian}" != "$DISTRIBUTION" ] && [ "$ARCHITECTURE" == "armhf" ]; then
DEBBUILDOPTS="-b"
DISTRIBUTION=${DISTRIBUTION#raspbian-}
fi
if [ -z "$DEB_SIGN_KEYID" ]; then
AUTO_DEBSIGN=${AUTO_DEBSIGN:-no}
fi
#-- choose dependency solver
#-- apt is better if target distribution contains apt >= 1.4~beta3
#-- aptitude is better if target distribution contains apt < 1.4~beta3
case "$DISTRIBUTION" in
sid|buster|stretch|\
ceres|beowulf|ascii|\
bionic|artful|zesty)
PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-apt"
;;
jessie|wheezy|squeeze|lenny|etch|sarge|\
raspbian-jessie|raspbian-wheezy|\
yakkety|xenial|wily|vivid|utopic|trusty|saucy|raring|quantal|precise|oneiric|natty|maverick|lucid)
PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-aptitude"
;;
esac

@ -1,7 +0,0 @@
#!/bin/sh
# invoke shell if build fails.
apt-get install -y --force-yes vim less bash mc
cd /tmp/buildd/*/debian/..
echo "Build FAILED! Invoking shell"
/bin/bash < /dev/tty > /dev/tty 2> /dev/tty

@ -1,152 +0,0 @@
#!/bin/bash
#-- this is necessary for running 'apt-ftparchive' in a hook script if required
EXTRAPACKAGES="$EXTRAPACKAGES apt-utils"
#-- get desired distribution and architecture
if [ "$DIST" = "" ]; then
DIST=$(lsb_release -cs)
fi
DISTRIBUTION=$DIST
if [ "$ARCH" = "" ]; then
ARCH=$(dpkg --print-architecture)
fi
ARCHITECTURE=$ARCH
if [ -n "$T" ]; then
if [ "${T##*-}" = "vm" ]; then
BUILD_VM="qemu"
T=${T%-vm}
fi
if [ "${T#*@}" = "$T" ]; then
DISTRIBUTION=${T%-*}
ARCHITECTURE=${T##*-}
else
DISTRIBUTION=${T%@*}
ARCHITECTURE=${T##*@}
fi
if [ "${DISTRIBUTION##*-}" = "backports" ]; then
BACKPORTS="yes"
DISTRIBUTION=${DISTRIBUTION%-backports}
fi
fi
BASETGZ="/var/cache/pbuilder/base-$DIST@$ARCH.tgz"
#-- select apt components
case "$DISTRIBUTION" in
sid|buster|stretch|jessie|wheezy|squeeze|lenny|etch|sarge)
COMPONENTS="main contrib non-free"
;;
ceres|beowulf|ascii)
COMPONENTS="main"
DEBOOTSTRAPOPTS=(--variant=buildd --no-check-gpg)
EXTRAPACKAGES="$EXTRAPACKAGES devuan-keyring"
;;
bionic|artful|zesty|yakkety|xenial|wily|vivid|utopic|trusty|saucy|raring|quantal|precise|oneiric|natty|maverick|lucid)
COMPONENTS="main restricted universe multiverse"
DEBOOTSTRAPOPTS=(--variant=buildd)
;;
raspbian-jessie|raspbian-wheezy)
COMPONENTS="main contrib non-free"
DEBOOTSTRAPOPTS=(--variant=buildd --no-check-gpg)
;;
esac
EXTRAPACKAGES="$EXTRAPACKAGES fakeroot apt-transport-https ca-certificates"
#-- select base apt sources
case "$DISTRIBUTION" in
sid|buster|stretch|jessie|wheezy)
MIRRORSITE=http://deb.debian.org/debian
;;
squeeze|lenny|etch|sarge)
MIRRORSITE=http://archive.debian.org/debian
OTHERMIRROR="#deb http://archive.debian.org/backports.org $DISTRIBUTION-backports main contrib non-free"
;;
ceres|beowulf|ascii)
MIRRORSITE=http://auto.mirror.devuan.org/merged
;;
bionic|artful|zesty|yakkety|wily|vivid|trusty|precise)
if [ "$ARCHITECTURE" = "amd64" ] || [ "$ARCHITECTURE" = "i386" ]; then
MIRRORSITE=http://archive.ubuntu.com/ubuntu
else
MIRRORSITE=http://ports.ubuntu.com
fi
;;
xenial)
if [ "$ARCHITECTURE" = "amd64" ] || [ "$ARCHITECTURE" = "i386" ]; then
MIRRORSITE=http://archive.ubuntu.com/ubuntu
else
MIRRORSITE=http://ports.ubuntu.com
fi
OTHERMIRROR="deb $MIRRORSITE $DISTRIBUTION-updates $COMPONENTS"
;;
utopic|saucy|raring|quantal|oneiric|natty|maverick|lucid)
MIRRORSITE=http://old-releases.ubuntu.com/ubuntu
;;
raspbian-jessie|raspbian-wheezy)
MIRRORSITE=http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian
;;
esac
if [ -n "${ARCH}" ]; then
NAME="$NAME-$ARCH"
DEBOOTSTRAPOPTS=("--arch" "$ARCH" "${DEBOOTSTRAPOPTS[@]}")
fi
BUILDRESULT="/var/cache/pbuilder/$DISTRIBUTION/result/"
APTCACHE="/var/cache/pbuilder/aptcache/$DISTRIBUTION/"
if [ -n "$APTCACHE" ] && [ ! -d "$APTCACHE" ]; then
mkdir $APTCACHE
fi
if [ -z "$(stat -L --print "%d\n" $APTCACHE/. /var/cache/pbuilder/build/. | uniq -d)" ]; then
# apt cache for build on tmpfs is managed by hook scripts
BINDMOUNTS="${BINDMOUNTS} ${APTCACHE}"
APTCACHEHARDLINK=no
APTCACHE=""
fi
BUILDPLACE="/var/cache/pbuilder/build/"
# default PKGNAME_LOGFILE
PKGNAME_LOGFILE="__build__.log"
#-- extra options
DEBBUILDOPTS="-B"
if [ "$ARCHITECTURE" == "amd64" ]; then
DEBBUILDOPTS="-b"
fi
if [ "${DISTRIBUTION#raspbian}" != "$DISTRIBUTION" ] && [ "$ARCHITECTURE" == "armhf" ]; then
DEBBUILDOPTS="-b"
DISTRIBUTION=${DISTRIBUTION#raspbian-}
fi
if [ -z "$DEB_SIGN_KEYID" ]; then
AUTO_DEBSIGN=${AUTO_DEBSIGN:-no}
fi
#-- choose dependency solver
#-- apt is better if target distribution contains apt >= 1.4~beta3
#-- aptitude is better if target distribution contains apt < 1.4~beta3
case "$DISTRIBUTION" in
sid|buster|stretch|\
ceres|beowulf|ascii|\
bionic|artful|zesty)
PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-apt"
;;
jessie|wheezy|squeeze|lenny|etch|sarge|\
raspbian-jessie|raspbian-wheezy|\
yakkety|xenial|wily|vivid|utopic|trusty|saucy|raring|quantal|precise|oneiric|natty|maverick|lucid)
PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-aptitude"
;;
esac

@ -4,15 +4,23 @@
#
# Load common code and initialization
. ./_build_common.sh
. ./internals/_build_common.sh
init_common
# Timer settings
_BUILDMOD_TIMER_NUM=0
_BUILDMOD_TIME="--/--:--:--:---"
exec_time_start $_BUILDMOD_TIMER_NUM
# Need sudo for non-root users
SUDO_CMD=""
if [ "`whoami`" != "root" ]; then
SUDO_CMD="sudo -E"
fi
#----------------------------
# Parameters:
# $1 - error code
@ -40,7 +48,7 @@ function do_exit()
function recreate_folder()
{
if [ -d "$1" ]; then
rm -R "$1"
$SUDO_CMD rm -R "$1"
fi
mkdir -p "$1"
}
@ -97,33 +105,18 @@ if [ "bool_BUILD_LOCALLY" = "y" ]; then
bool_SHELL_HOOK="n"
bool_INTERNAL_PBUILDER="n"
fi
# pbuilder absolute paths
PBUILDER_HOOK_DIR="$HOME/.pbuilder/hooks"
PBUILDER_SHELL_HOOK="$PBUILDER_HOOK_DIR/C10shell"
PBUILDER_SHELL_HOOK_TEMPLATE="$PBUILDER_HOOK_DIR/__template_C10shell"
PBUILDER_DEPS_HOOK="$PBUILDER_HOOK_DIR/D05deps"
PBUILDER_DEPS_HOOK_SAVE="$PBUILDER_HOOK_DIR/__saved_D05deps"
export bool_SHELL_HOOK
export bool_INTERNAL_PBUILDER
# Local option variables
# - internal pbuilder
OPT_INTERNAL_PBUILDER=""
if [ "$bool_INTERNAL_PBUILDER" = "y" ]; then
OPT_INTERNAL_PBUILDER="--use-pdebuild-internal"
fi
# - sign packages
OPT_SIGN_PKG_LOCAL="-uc -us"
OPT_SIGN_PKG_PBUILDER=""
if [ ! -z "$GPG_SIGN_KEYID" ]; then
OPT_SIGN_PKG_LOCAL="-k$GPG_SIGN_KEYID"
OPT_SIGN_PKG_PBUILDER="--auto-debsign --debsign-k $GPG_SIGN_KEYID"
fi
# - show logs
OPT_SHOW_LOGS="&>"
if [ "$bool_SHOW_BUILD_LOGS" = "y" ]; then
OPT_SHOW_LOGS=" | tee "
fi
export OPT_SHOW_LOGS
# Log start
echo -e "${CLightCyan}#### Processing module \"$MOD_NAME\" ####${CNone}"
@ -154,7 +147,7 @@ done
# Build output directories
#----------------------------
TDE_BUILD_DIR="$TDE_DIR/$CFG_TDE_BUILD_DIR"
TDE_DEBS_DIR="$TDE_DIR/$CFG_TDE_DEBS_DIR"
export TDE_DEBS_DIR="$TDE_DIR/$CFG_TDE_DEBS_DIR"
BUILD_DIRS=("TDE_BUILD_DIR" "TDE_DEBS_DIR")
@ -218,14 +211,13 @@ else
fi
fi
#----------------------------
# Prepare source files
#----------------------------
# remove output from previous build
if [ -d "$MOD_BUILD_PATH" ]; then
cd "$MOD_BUILD_PATH/.."
rm *.deb *.dsc *.changes *.tar.bz2 *.tar.gz *.tar.xz *.log *.buildinfo &>/dev/null
$SUDO_CMD rm *.deb *.dsc *.changes *.tar.bz2 *.tar.gz *.tar.xz *.log *.buildinfo &>/dev/null
fi
# copy main repo source files, if needed
@ -249,7 +241,7 @@ if [ "$bool_COPY_MOD_SRC" = "y" ]; then
# Extra dependency module
if [ `find "$MOD_GIT_PATH" -name '*.dsc' | wc -l` == 1 ]; then
if [ -d "$MOD_BUILD_PATH" ]; then
rm -R "$MOD_BUILD_PATH"
$SUDO_CMD rm -R "$MOD_BUILD_PATH"
fi
eval dpkg-source --no-copy --no-check -x `find "$MOD_GIT_PATH" -name '*.dsc'` \
\"$MOD_BUILD_PATH\" $OPT_SHOW_LOGS/dev/null
@ -267,7 +259,7 @@ fi
# copying packaging scripts, if needed
if [ "$bool_EXTRADEP_MOD" != "y" -a "$bool_COPY_PKGING_FILES" = "y" ]; then
if [ -d "$MOD_BUILD_PKGING_PATH" ]; then
rm -R $MOD_BUILD_PKGING_PATH
$SUDO_CMD rm -R $MOD_BUILD_PKGING_PATH
fi
cp -R "$MOD_GIT_PKGING_PATH" "$MOD_BUILD_PKGING_PATH"
@ -327,7 +319,7 @@ if [ "$bool_EXTRADEP_MOD" != "y" -a "$bool_COPY_PKGING_FILES" = "y" ]; then
fi
# prepare destination directory for building
MOD_DEB_PATH="$TDE_DEBS_DIR/$MOD_NAME"
export MOD_DEB_PATH="$TDE_DEBS_DIR/$MOD_NAME"
MOD_DEBSRC_PATH="$MOD_DEB_PATH/src"
recreate_folder "$MOD_DEB_PATH"
recreate_folder "$MOD_DEBSRC_PATH"
@ -390,57 +382,26 @@ fi
#----------------------------
# Build
#----------------------------
BUILDING_LOG_FILE="$MOD_DEBSRC_PATH/__build__.log"
export BUILDING_LOG_FILE="$MOD_DEBSRC_PATH/__build__.log"
cd "$MOD_BUILD_PATH"
if [ "$bool_BUILD_LOCALLY" = "y" ]; then
## Build module locally
echo -e "${CYellow}> Building locally${CNone}"
eval dpkg-buildpackage $OPT_SIGN_PKG_LOCAL $OPT_SHOW_LOGS\"$BUILDING_LOG_FILE\"
build_retval=$?
#
build_retval=$?
else
## Build module in a clean chroot environment using pbuilder
# Install/remove shell hook
if [ "$bool_SHELL_HOOK" = "y" ]; then
if [ -x "$PBUILDER_SHELL_HOOK_TEMPLATE" ]; then
cp "$PBUILDER_SHELL_HOOK_TEMPLATE" "$PBUILDER_SHELL_HOOK" &>/dev/null
else
echo "pbuilder shell hook template file not found! ($PBUILDER_SHELL_HOOK_TEMPLATE)"
fi
else
if [ -x "$PBUILDER_SHELL_HOOK" ]; then
rm "$PBUILDER_SHELL_HOOK" &>/dev/null
fi
fi
# Build using pbuilder
echo -e "${CYellow}> Building using pbuilder${CNone}"
# Create pbuilder hook to make sure all available packages are scanned
# Store any existing D05 hook as a temporary file, this will be reinstated at the end
if [ -x "$PBUILDER_DEPS_HOOK" ]; then
mv "$PBUILDER_DEPS_HOOK" "$PBUILDER_DEPS_HOOK_SAVE"
fi
cat <<END_D05 > "$PBUILDER_DEPS_HOOK"
#!/bin/sh
(cd "$TDE_DEBS_DIR"; apt-ftparchive packages . > Packages)
echo "deb [trusted=yes] file://$TDE_DEBS_DIR ./" >> /etc/apt/sources.list
apt-get update
END_D05
chmod a+x "$PBUILDER_DEPS_HOOK"
# Build
eval pdebuild $OPT_INTERNAL_PBUILDER $OPT_SIGN_PKG_PBUILDER \
--architecture $ARCHITECTURE \
--buildresult \"$MOD_DEB_PATH\" \
--pbuilderroot \"sudo DIST=$DISTRO_NAME ARCH=$ARCHITECTURE\" \
--logfile \"$BUILDING_LOG_FILE\" \
-- \
--bindmounts \"$TDE_DEBS_DIR\" \
--hookdir \"$PBUILDER_HOOK_DIR\" \
$OPT_SHOW_LOGS\"$BUILDING_LOG_FILE\"
build_retval=$?
if [ -x "$PBUILDER_DEPS_HOOK" ]; then
mv "$PBUILDER_DEPS_HOOK_SAVE" "$PBUILDER_DEPS_HOOK"
fi
$SUDO_CMD "$SCRIPT_DIR/internals/_pbuilder.sh"
build_retval=$?
if [ "`whoami`" != "root" ]; then
cd "$MOD_DEB_PATH"
$SUDO_CMD chown -R `id -u -n`:`id -g -n` . &>/dev/null
cd "$MOD_BUILD_PATH/.."
$SUDO_CMD chown `id -u -n`:`id -g -n` *.dsc *.changes *.tar.xz *.tar.bz2 *.tar.gz *.buildinfo &>/dev/null
$SUDO_CMD chown `id -u -n`:`id -g -n` "$TDE_DEBS_DIR/Packages" &>/dev/null
cd "$MOD_BUILD_PATH"
fi
fi
if [ $build_retval -eq 0 ]; then

@ -4,7 +4,7 @@
#
# Load common code and initialization
. ./_build_common.sh
. ./internals/_build_common.sh
init_common
#----------------------------

@ -0,0 +1,18 @@
#!/bin/sh
# Color table
CPurple='\e[0;35m'
CWhite='\e[1;37m'
CNone='\e[0m'
# Invoke shell if build fails.
apt-get install -y --force-yes vim less bash mc
cd /tmp/buildd/*/debian/..
echo "${CPurple}---------------------------------------------------------${CNone}" >/dev/tty
echo "${CPurple} !!! Build FAILED !!! ${CNone}" >/dev/tty
echo "${CPurple} You are now inside a shell in the building environment. ${CNone}" >/dev/tty
echo "${CPurple} ${CWhite}mc, vim, less${CPurple} are available for your convenience. ${CNone}" >/dev/tty
echo "${CPurple} Type ${CWhite}exit${CPurple} to leave the shell when you are done. ${CNone}" >/dev/tty
echo "${CPurple}---------------------------------------------------------${CNone}" >/dev/tty
/bin/bash </dev/tty >/dev/tty 2>/dev/tty

@ -13,6 +13,7 @@
# Brown 0;33 Yellow 1;33
# Light Gray 0;37 White 1;37
# No Color 0
set -a
CBlack='\e[0;30m'
CDarkGray='\e[1;30m'
CBlue='\e[0;34m'
@ -30,13 +31,13 @@ CYellow='\e[1;33m'
CGray='\e[0;37m'
CWhite='\e[1;37m'
CNone='\e[0m'
set +a
#----------------------------
function init_common()
{
# Check script folder
SCRIPT_DIR=$(dirname $(readlink -f "$0"))
export SCRIPT_DIR=$(dirname $(readlink -f "$0"))
# Prevent the script to be run from TDE packaging repo
REPO_URL=$(git config --get remote.origin.url 2>/dev/null)
@ -48,14 +49,14 @@ function init_common()
fi
# Read config settings
CFG_FILE=$SCRIPT_DIR/build_config.sh
CFG_FILE=$SCRIPT_DIR/_config.sh
if [ -f "$CFG_FILE" ]; then
. "$CFG_FILE"
else
echo -e "${CYellow} --- NOTE ---${CNone}"
echo "Creating TDE build configuration file from template as $CFG_FILE."
echo "Please check and modify as required, then rerun this script."
cp "$SCRIPT_DIR/_build_config_template.sh" "$CFG_FILE"
cp "$SCRIPT_DIR/internals/_config_template.sh" "$CFG_FILE"
exit 2
fi
@ -91,7 +92,7 @@ function init_common()
fi
# Make sure we have selected a supported distribution
DISTS_FILE=$SCRIPT_DIR/distro_list.txt
DISTS_FILE=$SCRIPT_DIR/internals/distro_list.txt
if [ ! -f "$DISTS_FILE" ]; then
echo -e "${CYellow} --- NOTE ---${CNone}"
echo "Could not find the list of supported distributions."
@ -120,8 +121,8 @@ function init_common()
exit 7
fi
SCRIPT_LOG_DIR=$TDE_DIR/$CFG_SCRIPT_LOG_DIR
LOG_RESULT_FILENAME="$SCRIPT_LOG_DIR/build_result.log" # Log result into the common build logfile
export SCRIPT_LOG_DIR=$TDE_DIR/$CFG_SCRIPT_LOG_DIR
export LOG_RESULT_FILENAME="$SCRIPT_LOG_DIR/build_result.log" # Log result into the common build logfile
cd "$SCRIPT_DIR"
}

@ -1,7 +1,7 @@
#!/bin/bash
# Load common code and initialization
. ./_build_common.sh
. ./internals/_build_common.sh
init_common
_BUILDSET_TIMER=1

@ -1,4 +1,5 @@
#!/bin/bash
set -a
#----------------------------
# Build information
@ -36,3 +37,6 @@ CFG_HOOK_DIR="$CFG_GIT_DIR/$CFG_HOOKS_DIR"
#----------------------------
CFG_TDE_BUILD_DIR="$CFG_BUILD_DIR/build"
CFG_TDE_DEBS_DIR="$CFG_BUILD_DIR/debs"
#----------------------------
set +a

@ -0,0 +1,75 @@
#!/bin/bash
function run_pdebuild()
{
# pbuilder absolute paths
PBUILDER_HOOK_DIR="/var/cache/pbuilder/hooks"
PBUILDER_SHELL_HOOK_TEMPLATE="$SCRIPT_DIR/internals/C10shell"
PBUILDER_SHELL_HOOK="$PBUILDER_HOOK_DIR/C10shell"
PBUILDER_DEPS_HOOK="$PBUILDER_HOOK_DIR/D05deps"
# Local option variables
# - internal pbuilder
OPT_INTERNAL_PBUILDER=""
if [ "$bool_INTERNAL_PBUILDER" = "y" ]; then
OPT_INTERNAL_PBUILDER="--use-pdebuild-internal"
fi
# Sign packages optino
OPT_SIGN_PKG_PBUILDER=""
if [ ! -z "$GPG_SIGN_KEYID" ]; then
OPT_SIGN_PKG_LOCAL="-k$GPG_SIGN_KEYID"
OPT_SIGN_PKG_PBUILDER="--auto-debsign --debsign-k $GPG_SIGN_KEYID"
fi
## Build module in a clean chroot environment using pbuilder
# Install/remove shell hook
if [ ! -d "$PBUILDER_HOOK_DIR" ]; then
mkdir -p "$PBUILDER_HOOK_DIR"
fi
if [ "$bool_SHELL_HOOK" = "y" ]; then
cp "$PBUILDER_SHELL_HOOK_TEMPLATE" "$PBUILDER_SHELL_HOOK"
chmod a+x "$PBUILDER_SHELL_HOOK"
else
if [ -x "$PBUILDER_SHELL_HOOK" ]; then
rm "$PBUILDER_SHELL_HOOK"
fi
fi
# Build using pbuilder
echo -e "${CYellow}> Building using pbuilder${CNone}"
# Create pbuilder hook to make sure all available packages are scanned
# Store any existing D05 hook as a temporary file, this will be reinstated at the end
cat <<END_D05 > "$PBUILDER_DEPS_HOOK"
#!/bin/sh
(cd "$TDE_DEBS_DIR"; apt-ftparchive packages . > Packages)
echo "deb [trusted=yes] file://$TDE_DEBS_DIR ./" >> /etc/apt/sources.list
apt-get update
END_D05
chmod a+x "$PBUILDER_DEPS_HOOK"
# Build
eval pdebuild $OPT_INTERNAL_PBUILDER $OPT_SIGN_PKG_PBUILDER \
--architecture $ARCHITECTURE \
--buildresult \"$MOD_DEB_PATH\" \
--pbuilderroot \"sudo DIST=$DISTRO_NAME ARCH=$ARCHITECTURE\" \
--logfile \"$BUILDING_LOG_FILE\" \
-- \
--bindmounts \"$TDE_DEBS_DIR\" \
--hookdir \"$PBUILDER_HOOK_DIR\" \
$OPT_SHOW_LOGS\"$BUILDING_LOG_FILE\"
PBUILDER_RETVAL=$?
# Remove shell hook if it was installed before the build
if [ -x "$PBUILDER_SHELL_HOOK" ]; then
rm "$PBUILDER_SHELL_HOOK"
fi
# Remove deps hook
if [ -x "$PBUILDER_DEPS_HOOK" ]; then
rm "$PBUILDER_DEPS_HOOK"
fi
# Return pdebuild return value to calling function
return $PBUILDER_RETVAL
}
#----------------------------
# The actual code is inside a function to allow the pdebuild return value to be
# correctly passed back to the calling script in all cases (root and non root users)
run_pdebuild

@ -1,12 +1,5 @@
#!/bin/bash
# Load common code
. ./_build_common.sh
#----------------------------
# Initialization
init_common
#----------------------------
# Update TDE main repository
echo -e "\n---- Updating main TDE GIT repo ----\n"

@ -1,10 +1,7 @@
#!/bin/bash
# Load common code
. ./_build_common.sh
#----------------------------
# Initialization
. ./internals/_build_common.sh
init_common
#----------------------------
@ -14,6 +11,6 @@ if [ ! -d "$SCRIPT_LOG_DIR" ]; then
exit 1
fi
echo "Update in progress..."
./_update_repositories-int.sh &>"$SCRIPT_LOG_DIR/update-repos.log"
. ./internals/_update_repositories.sh &>"$SCRIPT_LOG_DIR/update-repos.log"
echo "Update completed."

Loading…
Cancel
Save