Previous: Quick start Table of Contents Next: Profiles

larch – a do-it-yourself live Arch Linux CD



Overview of the larch build system


Warning

Before we get started I should point out that most of the scripts need to be run by the 'root' user. That is quite normal for an installer, but it is somewhat risky - a slight slip and all sorts of damage could be done. Ideally you would have a machine specially set aside for this sort of thing, i.e. one without any important data on it. You have been warned!

Installation of the larch build system

The larch package may be installed in the normal Arch Linux way using pacman. This method will only work on an Arch system, of course.

Alternatively, larch may be used without installing it. This should also work on non-Arch Linux systems. Download larch-setup to an empty working directory and run it (it is a shell script). This will in turn download and unpack the current larch package, setting up symlinks so that the build scripts can be run from this working directory. If you don't put this new directory on your 'PATH', you may well need to add a path prefix when running it (e.g. './mklarch' if your current directory is the one containing the script).

Using the larch build system

For building a live medium from a 'profile' you need, primarily, mklarch; for building from an existing installation (including rebuilding after using mklarch) larchify. mklarch uses inpacs to do the actual installation of the packages, and the functions for performing the live CD creation are in the file (larch/)buildlive.

Unlike a normal installation, that used by larch need not be a separate partition, it can be placed anywhere convenient. The default installation directory ('/home/larchroot', variable 'INSTLDIR' in the scripts) should normally be acceptable (the building work is done in the '.larch' sub-directory). Note that lots of space is necessary in ${INSTLDIR}, nearly 4GB for a 700MB CD.

Quick mklarch overview

That first stage is very much like a normal Arch installation to a partition - which is another possibility with the inpacs script. A couple of small tweaks are made to this base installation to make it more convenient for live CD/USB use: some glibc locales are generated (set by the file locale.gen in the profile) and, if openssh is installed, the initial ssh-keys are generated.

Before mklarch can be used a 'profile' for the live medium must be built. This is a folder containing configuration files determining which packages are installed, which glibc locales are installed, and generally which files should be customized. Further details are given in 'Profiles'.

A simple way to build an iso would then be:

./mklarch -p myprofile

To build a live USB-stick using GRUB to boot and using the pacman databases as they exist on the build host (rather than the default of downloading the latest databases using 'pacman -Sy'):

./mklarch -p myprofile -y /var/lib/pacman -ug

The build process can be stopped after the Arch system installation (before building the 'squashed' live system) by passing the '-a' option to mklarch. All the available options can be seen by running 'mklarch -h'.

The rest of the build process is handled by the functions in buildlive, primarily mklive, which puts all the various components together into a CD image at '${INSTLDIR}/.larch/cd', then using the build_iso function to create the iso or (using the usbboot script) to prepare a bootable USB-stick.

If you set up your configuration wrong (or if you or I made some other mistake ...), you might find you have destroyed some important data and/or your system doesn't work any more. But that's life - Share and Enjoy!

Quick larchify overview

larchify uses the functions in buildlive to build a live medium from an existing Arch Linux installation. It can also be used to rebuild a (posibly altered) live medium from the installation arising from the use of mklarch. For example, after building an iso using mklarch, you might want to build another with the same packages but with a changed overlay - maybe an alteration to rc.conf. In that case you could make the necessary changes to the profile and call:

./larchify -rp myprofile
The '-r' option causes the old squashed image to be used, which can save a lot time. To see all the options run './larchify -h'.

Minimal build system requirements

The larch scripts have been designed to work without heavy demands on the build system. Although they have been developed under Arch Linux, they should run on other GNU/Linux systems. The required software has been kept to a minimum by means of a sort of bootstrapping - some of the build functions are carried out on the newly installed Arch system using chroot. For example, you do not need support for squashfs or aufs on the build system. bash, mkisofs and wget are assumed to be available (for building a USB-stick version with syslinux you also need mkfs.vfat from the 'dosfstools' package). If you are not running Arch Linux, the installation script (larch-setup) will set up pacman using a special package available from the larch repository (pacman-packed.tar.gz), which contains in addition to the pacman binary the run-time libraries it needs.

'pacman.conf' for building and live system use

You also need an appropriate 'pacman.conf' file. On a non-Arch system, unless a pacman.conf is already present in the working directory, this will be created afresh and a dialog will be presented by larch-setup to select a mirror. Because of potential overwriting (see below) this file is also copied to pacman.conf.0 (also in the working directory).

On an Arch system it is assumed that you have already configured your mirrors appropriately, and the system '/etc/pacman.conf' can be taken.

When there is a 'pacman.conf' in the profile (base directory, not overlay), this will be copied to the working directory and used by mklarch for performing the installation, even if there was already a version in the working directory. The 'pacman.conf' in the resulting live system will also be this version if built using mklarch (i.e. taken from the resulting file in the working directory). If larchify is used the existing '/etc/pacman.conf' within the system to be larchified will be used. However, if the overlay in the profile has its own '/etc/pacman.conf' this will (in both cases) be used in preference.

Additionally, an 'IgnorePkg' line is added to the live system's '/etc/pacman.conf', for 'kernel26', 'aufs' and 'initscripts'. Packages listed in the 'noupdate' file in the profile, will also be added. This line will be removed on installation using larchin.

squashfs and aufs

The newly installed system is made into a squashed file-system in the file system.sqf. This is mounted as the lower layer of a union (aufs) file-system. On top of that there is a writable tmpfs so that the file-system as a whole appears writable. An 'overlay' archive is copied into the writable layer. As this top layer is compressed using lsop, this package must be installed in the target. Of course only the tmpfs top layer can actually be written to, and its contents disappear when the system reboots, so the writing is only temporary.

There is also a second overlay archive, a squashed file system which forms the middle layer of the union file-system, which increases the flexibillity of the session saving facility.

In order to boot into a system constructed in that manner, you need an initramfs which deals with mounting all the various bits in the right way before entering the normal boot sequence. In larch the initramfs is built using the standard Arch 'mkinitcpio' system, for which special 'hooks' (essentially code plug-ins) have been developed to manage the requirements of a larch live system.

aufs as an alternative to unionfs

larch was initially designed around unionfs, but since then aufs has been generally a more reliable alternative, and is now available as a package in Arch Linux. In order to use aufs its kernel module needs to be installed (which, if using mklarch, can be achieved by placing package 'aufs' in addedpacks), the rest will then be handled automatically. N.B. As unionfs hasn't been tested for a long time (all development work is done using aufs) and the current 'overlay merge' only works with aufs, it is strongly recommended that aufs be used.

Partial rebuilding

If you make some changes to a build which don't require a renewed installation (the Arch installation in '${INSTLDIR}'), you can use the larchify script to rebuild the system. Such changes might be tweaks to the overlay, or 'rcconfx', for example (but note that the glibc locales are in the installation so if you want to change these, you have to rerun the whole installation, or handle that manually). If - as is likely - the squashed system from the previous build is still there and is still valid, this can be reused by passing the '-r' option (thus saving the time required to squash the system).

If you only make changes to files in '${INSTLDIR}/.larch/cd', you can rebuild the iso using the '-i' option to larchify. But be aware that some of the files in the directories 'isolinux' (when using the isolinux bootloader) and 'boot' (when using the GRUB bootloader) are renewed from 'larch/cd-root' and the 'cd-root' directory in the profile.

N.B. Even though the Arch installation is in principle now redundant, you shouldn't delete it if you want to do a quick rebuild using larchify, because this installation is used (via 'chroot') for some of the build functions.

Custom packages

It is possible to include your own 'custom' packages in the installation. Maybe ones you have compiled yourself, e.g. from the AUR, or modified versions (fixes or customizations) of standard packages. To do this you need to put your packages in a directory and run gen-repo - which requires python - on this directory (run it without arguments to get usage instructions). Then place an entry for this new repository in your 'pacman.conf' (in the profile or else in the working directory). If your packages replace some in the existing repositories, your custom repository needs to come before those repositories in 'pacman.conf'. Any packages you want installed now just need to be listed in addedpacks.

It is not necessary to build a custom kernel for larch, the standard kernel ('kernel26') can be used. It includes unionfs and squashfs modules, and is now included in the set of 'base' packages. If aufs is desired (which is likely, as at the time of writing, unionfs didn't work very well), this must be included in addedpacks.

Generating the basepacks list

In case the base package set changes, inpacs uses the list provided by 'pacman -Sg base' by default. However, if there is a 'basepacks' file in the profile this will be used to supply the list of base packages (one package per line).

inpacs can also be used separately for Arch installation. To get a full listing of possible options run it without arguments (note that it pays no attention to profiles, all options must be passed explicitly).

It is possible to filter out certain unwanted packages from the base set. In most cases the default setting will be satisfactory, but you can change it by placing a 'baseveto' file (one package per line) in the profile directory. Be aware, however, that vetoed packages will be installed anyway if they are required by some other installed package.

addedpacks

This group of packages is the main place for defining your system. Here you can enter all the applications you would like in your live CD/USB system (and subsequently installed to a hard disk partition, if that was your intention). Thanks to pacman you don't need to sort out dependencies, these should all be included automatically. It is (still) possible to squeeze (or rather 'squash') quite a respectable quantity of software onto a CD - my first experiments included most of KDE, OpenOffice, K3b, GIMP, and quite a lot besides.

Installation of the initial system

The foundation of the larch live CD/USB system is an initial, potentially unconfigured Arch Linux system squashed up in the file 'system.sqf'. The installation of this system is can be handled by mklarch, which uses inpacs, which in turn uses pacman to install all the constituent packages using the '-r' option to install to the chosen location, which can be anywhere where there is enough space.

By default the package cache on the build host will be used, so that only packages which have not already been downloaded will be fetched from the chosen mirror. The '-c' option to mklarch allows another cache location to be used. Note that this can also be on a remote machine, mounted using sshfs or NFS.

Making a live CD from an existing Arch installation

As mentioned above, larchify can be used to make a live medium from an existing Arch installation, using the mount point of the installation as argument. The installation must already be mounted, including any sub-mounts (e.g. /home on another partition). The main mount must be with options 'exec,dev', because some of the building is done via a chroot to the installation.

It can also 'livify' the currently running (Arch only!) system, by passing '/' as argument.

All aspects of the larch building process connected with the installation are in this case irrelevant, as it is assumed that the installation is already complete. The handling of 'profiles' is also different as these are largely concerned with the installation process. A profile may still be used, though, maybe to customize the 'mkinitcpio.conf' used to build the live system's initramfs, or to add extra content by means of the cd-root folder, for example.

In order to support building a larch system, certain packages outside the Arch base group must be installed in the system to be 'larchified':

squashfs-tools, lzop, larch-live,
aufs (if - recommended - you want to use that instead of unionfs),
syslinux (if using isolinux or syslinux),
cdrkit (for building an iso),
eject (to eject a CD at shutdown).

for the hard-disk installer - larchin, python, pygtk,
parted, ntfsprogs,
(optional, but recommended) gparted.

for this documentation, and for the capability of doing complete
rebuilds - larch

Note that some things in '/var' will not be included in the 'live' system. Firstly, the standard pacman package cache, '/var/cache/pacman/pkg'. Also the log files (in '/var/log') and '/var/tmp' (temporary files, like '/tmp') are not saved. As some files in '/var/log' are required for certain aspects of logging to function, these are recreated (as empty files) in the resulting 'system.sqf'.

Previous: Quick start Table of Contents Next: Profiles