Speeding Up the Buildroot Build Process

Posted Sep 18, 2024

Speeding up buildroot build

By Nayab Sayed

4 min read

I frequently need to build a fresh Buildroot target image, and it can be quite time-consuming, particularly when dealing with toolchain-related changes that necessitate starting from scratch.

According to Buildroot’s documentation, there are few ways to expedite the build process for the Buildroot system. The analysis that follows was conducted by building a target image for the Raspberry Pi 4 using it’s default configuration on a 16 thread AMD machine.

The typical steps for building an embedded Linux image for the Raspberry Pi 4 using Buildroot are as follows:

  
git clone https://gitlab.com/buildroot.org/buildroot.git
cd buildroot/
make raspberrypi4_64_defconfig
make -j`nproc`

The build process took approximately 37m12s to complete.

Speeding up the initial build

Using top level parallel build

By default, using make -jN isn’t effective in Buildroot. However, there’s an experimental feature that can be enabled to make it effective. To do this, run make menuconfig, search for the string BR2_PER_PACKAGE_DIRECTORIES, enable it, and save the configuration.

Now, run make clean && make -j`nproc` to perform a top-level parallel build.

The build process took approximately 26m7s to complete. For this testing purpose, intentionally I deleted download directory so that all the sources will be downloaded again.

Using the external SDK

You have the option to instruct Buildroot to use an external toolchain. If you don’t have one available, Buildroot can even create one for you.

Buildroot SDK creation steps

First, Be sure to make a record of the toolchain settings from make menuconfig -> Toolchain. You can also take a screenshot for future reference.
In the System configuration -> Init system section, choose None.
In the System configuration -> /bin/sh section, choose none.
Disable Target packages -> Busybox section.
Disable File system images -> tar the root filesystem
Run make sdk

The SDK/toolchain will be generated in the output/images directory. In my case, it’s located at output/images/aarch64-buildroot-linux-gnu_sdk-buildroot.tar.gz. Make sure to copy this file to another location. Rasberry Pi 4 Buildroot Toolchain Configuration

Using the external toolchain

Go to make menuconfig -> Toolchain,

Set Toolchain type to External toolchain
Set Toolchain to Custom toolchain
Set Toolchain origin to Toolchain to be downloaded and installed
Set Toolchain URL to the SDK downloaded path. In my case it is: file:///home/nayab/aarch64-buildroot-linux-gnu_sdk-buildroot.tar.gz
Set External toolchain gcc version -> Taken from above step. (ex: 11.x)
Set External toolchain kernel headers series -> Taken from above step. (ex: 5.10.x)
Set External toolchain C library -> Taken from above step (ex: glibc)
Set Toolchain has C++ support? -> Taken from above step. (ex: Enabled in my case)
Set Toolchain has RPC support? -> Disabled.

After pointing Buildroot to the external toolchain, run the command make clean && make -j`nproc` .The build process took approximately 14m47s to complete. For this testing purpose, intentionally I deleted download directory before running make command.

Speeding up incremental/parallel builds

Using the compiler cache (ccache)

To enable ccache, navigate to make menuconfig -> Build options -> Enable compiler cache. With ccache enabled, the build process stores object files created from source files in the default cache directory $HOME/.buildroot-ccache. Any subsequent or incremental builds with ccache enabled will reduce build time nicely.

In my case, when I initiated an incremental build using the commands make clean and make -j`nproc`, the entire build process only took 5 minutes and 24 seconds. That’s incredibly fast. Now also, the dl directory was deleted manually before building.

Note that, make clean won’t delete compiler ccache directory. This need to be deleted manually if you want to get rid of it.

The compiler cache can be shared among multiple parallel builds, resulting in a significant reduction in build times, no matter how many builds are concurrently taking place.

Re-using the download directory

This is especially useful when you need to build for multiple targets simultaneously. In Buildroot, there’s a configuration option, BR2_DL_DIR, where you can set the value to use a shared downloads path for all target builds. I configured this to use the downloads directory where all the files were already downloaded.

The build process took approximately 3m4s to complete.

Buildroot Building out of tree feature lets you construct images for multiple targets using the same Buildroot source tree. This way source cloning (ex: git clone) time will be saved.

For instance, assuming the current working directory is the Buildroot source tree:

In one Terminal tab, run:

  
mkdir rpi; make O=rpi raspberrypi4_64_defconfig

In another Terminal tab, run:

  
mkdir mchp; make O=mchp microchip_sam9x60ek_mmc_defconfig

Now there are two output directories, named rpi and mchp, for each target, all utilizing the same Buildroot source tree.

Comparison table

BR2_PER_PACKAGE_DIRECTORIES	make -j`nproc`	SDK	ccache	Downloads	Build Time
Disabled	Yes	Compile	Disabled	NA	37m12s
Enabled	Yes	Compile	Disabled	NA	26m7s
Enabled	Yes	External	Disabled	NA	14m47s
Enabled	Yes	External	Enabled	NA	5m24s
Enabled	Yes	External	Enabled	Available	3m4s

Miscellaneous

Investing in better hardware can significantly reduce build times. Consider getting a faster SSD and increasing RAM.
Building within a virtual machine can lead to notably poor performance.

References

linux, buildroot

This post is licensed under CC BY 4.0 by the author.