Initrd Size Issues

What is an INITRD?
A small file used to create a small environment inside a ram disk that is used to prepare the system before starting the full system. The main task is to load any drivers needed to mount the rootfs, then mount it and start the system.

More elaborate systems can also run in an intrd/initramfs. For example, the Debian netinstallers run entirely from an initrd image.

see https://wiki.debian.org/initramfs

Why are they used?
Using an initrd/initramfs allows the use of a smaller, more generic kernel by loading any needed kernel modules needed to mount the rootfs from a custom initrd/initramfs rather than compiling every possible disk/filesystem driver into the kernel. This saves on memory which has historically been vital and remains important for embedded systems. It also allows for other complex/dynamic operations such as an installer or recovery environment to operate without disks/etc.

Address passed by Bootloader
On ARM systems the bootloader typically passes the memory address where the initrd/initramfs is loaded inside of a data structure that it passes to the kernel (ATAGs). The kernel typically ignores this if the address is passed via the kernel cmdline ("INITRD=...."). It also passes the size of the initrd/initramfs.

see: http://www.simtec.co.uk/products/SWLINUX/files/booting_article.html#ATAG_INITRD2

passed by bootloader
The address can be set via a INITRD= element to the kernel command line. In the case of arm devices and U-Boot this is passed via the ATAG_CMDLINE tag.

hardcoded in kernel
You can hard code a cmdline at compile time by setting "CONFIG_CMDLINE".

set/appended by device-tree
The kernel will include cmdline items that are specified in the "choosen" section of the device tree. To use that command line I believe you must either disable ATAG support (CONFIG_ATAGS) or tell it to merge the ATAG and DTB command line (CONFIG_ARM_ATAG_DTB_COMPAT_CMDLINE_EXTEND). I believe the default behavior of CONFIG_ARM_ATAG_DTB_COMPAT is to overwrite the DTB command line.

Why do they fail with newer kernels?
The size of the linux kernel has greatly increased over the years. The bootloaders found in many older ARM devices were built/configured with assumptions about kernel size which no longer work with today's kernel. This also applies to kernel modules that go into initrd files as well as the growth of the utilities inside an initrd.

For some of the oldest (orion5x) devices I've worked with uboot will crash if you try to load a file larger than 7MB. Looking at it happen via serial console and comparing the error to the source code that is available it appears to be an issue with the Marvell Sata driver in uboot for those older versions. I don't have a solution for that issue beyond working to reduce initrd sizes where possible.

Another issue typical of these systems is simply loading the initrd to close to the beginning of memory. The compressed kernel decompresses itself to the beginning of memory, if it determines that the decompressed kernel will overwrite the compressed kernel it first relocates itself down farther into memory before it begins. That process does not take the location of the initrd into account and will overwrite the initrd if it is stored above (beginning of memory + decompressed kernel size + 1mb reserved for DTB + compressed kernel size) ....give or take.

reduce size by excluding components
The simplest way to make the initrd/initramfs smaller is to limit the kernel modules that are included via configuration:

/usr/share/initramfs-tools/conf.d/modules:

MODULES=dep

Modern versions of initramfs-tools includes a lot of libraries to support features like kernel signing which take up a lot of space and are usually not supported by older embedded devices. There are alternate systems for generating initrd/initramfs that are likely smaller, you can also manually build an even more minimum one though I don't have experience with that currently.

see https://www.linuxfromscratch.org/blfs/view/svn/postlfs/initramfs.html

reduce size with compression
Debian's initramfs-tool defaults to gzip compression, the size of the initrd/initramfs can be reduced dramatically by switching to XZ compression. This requires a lot more CPU time to compress and a lot more memory to decompress.

The following settings have been found to provide the smallest possible file while still working on devices with as little as 64MB ram.

/usr/share/initramfs-tools/conf.d/compress:

COMPRESS=xz XZ_OPT=-2e export XZ_OPT