Comparing Linux 2.4 and Linux 2.6 Kernels:
Many customers ask for support of Linux Kernel version 2.6 for their embedded projects. This is easy enough to understand: after all, 2.6 is the latest and greatest version of the Linux kernel, and who wants to base a new project on old stuff? They are surprised when we tell them that we don't recommend to use Linux 2.6 for most embedded systems.
Here is a comparison of 2.4 and 2.6 on real embedded hardware that explains why:
Build and Test Environment, Kernel Versions and Test Hardware:
The ELDK 3.1 toolchain was used to build the Linux kernels and to provide the necessary test environment (NFS root filesystem, ramdisk images).
The following versions of the Linux Kernel were used in the tests:
- Kernel Version 2.4.25 from CVS server at snapshot of 2005-04-23-00:00
- Kernel plus
The following hardware was used for the tests:
- Sandpoint 8240 (MPC8240 at 132 MHz CPU clock, 64 MB RAM)
- TQM860L (MPC860-T at 50 MHz CPU clock, 16 MB RAM)
Building the Kernels:
2.4.25:- Sandpoint Board:
-> export CROSS_COMPILE=ppc_82xx- -> make sandpoint_config -> make oldconfig -> time make -j8 dep uImage ... Image Name: Linux-2.4.25 Created: Sat Apr 23 16:11:01 2005 Image Type: PowerPC Linux Kernel Image (gzip compressed) Data Size: 909254 Bytes = 887.94 kB = 0.87 MB ... real 1m17.691s user 4m28.245s sys 0m28.346s -> size vmlinux text data bss dec hex filename 1838116 125624 172708 2136448 209980 vmlinux
- : Linux 2.4.25 config file for Sandpoint board
- : Linux 2.4.25 U-Boot image for Sandpoint board
- TQM860L Board:
-> export CROSS_COMPILE=ppc_8xx- -> cp ../config-2.4.25-TQM860L .config -> make oldconfig -> time make -j8 dep uImage ... Image Name: Linux-2.4.25 Created: Sat Apr 23 17:04:44 2005 Image Type: PowerPC Linux Kernel Image (gzip compressed) Data Size: 507688 Bytes = 495.79 kB = 0.48 MB ... real 0m50.348s user 2m55.328s sys 0m19.589s -> size vmlinux text data bss dec hex filename 1025524 50008 142596 1218128 129650 vmlinux
- : Linux 2.4.25 config file for TQM860L board
- : Linux 2.4.25 U-Boot image for TQM860L board
2.6.11.7:
- Sandpoint Board:
-> export CROSS_COMPILE=ppc_82xx- -> make ARCH=ppc CROSS_COMPILE=ppc_82xx- sandpoint_defconfig -> time make -j8 ARCH=ppc CROSS_COMPILE=ppc_82xx- uImage ... Image Name: Linux-2.6.11.7 Created: Sat Apr 23 16:42:11 2005 Image Type: PowerPC Linux Kernel Image (gzip compressed) Data Size: 1150438 Bytes = 1123.47 kB = 1.10 MB ... real 1m41.571s user 5m48.509s sys 0m39.444s -> size vmlinux text data bss dec hex filename 2253870 226316 93720 2573906 274652 vmlinux
- : Linux 2.6.11.7 config file for Sandpoint board
- : Linux 2.6.11.7 U-Boot image for Sandpoint board
- TQM860L Board:
-> export CROSS_COMPILE=ppc_8xx- -> cp ../config-2.6.11.7-TQM860L .config -> make ARCH=ppc CROSS_COMPILE=ppc_8xx- oldconfig -> time make -j8 ARCH=ppc CROSS_COMPILE=ppc_8xx- uImage ... Image Name: Linux-2.6.11.7 Created: Sat Apr 23 17:14:19 2005 Image Type: PowerPC Linux Kernel Image (gzip compressed) Data Size: 723718 Bytes = 706.76 kB = 0.69 MB ... real 1m12.198s user 4m11.517s sys 0m29.351s -> size vmlinux text data bss dec hex filename 1497745 79248 64248 1641241 190b19 vmlinux
- : Linux 2.6.11.7 config file for TQM860L board
- : Linux 2.6.11.7 U-Boot image for TQM860L board
Results:
| real | 77.7s | 101.6s | + 31 % |
| user | 268.2s | 348.6s | + 30 % |
| sys | 28.3s | 39.4s | + 39 % |
| Compressed Kernel | 887.9kB | 1123.5kB | + 27 % |
| Uncompressed kernel | 2086.4kB | 2513.6kB | + 20 % |
| real | 50.3s | 72.2s | + 43 % |
| user | 175.3s | 251.5s | + 43 % |
| sys | 19.6s | 29.4s | + 50 % |
| Compressed Kernel | 495.8kB | 706.8kB | + 43 % |
| Uncompressed kernel | 1189.6kB | 1602.8kB | + 35 % |
Booting Linux:
The kernel images were booted in the same environment with root filesystem over NFS in an idle network. The timings were recorded in the following log files:
- : Timed bootlog for Linux 2.4.25 on TQM860L board
- : Timed bootlog for Linux 2.4.25 on Sandpoint board
- : Timed bootlog for Linux 2.6.11.7 on TQM860L board
- : Timed bootlog for Linux 2.6.11.7 on Sandpoint board
Results:
| Time from begin of loading the kernel image into RAM ... | 2.4.25 | 2.6.11.7 | Delta |
|---|---|---|---|
| ... until "Linux version" message | |||
| TQM860L | 1.9s | 2.0s | + 6 % |
| Sandpoint | 13.0s | 15.0s | + 15 % |
| ... until "Freeing unused kernel memory" message (= enter user space) | |||
| TQM860L | 3.7s | 3.8s | + 1 % |
| Sandpoint | 23.0s | 22.0s | - 4 % |
| ... until "login:" message (= full multi-user mode) | |||
| TQM860L | 49.7s | 58.0s | + 17 % |
| Sandpoint | 56.2s | 59.6s | + 6 % |
The slight advantage for the 2.6 when booting is at least partially due to the fact that the configurations were slightly different: in 2.4, the SCSI driver was enabled, which was not the case in 2.6
Running with ramdisk based root file system:
The systems were booted using the ramdisk images provided by the ELDK. The main purpose was to find out how much memory remained free for application code. A simple memory allocator () was used to test for free (= allocateable by an application process) memory.
2.4.25:
- Sandpoint Board:
- TQM860L Board:
... Memory: 13396k available (892k kernel code, 252k data, 52k init, 0k highmem) ... # free total used free shared buffers Mem: 15032 7112 7920 0 108 Swap: 0 0 0 Total: 15032 7112 7920 # ./mem_eat ... Got total = 9240576 = 9024kB __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) VM: killing process mem_eat
- Sandpoint Board:
- TQM860L Board:
... Memory: 12980k available (1240k kernel code, 288k data, 88k init, 0k highmem) ... # free total used free shared buffers Mem: 14676 6572 8104 0 4092 Swap: 0 0 0 Total: 14676 6572 8104 # ./mem_eat ... Got total = 8519680 = 8320kB oom-killer: gfp_mask=0xd2 ...
Results:
| Kernel Code + Data | kB | kB | + % |
| total "free" | kB | kB | - % = - kB |
| Available for malloc() | kB | kB | - % = - kB |
| Kernel Code + Data | 1144kB | 1528kB | + 34 % |
| total "free" | 15032kB | 14676kB | - 2 % = - 356kB |
| Available for malloc() | 9024kB | 8320kB | - 8 % = - 704kB |
Running the lmbench benchmark:
To get some more extensive performance data the lmbench benchmark (version 3.0-a4) was run on the systems). The give a pretty clear and somewhat disappointing picture:
- Even in tests that should not depend on the OS environment at all (like basic integer or float operations) the 2.6 kernel 1...3% slower thanthe 2.4 kernel. This is probably casued by the bigger kernel size which, especially in combination with the small caches on embedded processors, results in some loss of performance.
- Context switching which is essential for the efficient operation of a multi-tasking OS is on average 55% slower (range: 10...94%).
- Local communication latencies are significantly slower, too: 80% for pipes, 79% for UNIX domain sockets, 34% for UDP, 12% for RPC over UDP, 25% for TCP, 12% for RPC over TCP, and 25% for TCP connections.
- File system and VM system latencies go up dramatically, too: 76% for file creation, 17% for file deletion, 32% for the mmap() latency, 6% for page faults and 2% for select() on 100 file descriptors.
- Local communication bandwidth suffers hard, too, but the results differ depending on the processor used:
- Pipes become slightly (3%) faster on the MPC860, but reach only 49% on the MPC8240
- UNIX domain sockets are 11 slower on the MPC8240, but same speed on MPC860
- TCP is 2% faster on the MPC8240, but 31% slower on the MPC860
- File re-reads drop by 6%
Note that some tests did not run under 2.6 on the MPC860 system because the memory was exhausted (the test system had only 16 MB RAM).
Management Summary:
Using the 2.6 kernel on embedded systems implicates the following disadvantages:- Slow to build: 2.6 takes 30...40% longer to compile
- Big memory footprint in flash: the 2.6 compressed kernel image is 30...40% bigger
- Big memory footprint in RAM: the 2.6 kernel needs 30...40% more RAM; the available RAM size for applications is 700kB smaller
- Slow to boot: 2.6 takes 5...15% longer to boot into multi-user mode
- Slow to run: context switches up to 96% slower, local communication latencies up to 80% slower, file system latencies up to 76% slower, local communication bandwidth less than 50% in some cases.
There may be a few cases where the use of Linux kernel version 2.6 makes sense even for embedded systems (typically on "bigger" systems with more powerful processors), but if memory footprint or system performance are important you probably want to stick with a 2.4 kernel for now.
-- - 24 Apr 2005