分类： LINUX
2010-09-20 01:31:50
In recent months I played with QEMU emulation of an ARM Versatile Platform Board, making it run , the boot-loader and a complete with a . I tried to put everything together to emulate a complete boot procedure, but it was not so simple. What follows is a description of what I’ve done to emulate a complete boot for an emulated ARM system, and the applied principles can be easily transferred to other different platforms.
qemu-system-arm: can be installed on Ubuntu with “sudo apt-get install qemu-kvm-extras“, on Debian with “aptitude install qemu” as root.mkImage: can be installed with the package uboot-mkimage. Alternatively, it is compiled from U-Boot source.arm-none-eabi toolchain: can be downloaded from the the pagezImage: the Linux kernel created in my rootfs.img.gz: the Busybox-based file system created in my On real, physical boards the boot process usually involves a non-volatile memory (e.g. a Flash) containing a boot-loader and the operating system. On power on, the core loads and runs the boot-loader, that in turn loads and runs the operating system. QEMU has the possibility to emulate Flash memory on many platforms, but not on the VersatilePB. There are that can add flash support, but for now I wanted to leave QEMU as it is.
QEMU can load a Linux kernel using the -kernel and -initrd
options; at a low level, these options have the effect of loading two
binary files into the emulated memory: the kernel binary at address 0x10000 (64KiB) and the ramdisk binary at address 0x800000 (8MiB). Then QEMU prepares the kernel arguments and jumps at 0x10000 (64KiB)
to execute Linux. I wanted to recreate this same situation using
U-Boot, and to keep the situation similar to a real one I wanted to
create a single binary image containing the whole system, just like
having a Flash on board. The -kernel option in QEMU will be
used to load the Flash binary into the emulated memory, and this means
the starting address of the binary image will be 0x10000 (64KiB).
Understanding memory usage during the boot process is important
because there is the risk of overwriting something during memory copy
and relocation. One feature of U-Boot is self-relocation, which means
that on execution the code copies itself into another address, which by
default is 0x1000000 (16MiB). This feature comes handy in
our scenario because it frees lower memory space in order to copy the
Linux kernel. The compressed kernel image size is about 1.5MiB, so the
first 1.5MiB from the start address must be free and usable when U-Boot
copies the kernel. The following figure shows the solution I came up
with:
Timeline of the memory usage
At the beginning we have three binary images together: U-Boot (about
80KiB), Linux kernel (about 1.5MiB) and the root file system ramdisk
(about 1.1MiB). The images are placed at a distance of 2MiB, starting
from address 0x10000. At run-time U-boot relocates itself to address 0x1000000, thus freeing 2MiB of memory from the start address. The U-Boot command bootm then copies the kernel image into 0x10000 and the root filesystem into 0x800000; after that then jumps at the beginning of the kernel, thus creating the same situation as when QEMU starts with the -kernel and -initrd options.
The problem with this solution is that U-Boot, when configured to be
built for VersatilePB, does not support ramdisk usage, which means that
it does not copy the ramdisk during the bootm command, and
it does not give any information about the ramdisk to the kernel. In
order to give it the functionality I need, I patched the original source
code of U-Boot before compilation. The following code is the patch to
apply to u-boot-2010.03 source tree:
01 | diff -rupN u-boot-2010.03.orig/common/image.c u-boot-2010.03/common/image.c |
02 | --- u-boot-2010.03.orig/common/image.c 2010-03-31 23:54:39.000000000 +0200 |
03 | +++ u-boot-2010.03/common/image.c 2010-04-12 15:42:15.911858000 +0200 |
04 | @@ -941,7 +941,7 @@ int boot_get_ramdisk (int argc, char *ar |
05 | return 1; |
06 | } |
07 |
08 | -#if defined(CONFIG_B2) || defined(CONFIG_EVB4510) || defined(CONFIG_ARMADILLO) |
09 | +#if defined(CONFIG_B2) || defined(CONFIG_EVB4510) || defined(CONFIG_ARMADILLO) || defined(CONFIG_VERSATILE) |
10 | /* |
11 | * We need to copy the ramdisk to SRAM to let Linux boot |
12 | */ |
13 | diff -rupN u-boot-2010.03.orig/include/configs/versatile.h u-boot-2010.03/include/configs/versatile.h |
14 | --- u-boot-2010.03.orig/include/configs/versatile.h 2010-03-31 23:54:39.000000000 +0200 |
15 | +++ u-boot-2010.03/include/configs/versatile.h 2010-04-12 15:43:01.514733000 +0200 |
16 | @@ -124,8 +124,11 @@ |
17 | #define CONFIG_BOOTP_SUBNETMASK |
18 |
19 | #define CONFIG_BOOTDELAY 2 |
20 | -#define CONFIG_BOOTARGS "root=/dev/nfs mem=128M ip=dhcp "\ |
21 | - "netdev=25,0,0xf1010000,0xf1010010,eth0" |
22 | +/*#define CONFIG_BOOTARGS "root=/dev/nfs mem=128M ip=dhcp "\ |
23 | + "netdev=25,0,0xf1010000,0xf1010010,eth0"*/ |
24 | +#define CONFIG_BOOTARGS "root=/dev/ram mem=128M rdinit=/sbin/init" |
25 | +#define CONFIG_BOOTCOMMAND "bootm 0x210000 0x410000" |
26 | +#define CONFIG_INITRD_TAG 1 |
27 |
28 | /* |
29 | * Static configuration when assigning fixed address |
I also changed the boot arguments (CONFIG_BOOTARGS)so that they are the same as those given from QEMU command line, and then added a command (CONFIG_BOOTCOMMAND) to start the Linux boot automatically. To apply the patch:
~/u-boot-2010.03.patch~/u-boot-2010.03cd into the source tree directorypatch -p1 < ~/u-boot-2010.03.patch“After applying the patch, U-Boot can be built as seen :
1 | make CROSS_COMPILE=arm-none-eabi- versatilepb_config |
2 | make CROSS_COMPILE=arm-none-eabi- all |
The building process will create a u-boot.bin image that supports ramdisks for the VersatilePB. Incidentally, it will also build the mkimage executable in the tools directory; it can be used instead of the one installed with Debian/Ubuntu packages.
As I said earlier, I need to create a flash image in which the three
binary images are placed at a distance of 2MiB. U-Boot needs to work
with binary images wrapped with a custom header, created using the mkimage tool. After creating the Linux and root file system images, we can write them inside a big binary at a given address with the dd command. Assuming that we have in the same directory: u-boot.bin, zImage and rootfs.img.gz, the list of commands to run are:
1 | mkimage -A arm -C none -O linux -T kernel -d zImage -a 0x00010000 -e 0x00010000 zImage.uimg |
2 | mkimage -A arm -C none -O linux -T ramdisk -d rootfs.img.gz -a 0x00800000 -e 0x00800000 rootfs.uimg |
3 | dd if=/dev/zero of=flash.bin bs=1 count=6M |
4 | dd if=u-boot.bin of=flash.bin conv=notrunc bs=1 |
5 | dd if=zImage.uimg of=flash.bin conv=notrunc bs=1 seek=2M |
6 | dd if=rootfs.uimg of=flash.bin conv=notrunc bs=1 seek=4M |
These commands do the following:
zImage.uimg and rootfs.uimg, that contain also information on where to relocate themflash.binu-boot.bin at the beginning of flash.binzImage.uimg at 2MiB from the beginning of flash.binrootfs.uimg at 4MiB from the beginning of flash.binAt the end we have a binary image, flash.bin, containing the memory layout that I had in mind.
To boot Linux we can finally call:
1 | qemu-system-arm -M versatilepb -m 128M -kernel flash.bin -serial stdio |
The U-Boot-related messages will appear on the console:
01 | U-Boot 2010.03 (Apr 12 2010 - 15:45:31) |
02 |
03 | DRAM: 0 kB |
04 | ## Unknown FLASH on Bank 1 - Size = 0x00000000 = 0 MB |
05 | Flash: 0 kB |
06 | *** Warning - bad CRC, using default environment |
07 |
08 | In: serial |
09 | Out: serial |
10 | Err: serial |
11 | Net: SMC91111-0 |
12 | Hit any key to stop autoboot: 0 |
13 | ## Booting kernel from Legacy Image at 00210000 ... |
14 | Image Name: |
15 | Image Type: ARM Linux Kernel Image (uncompressed) |
16 | Data Size: 1492328 Bytes = 1.4 MB |
17 | Load Address: 00010000 |
18 | Entry Point: 00010000 |
19 | ## Loading init Ramdisk from Legacy Image at 00410000 ... |
20 | Image Name: |
21 | Image Type: ARM Linux RAMDisk Image (uncompressed) |
22 | Data Size: 1082127 Bytes = 1 MB |
23 | Load Address: 00800000 |
24 | Entry Point: 00800000 |
25 | Loading Kernel Image ... OK |
26 | OK |
27 |
28 | Starting kernel ... |
29 |
30 | Uncompressing Linux... done, booting the kernel. |
Then the Linux kernel will execute inside the emulated screen and the message “Please press Enter to activate this console”
will appear, indicating that the root file system is working and so the
boot process completed successfully. If something doesn’t work, one can
always check that the system works without U-Boot, with the following
command:
1 | qemu-system-arm
-M versatilepb -m 128M -kernel zImage -initrd rootfs.img.gz -append
"root=/dev/ram mem=128M rdinit=/sbin/init" -serial stdio |
The kernel should uncompress and execute up to the activation of the console.
This procedure has room for improvements and optimizations, for
example there’s too much memory copying here and there, where mostly
everything can be executed in place. It is anyway a nice exercise and a
good starting point that reveals interesting details about the boot
process in embedded systems. As usual, this is possible mainly due to
the fact that all the tools are free and open source.
Posted on 2010/03/22 by
Last time I experimented on programs and ; now I want to compile a Linux kernel for an ARM architecture from scratch. I don’t have a physical ARM device handy, so I’m using QEMU instead, as I’ve already done before.
Both the mainline kernel and QEMU support the platform, so that’s the target I chose for my tests. The toolchain I’ll be using is the . [edit] From version 2010q1 of the toolchain, the manual explicitly says that the compiler is not intended for Linux kernel development; it is anyway possible to use the GNU/Linux toolchain for the same scope. [/edit]
The vanilla kernel can be downloaded from ; I took the latest at the moment () and extracted it in a folder. From that folder I ran:
1 | make ARCH=arm versatile_defconfig |
This command sets a predefined configuration, used in compilation, that is capable of building a kernel targeted to run inside the VersatilePB board. I wanted to tweak it a little bit, so I ran:
1 | make ARCH=arm menuconfig |
I removed module support (for simplicity) and enabled EABI support as a binary format (allowing also old ABI). This is necessary to run software compiled with the CodeSourcery toolchain. I exited the menu and saved the configuration, then I ran:
1 | make ARCH=arm CROSS_COMPILE=arm-none-eabi- all |
[edit] If using the GNU/Linux toolchain, the command that must be used is, instead:
1 | make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- all |
[/edit]
This will start the building of the kernel using the correct ARM
compiler; the build will create, among other binaries, a compressed
kernel in a file called zImage located in “arch/arm/boot“. This image can be run in QEMU using the following command (assuming that you are in the “arch/arm/boot” directory):
1 | qemu-system-arm -M versatilepb -m 128M -kernel zImage |
QEMU will execute the Linux image: the kernel will display many boot messages and then it will complain that it didn’t find a root filesystem. Let’s then create the simplest filesystem we can do: it consists of a single “Hello World” executable, that can be built using the .
1 | #include |
2 |
3 | void main() { |
4 | printf("Hello World!\n"); |
5 | while(1); |
6 | } |
Note: an infinite loop is introduced because when Linux executes the first program in the root filesystem, it expects that the program does not exit.
Having the GNU/Linux ARM toolchain installed (be aware that it is different from the bare EABI toolchain) I ran:
1 | arm-none-linux-gnueabi-gcc -static test.c -o test |
This creates an executable ELF program for ARM, statically linked
(all the libraries that it needs are linked together in a single
binary). We can now create a simple filesystem using the cpio tool as follows:
1 | echo test | cpio -o --format=newc > rootfs |
The cpio tool takes a list of files and outputs an archive; the newc format is the format of the initramfs filesystem, that the Linux kernel recognizes. The rootfs
file in our case is a binary image of a filesystem containing a single
file, that is the test ELF program. QEMU can pass the filesystem binary
image to the kernel using the initrd parameter; the kernel must also know that the root filesystem will be located in RAM (because that’s where QEMU writes the initrd binary) and that the program to launch is our test executable, so the command line becomes:
1 | qemu-system-arm -M versatilepb -m 128M -kernel zImage -initrd rootfs -append "root=/dev/ram rdinit=/test" |
The QEMU window will show the boot messages we saw before, but at the end of the execution a “Hello World!” will be displayed. The next step would be to create a working filesystem with a command shell and at least basic functionality.
Posted on 2010/03/27 by
is a solution for embedded Linux designs that need a compact
filesystem: the trick is compiling and linking many system utilities
into a single binary that behaves differently based on the name it was
used to execute it. A working Linux root filesystem then consists in a
small directory tree (/bin, /sbin, /usr/bin, …), a single executable binary in /bin/busybox, and many symbolic links to the Busybox binary (/bin/ls, /bin/sh, /sbin/ifconfig, …), and using a typical configuration it can be as small as 2MB (1MB if compressed).
I compiled a Linux kernel and a minimal root filesystem containing just a “Hello World” program. Now we can build a reasonably working root filesystem and test it using . Note that, in order to follow this example, you need:
qemu on Debian or qemu-kvm-extras on Ubuntu)libncurses5-dev) to compile the menu configurationI downloaded and extracted it. Inside the extracted directory I ran:
1 | $ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- defconfig |
A default configuration file is created. To change it to our needs:
1 | $ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- menuconfig |
I checked the option to compile Busybox as a static executable, so
that we don’t have to copy the dynamic libraries inside the root
filesystem. The setting can be found in “Busybox Settings --> Build Options“. Then, the following command builds Busybox and creates a directory called _install containing the root filesystem tree:
1 | $ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- install |
We can now create a root filesystem image using the cpio tool, and in order to compact the filesystem even more, we can run gzip on it to create a compressed image:
1 | $ cd _install |
2 | $ find . | cpio -o --format=newc > ../rootfs.img |
3 | $ cd .. |
4 | $ gzip -c rootfs.img > rootfs.img.gz |
In my case, the compressed image size is about 1MB. To test Busybox,
we can emulate an ARM platform with QEMU using the following command,
assuming we copy the Linux kernel zImage in the current directory:
1 | $ qemu-system-arm -M versatilepb -m 128M -kernel zImage -initrd rootfs.img.gz -append "root=/dev/ram rdinit=/bin/sh" |
The Linux kernel will boot, and the shell /bin/sh will be executed as specified by the boot parameter rdinit, showing the common “#” prompt. The shell can be used normally, for example you can run ls to find the same directory structure of the Busybox _install directory, but using commands like ps and mount we can see that not everything is in place: both programs complain abount the /proc directory. We can create and populate the /proc directory running these commands inside the QEMU emulated system prompt:
1 | # mkdir /proc |
2 | # mount -t proc none /proc |
After that, the ps and mount programs work fine. We can also note that the /dev directory is almost empty (the only exception being the console device). To populate it we need to mount also the /sys directory, so that we can use the mdev tool; run inside QEMU:
1 | # mkdir /sys |
2 | # mount -t sysfs none /sys |
3 | # mdev -s |
The /sys and /dev directory are now populated. To execute these steps every time, we can use /sbin/init
functionality: this program is usually the first run by the Linux
kernel, and its default behavior is to execute the initialization file
with path /etc/init.d/rcS. In the host computer this time, in the folder where we compiled Busybox, we create the missing directories:
1 | $ cd _install |
2 | $ mkdir proc sys dev etc etc/init.d |
3 | $ cd .. |
Now we create a new _install/etc/init.d/rcS with the following content:
1 | #!/bin/sh |
2 | mount -t proc none /proc |
3 | mount -t sysfs none /sys |
4 | /sbin/mdev -s |
We can then recreate an updated root filesystem compressed image as before, and run:
1 | $ qemu-system-arm -M versatilepb -m 128M -kernel zImage -initrd rootfs.img.gz -append "root=/dev/ram rdinit=/sbin/init" |
The Linux kernel will boot again, but this time it is asking to press
Enter, and when we do the usual console prompt will appear. The system
directories are fully populated, meaning that the script rcS has been executed; other than that, /sbin/init also spawns shell terminals using getty.
In our example the Busybox filesystem is accessed using a ram disk,
but it can also be read from a physical disk storage or loaded from the
network using NFS protocol. Maybe another time I will describe in
details how to run Busybox with NFS inside QEMU.
Posted on 2010/04/27 by
Developing a root filesystem for embedded Linux architectures can present some inconveniences: for example if the filesystem should be loaded into the embedded system every time and it’s not always an easy/quick task, or if the disk space is very low. The Linux kernel has the ability to read the root filesystem from the network, specifically from a NFS share, and this can be exploited to solve these problems. A common setup is having an embedded board running a Linux kernel, connected via Ethernet to a server with a directory exported through NFS and a cross-compiler installed. Testing a new root filesystem becomes as easy as resetting the board. When using QEMU as a target instead of a real board, the inconvenience is similar: the root filesystem image should be created every time. Fortunately, QEMU offers a network model that allows to access the host as thought it was connected with an Ethernet network. The details of the default connection can be found in the documentation; briefly, a virtual LAN is created where the guest virtual system is given address 10.0.2.15 while the host workstation can be accessed using address 10.0.2.2.
qemu-system-arm: can be installed on Ubuntu with “sudo apt-get install qemu-kvm-extras“, on Debian with “aptitude install qemu” as rootNFS server: can be installed on Ubuntu or Debian with the nfs-kernel-server packagecurses developer libraries: can be installed on Ubuntu or Debian with the libncurses-dev packagearm-none-linux-gnueabi toolchain: can be downloaded from the the pagezImage: the Linux kernel created in my Busybox _install: the Busybox-based file system created in my We will create an exported folder with our user permissions: in this
way the user is free to compile ARM software and copy it inside the
shared folder. For QEMU to be able to read/write the folder, it should
gain the same access permissions as our user. This can be done using
NFS functionality that maps the anonymous access to a custom user. Find
your uid and gid numeric values with the id command; for example, mine are both 1000. Then from the command line, as root, run:
1 | # mkdir -p /srv/nfs/ |
2 | # chown 1000:1000 /srv/nfs |
Then edit the file /etc/exports adding the following line:
1 | /srv/nfs 127.0.0.1(rw,sync,no_subtree_check,all_squash,insecure,anonuid=1000,anongid=1000) |
Run the command as root:
1 | # exportfs -av |
You can check that the folder is effectively exported by running as root:
1 | # mkdir ~/nfs_test |
2 | # mount localhost:/srv/nfs ~/nfs_test |
3 | # umount ~/nfs_test |
Now we can populate the folder with an ARM root filesystem; we will use a Busybox root tree, that can be built by following my , and add some initialization files. Copy the _install directory that is generated with Busybox build into the NFS share, in order to have the /srv/nfs/_install folder containing the roof filesystem. Then create some missing folders as your user (not as root!):
1 | $ cd /srv/nfs/_install |
2 | $ mkdir -p dev proc sys etc/init.d |
The first program that the Linux kernel executes is /sbin/init;
this program reads a setting file (if available) to know what to do. In
our case we want the initialization simply to run a shell script and
open a console. Create the file /srv/nfs/_install/etc/inittab with the following contents:
1 | ::sysinit:/etc/init.d/rcS |
2 | ::respawn:/sbin/getty -n -l /bin/sh -L 115200 tty1 vt100 |
3 | ::restart:/sbin/init |
The rcS file, as seen in my
about Busybox, can be used to populate the system directories. In our
case, since the root tree is owned by a user in the host workstation, we
cannot create device nodes; a possible workaround is to mount a
temporary filesystem into the /dev directory. Create a new file called /srv/nfs/_install/etc/init.d/rcS with execute permissions (using “chmod +x“), that includes the following content:
1 | mount -t proc none /proc |
2 | mount -t sysfs none /sys |
3 | mount -t ramfs ramfs /dev |
4 | /sbin/mdev -s |
Now we are ready to test the boot process. Run the following command from the directory where the Linux kernel image is located:
1 | $
qemu-system-arm -M versatilepb -m 128M -kernel zImage -append
"root=/dev/nfs nfsroot=10.0.2.2:/srv/nfs/_install rw
ip=10.0.2.15::10.0.2.1:255.255.255.0 init=/sbin/init" |
The kernel should perform the boot and then a shell should appear. You can try to create a file inside QEMU and see that it appears also in the host system; note that the file’s owner is the user set in the exports file. At the same time, you can write files into the exported folders and they can be accessed by the virtual guest ARM system as well.
To troubleshoot the NFS connection, the log file /var/log/syslog in the host system can be examined.
Recently I wanted to debug a Linux program running inside an ARM system emulated with . I went into some troubles, so I’m going to write here the procedure that worked for me. I wanted to use gdbserver to run a program inside QEMU, and then connect to it from a GDB instance running on my PC, using a TCP link. gdbserver is a piece of software that implements some of the functionalities (the debugging stubs) and then offers the possibility to connect to a full GDB instance through the network (or through a serial port). What I wanted to obtain is illustrated in the following figure.
Using gdbserver inside QEMU
The color blue indicates software compiled to run on my PC (32-bit x86) while the color green indicates software compiled to run on ARM. qemu-system-arm is the software that emulates a VersatilePB platform; I tried to use the one that can be installed through Ubuntu repositories (the package is qemu-kvm-extras) but it froze running the last version of Linux (2.6.35). For this reason I decided to compile the upstream version and use that one. The GDB server and “client” come from the , as well as the compiler used to cross-compile the software for ARM. The program I’ll debug in this example is the simple , that doesn’t do very much beyond printing “Hello World”, but is a nice example of cross-compilation with GNU Autotools.
In order to follow this procedure, I installed:
$ sudo apt-get install build-essential ddd cpio libncurses5-dev libsdl-dev zlib1g-dev $ wget $ chmod +x arm-2010q1-202-arm-none-linux-gnueabi.bin $ ./arm-2010q1-202-arm-none-linux-gnueabi.bin
I installed the toolchain in the default directory “~/CodeSourcery”. The gdbserver executable in my case can be found at the path “/home/francesco/CodeSourcery/Sourcery_G++_Lite/arm-none-linux-gnueabi/libc/usr/bin/gdbserver”.
Note that the following procedure is run in a dedicated folder, and no root access is required from now on. At the end of the procedure about 1 Gigabyte of hard disk space was used.
First of all, I took the new kernel version from the .
$ wget $ tar xjf linux-2.6.35.tar.bz2 $ cd linux-2.6.35/ $ make ARCH=arm versatile_defconfig $ make ARCH=arm menuconfig
When the menu appears, I go into the “Kernel Features” section and enable EABI support; then I exit (saving the configuration) and compile:
$ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- all $ cd ..
The result is a compressed kernel image in “./linux-2.6.35/arch/arm/boot/zImage”.
Next, I take the latest version of ; in a I compiled it statically, but this time I will not, because gdbserver (that I plan to use) needs shared libraries anyway.
$ wget $ tar xjf busybox-1.17.1.tar.bz2 $ cd busybox-1.17.1 $ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- defconfig $ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- install $ cd ..
The result is the folder “busybox-1.17.1/_install”, containing a minimal root filesystem that lacks shared libraries.
I recompiled from source only the QEMU binaries needed to emulate an ARM system.
$ wget $ tar xzf qemu-0.12.5.tar.gz $ cd qemu-0.12.5 $ ./configure --enable-sdl --disable-kvm --enable-debug --target-list=arm-softmmu $ ./make $ cd ..
The relevant result is the program “./qemu-0.12.5/arm-softmmu/qemu-system-arm” that will be used to emulate the VersatilePB platform.
This package needs to be configured for cross-compilation; turns out it’s very easy to do: it needs just the prefix of the cross-compiler.
$ wget $ tar xzf hello-2.6.tar.gz $ cd hello-2.6 $ ./configure --host=arm-none-linux-gnueabi $ make $ cd ..
The result is the ARM-executable “hello-2.6/src/hello”.
All the ARM binaries involved (busybox, gdbserver, hello) need shared libraries. The Codesourcery toolchain offers these libraries in a subfolder of its installation. In my case it’s “/home/francesco/CodeSourcery/Sourcery_G++_Lite/arm-none-linux-gnueabi/libc/lib/”. In order to discover what are the needed libraries I used the readelf tool distributed in the CodeSourcery toolchain. In particular, I ran:
$ arm-none-linux-gnueabi-readelf hello-2.6/src/hello -a |grep lib [Requesting program interpreter: /lib/ld-linux.so.3] 0x00000001 (NEEDED) Shared library: [libgcc_s.so.1] 0x00000001 (NEEDED) Shared library: [libc.so.6] 00010694 00000216 R_ARM_JUMP_SLOT 0000835c __libc_start_main 2: 0000835c 0 FUNC GLOBAL DEFAULT UND __libc_start_main@GLIBC_2.4 (2) 89: 0000844c 4 FUNC GLOBAL DEFAULT 12 __libc_csu_fini 91: 0000835c 0 FUNC GLOBAL DEFAULT UND __libc_start_main@@GLIBC_ 101: 00008450 204 FUNC GLOBAL DEFAULT 12 __libc_csu_init 000000: Version: 1 File: libgcc_s.so.1 Cnt: 1 0x0020: Version: 1 File: libc.so.6 Cnt: 1
The hello binary requires three shared libraries: “ld-linux.so.3″, “libgcc_s.so.1″ and “libc.so.6″. I executed this command for all three binaries, and I copied the required libraries into the Busybox filesystem, together with the gdbserver executable and the hello executable.
$ cd busybox-1.17.1/_install $ mkdir -p lib $ cp /home/francesco/CodeSourcery/Sourcery_G++_Lite/arm-none-linux-gnueabi/libc/lib/ld-linux.so.3 lib/ $ cp /home/francesco/CodeSourcery/Sourcery_G++_Lite/arm-none-linux-gnueabi/libc/lib/libgcc_s.so.1 lib/ $ cp /home/francesco/CodeSourcery/Sourcery_G++_Lite/arm-none-linux-gnueabi/libc/lib/libm.so.6 lib/ $ cp /home/francesco/CodeSourcery/Sourcery_G++_Lite/arm-none-linux-gnueabi/libc/lib/libc.so.6 lib/ $ cp /home/francesco/CodeSourcery/Sourcery_G++_Lite/arm-none-linux-gnueabi/libc/lib/libdl.so.2 lib/ $ cp /home/francesco/CodeSourcery/Sourcery_G++_Lite/arm-none-linux-gnueabi/libc/usr/bin/gdbserver usr/bin/ $ cp ../../hello-2.6/src/hello usr/bin/ $ cd ../../
For my experiment I need a working network from the guest ARM system side, so I prepared an initialization script to enable it. Extending from , here is the script “rcS” that i used.
#!/bin/sh mount -t proc none /proc mount -t sysfs none /sys /sbin/mdev -s ifconfig lo up ifconfig eth0 10.0.2.15 netmask 255.255.255.0 route add default gw 10.0.2.1
Next, I copy rcS it inside the “etc/init.d” directory of the Busybox filesystem and create a compressed filesystem image:
$ cd busybox-1.17.1/_install $ mkdir -p proc sys dev etc etc/init.d $ cp ../../rcS etc/init.d $ chmod +x etc/init.d/rcS $ find . | cpio -o --format=newc | gzip > ../../rootfs.img.gz $ cd ../../
Now I have put everything in place:
To run the plaform the command line is:
$ ./qemu-0.12.5/arm-softmmu/qemu-system-arm -M versatilepb -m 128M -kernel ./linux-2.6.35/arch/arm/boot/zImage -initrd ./rootfs.img.gz -append "root=/dev/ram rdinit=/sbin/init" -redir tcp:1234::1234
The “redir” option will redirect any TCP communication from port 1234 of my Ubuntu PC to port 1234 of the guest ARM system. The system will boot, and a console can be opened with root access. Inside the bash prompt, I run the debugging server:
# gdbserver --multi 10.0.2.15:1234
This command will launch a server that waits for a GDB connection from port 1234. On my PC I open the debugger:
$ ddd --debugger arm-none-linux-gnueabi-gdb
It is also possible to run the arm-none-linux-gnueabi-gdb command alone or connected to another front-end. In order to debug the remote program, I need to tell GDB to consider the ARM shared libraries instead of the local ones (that are for 32-bit x86); otherwise on execution the debugger will complain that the libraries don’t match.
set solib-absolute-prefix nonexistantpath set solib-search-path /home/francesco/CodeSourcery/Sourcery_G++_Lite/arm-none-linux-gnueabi/libc/lib/ file ./hello-2.6/src/hello target extended-remote localhost:1234 set remote exec-file /usr/bin/hello break main run
At this point the debugging goes on as usual.
Posted on 2010/04/12 by