SystemTap Benchmark on N800/N810 HOWTO
Systemtap benchmark tests (located inside testsuite/ folder) are performed running make check (unpriviledged test subset) make installcheck (privileged probe execution tests) command inside systemtap source directory. Actually, there are various ways in order to get systemtap tests running inside your device. Below is a table of possibilities for each device:
SystemTap install methods
NFS directory export from desktop
Systemtap and kernel directories stored on removable memory card
Systemtap and kernel directories stored on internal memory card
Step 1: Kernel configuration
First, you need to get OS2008 kernel source. It is available here. Run the following commands inside scratchbox:
[sbox-ARMEL: ~] > wget http://repository.maemo.org/pool/os2008/free/source/k/kernel-source-rx-34/kernel-source-rx-34_220.127.116.11.orig.tar.gz [sbox-ARMEL: ~] > wget http://repository.maemo.org/pool/os2008/free/source/k/kernel-source-rx-34/kernel-source-rx-34_18.104.22.168-osso71.diff.gz [sbox-ARMEL: ~] > wget http://repository.maemo.org/pool/os2008/free/source/k/kernel-source-rx-34/kernel-source-rx-34_22.214.171.124-osso71.dsc
After downloading kernel source files, you need to unpack them inside a directory. For this, run the command below:
[sbox-ARMEL: ~] > dpkg-source -x kernel-source-rx-34_126.96.36.199-osso71.dsc
A new directory kernel-source-rx-34-188.8.131.52 is created. Enter it:
[sbox-ARMEL: ~] > cd kernel-source-rx-34-184.108.40.206
By default this kernel doesn't support Kprobes. This patch adds Kprobes to ARM kernel. Download it, save it as kernel-arm-kprobes.patch and patch kernel using this command:
[sbox-ARMEL: ~/kernel-source-rx-34-220.127.116.11] > patch -p1 < kernel-arm-kprobes.patch
Now you need to setup kernel initial parameters:
[sbox-ARMEL: ~/kernel-source-rx-34-18.104.22.168] > make nokia_2420_defconfig [sbox-ARMEL: ~/kernel-source-rx-34-22.214.171.124] > make oldconfig
Edit .config file:
[sbox-ARMEL: ~/kernel-source-rx-34-126.96.36.199] > vi .config
Modify the entries below, switching their values to y:
CONFIG_RELAY=y CONFIG_DEBUG_FS=y CONFIG_DEBUG_INFO=y CONFIG_KPROBES=y
If some options are not set then you should replace strings like # CONFIG_KPROBES is not set by CONFIG_KPROBES=y. It's important to delete comments, because otherwise nothing will work.
Complete the process running make oldconfig again (press Enter for all questions):
[sbox-ARMEL: ~/kernel-source-rx-34-188.8.131.52] > make oldconfig
Now your kernel is ready to be compiled. Run make to finish the process:
[sbox-ARMEL: ~/kernel-source-rx-34-184.108.40.206] > make zImage
zImage kernel image is created and located in arch/arm/boot directory inside kernel-tree. In the root directory, copy vmlinux to some other directory for later usage inside device. Now you need to flash built kernel into maemo device. In order to do this flasher tool could be used. You can download it from this site. Then the following steps should be performed:
- Check that the battery on your maemo device is not over.
- Switch off maemo device.
- Connect maemo device with your x86 computer using usb cable.
- Run this command on your x86 computer under root account:
./flasher -f -k zImage
- Switch on maemo device.
- After flashing is done unplug device. It will boot up.
More information about flasher could be found here.
Step 2: Device configuration
Now you need to insert additional repositories on your device in order to install needed packages. Add these two repositories using Application Manager or simply adding these lines to /etc/apt/sources.list file:
$ echo "deb http://repository.maemo.org free non-free" >> /etc/apt/sources.list $ echo "deb http://repository.maemo.org/extras free non-free" >> /etc/apt/sources.list $ apt-get update
Now install the following packages:
$ apt-get install libcap1 libcap-bin openssh-server build-essential binutils
Systemtap needs dejagnu and expect to be installed on your device, but since they doesn't appear on the repositories we at INdT produced some packages for these applications that you can use in order to get systemtap running:
Copy all these packages to your device and install them using dpkg (i.e. dpkg -i <package>.deb).
In order to get your device connected to your computer using the USB cable, you need to setup usbnet kernel module. Your device also need to be R&D mode enabled to get root access.
Inside your device, run the following commands:
$ sudo gainroot $ /etc/init.d/ke-recv stop $ rmmod g_file_storage $ insmod /mnt/initfs/lib/modules/2.6.21-omap1/g_ether.ko $ ifup usb0
Tip: You can save into a shell script for future use.
Now plug in the USB cable on your device and on your computer, and run the following commands inside your computer:
$ sudo modprobe cdc_ether $ sudo ifup usb0
By default, your computer uses 192.168.2.14 IP address and your device gets 192.168.2.15.
More info about this subject can be found on Maemo's wiki: Setting up USB Networking.
Step 3: Kernel and systemtap directories
Method 1: NFS
If you chose to use NFS to export both kernel and systemtap directories from your desktop, you need to have nfs-kernel-server package installed on your computer. Exit scratchbox and edit /etc/exports file:
$ sudo vim /etc/exports
Add the following lines:
/scratchbox/users/<USER>/home/<USER>/kernel-source-rx-34-220.127.116.11 *(rw,no_root_squash,no_subtree_check,sync) /scratchbox/users/<USER>/home/<USER>/systemtap-20080202 *(rw,no_root_squash,no_subtree_check,sync)
Where <USER> corresponds to the scratchbox user login. Now you need to restart NFS server:
$ sudo /etc/init.d/nfs-kernel-server restart
Your device doesn't have nfs kernel modules by default. You need to install an additional package rx-34-kernel-modules-extra. A good advantage of using usbnet is that your computer "automagically" shares its internet connection with the device. Knowing this, run the command below:
$ apt-get install rx-34-kernel-modules-extra
Now, with the kernel modules installed, you need to insert them into kernel:
$ insmod /lib/modules/2.6.21-omap1/extra/sunrpc.ko $ insmod /lib/modules/2.6.21-omap1/extra/lockd.ko $ insmod /lib/modules/2.6.21-omap1/extra/nfs.ko
NFS kernel modules are now inserted into kernel. You can now mount both kernel and systemtap directories from your computer inside the device:
$ mkdir -p /mnt/kernel $ mount -t nfs 192.168.2.14:/scratchbox/users/<USER>/home/<USER>/kernel-source-rx-34-18.104.22.168 /mnt/kernel $ mkdir -p /mnt/stap $ mount -t nfs 192.168.2.14:/scratchbox/users/<USER>/home/<USER>/kernel-source-rx-34-22.214.171.124 /mnt/stap
Where <USER> corresponds to the scratchbox user login.
Method 2: Memory card
Using this method, we can store both systemtap and kernel source directories inside a memory card. On N810, we could use the internal memory card (2Gb) for this. Follow the steps below (N810 internal memory card):
Inside your device, open "X Terminal" and run this command to gain root access:
$ sudo gainroot
Note: You need to have "research and development" flag enabled on your device. See flasher info for more details about this.
You need to install these additional packages (in order to format memory card with ext2 partition type):
$ apt-get install e2fslibs e2fsprogs
Now you need to umount the previously mounted internal memory card:
$ umount /media/mmc2
By default, the internal memory card is partitioned as vfat. We need to change this to ext2. Use 'sfdisk' to do that:
$ sfdisk /dev/mmcblk0
Press "Enter" four times then press y. Now your memory card is partitioned as ext2. If you want to check if it is correct, use the following command:
$ sfdisk -l /dev/mmcblk0 /dev/mmcblk0p1 0+ 61439 61440- 1966079+ 83 Linux /dev/mmcblk0p2 0 - 0 0 0 Empty /dev/mmcblk0p3 0 - 0 0 0 Empty /dev/mmcblk0p4 0 - 0 0 0 Empty
Now you need to format this new partition with ext2 filesystem:
$ mke2fs /dev/mmcblk0p1
Your filesystem is now ready to be mounted! Mount it with this command:
$ mount -t ext2 /dev/mmcblk0 /media/mmc2
There's now a file you need to modify in order to alter the mounting sequence of your device. Open /etc/fstab and edit the line that starts with /dev/mmcblk0p1 and modify it to this:
/dev/mmcblk0p1 /media/mmc2 ext2 defaults 0 0
That's it! You can now copy both kernel and systemtap directories inside your memory card using scp. As an example of scp usage, it works with a command like this:
$ scp <USER>@192.168.2.14:/scratchbox/users/<USER>/home/<USER>/kernel /media/mmc2/kernel $ scp <USER>@192.168.2.14:/scratchbox/users/<USER>/home/<USER>/systemtap /media/mmc2/systemtap
Where <USER> is the user login from your computer.
Step 4: Additional files/directories
Your device is almost ready to run systemtap benchmark tests. You need to create symbolic links inside the device's filesystem in order to achieve systemtap compilation rules:
$ ln -s /mnt/kernel/vmlinux /boot/vmlinux-2.6.21-omap1 $ ln -s /mnt/kernel/vmlinux /lib/modules/2.6.21-omap1/vmlinux $ ln -s /mnt/kernel /lib/modules/2.6.21-omap1/build $ ln -s /mnt/kernel /lib/modules/2.6.21-omap1/kernel $ ln -s /mnt/kernel /lib/modules/2.6.21-omap1/source
Note: If you chosed to use the memory card method, remember to change /mnt/kernel with /media/mmc2/kernel directory.
Now you can install systemtap package that you've created previously on scratchbox inside your device:
$ dpkg -i systemtap_20080405-1_armel.deb
Also from scratchbox, you need to copy the following binaries (some of these may already exist on your device filesystem, but on a smaller version):
Insert them inside /bin directory. Notice that some of these applications may be already available inside the device.
Now, before actually running systemtap, your device may have low resources in terms of memory and processor idle time. In order to minimize that problem, you can stop some maemo services that won't be necessary for that moment:
$ /etc/init.d/metalayer_crawler0 stop $ /etc/init.d/hildon-desktop stop $ /etc/init.d/af-base-apps stop
Your device is now ready to run make check and/or make installcheck inside systemtap directory.
More information about systemtap test-suite execution can be found here.