Adding A System Call In Linux

System call is a set of special APIs, as the interface between operating system’s kernel  and other programs. For example, if a program wants to create a new process, it should actually call a system call to achieve that.

In this blog, we will discuss how to add a system call in Linux. And all the operations are done on Ubuntu 10.04 with 2.6.22.9 kernel and a X86 machine.

1. Add a new system call in Linux

In the following, we will add a new system call (mysyscall) with system call number being 324, in eleven simple steps.

1. Download the kernel source code kernel-source-2.6.22.9.tar.gz from Ubuntu. As tested, the original official copy of kernel 2.6.23.9 cannot work with SATA hard disk and Ubuntu 10.04. And the one from Ubuntu is already patched.
2. Exact the files to /usr/src, or any other folder you want. And change the current working directory to the source code folder, such as /usr/src/linux-source-2.6.22.
3. Edit the file ./kernel/sys.c , add a new function as following:
   
   C

   1 2 3 4 5

   asmlinkage int mysyscall(int number) {     printk("This is an example of systemcall: %dn", number);     return number*6; }

   
   asmlinkage is a macro definition as: “#define asmlinkage CPP_ASMLINKAGE __attribute__((regparm(0)))”. It tells the compiler, the parameters should be found in stack, rather than registers. But in x86-64 machines, this modifier does nothing with the parameters passing. More details could be found here.
   And the function printk is a version of printf in Linux kernel.
4. Edit the head file ./include/asm-i386/unistd.h:
   Add a new line as: “#define __NR_mysyscall 324”. This new line indicates the system call number of our function is 324.
   Change the line “#define NR_syscalls 324“ to “#define NR_syscalls 325“. This means the total number of system calls is 325 (system call number starts with 0).
5. Edit the file ./arch/i386/kernel/syscall_table.S to add a new line at the end:
   .long mysyscall
6. Copy the current using config file to current folder: cp /boot/config-$(uname -r) ./.config
7. make oldconfig
8. make-kpkg clean
9. fakeroot make-kpkg –initrd –revision=custom.1.0 kernel_image modules_image
   This command will create a install package at the upper-level folder, such as /usr/src/linux-image-2.6.22.9_custom.1.0_i386.deb
10. Install the new Linux kernel: dpkg –i /usr/src/linux-image-2.6.22.9_custom.1.0_i386.deb
11. Reboot, select to use the new kernel, and enjoy it!

2. Testing

In this section, we will write a small program to test the new system call.

1. Edit the test file:
   
   Assembly (x86)

   1 2 3 4 5 6 7 8 9 10 11 12

   ; tiny.asm BITS 32 GLOBAL _start SECTION .text _start: mov eax, 324 mov ebx, 9 int 0x80 push eax mov eax, 1 pop ebx int 0x80

   As we discussed previously, on X86-32 machine, the parameters of system calls should be located in stack (here). But why we just push the parameter into register when we call it? An interrupt by “int 0x80” does not lead us directly to our system call function. It leads to a interrupt handler “system_call()”, which is hardware platform dependent. And the system_call() will do these preparation works for the actual system call functions.

2. Compile and link it:
   nasm -f elf test.s
   gcc -o test2 -Wall -s -nostdlib test.o
3. run the test progrom and check it exit code.
   ./test2 ; echo $?
   As expected, the exit code should be 54.
4. We also could check the information in kernel messages cache. With command “dmesg |tail -1”, we could get the following message:
   This is an example of systemcall: 9

PS: I wrote this article in 2007. So it cannot be directly applied to the latest Linux kernel.