樹莓派 2 透過 linux sys 設定 gpio

利用 linux sys 檔案系統來設定 gpio 很簡單, 不簡單的是 gpio 編號是多少呢? 我找了好久終於找到樹莓派 2 的 gpio pin 圖。

發現可以用 gpio pin 17, 27 來測試。

樹莓派 2 gpio pin

cd /sys/class/gpio
echo 27 > export
echo out > gpio27/direction
echo 1 > gpio27/value

cd /sys/class/gpio
echo 17 > export
echo out > gpio17/direction
echo 1 > gpio17/value

由於我沒有接上 led, 所以透過 LA 來確認有無正確設定這 2 個 gpio pin。



觀察 gpio 編號

mount -t debugfs none /sys/kernel/debug/
cat /sys/kernel/debug/gpio

#!/bin/sh
cd /sys/class/gpio/
echo 17 > export
echo out >  gpio17/direction

echo 27 > export
echo out >  gpio27/direction

while true; do
    echo 1 > /sys/class/gpio/gpio17/value
    echo 0 > /sys/class/gpio/gpio27/value
    #sleep 0.5
    echo 0 > /sys/class/gpio/gpio17/value
    echo 1 > /sys/class/gpio/gpio27/value
    #sleep 0.5
done

LA 抓到的波型

使用 gdb+qemu 來執行/除錯 raspberry pi linux kernel

the 1st edition: 20150408
the 2nd edition: 20150711 add rpi2 part, Initramfs source files conf
the 3rd edition: 20151001 /dev/ram0
the 4th edition: 20170320 使用 4.4.y linux kernel

看了這個《Linux 内核分析》課程之後, 我知道可以用 qemu + gdb 來 debug linux kernel。

課程所提供的 rootfs 在這裡, 很精簡, 對學習很有幫助:

git clone  https://github.com/mengning/menu.git

我想如法泡製在 raspberry pi 上, 因為我沒有實體的 pi 開發版 (後來有了), 所以用這方法來 trace linux kernel。不過要先解決如何編譯 linux kernel for qemu/pi。

《raspberry pi Kernel Building》這份文件有教學, 不過編譯出來的 kernel 無法在 qemu 上執行, 要參考這篇《build raspberrypi kernel for qemu》(需要 patch 並選擇一些選項)

舊的 xecdesign.com 資料已經不存在了, 也無法適用於 4.4.y 的 kernel, https://github.com/dhruvvyas90/qemu-rpi-kernel 這裡提供的 linux-arm.patch 可以讓 rpi 4.4.y 的 kernel 在 qemu 中啟動, tools/build-kernel-qemu 提供了 script 自動編譯 kernel。

1. wget http://xecdesign.com/downloads/linux-qemu/linux-arm.patch
2. patch -p1 -d linux/ < linux-arm.patch

記得加入 debug option, 這樣編譯出來的 kernel 才有 debug symbol, 可以配合 gdb 做 source code level debug。

add debug option

kernel hacking—>
   Compile-time checks and compiler options ->
     [*] compile the kernel with debug info

這是我的 linux kernel config 和下載的 patch:
config and patch

https://github.com/descent/linux/tree/rpi-4.4.y-qemu 有我 patch 好的 kernel, qemu-config 是給 qemu 用的 config file。

測試環境:

git clone https://github.com/raspberrypi/linux
git commit 8362c08dfc8dd9e54745b3f1e5e4ff0a1fb30614

再來的問題是, 如何載入 initrd?

和 x86 的版本有點不同, 需要 -append 加入參數,《Compiling Linux kernel for QEMU ARM emulator》提供了一個辦法, 不過 -append 裡頭的參數還是無法讓我正常使用《Linux内核分析》裡頭介紹的 rootfs initrd image, 其文章提供的 hello rootfs initrd image 當然也無法使用, 最後找到以下指令。

qemu-system-arm -M versatilepb -cpu arm1176 -kernel /media/2/pi/linux/arch/arm/boot/zImage -m 128 -initrd rootfs  -append "initrd=rootfs"
qemu-system-arm -M versatilepb -cpu arm1176 -kernel /media/2/pi/linux/arch/arm/boot/zImage -m 128 -initrd m/rootfs.img  -append "initrd=rootfs.img"

menuos for raspberry pi in qemu

寄件者 ??

雖然有了 arm 的 debug 環境, 不過我還有些疑惑? linux 被載入的位址為何? 固定在那個位址嗎? 它可以被移動到其他位址嗎?

x86 有個 relocatable kernel 的選項。那他怎麼做 relocation 的動作?

x86 linux option

1 Processor type and features
2   (0x1000000) Physical address where the kernel is loaded
3   [*] Build a relocatable kernel

對應的變數是:
CONFIG_RELOCATABLE=y

arm 版本沒有 relocatable kernel 的選項, 不過有個 AUTO_ZRELADDR=y, 在 Enable "AUTO_ZRELADDR" support under "Boot" options, 好像是等同於 RELOCATABLE。

The Linux Kernel: Configuring the Kernel Part 5 提到:

This next kernel option (Build a relocatable kernel (RELOCATABLE)) allows the kernel to be placed somewhere else in the memory. The kernel file will be 10% larger,

深入探索 Kdump，第 3 部分: Kdump 原理探秘

可重定位內核（relocatable kernel）
可重定位內核的意義

在 kdump 出現之前，內核只能從一個固定的物理地址上啟動。這對 kdump 來說是一種限制。因為為了收集生產內核的內存鏡像，捕獲內核不能從生產內核使用的啟動地址上啟動。因此就需要另編譯一個從一個不同的地址啟動的內核來作捕獲內核。這就是為什麼 RHEL5 中有一個包叫 kernel-kdump 的原因。技術的創新往往來自對方便的追求。開發人員為了不用費心多編譯一個內核，為 kernel 實現了「可重定向」這個特性。
實現原理

x86_64: 運行時修改 text 段及 data 段的眏射

kernel 在啟動以後，會檢測自己被加載到了什麼位置。然後根據這個來更新自己的內存頁表以反映 kernel 的 text 段和 data 段中虛擬地址與物理地址之間正確的映射關係。

i386: 使用預先生成的重定位信息

i386 中的 text 和 data 段是已經寫死的線性映射區的一部分，要想使用修改頁表的辦法支持重定向是比較困難的。於是在編譯內核時，另生成一份所有需要重定位的 symbol 的位置信息，放進 bzimage 格式的內核中。內核啟動解壓縮後，根據加載的地址和這份表來時行重定位。

powerpc: 將 vmlinuz 鏈接為「position-independent executable」形式

與 x86 體系不同，在 powerpc 體系中，/boot/vmlinuz 並不是一個 bzimage 格式的文件，它就是一個 ELF 格式的文件，而且啟動機理也不盡相同。因此，在 powerpc 上主要是利用了「位置無關可執行」格式這一成熟技術來實現可重定位。

以下是我的 qemu command, 搭配 linux/vmlinux 就可以 gdb single step linux kernel。

qemu-system-arm -M versatilepb -cpu arm1176 -kernel linux/arch/arm/boot/zImage -m 128 -initrd m/rootfs.img  -append "initrd=rootfs.img" -s -S

不過我想從第一個指令開始看起, 無法成功, 只好照著課程步驟, 把 break point 設在 start_kernel, 可以正常 debug, 和課程提到的是一樣的結果。以下是開機時暫存器的值。

gdb.sh 示範如果使用 gdb 連上 qemu:

gdb.sh

 1 descent@debian64:linux-4.4.1$ arm-linux-gnueabihf-gdb linux/vmlinux
 2 GNU gdb (Debian 7.10-1+b1) 7.10
 3 Copyright (C) 2015 Free Software Foundation, Inc.
 4 License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 5 This is free software: you are free to change and redistribute it.
 6 There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
 7 and "show warranty" for details.
 8 This GDB was configured as "x86_64-linux-gnu".
 9 Type "show configuration" for configuration details.
10 For bug reporting instructions, please see:
11 <http://www.gnu.org/software/gdb/bugs/>.
12 Find the GDB manual and other documentation resources online at:
13 <http://www.gnu.org/software/gdb/documentation/>.
14 For help, type "help".
15 Type "apropos word" to search for commands related to "word"...
16 Reading symbols from vmlinux...done.
17 (gdb) target remote localhost:1234
18 Remote debugging using localhost:1234
19 0x0000fff0 in ?? ()
20 (gdb) b start_kernel
21 Breakpoint 1 at 0xc1846740: file init/main.c, line 498.
22 (gdb) c
23 Continuing.
24 
25 Breakpoint 1, start_kernel () at init/main.c:498
26 498 {
27 (gdb) 

rpi

 1 (gdb) i r
 2 r0             0x0    0
 3 r1             0x0    0
 4 r2             0x0    0
 5 r3             0x0    0
 6 r4             0x0    0
 7 r5             0x0    0
 8 r6             0x0    0
 9 r7             0x0    0
10 r8             0x0    0
11 r9             0x0    0
12 r10            0x0    0
13 r11            0x0    0
14 r12            0x0    0
15 sp             0x0    0x0 <__vectors_start>
16 lr             0x0    0
17 pc             0x0    0x0 <__vectors_start>
18 cpsr           0x400001d3     1073742291

使用 Initramfs source files conf 的選項可以將 initramfs 直接編入 kernel, 這樣就不需要使用 qemu 的 -initrd。

Initramfs source files conf

1 General setup  --->
2   [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
3   (/media/2/linux_kernel/m/rootfs) Initramfs source file(s)  

重要:
不過需要將 /dev/console 複製到 Initramfs source file (我的例子是: /media/2/linux_kernel/m/rootfs), 否則開機會有 console 相關的錯誤訊息。

arm(rpi) 的 qemu 指令

qemu-system-arm -M versatilepb -cpu arm1176 -kernel arch/arm/boot/zImage -m 128

descent@debian32:~/linux-3.10.27/arch/x86/boot$ ls ~/git/FS/target2/dev/
console  mtd1   mtd13  mtd3  mtd7       mtdblock1   mtdblock13  mtdblock3  mtdblock7  ptmx    rtc0  tty2  tty6   ttyS1
kmem     mtd10  mtd14  mtd4  mtd8       mtdblock10  mtdblock14  mtdblock4  mtdblock8  pts     tty   tty3  tty7   ttyS2
mem      mtd11  mtd15  mtd5  mtd9       mtdblock11  mtdblock15  mtdblock5  mtdblock9  ram0    tty0  tty4  tty8   urandom
mtd0     mtd12  mtd2   mtd6  mtdblock0  mtdblock12  mtdblock2   mtdblock6  null       random  tty1  tty5  ttyS0  zero

init 放在 /init, 有些環境可能要放在 /sbin/init

若要在終端機秀出 qemu 訊息, 使用以下指令, 重點在指定 -nographic -append "console=ttyAMA0 這兩個參數。

qemu-system-arm -M versatilepb -cpu arm1176 -kernel arch/arm/boot/zImage -m 128 -nographic -append "console=ttyAMA0"

qemu rpi message for terminal

  1 descent@debian64:linux$ qemu-system-arm -M versatilepb -cpu arm1176 -kernel arch/arm/boot/zImage -m 128 -nographic  -append "console=ttyAMA0"
  2 
  3 (process:31080): GLib-WARNING **: /build/glib2.0-94amRy/glib2.0-2.50.1/./glib/gmem.c:483: custom memory allocation vtable not supported
  4 pulseaudio: set_sink_input_volume() failed
  5 pulseaudio: Reason: Invalid argument
  6 pulseaudio: set_sink_input_mute() failed
  7 pulseaudio: Reason: Invalid argument
  8 Uncompressing Linux... done, booting the kernel.
  9 Booting Linux on physical CPU 0x0
 10 Initializing cgroup subsys cpuset
 11 Initializing cgroup subsys cpu
 12 Initializing cgroup subsys cpuacct
 13 Linux version 4.4.30-rt5+ (descent@debian64) (gcc version 4.8.3 20140303 (prerelease) (crosstool-NG linaro-1.13.1+bzr2650 - Linaro GCC 2014.03) ) #9 Mon Mar 20 17:27:35 CST 2017
 14 CPU: ARMv6-compatible processor [410fb767] revision 7 (ARMv7), cr=00c5387d
 15 CPU: VIPT aliasing data cache, unknown instruction cache
 16 Machine: ARM-Versatile PB
 17 Memory policy: Data cache writeback
 18 sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
 19 Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32480
 20 Kernel command line: console=ttyAMA0
 21 PID hash table entries: 512 (order: -1, 2048 bytes)
 22 Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
 23 Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
 24 Memory: 124120K/131072K available (4039K kernel code, 185K rwdata, 1088K rodata, 164K init, 139K bss, 6952K reserved, 0K cma-reserved)
 25 Virtual kernel memory layout:
 26     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
 27     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
 28     vmalloc : 0xc8800000 - 0xff800000   ( 880 MB)
 29     lowmem  : 0xc0000000 - 0xc8000000   ( 128 MB)
 30     modules : 0xbf000000 - 0xc0000000   (  16 MB)
 31       .text : 0xc0008000 - 0xc050a1bc   (5129 kB)
 32       .init : 0xc050b000 - 0xc0534000   ( 164 kB)
 33       .data : 0xc0534000 - 0xc05626c0   ( 186 kB)
 34        .bss : 0xc05626c0 - 0xc0585460   ( 140 kB)
 35 NR_IRQS:224
 36 VIC @f1140000: id 0x00041190, vendor 0x41
 37 FPGA IRQ chip 0 "SIC" @ f1003000, 13 irqs, parent IRQ: 63
 38 clocksource: timer3: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns
 39 Console: colour dummy device 80x30
 40 Calibrating delay loop... 735.23 BogoMIPS (lpj=3676160)
 41 pid_max: default: 32768 minimum: 301
 42 Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
 43 Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
 44 Disabling cpuset control group subsystem
 45 Initializing cgroup subsys memory
 46 Initializing cgroup subsys devices
 47 Initializing cgroup subsys freezer
 48 CPU: Testing write buffer coherency: ok
 49 Setting up static identity map for 0x8220 - 0x827c
 50 devtmpfs: initialized
 51 VFP support v0.3: implementor 41 architecture 1 part 20 variant b rev 5
 52 clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
 53 NET: Registered protocol family 16
 54 DMA: preallocated 256 KiB pool for atomic coherent allocations
 55 Serial: AMBA PL011 UART driver
 56 dev:f1: ttyAMA0 at MMIO 0x101f1000 (irq = 44, base_baud = 0) is a PL011 rev1
 57 console [ttyAMA0] enabled
 58 dev:f2: ttyAMA1 at MMIO 0x101f2000 (irq = 45, base_baud = 0) is a PL011 rev1
 59 dev:f3: ttyAMA2 at MMIO 0x101f3000 (irq = 46, base_baud = 0) is a PL011 rev1
 60 fpga:09: ttyAMA3 at MMIO 0x10009000 (irq = 70, base_baud = 0) is a PL011 rev1
 61 PCI core found (slot 11)
 62 PCI host bridge to bus 0000:00
 63 pci_bus 0000:00: root bus resource [mem 0x50000000-0x5fffffff]
 64 pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff pref]
 65 pci_bus 0000:00: root bus resource [io  0x1000-0xffff]
 66 pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
 67 PCI: bus0: Fast back to back transfers disabled
 68 pci 0000:00:0c.0: BAR 2: assigned [mem 0x50000000-0x50001fff]
 69 pci 0000:00:0c.0: BAR 1: assigned [mem 0x50002000-0x500023ff]
 70 pci 0000:00:0c.0: BAR 0: assigned [io  0x1000-0x10ff]
 71 vgaarb: loaded
 72 SCSI subsystem initialized
 73 clocksource: Switched to clocksource timer3
 74 NET: Registered protocol family 2
 75 TCP established hash table entries: 1024 (order: 0, 4096 bytes)
 76 TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
 77 TCP: Hash tables configured (established 1024 bind 1024)
 78 UDP hash table entries: 256 (order: 0, 4096 bytes)
 79 UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
 80 NET: Registered protocol family 1
 81 RPC: Registered named UNIX socket transport module.
 82 RPC: Registered udp transport module.
 83 RPC: Registered tcp transport module.
 84 RPC: Registered tcp NFSv4.1 backchannel transport module.
 85 NetWinder Floating Point Emulator V0.97 (double precision)
 86 futex hash table entries: 256 (order: -1, 3072 bytes)
 87 Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
 88 jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
 89 romfs: ROMFS MTD (C) 2007 Red Hat, Inc.
 90 Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
 91 io scheduler noop registered
 92 io scheduler deadline registered
 93 io scheduler cfq registered (default)
 94 pl061_gpio dev:e4: PL061 GPIO chip @0x101e4000 registered
 95 pl061_gpio dev:e5: PL061 GPIO chip @0x101e5000 registered
 96 pl061_gpio dev:e6: PL061 GPIO chip @0x101e6000 registered
 97 pl061_gpio dev:e7: PL061 GPIO chip @0x101e7000 registered
 98 clcd-pl11x dev:20: PL110 rev0 at 0x10120000
 99 clcd-pl11x dev:20: Versatile hardware, VGA display
100 Console: switching to colour frame buffer device 80x30
101 brd: module loaded
102 sym53c8xx 0000:00:0c.0: enabling device (0100 -> 0103)
103 sym0: <895a> rev 0x0 at pci 0000:00:0c.0 irq 93
104 sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking
105 sym0: SCSI BUS has been reset.
106 scsi host0: sym-2.2.3
107 scsi 0:0:2:0: CD-ROM            QEMU     QEMU CD-ROM      2.3. PQ: 0 ANSI: 5
108 scsi target0:0:2: tagged command queuing enabled, command queue depth 16.
109 scsi target0:0:2: Beginning Domain Validation
110 scsi target0:0:2: Domain Validation skipping write tests
111 scsi target0:0:2: Ending Domain Validation
112 sr 0:0:2:0: [sr0] scsi3-mmc drive: 16x/50x cd/rw xa/form2 cdda tray
113 cdrom: Uniform CD-ROM driver Revision: 3.20
114 physmap platform flash device: 04000000 at 34000000
115 physmap-flash.0: Found 1 x32 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
116 Intel/Sharp Extended Query Table at 0x0031
117 Using buffer write method
118 smc91x.c: v1.1, sep 22 2004 by Nicolas Pitre <nico@fluxnic.net>
119 smc91x smc91x.0 eth0: SMC91C11xFD (rev 1) at c8a5a000 IRQ 57
120  [nowait]
121 smc91x smc91x.0 eth0: Ethernet addr: 52:54:00:12:34:56
122 mousedev: PS/2 mouse device common for all mice
123 ledtrig-cpu: registered to indicate activity on CPUs
124 Netfilter messages via NETLINK v0.30.
125 nf_conntrack version 0.5.0 (1939 buckets, 7756 max)
126 ip_tables: (C) 2000-2006 Netfilter Core Team
127 NET: Registered protocol family 17
128 bridge: automatic filtering via arp/ip/ip6tables has been deprecated. Update your scripts to load br_netfilter if you need this.
129 Bridge firewalling registered
130 input: AT Raw Set 2 keyboard as /devices/fpga:06/serio0/input/input0
131 input: ImExPS/2 Generic Explorer Mouse as /devices/fpga:07/serio1/input/input2
132 VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
133 Please append a correct "root=" boot option; here are the available partitions:
134 0100            4096 ram0  (driver?)
135 0101            4096 ram1  (driver?)
136 0102            4096 ram2  (driver?)
137 0103            4096 ram3  (driver?)
138 0104            4096 ram4  (driver?)
139 0105            4096 ram5  (driver?)
140 0106            4096 ram6  (driver?)
141 0107            4096 ram7  (driver?)
142 0108            4096 ram8  (driver?)
143 0109            4096 ram9  (driver?)
144 010a            4096 ram10  (driver?)
145 010b            4096 ram11  (driver?)
146 010c            4096 ram12  (driver?)
147 010d            4096 ram13  (driver?)
148 010e            4096 ram14  (driver?)
149 010f            4096 ram15  (driver?)
150 0b00         1048575 sr0  driver: sr

qemu-system-i386

  1 Script started on Wed 05 Aug 2015 04:59:52 PM CST
  2 descent@debian32:~/linux-3.10.27/arch/x86/boot$ qemu-system-i386 -nographic -kernel bzImage -append "console=ttyS0"
  3 [    0.000000] Initializing cgroup subsys cpuset
  4 [    0.000000] Initializing cgroup subsys cpu
  5 [    0.000000] Initializing cgroup subsys cpuacct
  6 [    0.000000] Linux version 3.10.27 (descent@debian32) (gcc version 4.8.5 (Debian 4.8.5-1) ) #2 SMP Wed Aug 5 16:41:20 CST 2015
  7 [    0.000000] e820: BIOS-provided physical RAM map:
  8 [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
  9 [    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
 10 [    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
 11 [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000007fdffff] usable
 12 [    0.000000] BIOS-e820: [mem 0x0000000007fe0000-0x0000000007ffffff] reserved
 13 [    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
 14 [    0.000000] Notice: NX (Execute Disable) protection missing in CPU!
 15 [    0.000000] SMBIOS 2.8 present.
 16 [    0.000000] e820: last_pfn = 0x7fe0 max_arch_pfn = 0x100000
 17 [    0.000000] found SMP MP-table at [mem 0x000f6640-0x000f664f] mapped at [c00f6640]
 18 [    0.000000] Scanning 1 areas for low memory corruption
 19 [    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
 20 [    0.000000] init_memory_mapping: [mem 0x07800000-0x07bfffff]
 21 [    0.000000] init_memory_mapping: [mem 0x00100000-0x077fffff]
 22 [    0.000000] init_memory_mapping: [mem 0x07c00000-0x07fdffff]
 23 [    0.000000] ACPI: RSDP 000f6470 00014 (v00 BOCHS )
 24 [    0.000000] ACPI: RSDT 07fe16a9 00034 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
 25 [    0.000000] ACPI: FACP 07fe0bda 00074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
 26 [    0.000000] ACPI: DSDT 07fe0040 00B9A (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
 27 [    0.000000] ACPI: FACS 07fe0000 00040
 28 [    0.000000] ACPI: SSDT 07fe0c4e 009AB (v01 BOCHS  BXPCSSDT 00000001 BXPC 00000001)
 29 [    0.000000] ACPI: APIC 07fe15f9 00078 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
 30 [    0.000000] ACPI: HPET 07fe1671 00038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
 31 [    0.000000] 0MB HIGHMEM available.
 32 [    0.000000] 127MB LOWMEM available.
 33 [    0.000000]   mapped low ram: 0 - 07fe0000
 34 [    0.000000]   low ram: 0 - 07fe0000
 35 [    0.000000] Zone ranges:
 36 [    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
 37 [    0.000000]   Normal   [mem 0x01000000-0x07fdffff]
 38 [    0.000000]   HighMem  empty
 39 [    0.000000] Movable zone start for each node
 40 [    0.000000] Early memory node ranges
 41 [    0.000000]   node   0: [mem 0x00001000-0x0009efff]
 42 [    0.000000]   node   0: [mem 0x00100000-0x07fdffff]
 43 [    0.000000] Using APIC driver default
 44 [    0.000000] ACPI: PM-Timer IO Port: 0x608
 45 [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
 46 [    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
 47 [    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
 48 [    0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
 49 [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
 50 [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
 51 [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
 52 [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
 53 [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
 54 [    0.000000] Using ACPI (MADT) for SMP configuration information
 55 [    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
 56 [    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
 57 [    0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
 58 [    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
 59 [    0.000000] PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
 60 [    0.000000] e820: [mem 0x08000000-0xfffbffff] available for PCI devices
 61 [    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:1 nr_node_ids:1
 62 [    0.000000] PERCPU: Embedded 13 pages/cpu @c7ed0000 s32064 r0 d21184 u53248
 63 [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32382
 64 [    0.000000] Kernel command line: console=ttyS0
 65 [    0.000000] PID hash table entries: 512 (order: -1, 2048 bytes)
 66 [    0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
 67 [    0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
 68 [    0.000000] Initializing CPU#0
 69 [    0.000000] Initializing HighMem for node 0 (00000000:00000000)
 70 [    0.000000] Memory: 118524k/130944k available (5775k kernel code, 12028k reserved, 2092k data, 2124k init, 0k highmem)
 71 [    0.000000] virtual kernel memory layout:
 72 [    0.000000]     fixmap  : 0xfff15000 - 0xfffff000   ( 936 kB)
 73 [    0.000000]     pkmap   : 0xff800000 - 0xffc00000   (4096 kB)
 74 [    0.000000]     vmalloc : 0xc87e0000 - 0xff7fe000   ( 880 MB)
 75 [    0.000000]     lowmem  : 0xc0000000 - 0xc7fe0000   ( 127 MB)
 76 [    0.000000]       .init : 0xc17b0000 - 0xc19c3000   (2124 kB)
 77 [    0.000000]       .data : 0xc15a3f7b - 0xc17af0c0   (2092 kB)
 78 [    0.000000]       .text : 0xc1000000 - 0xc15a3f7b   (5775 kB)
 79 [    0.000000] Checking if this processor honours the WP bit even in supervisor mode...Ok.
 80 [    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
 81 [    0.000000] Hierarchical RCU implementation.
 82 [    0.000000]  RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
 83 [    0.000000] NR_IRQS:2304 nr_irqs:256 16
 84 [    0.000000] Console: colour VGA+ 80x25
 85 [    0.000000] console [ttyS0] enabled
 86 [    0.000000] tsc: Fast TSC calibration failed
 87 [    0.000000] tsc: Unable to calibrate against PIT
 88 [    0.000000] tsc: using HPET reference calibration
 89 [    0.000000] tsc: Detected 3594.682 MHz processor
 90 [    0.009236] Calibrating delay loop (skipped), value calculated using timer frequency.. 7189.36 BogoMIPS (lpj=3594682)
 91 [    0.011114] pid_max: default: 32768 minimum: 301
 92 [    0.013258] Security Framework initialized
 93 [    0.015457] SELinux:  Initializing.
 94 [    0.017211] Mount-cache hash table entries: 512
 95 [    0.024728] Initializing cgroup subsys freezer
 96 [    0.027285] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
 97 [    0.027285] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0
 98 [    0.027285] tlb_flushall_shift: 6
 99 [    0.067329] Freeing SMP alternatives: 24k freed
100 [    0.068019] ACPI: Core revision 20130328
101 [    0.082594] ACPI: All ACPI Tables successfully acquired
102 [    0.092004] Enabling APIC mode:  Flat.  Using 1 I/O APICs
103 [    0.095000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
104 [    0.107000] smpboot: CPU0: Intel QEMU Virtual CPU version 2.3.0 (fam: 06, model: 06, stepping: 03)
105 [    0.110000] APIC calibration not consistent with PM-Timer: 107ms instead of 100ms
106 [    0.110000] APIC delta adjusted to PM-Timer: 6249946 (6741014)
107 [    0.110658] Performance Events: Broken PMU hardware detected, using software events only.
108 [    0.112041] Failed to access perfctr msr (MSR c1 is 0)
109 [    0.121634] Brought up 1 CPUs
110 [    0.122075] smpboot: Total of 1 processors activated (7189.36 BogoMIPS)
111 [    0.136000] RTC time:  8:59:58, date: 08/05/15
112 [    0.138000] NET: Registered protocol family 16
113 [    0.140000] kworker/u2:0 (13) used greatest stack depth: 7312 bytes left
114 [    0.145499] kworker/u2:0 (17) used greatest stack depth: 7188 bytes left
115 [    0.149506] ACPI: bus type PCI registered
116 [    0.152247] PCI: PCI BIOS revision 2.10 entry at 0xfd40f, last bus=0
117 [    0.153054] PCI: Using configuration type 1 for base access
118 [    0.158222] kworker/u2:0 (28) used greatest stack depth: 7176 bytes left
119 [    0.224664] bio: create slab <bio-0> at 0
120 [    0.228120] ACPI: Added _OSI(Module Device)
121 [    0.229000] ACPI: Added _OSI(Processor Device)
122 [    0.229020] ACPI: Added _OSI(3.0 _SCP Extensions)
123 [    0.230053] ACPI: Added _OSI(Processor Aggregator Device)
124 [    0.248036] ACPI: Interpreter enabled
125 [    0.249202] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20130328/hwxface-568)
126 [    0.251100] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20130328/hwxface-568)
127 [    0.253401] ACPI: (supports S0 S3 S4 S5)
128 [    0.254000] ACPI: Using IOAPIC for interrupt routing
129 [    0.256325] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
130 [    0.260855] ACPI: No dock devices found.
131 [    0.293274] kworker/u2:0 (303) used greatest stack depth: 7164 bytes left
132 [    0.307297] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
133 [    0.309451] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
134 [    0.312217] PCI host bridge to bus 0000:00
135 [    0.313176] pci_bus 0000:00: root bus resource [bus 00-ff]
136 [    0.314220] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
137 [    0.315057] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
138 [    0.316110] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
139 [    0.317059] pci_bus 0000:00: root bus resource [mem 0x08000000-0xfebfffff]
140 [    0.325852] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI
141 [    0.326111] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB
142 [    0.343104] acpi PNP0A03:00: ACPI _OSC support notification failed, disabling PCIe ASPM
143 [    0.344070] acpi PNP0A03:00: Unable to request _OSC control (_OSC support mask: 0x08)
144 [    0.350525] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
145 [    0.351557] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
146 [    0.353559] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
147 [    0.355146] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
148 [    0.357429] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
149 [    0.360731] ACPI: Enabled 16 GPEs in block 00 to 0F
150 [    0.364861] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
151 [    0.365073] vgaarb: loaded
152 [    0.366034] vgaarb: bridge control possible 0000:00:02.0
153 [    0.369180] SCSI subsystem initialized
154 [    0.370104] ACPI: bus type ATA registered
155 [    0.374000] pps_core: LinuxPPS API ver. 1 registered
156 [    0.374042] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
157 [    0.376055] PTP clock support registered
158 [    0.377651] PCI: Using ACPI for IRQ routing
159 [    0.387092] cfg80211: Calling CRDA to update world regulatory domain
160 [    0.390840] NetLabel: Initializing
161 [    0.391045] NetLabel:  domain hash size = 128
162 [    0.392028] NetLabel:  protocols = UNLABELED CIPSOv4
163 [    0.394131] NetLabel:  unlabeled traffic allowed by default
164 [    0.396357] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
165 [    0.397000] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
166 [    0.399258] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
167 [    0.403000] Switching to clocksource hpet
168 [    0.459290] pnp: PnP ACPI init
169 [    0.460062] ACPI: bus type PNP registered
170 [    0.472349] pnp: PnP ACPI: found 10 devices
171 [    0.473069] ACPI: bus type PNP unregistered
172 [    0.476546] kworker/u2:0 (369) used greatest stack depth: 7020 bytes left
173 [    0.549015] NET: Registered protocol family 2
174 [    0.553424] TCP established hash table entries: 1024 (order: 1, 8192 bytes)
175 [    0.555696] TCP bind hash table entries: 1024 (order: 1, 8192 bytes)
176 [    0.557204] TCP: Hash tables configured (established 1024 bind 1024)
177 [    0.559307] TCP: reno registered
178 [    0.561870] UDP hash table entries: 256 (order: 1, 8192 bytes)
179 [    0.563621] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
180 [    0.566137] NET: Registered protocol family 1
181 [    0.568983] RPC: Registered named UNIX socket transport module.
182 [    0.570720] RPC: Registered udp transport module.
183 [    0.571742] RPC: Registered tcp transport module.
184 [    0.572788] RPC: Registered tcp NFSv4.1 backchannel transport module.
185 [    0.576091] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
186 [    0.576429] pci 0000:00:01.0: PIIX3: Enabling Passive Release
187 [    0.581731] pci 0000:00:01.0: Activating ISA DMA hang workarounds
188 [    0.622091] microcode: CPU0 sig=0x663, pf=0x1, revision=0x0
189 [    0.624728] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
190 [    0.626546] Scanning for low memory corruption every 60 seconds
191 [    0.633912] audit: initializing netlink socket (disabled)
192 [    0.636554] type=2000 audit(1438765198.636:1): initialized
193 [    0.708201] HugeTLB registered 4 MB page size, pre-allocated 0 pages
194 [    0.743534] VFS: Disk quotas dquot_6.5.2
195 [    0.745062] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
196 [    0.752887] kworker/u2:0 (544) used greatest stack depth: 6980 bytes left
197 [    0.761762] NFS: Registering the id_resolver key type
198 [    0.763958] Key type id_resolver registered
199 [    0.764838] Key type id_legacy registered
200 [    0.766713] msgmni has been set to 231
201 [    0.777784] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
202 [    0.780426] io scheduler noop registered
203 [    0.781273] io scheduler deadline registered
204 [    0.783633] io scheduler cfq registered (default)
205 [    0.786663] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
206 [    0.791211] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
207 [    0.793137] ACPI: Power Button [PWRF]
208 [    0.814357] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
209 [    0.839611] 00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
210 [    0.852105] Non-volatile memory driver v1.3
211 [    0.853475] Linux agpgart interface v0.103
212 [    0.857962] [drm] Initialized drm 1.1.0 20060810
213 [    0.869001] loop: module loaded
214 [    0.886918] scsi0 : ata_piix
215 [    0.889793] scsi1 : ata_piix
216 [    0.892069] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc040 irq 14
217 [    0.893396] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc048 irq 15
218 [    0.903225] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
219 [    0.907034] serio: i8042 KBD port at 0x60,0x64 irq 1
220 [    0.908692] serio: i8042 AUX port at 0x60,0x64 irq 12
221 [    0.917325] mousedev: PS/2 mouse device common for all mice
222 [    0.921552] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
223 [    0.928758] device-mapper: ioctl: 4.24.0-ioctl (2013-01-15) initialised: dm-devel@redhat.com
224 [    0.931408] cpuidle: using governor ladder
225 [    0.932440] cpuidle: using governor menu
226 [    0.934504] hidraw: raw HID events driver (C) Jiri Kosina
227 [    0.941414] Netfilter messages via NETLINK v0.30.
228 [    0.944016] nf_conntrack version 0.5.0 (1852 buckets, 7408 max)
229 [    0.952440] ctnetlink v0.93: registering with nfnetlink.
230 [    0.956965] ip_tables: (C) 2000-2006 Netfilter Core Team
231 [    0.959743] TCP: cubic registered
232 [    0.960075] Initializing XFRM netlink socket
233 [    0.963648] NET: Registered protocol family 10
234 [    0.968998] ip6_tables: (C) 2000-2006 Netfilter Core Team
235 [    0.972067] sit: IPv6 over IPv4 tunneling driver
236 [    0.975865] NET: Registered protocol family 17
237 [    0.977894] Key type dns_resolver registered
238 [    0.980079] Using IPI No-Shortcut mode
239 [    0.987210] registered taskstats version 1
240 [    0.990722]   Magic number: 11:675:977
241 [    1.053051] ata2.00: ATAPI: QEMU DVD-ROM, 2.3.0, max UDMA/100
242 [    1.056210] ata2.00: configured for MWDMA2
243 [    1.064049] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.3. PQ: 0 ANSI: 5
244 [    1.072887] sr0: scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
245 [    1.074694] cdrom: Uniform CD-ROM driver Revision: 3.20
246 [    1.079892] sr 1:0:0:0: Attached scsi generic sg0 type 5
247 [    1.088087] Freeing unused kernel memory: 2124k freed
248 [    1.128077] Write protecting the kernel text: 5776k
249 [    1.129247] Write protecting the kernel read-only data: 1696k
250                                                             
251   *    *                                   ****       ****  
252  ***  ***     **        **      *    *    *    *     **     
253  * *  * *    *  *      *  *     *    *   *      *   **      
254  * *  * *   *    *    *    *    *    *   *      *    ***    
255  *  **  *   ******    *    *    *    *   *      *      **   
256  *      *   *         *    *    *    *   *      *       **  
257  *      *    *        *    *     *  **    *    *       **   
258  *      *     ***     *    *      **  *    ****     ****    
259                                                             
260 MenuOS>>[    1.536217] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2
261 [    1.614615] tsc: Refined TSC clocksource calibration: 3594.685 MHz
262 [    1.617405] Switching to clocksource tsc
263 
264 MenuOS>>help
265 help - Menu List
266     * help - Menu List
267     * version - MenuOS V1.0(Based on Linux 3.18.6)
268     * quit - Quit from MenuOS
269     * time - Show System Time
270     * time-asm - Show System Time(asm)
271 MenuOS>>quit
272 quit - Quit from MenuOS
273 MenuOS>>qemu: terminating on signal 15 from pid 5972
274 descent@debian32:~/linux-3.10.27/arch/x86/boot$ exit
275 
276 Script done on Wed 05 Aug 2015 05:00:09 PM CST

rootfs 是《Linux 内核分析》提供的, 從這裡下載:
git clone https://github.com/mengning/menu.git

我做了個小修改使其可以在 arm kernel 上執行。

使用以下指令編譯:

gcc -o init linktable.c menu.c test.c -m32 -static -lpthread

編譯相關指令

make i386_defconfig

make ARCH=arm menuconfig

make ARCH=arm versatile_defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcmrpi_defconfig

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=mnt/ext4 modules
sudo make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=mnt/ext4 modules_install

模擬器果然只是模擬器, 在 rpi 2 的真實機器上並沒有正常呈現畫面, 我的 dev 加入不少東西, 我也不知道哪個才是必要的。init 則要放在 /sbin/init

rpi2-rfs

 1 root@NB-debian:/media/2# ls /dev/ sbin/
 2 /dev/:
 3 autofs   mapper       sdb1      tty22  tty5     vboxdrvu
 4 block   mcelog       sdb2      tty23  tty50    vboxnetctl
 5 bsg   media0       sdb3      tty24  tty51    vboxusb
 6 btrfs-control  mei       serial    tty25  tty52    vcs
 7 bus   mem       sg0       tty26  tty53    vcs1
 8 cdrom   mmcblk0      sg1       tty27  tty54    vcs2
 9 cdrw   mmcblk0p1      sg2       tty28  tty55    vcs3
10 char   mmcblk0p2      shm       tty29  tty56    vcs4
11 console   mqueue       snapshot  tty3   tty57    vcs5
12 core   net       snd       tty30  tty58    vcs6
13 cpu   network_latency     sr0       tty31  tty59    vcs7
14 cpu_dma_latency  network_throughput  stderr    tty32  tty6     vcsa
15 cuse   null       stdin     tty33  tty60    vcsa1
16 disk   port       stdout    tty34  tty61    vcsa2
17 dri   ppp       tty       tty35  tty62    vcsa3
18 dvd   psaux       tty0      tty36  tty63    vcsa4
19 dvdrw   ptmx       tty1      tty37  tty7     vcsa5
20 fb0   pts       tty10     tty38  tty8     vcsa6
21 fd   random       tty11     tty39  tty9     vcsa7
22 full   rfkill       tty12     tty4   ttyS0    vfio
23 fuse   rtc       tty13     tty40  ttyS1    vga_arbiter
24 hidraw0   rtc0       tty14     tty41  ttyS2    vhci
25 hpet   sda       tty15     tty42  ttyS3    vhost-net
26 hugepages  sda1       tty16     tty43  ttyUSB0  video0
27 initctl   sda2       tty17     tty44  uhid     watchdog
28 input   sda3       tty18     tty45  uinput   watchdog0
29 kmsg   sda4       tty19     tty46  urandom  xconsole
30 kvm   sda5       tty2      tty47  usb      zero
31 log   sda6       tty20     tty48  v4l
32 loop-control  sdb       tty21     tty49  vboxdrv
33 
34 sbin/:
35 init 

以下是真實機器版本:

寄件者 20150614 raspberry pi 2

總算搞出來了, sd card 抽換的手快抽筋了。

在 arm 平台假如還有問題的話, 有可能還需要 /dev/ram0 並指定 kernel 參數 (也許不用, 但 /dev/ram0 一定要)
root=/dev/ram0

ex:
bootargs root=/dev/ram0 rw console=ttyS0,115200 mem=768M@0x00000000

ref:

• 用QEMU模拟树莓派Raspberry Pi的方法
• https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=6685
• http://raspberrypi.stackexchange.com/questions/10489/how-does-raspberry-pi-boot
• http://chaotangwang.blogspot.tw/2014/01/raspberry-pi-project-how-does-raspberry.html
• http://wiki.beyondlogic.org/index.php?title=Understanding_RaspberryPi_Boot_Process
• http://www.linuxquestions.org/questions/linux-software-2/creating-initramfs-as-inbuilt-to-the-kernel-and-as-permanent-rootfs-843651/
• Raspberry Pi bare metal forum
• Raspberry Pi firmware source code

製作 raspberry pi 2 linux 的檔案系統

如果要自己製作 linux 的檔案系統, 通常是一個小系統, 要不然官方提供的版本已經很好用了, 何必要自己做一個, 既然要小, 就會以 busybox 為主, 通常我都參考 build busybox and glic to root file system 來製作檔案系統。

busybox

wget http://busybox.net/downloads/busybox-1.23.2.tar.bz2
tar -xjf busybox-1.23.2.tar.bz2
cd busybox-1.23.2/
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- install CONFIG_PREFIX=/home/export/rootfs

glibc

wget http://ftp.gnu.org/gnu/libc/glibc-2.21.tar.xz
tar -xJf glibc-2.21.tar.xz
mkdir glibc-build
cd glibc-build/
../glibc-2.21/configure arm-linux-gnueabi --target=arm-linux-gnueabi --build=i686-pc-linux-gnu --prefix= --enable-add-ons

在我使用的 toolchain 中, 要用以下的指令
../glibc-2.21/configure arm-linux-gnueabihf --target=arm-linux-gnueabihf --build=i686-pc-linux-gnu --prefix= --enable-add-ons
make
make install install_root=/home/export/rootfs

再安裝 glibc 後就會有一些工具可用, ldconfig, ldd 之類的, 對於查詢執行檔能不能正常運作很有用, 通常我會再安裝 bash。bash 的 cross compile 比較麻煩, 我用 native compile 搞定一個 static 版本。

然後再從 toolchain 複製 c++ library (libstdc++*) 到 root filesystem usr/lib, 其實這不是好方法, 比較正確的方式是從 source code 建立 toolchain, 再複製 toolchain 所需的 library 到 root filesystem, 可是因為太麻煩, 也不容易, 改用這樣的方式比較簡單。

刪除一些 glibc 多語系和 .a 的檔案。產生 /dev files (可能需要 root 權限), 從使用的 linux 複製過來即可。

wget https://matt.ucc.asn.au/dropbear/releases/dropbear-2015.68.tar.bz2
tar -xjf dropbear-2015.68.tar.bz2
cd dropbear-2015.68
./configure --host=arm-linux-gnueabi --prefix=/ --disable-zlib CC=arm-linux-gnueabi-gcc LD=arm-linux-gnueabi-ld
make
make install DESTDIR=/home/export/rootfs

我用 chroot 來做以下的 dropbear 相關設定 (x86), rfs 是 root file system 目錄:
chroot  rfs/ /bin/sh

產生 ssh key, 需要 /dev/urandom
mkdir /etc/dropbear
dropbearkey -t dss -f /etc/dropbear/dropbear_dss_host_key  
dropbearkey -t rsa -f /etc/dropbear/dropbear_rsa_host_key 

Unless otherwise specified, root will be given a default home directory of /home/root. However as this doesn't exist, DropBear will close your connection immediately after successfully logging in. To address this, simply create a home directory for root:
mkdir /home /home/root

產生密碼檔
touch /etc/passwd # /bin/bash 改成 /bin/sh
touch /etc/group
adduser root -u 0

建立 pty device, ref: http://www.denx.de/wiki/view/DULG/TelnetServerNotWorking
# mkdir /dev/pts
# mknod c 5 2 /dev/ptmx
# mount -t devpts devpts /dev/pts

DropBear can now be started by running:

dropbear -E 

同常都是用這個設定檔, 取消 busybox "Please press Enter to activate this console" 不需要按下 enter 即可進入 console

ref: linux开机自启动，去掉“Please press Enter to activate this console“

busyboxy /etc/inittab

 1 ::sysinit:/etc/init.d/rcS
 2
 3 # /bin/ash
 4 #
 5 # Start an "askfirst" shell on the serial port
 6 console::respawn:-/bin/ash
 7 #console::askfirst:-/bin/ash # Please press Enter to activate this console
 8 #console::askfirst:-/bin/login
 9
10 # Stuff to do when restarting the init process
11 ::restart:/sbin/init
12
13 # Stuff to do before rebooting
14 ::ctrlaltdel:/sbin/reboot
15 ::shutdown:/bin/umount -a -r
16 ::shutdown:/sbin/swapoff -a

我在幾個 arm cortex A9 平台上都很順利, 惟獨在 rpi2 上, 製作的檔案系統無法正常開到 login/shell 畫面。製作一個 linux 檔案系統的學問很大, 通常做完開不了機是很正常的, 至少有 10 種以上的原因會造成這個結果, 這次遇到一個我不知道的原因, 所以還要再加上一個。

錯誤訊息是這樣:

[ 3.052569] smsc95xx v1.0.4
[ 3.116771] smsc95xx 1-1.1:1.0: eth0: register 'smsc95xx' at usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:f7:69:6d

在開到網路這部份就 hang 住了, 把訊息拿去 google, 有一堆, 有的說要改 config.txt, 有的要改 cmdline.txt, 當然都無法解決我的問題, 我猜測了幾個原因:

1. init 是不是又放錯位置了 (/sbin/init 才對)
2. init 沒有編好, 無法正常執行
3. init 需要的 library 沒有安置好
4. /dev/ 沒有需要的 device file
5. kernel 開機參數是不是有問題
6. root filesystem 沒有正確 mount
7. root filesystem 製作有問題
8. kernel 沒有正確編譯
9. sd partition 是不是有問題
10. sd 檔案系統是不是有問題
11. /etc/ 的設定是不是有問題

真的有 10 種以上的可能, 沒一個中 ...

linux_src/init/main.c L966 ~ 969 告訴我們 linux 怎麼啟動 init。

linux_src/init/main.c

 931 static int __ref kernel_init(void *unused)
 932 {
 933  int ret;
 934 
 935  kernel_init_freeable();
 936  /* need to finish all async __init code before freeing the memory */
 937  async_synchronize_full();
 938  free_initmem();
 939  mark_rodata_ro();
 940  system_state = SYSTEM_RUNNING;
 941  numa_default_policy();
 942 
 943  flush_delayed_fput();
 944 
 945  if (ramdisk_execute_command) {
 946   ret = run_init_process(ramdisk_execute_command);
 947   if (!ret)
 948    return 0;
 949   pr_err("Failed to execute %s (error %d)\n",
 950          ramdisk_execute_command, ret);
 951  }
 952 
 953  /*
 954   * We try each of these until one succeeds.
 955   *
 956   * The Bourne shell can be used instead of init if we are
 957   * trying to recover a really broken machine.
 958   */
 959  if (execute_command) {
 960   ret = run_init_process(execute_command);
 961   if (!ret)
 962    return 0;
 963   pr_err("Failed to execute %s (error %d).  Attempting defaults...\n",
 964    execute_command, ret);
 965  }
 966  if (!try_to_run_init_process("/sbin/init") ||
 967      !try_to_run_init_process("/etc/init") ||
 968      !try_to_run_init_process("/bin/init") ||
 969      !try_to_run_init_process("/bin/sh"))
 970   return 0;
 971 
 972  panic("No working init found.  Try passing init= option to kernel. "
 973        "See Linux Documentation/init.txt for guidance.");
 974 }

官方的檔案系統沒問題, 不過是 systemd 的設定, 不熟, 很難比對。

/etc/inittab L1 ~ L15 是原本的設定, 其結果是在 kernel 開機後就會直接進入 console, 不需要帳號/密碼。

/etc/inittab

 1 ::sysinit:/etc/init.d/rcS 
 2 
 3 # /bin/ash
 4 #
 5 # Start an "askfirst" shell on the serial port
 6 #console::askfirst:-/bin/ash
 7 #console::askfirst:-/bin/login
 8 
 9 # Stuff to do when restarting the init process
10 ::restart:/sbin/init
11 
12 # Stuff to do before rebooting
13 #::ctrlaltdel:/sbin/reboot
14 #::shutdown:/bin/umount -a -r
15 #::shutdown:/sbin/swapoff -a
16 
17 # /etc/inittab
18 #
19 # Copyright (C) 2001 Erik Andersen <andersen@codepoet.org>
20 #
21 # Note: BusyBox init doesn't support runlevels.  The runlevels field is
22 # completely ignored by BusyBox init. If you want runlevels, use
23 # sysvinit.
24 #
25 # Format for each entry: <id>:<runlevels>:<action>:<process>
26 #
27 # id        == tty to run on, or empty for /dev/console
28 # runlevels == ignored
29 # action    == one of sysinit, respawn, askfirst, wait, and once
30 # process   == program to run
31 
32 # Startup the system
33 null::sysinit:/bin/mount -t proc proc /proc
34 null::sysinit:/bin/mount -o remount,rw / # REMOUNT_ROOTFS_RW
35 null::sysinit:/bin/mkdir -p /dev/pts
36 null::sysinit:/bin/mkdir -p /dev/shm
37 null::sysinit:/bin/mount -a
38 null::sysinit:/bin/hostname -F /etc/hostname
39 # now run any rc scripts
40 ::sysinit:/etc/init.d/rcS
41 
42 # Put a getty on the serial port
43 tty1::respawn:/sbin/getty -L ttyAMA0 115200  # GENERIC_SERIAL
44 
45 # Stuff to do for the 3-finger salute
46 ::ctrlaltdel:/sbin/reboot
47 
48 # Stuff to do before rebooting
49 null::shutdown:/etc/init.d/rcK
50 null::shutdown:/bin/umount -a -r
51 null::shutdown:/sbin/swapoff -a

/etc/inir.d/rcS 可以加上:

mount -t proc none /proc
mount -t sysfs none /sys

不過最後卻需要/etc/inittab L43 的 ttyAMA0 這個設定才能正常以帳號/密碼的方式進入 console。所以還需要 /etc/passwd, /etc/group (從已經存在的 linux 系統複製即可) 記得把 root 密碼清除, 免得無法登入。

這花了我兩天才找到問題。

通常還可以複製 /dev/ 的裝置檔, 不用自己 mknod。

要製作 initfs 可以用以下指令:
cd rfs
find . | cpio -o -Hnewc |gzip -9 > ../rootfs.img

在 raspberry pi 2 上使用 u-boot

一般 soc 都會提供 u-boot 來當作 bootloader, 不過 rpi2 並沒有, u-boot 有很多豐富的功能, 在操作 rpi2 時, 也許有額外的助益, 來看看怎麼安裝 u-boot。

ref:
http://elinux.org/RPi_U-Boot

git clone git://github.com/swarren/u-boot.git
git checkout -b rpi_dev origin/rpi_dev

git clone git://git.denx.de/u-boot.git
make rpi_2_defconfig
make 
cp u-boot.bin /sd_card/part1
cp /sd_card/part1/config.txt /sd_card/part1/config.txt.bak
echo 'kernel=u-boot.bin' > /sd_card/part1/config.txt

u-boot 環境變數

dhcpuboot=usb start; dhcp u-boot.uimg; bootm
distro_bootcmd=for target in ${boot_targets}; do run bootcmd_${target}; done
fdt_addr_r=0x02000000
fdtfile=bcm2836-rpi-2-b.dtb
kernel_addr_r=0x01000000
loadaddr=0x00200000
mmc_boot=if mmc dev ${devnum}; then setenv devtype mmc; run scan_dev_for_boot_part; fi
pxefile_addr_r=0x00100000
ramdisk_addr_r=0x02100000
scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${bootpart}...; for prefix in ${boot_prefixes}; do rune
scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists devplist || setenv deve
scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${bootpart} ${prefix}extlinux/extlinux.conf; then echi
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${bootpart} ${prefixe
scriptaddr=0x00000000
soc=bcm283x
stderr=serial,lcd
stdin=serial,usbkbd
stdout=serial,lcd
usb_boot=usb start; if usb dev ${devnum}; then setenv devtype usb; run scan_dev_for_boot_part; fi
usbethaddr=b8:27:eb:90:b1:7b
vendor=raspberrypi




Environment size: 2188/16380 bytes
U-Boot> 

可以用 uart 接出來看到 u-boot 畫面。當然電視的畫面也有, 不過有點歪歪的就是。

u-boot 畫面

zImage 是我自己編譯出來的 kernel, 複製到 sdcard 上, 使用以下 u-boot 指令載入 kernel:

載入 linux

 1 # Tell Linux that it is booting on a Raspberry Pi2
 2 setenv machid 0x00000c42
 3 # Set the kernel boot command line
 4 setenv bootargs "earlyprintk console=tty0 console=ttyAMA0 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait noinitrd"
 5 # Save these changes to u-boot's environment
 6 saveenv
 7 # Load the existing Linux kernel into RAM
 8 fatload mmc 0:1 ${kernel_addr_r} zImage
 9 # Boot the kernel we have just loaded
10 bootz ${kernel_addr_r}

u-boot 已經愈來愈複雜了, 熟悉它不是壞事, 很多 soc 有提供了 u-boot, 也提供很複雜的載入指令, 在接觸到其他 soc 時, 就不會那麼陌生。

20230210 補充:

另外隨著 qemu (QEMU emulator version 7.2.0 (Debian 1:7.2+dfsg-2) ) 的開發, 目前也有 rpi2 的模擬器, 可以執行這個 u-boot, 但是時靈時不靈, 運氣好 u-boot 可以跑起來, 如 list 1。

qemu-system-arm -machine raspi2b -no-reboot -nographic -kernel ./u-boot

list 1. u-boot on qemu rpi2
  1 descent@debian-vm:u-boot$ ./qu.sh
  2
  3 U-Boot 2023.04-rc1-00335-gd1f5dbe664 (Feb 09 2023 - 15:12:42 +0800)
  4
  5 DRAM:  960 MiB
  6 RPI 2 Model B (0xa21041)
  7 Core:  63 devices, 14 uclasses, devicetree: embed
  8 MMC:   mmc@7e202000: 0
  9 Loading Environment from FAT... WARNING at drivers/mmc/bcm2835_sdhost.c:414/bcm2835_send_command()!
 10 WARNING at drivers/mmc/bcm2835_sdhost.c:414/bcm2835_send_command()!
 11 Card did not respond to voltage select! : -110
 12 ** Bad device specification mmc 0 **
 13 In:    serial
 14 Out:   vidconsole
 15 Err:   vidconsole
 16 Net:   No ethernet found.
 17 starting USB...
 18 Bus usb@7e980000: USB DWC2
 19 scanning bus usb@7e980000 for devices... 1 USB Device(s) found
 20        scanning usb for storage devices... 0 Storage Device(s) found
 21 Hit any key to stop autoboot:  0
 22 WARNING at drivers/mmc/bcm2835_sdhost.c:414/bcm2835_send_command()!
 23 WARNING at drivers/mmc/bcm2835_sdhost.c:414/bcm2835_send_command()!
 24 WARNING at drivers/mmc/bcm2835_sdhost.c:414/bcm2835_send_command()!
 25 Card did not respond to voltage select! : -110
 26 MMC Device 1 not found
 27 no mmc device at slot 1
 28 MMC Device 2 not found
 29 no mmc device at slot 2
 30
 31 Device 0: unknown device
 32 No ethernet found.
 33 missing environment variable: pxeuuid
 34 Retrieving file: pxelinux.cfg/00000000
 35 No ethernet found.
 36 Retrieving file: pxelinux.cfg/0000000
 37 No ethernet found.
 38 Retrieving file: pxelinux.cfg/000000
 39 No ethernet found.
 40 Retrieving file: pxelinux.cfg/00000
 41 No ethernet found.
 42 Retrieving file: pxelinux.cfg/0000
 43 No ethernet found.
 44 Retrieving file: pxelinux.cfg/000
 45 No ethernet found.
 46 Retrieving file: pxelinux.cfg/00
 47 No ethernet found.
 48 Retrieving file: pxelinux.cfg/0
 49 No ethernet found.
 50 Retrieving file: pxelinux.cfg/default-arm-bcm283x-rpi
 51 No ethernet found.
 52 Retrieving file: pxelinux.cfg/default-arm-bcm283x
 53 No ethernet found.
 54 Retrieving file: pxelinux.cfg/default-arm
 55 No ethernet found.
 56 Retrieving file: pxelinux.cfg/default
 57 No ethernet found.
 58 Config file not found
 59 No ethernet found.
 60 No ethernet found.
 61 U-Boot> help
 62 ?         - alias for 'help'
 63 base      - print or set address offset
 64 bdinfo    - print Board Info structure
 65 blkcache  - block cache diagnostics and control
 66 boot      - boot default, i.e., run 'bootcmd'
 67 bootd     - boot default, i.e., run 'bootcmd'
 68 bootefi   - Boots an EFI payload from memory
 69 bootelf   - Boot from an ELF image in memory
 70 bootflow  - Boot flows
 71 bootm     - boot application image from memory
 72 bootp     - boot image via network using BOOTP/TFTP protocol
 73 bootvx    - Boot vxWorks from an ELF image
 74 bootz     - boot Linux zImage image from memory
 75 cls       - clear screen
 76 cmp       - memory compare
 77 coninfo   - print console devices and information
 78 cp        - memory copy
 79 crc32     - checksum calculation
 80 dhcp      - boot image via network using DHCP/TFTP protocol
 81 dm        - Driver model low level access
 82 echo      - echo args to console
 83 editenv   - edit environment variable
 84 env       - environment handling commands
 85 exit      - exit script
 86 ext2load  - load binary file from a Ext2 filesystem
 87 ext2ls    - list files in a directory (default /)
 88 ext4load  - load binary file from a Ext4 filesystem
 89 ext4ls    - list files in a directory (default /)
 90 ext4size  - determine a file's size
 91 false     - do nothing, unsuccessfully
 92 fatinfo   - print information about filesystem
 93 fatload   - load binary file from a dos filesystem
 94 fatls     - list files in a directory (default /)
 95 fatmkdir  - create a directory
 96 fatrm     - delete a file
 97 fatsize   - determine a file's size
 98 fatwrite  - write file into a dos filesystem
 99 fdt       - flattened device tree utility commands
100 fstype    - Look up a filesystem type
101 fstypes   - List supported filesystem types
102 fsuuid    - Look up a filesystem UUID
103 go        - start application at address 'addr'
104 gpio      - query and control gpio pins
105 help      - print command description/usage
106 iminfo    - print header information for application image
107 imxtract  - extract a part of a multi-image
108 itest     - return true/false on integer compare
109 lcdputs   - print string on video framebuffer
110 ln        - Create a symbolic link
111 load      - load binary file from a filesystem
112 loadb     - load binary file over serial line (kermit mode)
113 loads     - load S-Record file over serial line
114 loadx     - load binary file over serial line (xmodem mode)
115 loady     - load binary file over serial line (ymodem mode)
116 loop      - infinite loop on address range
117 ls        - list files in a directory (default /)
118 md        - memory display
119 mii       - MII utility commands
120 mm        - memory modify (auto-incrementing address)
121 mmc       - MMC sub system
122 mmcinfo   - display MMC info
123 mw        - memory write (fill)
124 net       - NET sub-system
125 nfs       - boot image via network using NFS protocol
126 nm        - memory modify (constant address)
127 panic     - Panic with optional message
128 part      - disk partition related commands
129 ping      - send ICMP ECHO_REQUEST to network host
130 pinmux    - show pin-controller muxing
131 printenv  - print environment variables
132 pxe       - commands to get and boot from pxe files
133 random    - fill memory with random pattern
134 reset     - Perform RESET of the CPU
135 run       - run commands in an environment variable
136 save      - save file to a filesystem
137 saveenv   - save environment variables to persistent storage
138 setcurs   - set cursor position within screen
139 setenv    - set environment variables
140 setexpr   - set environment variable as the result of eval expression
141 showvar   - print local hushshell variables
142 size      - determine a file's size
143 sleep     - delay execution for some time
144 source    - run script from memory
145 sysboot   - command to get and boot from syslinux files
146 test      - minimal test like /bin/sh
147 tftpboot  - load file via network using TFTP protocol
148 true      - do nothing, successfully
149 usb       - USB sub-system
150 usbboot   - boot from USB device
151 version   - print monitor, compiler and linker version
152 U-Boot>



list 2. bdinfo
 1 U-Boot> bdinfo
 2 boot_params = 0x00000100
 3 DRAM bank   = 0x00000000
 4 -> start    = 0x00000000
 5 -> size     = 0x3c000000
 6 flashstart  = 0x00000000
 7 flashsize   = 0x00000000
 8 flashoffset = 0x00000000
 9 baudrate    = 115200 bps
10 relocaddr   = 0x3bf70000
11 reloc off   = 0x3bf68000
12 Build       = 32-bit
13 current eth = unknown
14 ethaddr     = 52:54:00:12:34:57
15 IP addr     = <NULL>
16 fdt_blob    = 0x00073510
17 new_fdt     = 0x00000000
18 fdt_size    = 0x00000000
19 Video       = hdmi@7e902000 active
20 FB base     = 0x3c100000
21 FB size     = 640x480x32
22 lmb_dump_all:
23  memory.cnt  = 0x1
24  memory[0]	[0x0-0x3bffffff], 0x3c000000 bytes flags: 0
25  reserved.cnt  = 0x3
26  reserved[0]	[0x0-0xfff], 0x00001000 bytes flags: 0
27  reserved[1]	[0x3ab67000-0x3bffffff], 0x01499000 bytes flags: 0
28  reserved[2]	[0x3bb67b90-0x3bffffff], 0x00498470 bytes flags: 0
29 devicetree  = embed
30 arch_number = 0x00000000
31 TLB addr    = 0x3bff0000
32 irq_sp      = 0x3bb6bea0
33 sp start    = 0x3bb6be90
34 Early malloc usage: 374 / 400
35 U-Boot>



ref:
Booting a Raspberry Pi2, with u-boot and HYP enabled

實作 spinlock on raspberry pi 2

這應該是我學習 os 拼圖的最後一塊了, 我已經會實作開機程式, context switch, system call, fork, exec, elf loader, 記憶體管理, mmu 設定, romfs 檔案系統, 剩下應該就是 process 的同步機制。這些主題深入下去都會花上不少時間, 我只取最簡單的實作方式, 主要觀念有完成就可以, 實作並不完美, 但由於我的簡化, 他們都變的很容易理解, 這是簡化後帶來的價值。剩下的工作便是把所有的努力組合起來, 這也是件大工程。os 當然還有其他主題, 不過我認為這樣應該對 os 有達到「略懂」的境界了。

process 同步機制有 spinlock, mutex, semaphore, 我的學習方式是簡化再簡化, 然後用程式碼實作他們, 否則我只會有「名詞」上的理解, 而不會真的理解, 有了 spinlock 就有了基本的 process 同步機制。

由於工作時間就耗掉了我生活的不少時間, 幾乎沒有時間學習一個大主題, 我的 compiler 學習之路只好緩一點 (在 os 上花了很大時間, 想換換口味), 偷空學習這些被我化繁為簡的零星主題。

以下的參考資料幫助我完成這個實作:

• Linux内核同步机制之（一）：原子操作
• Linux内核同步机制之（四）：spin lock
• Linux设备驱动开发详解:基于最新的Linux4.0内核 p164 代碼清單 7.1, 以及第七章, 這部份把容易搞混的 memory barrier, compiler/cpu 的亂序執行做了一個很好的說明。

由於要分解這些功能並簡化他們, 通常我會設計簡單的測試方式並輔以實作的小程式來理解他們, spinlock 的測試讓我大傷腦筋, 之前的想法是要先實作 process switch, 才能測試 spinlock, 我也的確做了某些成果, 不過在我愈來愈會設計這些小實驗後, 已經變得很擅長這件事情。我想到一個更好的方法, 不用先實作 process switch, 我在 rpi2 寫個 pthread 程式, 然後呼叫我自己寫的 my_spin_lock, my_spin_unlock 取代 pthread_spin_lock, pthread_spin_unlock, 若行為一樣, 就代表我成功了。

spinlock.c

  1 #include "spinlock.h"
  2 
  3 #include <stdio.h>
  4 
 37 
 38 void spinlock_init(Spinlock *spinlock)
 39 {
 40   spinlock->val_ = 0;
 41 }
 42 
 43 int spin_lock(Spinlock *spinlock)
 44 {
 77 #if 0
 78 lock_mutex:
 79   if(spinlock->val_ == 1)
 80     goto lock_mutex;
 81   else
 82     spinlock->val_ = 1;
 83 
 84 #endif
103         unsigned long tmp;
104         int result;
105 
106         __asm__ __volatile__("@ atomic_add\n"
107 "1:     ldrex   %0, [%3]\n"
108 "       cmp     %0, #1\n"
109        "beq 1b\n"
110        "mov %0, #1\n"
111 "       strex   %1, %0, [%3]\n"
112 "       teq     %1, #0\n"
113 "       bne     1b"
114         : "=&r" (result), "=&r" (tmp), "+Qo"(spinlock->val_)
115         : "r" (&spinlock->val_)
116         : "cc");
118 }
119 
120 int spin_unlock(Spinlock *spinlock)
121 {
122   spinlock->val_ = 0;
123 }
124 
125 #ifdef TEST
126 #include <pthread.h>
127 #include <signal.h>
128 
129 FILE *fs;
130 Spinlock sp;
131 pthread_spinlock_t spinlock;
132 
133 int run=1;
134 
135 void* write_file_1(void *p)
136 {
137   int time=0;
138   pthread_t tid = pthread_self();
139 
140   while(run)
141   {
142 #ifdef PTHREAD_FUNC
143     pthread_spin_lock(&spinlock);
144 #else
145     spin_lock(&sp);
146 #endif
147     //printf("111\n");
148     fprintf(fs, "%d ## thread 1 ## %d\n", time, tid);
149     fprintf(fs, "%d ## thread 11\n", time);
150     fprintf(fs, "%d ## thread 111\n", time);
151 #if 1
152 #ifdef PTHREAD_FUNC
153     pthread_spin_unlock(&spinlock);
154 #else
155     spin_unlock(&sp);
156 #endif
157 #endif
158     ++time;
159   }
160   // printf("thread 1 exit\n");
161 }
162 
163 void* write_file_2(void *p)
164 {
165   int time = 0;
166   pthread_t tid = pthread_self();
167 
168   while(run)
169   {
170 #ifdef PTHREAD_FUNC
171     pthread_spin_lock(&spinlock);
172 #else
173     spin_lock(&sp);
174 #endif
175     //printf("222\n");
176     fprintf(fs, "%d ## thread 2 long string 0123456789 ## %d\n", time, tid);
177     fprintf(fs, "%d ## thread 22 long string 0123456789\n", time);
178     fprintf(fs, "%d ## thread 222 long string 0123456789\n", time);
179 #ifdef PTHREAD_FUNC
180     pthread_spin_unlock(&spinlock);
181 #else
182     spin_unlock(&sp);
183 #endif
184     ++time;
185   }
186   // printf("thread 2 exit\n");
187 }
188 
189 void* write_file_3(void *p)
190 {
191   int time = 0;
192   pthread_t tid = pthread_self();
193   int i;
194 
195   while(run)
196   {
197 #ifdef PTHREAD_FUNC
198     pthread_spin_lock(&spinlock);
199 #else
200     spin_lock(&sp);
201 #endif
202     fprintf(fs, "%d ## thread 3 long string ABCDEFGHIJKLMNOPQRSTUVWXYZ!@#$%^&*()~ ## %d\n", time, tid);
203     fprintf(fs, "%d ## thread 33 long string ABCDEFGHIJKLMNOPQRSTUVWXYZ!@#$%^&*()~\n", time);
204     fprintf(fs, "%d ## thread 333 long string ABCDEFGHIJKLMNOPQRSTUVWXYZ!@#$%^&*()~\n", time);
205 #ifdef PTHREAD_FUNC
206     pthread_spin_unlock(&spinlock);
207 #else
208     spin_unlock(&sp);
209 #endif
210     ++time;
211   }
212   // printf("thread 2 exit\n");
213 }
214 
215 static void* sig_thread (void *arg)
216 {
217   sigset_t *set = arg;
218   int s, sig;
219 
220   printf("sig thread pid: %d\n", getpid());
221 
222   for (;;)
223   {
224     s = sigwait (set, &sig);
225     if (s == 0)
226     {
227       printf ("Signal handling thread got signal %d\n", sig);
228       if (sig == SIGINT)
229       {
230         run = 0;
231         //break;
232       }
233     }
234   }
235 }
236 
237 int ret1, ret2, ret3;
238 
239 int main(int argc, char *argv[])
240 {
241   pthread_t thread0, thread1, thread2, thread;
242   sigset_t set;
243 
244 #ifdef PTHREAD_FUNC
245   pthread_spin_init(&spinlock, 0);
246   printf("init pthread spinlock\n");
247 #else
248   spinlock_init(&sp);
249   printf("init my spinlock\n");
250 #endif
251 
252   sigemptyset (&set);
253   sigaddset (&set, SIGQUIT);
254   sigaddset (&set, SIGINT);
255   int s = pthread_sigmask (SIG_BLOCK, &set, NULL);
256   if (s != 0)
257   {
258     perror("pthread_sigmask");
259     return -1;
260   }
261 
262 
263   printf("open %s to write\n", FN);
264   fs = fopen(FN, "w");
265   if (fs == NULL) 
266   {
267     perror("open fail");
268     return -1;
269   }
270 
271   pthread_create (&thread, NULL, &sig_thread, (void *) &set);
272   pthread_create(&thread0, NULL, write_file_1, NULL);
273   pthread_create(&thread1, NULL, write_file_2, NULL);
274   pthread_create(&thread2, NULL, write_file_3, NULL);
275 
276   pthread_join(thread0, (void **)&ret1);
277   pthread_join(thread1, (void **)&ret2);
278   pthread_join(thread2, (void **)&ret3);
279 
280   fclose(fs);
281   printf("test end\n");
282   
283   return 0;
284 }
285 #endif

spinlock.c L77 ~ 84 是 c 版本的演算法, 為什麼不能用這個版本, 因為需要 atomic 操作, 這大家都知道, 不多說了; 困難的是怎麼實作 atomic, spinlock.c L103 ~ 116 是我參考 linux 3.10.37 arch/arm/include/asm/atomic.h atomic_add 改出來的 (因為我自己寫的都有問題 ... 冏), 這 inline assembly 實在太難, 我沒能完全看懂, ldrex/strex 可以在 armv6, cortex armv7-A 上使用, 我在 rpi2 執行這個測試, 這個版本應該也可以在 cortex m3 上執行, 這才是我真正的目的。

x86 可參考 arch/x86/include/asm/atomic.h, 或是直接參考 arch/x86/include/asm/spinlock.h 的實作。

3.10.27/arch/arm/include/asm/spinlock.h

74 static inline void arch_spin_lock(arch_spinlock_t *lock)
75 {
76 unsigned long tmp;
77 u32 newval;
78 arch_spinlock_t lockval;
79
80 __asm__ __volatile__(
81 "1: ldrex %0, [%3]\n"
82 " add %1, %0, %4\n"
83 " strex %2, %1, [%3]\n"
84 " teq %2, #0\n"
85 " bne 1b"
86 : "=&r" (lockval), "=&r" (newval), "=&r" (tmp)
87 : "r" (&lock->slock), "I" (1 << TICKET_SHIFT)
88 : "cc");
89
90 while (lockval.tickets.next != lockval.tickets.owner) {
91 wfe();
92 lockval.tickets.owner = ACCESS_ONCE(lock->tickets.owner);
93 }
94
95 smp_mb();
96 }

3.10.27/arch/arm/include/asm/atomic.h

  1 /*
  2  *  arch/arm/include/asm/atomic.h
  3  *
  4  *  Copyright (C) 1996 Russell King.
  5  *  Copyright (C) 2002 Deep Blue Solutions Ltd.
  6  *
  7  * This program is free software; you can redistribute it and/or modify
  8  * it under the terms of the GNU General Public License version 2 as
  9  * published by the Free Software Foundation.
 10  */
 11 #ifndef __ASM_ARM_ATOMIC_H
 12 #define __ASM_ARM_ATOMIC_H
 13 
 14 #include <linux/compiler.h>
 15 #include <linux/types.h>
 16 #include <linux/irqflags.h>
 17 #include <asm/barrier.h>
 18 #include <asm/cmpxchg.h>
 19 
 20 #define ATOMIC_INIT(i) { (i) }
 21 
 22 #ifdef __KERNEL__
 23 
 24 /*
 25  * On ARM, ordinary assignment (str instruction) doesn't clear the local
 26  * strex/ldrex monitor on some implementations. The reason we can use it for
 27  * atomic_set() is the clrex or dummy strex done on every exception return.
 28  */
 29 #define atomic_read(v) (*(volatile int *)&(v)->counter)
 30 #define atomic_set(v,i) (((v)->counter) = (i))
 31 
 32 #if __LINUX_ARM_ARCH__ >= 6
 33 
 34 /*
 35  * ARMv6 UP and SMP safe atomic ops.  We use load exclusive and
 36  * store exclusive to ensure that these are atomic.  We may loop
 37  * to ensure that the update happens.
 38  */
 39 static inline void atomic_add(int i, atomic_t *v)
 40 {
 41  unsigned long tmp;
 42  int result;
 43 
 44  __asm__ __volatile__("@ atomic_add\n"
 45 "1: ldrex %0, [%3]\n"
 46 " add %0, %0, %4\n"
 47 " strex %1, %0, [%3]\n"
 48 " teq %1, #0\n"
 49 " bne 1b"
 50  : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter)
 51  : "r" (&v->counter), "Ir" (i)
 52  : "cc");
 53 }

程式有 3 個 thread 在寫同一個檔案 /tmp/xyz1, 按下 ctrl-c 會收到 SIGINT, 程式會正確的處理這個 SIGINT 然後結束整個程式 (這並不容易, 請參考《thread 和 signal》)。spinlock test result 為失敗與成功的內容。一開始的實作並不正確, 所以我特別用了 pthread_spin_lock/pthread_spin_unlock 來測試, 而使用pthread_spin_lock/pthread_spin_unlock 的確會得到正確的結果, 所以我一開始的實作是有問題的。

spinlock test result

失敗:
237 ## thread 1 ## 1985418336
237 ## thread 3 long string ABCDEFGHIJKLMNOPQRSTUVWXYZ!@#$%^&*()~ ## 1968641120
206 ## thread 2 long string 0123456789 ## 1977029728

成功:
6 ## thread 1 ## 1985877088
6 ## thread 11
6 ## thread 111
0 ## thread 2 long string 0123456789 ## 1977488480
0 ## thread 22 long string 0123456789
0 ## thread 222 long string 0123456789
7 ## thread 1 ## 1985877088
7 ## thread 11
7 ## thread 111
1 ## thread 2 long string 0123456789 ## 1977488480
1 ## thread 22 long string 0123456789
1 ## thread 222 long string 0123456789

每個 thread 應該以 3 行為輸出單位, 若是被混在一起, 表示雖然有某個 thread 取得了 spinlock, 但其他的 thread 並沒有 busy loop, 而是也取得了同樣的 spinlock 並進入了 critical section, 造成了失敗的結果。

check_result.c 則是用來檢查 /tmp/xyz1 是否是正確。整個測試在 rpi2 上完成。

不知道是不是還需要 memory barrier, dsb, isb, dmb 這些指令 (其實應該要的)。驗證 spinlock function 是否正確非常困難, 我也不確定這個版本一定是對的, 因為只要有一個失敗案例, 這個 spinlock 實作就不正確了, 若用上了有 bug 的 spinlock function, 那程式會很難除錯, 若核4 有這樣的程式碼應該會嚇死不少程式人。

spinlock 做出來了, 那 mutex 呢? 把 spinlock.c L109 改成讓出 cpu 的程式碼就可以了, 這就是俗稱的去「睡覺」。你說要怎麼做? 這在 process switch 的階段就已經會了, 所以你得搞懂那個才行。

實作完 spinlock 後還有使用 spinlock 的議題, 非常的複雜, 感覺起來 mutex 是比較好的, 那為什麼還要設計 spinlock, 又中斷部份的程式碼為什麼要 spinlock 而不用 mutex 呢?

source code:
https://github.com/descent/progs/tree/master/spinlock

ref:

• 理解 Memory barrier (内存屏障)
• Memory Barrier in Lock API
• Linux x86 spinlock 實現之分析
• 分析Linux中Spinlock在ARM及X86平台上的实现
• semaphore, mutex, spin lock

DMB, DSB, ISB:

• http://www.07net01.com/linux/ARM_Cortex_M3_xuexibiji_4_5__14436_1347159341.html
• http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0489c/CIHGHHIE.html

add new system call for arm linux

網路上有很多增加一個 system call for arm linux 的文章, 不過都有些問題, 不是過時了就是那裡怪怪的。

我用 raspberry pi 2 作為平台, linx 4.1.y 為練習的版本, 還蠻新的, 應該可以撐好幾年。

總共要改這些:
modified: arch/arm/include/asm/unistd.h
modified: arch/arm/include/uapi/asm/unistd.h
modified: include/linux/syscalls.h
modified: arch/arm/kernel/Makefile
modified: arch/arm/kernel/calls.S
new file: arch/arm/kernel/my_syscall.c

參考 add_arm_linux_new_syscall.patch, 值得注意的是 L10 從 388 改成 392, 奇怪, 不是才加入一個 system call 到 calls.S 嗎? 怎麼不是 389?? 大栽問?

add_arm_linux_new_syscall.patch

  1 diff --git a/arch/arm/include/asm/unistd.h b/arch/arm/include/asm/unistd.h
  2 index 32640c4..7cba573 100644
  3 --- a/arch/arm/include/asm/unistd.h
  4 +++ b/arch/arm/include/asm/unistd.h
  5 @@ -19,7 +19,7 @@
  6   * This may need to be greater than __NR_last_syscall+1 in order to
  7   * account for the padding in the syscall table
  8   */
  9 -#define __NR_syscalls  (388)
 10 +#define __NR_syscalls  (392)
 11  
 12  /*
 13   * *NOTE*: This is a ghost syscall private to the kernel.  Only the
 14 diff --git a/arch/arm/include/uapi/asm/unistd.h b/arch/arm/include/uapi/asm/unistd.h
 15 index 0c3f5a0..20031a2 100644
 16 --- a/arch/arm/include/uapi/asm/unistd.h
 17 +++ b/arch/arm/include/uapi/asm/unistd.h
 18 @@ -414,6 +414,7 @@
 19  #define __NR_memfd_create  (__NR_SYSCALL_BASE+385)
 20  #define __NR_bpf   (__NR_SYSCALL_BASE+386)
 21  #define __NR_execveat   (__NR_SYSCALL_BASE+387)
 22 +#define __NR_get_mmu_reg  (__NR_SYSCALL_BASE+388)
 23  
 24  /*
 25   * The following SWIs are ARM private.
 26 diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile
 27 index 752725d..c6fc56d 100644
 28 --- a/arch/arm/kernel/Makefile
 29 +++ b/arch/arm/kernel/Makefile
 30 @@ -18,7 +18,7 @@ CFLAGS_REMOVE_return_address.o = -pg
 31  obj-y  := elf.o entry-common.o irq.o opcodes.o \
 32       process.o ptrace.o reboot.o return_address.o \
 33       setup.o signal.o sigreturn_codes.o \
 34 -     stacktrace.o sys_arm.o time.o traps.o
 35 +     stacktrace.o sys_arm.o time.o traps.o my_syscall.o
 36  
 37  obj-$(CONFIG_ATAGS)  += atags_parse.o
 38  obj-$(CONFIG_ATAGS_PROC) += atags_proc.o
 39 diff --git a/arch/arm/kernel/calls.S b/arch/arm/kernel/calls.S
 40 index 05745eb..a715f70 100644
 41 --- a/arch/arm/kernel/calls.S
 42 +++ b/arch/arm/kernel/calls.S
 43 @@ -397,6 +397,7 @@
 44  /* 385 */ CALL(sys_memfd_create)
 45    CALL(sys_bpf)
 46    CALL(sys_execveat)
 47 +  CALL(sys_get_mmu_reg)
 48  #ifndef syscalls_counted
 49  .equ syscalls_padding, ((NR_syscalls + 3) & ~3) - NR_syscalls
 50  #define syscalls_counted
 51 diff --git a/arch/arm/kernel/my_syscall.c b/arch/arm/kernel/my_syscall.c
 52 new file mode 100644
 53 index 0000000..d61144f
 54 --- /dev/null
 55 +++ b/arch/arm/kernel/my_syscall.c
 56 @@ -0,0 +1,87 @@
 57 +#include <linux/linkage.h>
 58 +#include <linux/kernel.h>
 59 +#include <linux/syscalls.h>
 60 +
 61 +
 62 +typedef unsigned int u32;
 63 +
 64 +static void get_ttbr1(u32 *ttbr1)
 65 +{
 66 +  asm
 67 +  (
 68 +    "mrc p15,0,%0,c2,c0,1\n"
 69 +    :"=r"(*ttbr1)
 70 +    :
 71 +    :
 72 +  );
 73 +/*
 74 + * ref: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0211h/Bhchbicd.html
 75 +MRC p15, 0, <Rd>, c2, c0, 1            ; Read Translation Table Base Register 1
 76 +MCR p15, 0, <Rd>, c2, c0, 1            ; Write Translation Table Base Register 1 
 77 +*/
 78 +}
 79 +
 80 +static void get_ttbr0(u32 *ttbr0)
 81 +{
 82 +  u32 a;
 83 +  asm
 84 +  (
 85 +    "mrc p15,0,%0,c2,c0,0\n"
 86 +    :"=r"(a)
 87 +    :
 88 +    :
 89 +  );
 90 +  *ttbr0 = a;
 91 +  printk(KERN_ALERT "a: %x\n", a);
 92 +}
 93 +
 94 +
 95 +static void get_ttbcr(u32 *reg)
 96 +{
 97 +  asm
 98 +  (
 99 +    "mrc p15,0,%0,c2,c0,2\n"
100 +    :"=r"(*reg)
101 +    :
102 +    :
103 +  );
104 +
105 +#if 0
106 +http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0211k/Bihgfcgf.html
107 +MRC p15, 0, <Rd>, c2, c0, 2      ; Read Translation Table Base Control Register
108 +MCR p15, 0, <Rd>, c2, c0, 2      ; Write Translation Table Base Control Register
109 +#endif
110 +}
111 +
112 +
113 +SYSCALL_DEFINE1(get_mmu_reg, unsigned int, reg_type)
114 +{
115 +  int reg = 0x12345678;
116 +  printk(KERN_ALERT "reg_type: %x\n", reg_type);
117 +  switch (reg_type)
118 +  {
119 +    case 0:
120 +    {
121 +      get_ttbr0(&reg);
122 +      break;
123 +    }
124 +    case 1:
125 +    {
126 +      get_ttbr1(&reg);
127 +      break;
128 +    }
129 +    case 2:
130 +    {
131 +      get_ttbcr(&reg);
132 +      break;
133 +    }
134 +    default:
135 +    {
136 +      printk(KERN_ALERT "not support reg_type: %x\n", reg_type);
137 +      return -5;
138 +      break;
139 +    }
140 +  }
141 +  return reg;
142 +}
143 +
144 diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
145 index 76d1e38..f1aac22 100644
146 --- a/include/linux/syscalls.h
147 +++ b/include/linux/syscalls.h
148 @@ -701,6 +701,7 @@ asmlinkage long sys_sysfs(int option,
149  asmlinkage long sys_syslog(int type, char __user *buf, int len);
150  asmlinkage long sys_uselib(const char __user *library);
151  asmlinkage long sys_ni_syscall(void);
152 +asmlinkage long sys_get_mmu_reg(unsigned int reg_type);
153  asmlinkage long sys_ptrace(long request, long pid, unsigned long addr,
154        unsigned long data);
155  
156 @@ -825,6 +826,7 @@ asmlinkage long sys_fanotify_mark(int fanotify_fd, unsigned int flags,
157  asmlinkage long sys_syncfs(int fd);
158  
159  asmlinkage long sys_fork(void);
160 +asmlinkage long sys_get_mmu_reg(unsigned int reg_type);
161  asmlinkage long sys_vfork(void);
162  #ifdef CONFIG_CLONE_BACKWARDS
163  asmlinkage long sys_clone(unsigned long, unsigned long, int __user *, int,

如果單純的 +1, compile 很有可能會看到:

arch/arm/kernel/entry-common.S:110: Error: __NR_syscalls is not equal to the size of the syscall table

原因是 arch/arm/kernel/entry-common.S, arch/arm/kernel/calls.S 在計算 a.c L1043~L1045, 沒符合程式邏輯罷了, 解決方法很簡單, 拿到這段檢查程式就好了, 優雅的方法就是去計算其規則, 讓結果符合這個檢查, 392 就是我算出的結果。

cpp.sh L1 是我將 arch/arm/kernel/entry-common.S 展開, L2 是單純的用 as compile, 照著規則算出來看多少才會過關。

這段很難說明, 所以就不說了, a.c L1029 是關鍵, 補上需要的數量, 真的搞不定就拿掉吧!

a.c

 634 
 635  .equ NR_syscalls,0
 636 
 637 # 1 "arch/arm/kernel/calls.S" 1
 638 # 12 "arch/arm/kernel/calls.S"
 639          .equ NR_syscalls,NR_syscalls+1
 640   .equ NR_syscalls,NR_syscalls+1
 641   .equ NR_syscalls,NR_syscalls+1
 642   .equ NR_syscalls,NR_syscalls+1
 643   .equ NR_syscalls,NR_syscalls+1
 644          .equ NR_syscalls,NR_syscalls+1
 645   .equ NR_syscalls,NR_syscalls+1
 646   .equ NR_syscalls,NR_syscalls+1
 647   .equ NR_syscalls,NR_syscalls+1
 648   .equ NR_syscalls,NR_syscalls+1
 649          .equ NR_syscalls,NR_syscalls+1
 650   .equ NR_syscalls,NR_syscalls+1
 651   .equ NR_syscalls,NR_syscalls+1
 652   .equ NR_syscalls,NR_syscalls+1
 653   .equ NR_syscalls,NR_syscalls+1
 654          .equ NR_syscalls,NR_syscalls+1
 655   .equ NR_syscalls,NR_syscalls+1
 656   .equ NR_syscalls,NR_syscalls+1
 657   .equ NR_syscalls,NR_syscalls+1
 658   .equ NR_syscalls,NR_syscalls+1
 659          .equ NR_syscalls,NR_syscalls+1
 660   .equ NR_syscalls,NR_syscalls+1
 661   .equ NR_syscalls,NR_syscalls+1
 662   .equ NR_syscalls,NR_syscalls+1
 663   .equ NR_syscalls,NR_syscalls+1
 664          .equ NR_syscalls,NR_syscalls+1
 665   .equ NR_syscalls,NR_syscalls+1
 666   .equ NR_syscalls,NR_syscalls+1
 667   .equ NR_syscalls,NR_syscalls+1
 668   .equ NR_syscalls,NR_syscalls+1
 669          .equ NR_syscalls,NR_syscalls+1
 670   .equ NR_syscalls,NR_syscalls+1
 671   .equ NR_syscalls,NR_syscalls+1
 672   .equ NR_syscalls,NR_syscalls+1
 673   .equ NR_syscalls,NR_syscalls+1
 674          .equ NR_syscalls,NR_syscalls+1
 675   .equ NR_syscalls,NR_syscalls+1
 676   .equ NR_syscalls,NR_syscalls+1
 677   .equ NR_syscalls,NR_syscalls+1
 678   .equ NR_syscalls,NR_syscalls+1
 679          .equ NR_syscalls,NR_syscalls+1
 680   .equ NR_syscalls,NR_syscalls+1
 681   .equ NR_syscalls,NR_syscalls+1
 682   .equ NR_syscalls,NR_syscalls+1
 683   .equ NR_syscalls,NR_syscalls+1
 684          .equ NR_syscalls,NR_syscalls+1
 685   .equ NR_syscalls,NR_syscalls+1
 686   .equ NR_syscalls,NR_syscalls+1
 687   .equ NR_syscalls,NR_syscalls+1
 688   .equ NR_syscalls,NR_syscalls+1
 689          .equ NR_syscalls,NR_syscalls+1
 690   .equ NR_syscalls,NR_syscalls+1
 691   .equ NR_syscalls,NR_syscalls+1
 692   .equ NR_syscalls,NR_syscalls+1
 693   .equ NR_syscalls,NR_syscalls+1
 694          .equ NR_syscalls,NR_syscalls+1
 695   .equ NR_syscalls,NR_syscalls+1
 696   .equ NR_syscalls,NR_syscalls+1
 697   .equ NR_syscalls,NR_syscalls+1
 698   .equ NR_syscalls,NR_syscalls+1
 699          .equ NR_syscalls,NR_syscalls+1
 700   .equ NR_syscalls,NR_syscalls+1
 701   .equ NR_syscalls,NR_syscalls+1
 702   .equ NR_syscalls,NR_syscalls+1
 703   .equ NR_syscalls,NR_syscalls+1
 704          .equ NR_syscalls,NR_syscalls+1
 705   .equ NR_syscalls,NR_syscalls+1
 706   .equ NR_syscalls,NR_syscalls+1
 707   .equ NR_syscalls,NR_syscalls+1
 708   .equ NR_syscalls,NR_syscalls+1
 709          .equ NR_syscalls,NR_syscalls+1
 710   .equ NR_syscalls,NR_syscalls+1
 711   .equ NR_syscalls,NR_syscalls+1
 712   .equ NR_syscalls,NR_syscalls+1
 713   .equ NR_syscalls,NR_syscalls+1
 714          .equ NR_syscalls,NR_syscalls+1
 715   .equ NR_syscalls,NR_syscalls+1
 716   .equ NR_syscalls,NR_syscalls+1
 717   .equ NR_syscalls,NR_syscalls+1
 718   .equ NR_syscalls,NR_syscalls+1
 719          .equ NR_syscalls,NR_syscalls+1
 720   .equ NR_syscalls,NR_syscalls+1
 721   .equ NR_syscalls,NR_syscalls+1
 722   .equ NR_syscalls,NR_syscalls+1
 723   .equ NR_syscalls,NR_syscalls+1
 724          .equ NR_syscalls,NR_syscalls+1
 725   .equ NR_syscalls,NR_syscalls+1
 726   .equ NR_syscalls,NR_syscalls+1
 727   .equ NR_syscalls,NR_syscalls+1
 728   .equ NR_syscalls,NR_syscalls+1
 729          .equ NR_syscalls,NR_syscalls+1
 730   .equ NR_syscalls,NR_syscalls+1
 731   .equ NR_syscalls,NR_syscalls+1
 732   .equ NR_syscalls,NR_syscalls+1
 733   .equ NR_syscalls,NR_syscalls+1
 734          .equ NR_syscalls,NR_syscalls+1
 735   .equ NR_syscalls,NR_syscalls+1
 736   .equ NR_syscalls,NR_syscalls+1
 737   .equ NR_syscalls,NR_syscalls+1
 738   .equ NR_syscalls,NR_syscalls+1
 739           .equ NR_syscalls,NR_syscalls+1
 740   .equ NR_syscalls,NR_syscalls+1
 741   .equ NR_syscalls,NR_syscalls+1
 742   .equ NR_syscalls,NR_syscalls+1
 743   .equ NR_syscalls,NR_syscalls+1
 744           .equ NR_syscalls,NR_syscalls+1
 745   .equ NR_syscalls,NR_syscalls+1
 746   .equ NR_syscalls,NR_syscalls+1
 747   .equ NR_syscalls,NR_syscalls+1
 748   .equ NR_syscalls,NR_syscalls+1
 749           .equ NR_syscalls,NR_syscalls+1
 750   .equ NR_syscalls,NR_syscalls+1
 751   .equ NR_syscalls,NR_syscalls+1
 752   .equ NR_syscalls,NR_syscalls+1
 753   .equ NR_syscalls,NR_syscalls+1
 754           .equ NR_syscalls,NR_syscalls+1
 755   .equ NR_syscalls,NR_syscalls+1
 756   .equ NR_syscalls,NR_syscalls+1
 757   .equ NR_syscalls,NR_syscalls+1
 758   .equ NR_syscalls,NR_syscalls+1
 759           .equ NR_syscalls,NR_syscalls+1
 760   .equ NR_syscalls,NR_syscalls+1
 761   .equ NR_syscalls,NR_syscalls+1
 762   .equ NR_syscalls,NR_syscalls+1
 763   .equ NR_syscalls,NR_syscalls+1
 764           .equ NR_syscalls,NR_syscalls+1
 765   .equ NR_syscalls,NR_syscalls+1
 766   .equ NR_syscalls,NR_syscalls+1
 767   .equ NR_syscalls,NR_syscalls+1
 768   .equ NR_syscalls,NR_syscalls+1
 769           .equ NR_syscalls,NR_syscalls+1
 770   .equ NR_syscalls,NR_syscalls+1
 771   .equ NR_syscalls,NR_syscalls+1
 772   .equ NR_syscalls,NR_syscalls+1
 773   .equ NR_syscalls,NR_syscalls+1
 774           .equ NR_syscalls,NR_syscalls+1
 775   .equ NR_syscalls,NR_syscalls+1
 776   .equ NR_syscalls,NR_syscalls+1
 777   .equ NR_syscalls,NR_syscalls+1
 778   .equ NR_syscalls,NR_syscalls+1
 779           .equ NR_syscalls,NR_syscalls+1
 780   .equ NR_syscalls,NR_syscalls+1
 781   .equ NR_syscalls,NR_syscalls+1
 782   .equ NR_syscalls,NR_syscalls+1
 783   .equ NR_syscalls,NR_syscalls+1
 784           .equ NR_syscalls,NR_syscalls+1
 785   .equ NR_syscalls,NR_syscalls+1
 786   .equ NR_syscalls,NR_syscalls+1
 787   .equ NR_syscalls,NR_syscalls+1
 788   .equ NR_syscalls,NR_syscalls+1
 789           .equ NR_syscalls,NR_syscalls+1
 790   .equ NR_syscalls,NR_syscalls+1
 791   .equ NR_syscalls,NR_syscalls+1
 792   .equ NR_syscalls,NR_syscalls+1
 793   .equ NR_syscalls,NR_syscalls+1
 794           .equ NR_syscalls,NR_syscalls+1
 795   .equ NR_syscalls,NR_syscalls+1
 796   .equ NR_syscalls,NR_syscalls+1
 797   .equ NR_syscalls,NR_syscalls+1
 798   .equ NR_syscalls,NR_syscalls+1
 799           .equ NR_syscalls,NR_syscalls+1
 800   .equ NR_syscalls,NR_syscalls+1
 801   .equ NR_syscalls,NR_syscalls+1
 802   .equ NR_syscalls,NR_syscalls+1
 803   .equ NR_syscalls,NR_syscalls+1
 804           .equ NR_syscalls,NR_syscalls+1
 805   .equ NR_syscalls,NR_syscalls+1
 806   .equ NR_syscalls,NR_syscalls+1
 807   .equ NR_syscalls,NR_syscalls+1
 808   .equ NR_syscalls,NR_syscalls+1
 809           .equ NR_syscalls,NR_syscalls+1
 810   .equ NR_syscalls,NR_syscalls+1
 811   .equ NR_syscalls,NR_syscalls+1
 812   .equ NR_syscalls,NR_syscalls+1
 813   .equ NR_syscalls,NR_syscalls+1
 814           .equ NR_syscalls,NR_syscalls+1
 815   .equ NR_syscalls,NR_syscalls+1
 816   .equ NR_syscalls,NR_syscalls+1
 817   .equ NR_syscalls,NR_syscalls+1
 818   .equ NR_syscalls,NR_syscalls+1
 819           .equ NR_syscalls,NR_syscalls+1
 820   .equ NR_syscalls,NR_syscalls+1
 821   .equ NR_syscalls,NR_syscalls+1
 822   .equ NR_syscalls,NR_syscalls+1
 823   .equ NR_syscalls,NR_syscalls+1
 824           .equ NR_syscalls,NR_syscalls+1
 825   .equ NR_syscalls,NR_syscalls+1
 826   .equ NR_syscalls,NR_syscalls+1
 827   .equ NR_syscalls,NR_syscalls+1
 828   .equ NR_syscalls,NR_syscalls+1
 829           .equ NR_syscalls,NR_syscalls+1
 830   .equ NR_syscalls,NR_syscalls+1
 831   .equ NR_syscalls,NR_syscalls+1
 832   .equ NR_syscalls,NR_syscalls+1
 833   .equ NR_syscalls,NR_syscalls+1
 834           .equ NR_syscalls,NR_syscalls+1
 835   .equ NR_syscalls,NR_syscalls+1
 836   .equ NR_syscalls,NR_syscalls+1
 837   .equ NR_syscalls,NR_syscalls+1
 838   .equ NR_syscalls,NR_syscalls+1
 839           .equ NR_syscalls,NR_syscalls+1
 840   .equ NR_syscalls,NR_syscalls+1
 841   .equ NR_syscalls,NR_syscalls+1
 842   .equ NR_syscalls,NR_syscalls+1
 843   .equ NR_syscalls,NR_syscalls+1
 844           .equ NR_syscalls,NR_syscalls+1
 845   .equ NR_syscalls,NR_syscalls+1
 846   .equ NR_syscalls,NR_syscalls+1
 847   .equ NR_syscalls,NR_syscalls+1
 848   .equ NR_syscalls,NR_syscalls+1
 849           .equ NR_syscalls,NR_syscalls+1
 850   .equ NR_syscalls,NR_syscalls+1
 851   .equ NR_syscalls,NR_syscalls+1
 852   .equ NR_syscalls,NR_syscalls+1
 853   .equ NR_syscalls,NR_syscalls+1
 854           .equ NR_syscalls,NR_syscalls+1
 855   .equ NR_syscalls,NR_syscalls+1
 856   .equ NR_syscalls,NR_syscalls+1
 857   .equ NR_syscalls,NR_syscalls+1
 858   .equ NR_syscalls,NR_syscalls+1
 859           .equ NR_syscalls,NR_syscalls+1
 860   .equ NR_syscalls,NR_syscalls+1
 861   .equ NR_syscalls,NR_syscalls+1
 862   .equ NR_syscalls,NR_syscalls+1
 863   .equ NR_syscalls,NR_syscalls+1
 864           .equ NR_syscalls,NR_syscalls+1
 865   .equ NR_syscalls,NR_syscalls+1
 866   .equ NR_syscalls,NR_syscalls+1
 867   .equ NR_syscalls,NR_syscalls+1
 868   .equ NR_syscalls,NR_syscalls+1
 869           .equ NR_syscalls,NR_syscalls+1
 870   .equ NR_syscalls,NR_syscalls+1
 871   .equ NR_syscalls,NR_syscalls+1
 872   .equ NR_syscalls,NR_syscalls+1
 873   .equ NR_syscalls,NR_syscalls+1
 874           .equ NR_syscalls,NR_syscalls+1
 875   .equ NR_syscalls,NR_syscalls+1
 876   .equ NR_syscalls,NR_syscalls+1
 877   .equ NR_syscalls,NR_syscalls+1
 878   .equ NR_syscalls,NR_syscalls+1
 879           .equ NR_syscalls,NR_syscalls+1
 880   .equ NR_syscalls,NR_syscalls+1
 881   .equ NR_syscalls,NR_syscalls+1
 882   .equ NR_syscalls,NR_syscalls+1
 883   .equ NR_syscalls,NR_syscalls+1
 884           .equ NR_syscalls,NR_syscalls+1
 885   .equ NR_syscalls,NR_syscalls+1
 886   .equ NR_syscalls,NR_syscalls+1
 887   .equ NR_syscalls,NR_syscalls+1
 888   .equ NR_syscalls,NR_syscalls+1
 889           .equ NR_syscalls,NR_syscalls+1
 890   .equ NR_syscalls,NR_syscalls+1
 891   .equ NR_syscalls,NR_syscalls+1
 892   .equ NR_syscalls,NR_syscalls+1
 893   .equ NR_syscalls,NR_syscalls+1
 894           .equ NR_syscalls,NR_syscalls+1
 895   .equ NR_syscalls,NR_syscalls+1
 896   .equ NR_syscalls,NR_syscalls+1
 897   .equ NR_syscalls,NR_syscalls+1
 898   .equ NR_syscalls,NR_syscalls+1
 899           .equ NR_syscalls,NR_syscalls+1
 900   .equ NR_syscalls,NR_syscalls+1
 901   .equ NR_syscalls,NR_syscalls+1
 902   .equ NR_syscalls,NR_syscalls+1
 903   .equ NR_syscalls,NR_syscalls+1
 904           .equ NR_syscalls,NR_syscalls+1
 905   .equ NR_syscalls,NR_syscalls+1
 906   .equ NR_syscalls,NR_syscalls+1
 907   .equ NR_syscalls,NR_syscalls+1
 908   .equ NR_syscalls,NR_syscalls+1
 909           .equ NR_syscalls,NR_syscalls+1
 910   .equ NR_syscalls,NR_syscalls+1
 911   .equ NR_syscalls,NR_syscalls+1
 912   .equ NR_syscalls,NR_syscalls+1
 913   .equ NR_syscalls,NR_syscalls+1
 914           .equ NR_syscalls,NR_syscalls+1
 915   .equ NR_syscalls,NR_syscalls+1
 916   .equ NR_syscalls,NR_syscalls+1
 917   .equ NR_syscalls,NR_syscalls+1
 918   .equ NR_syscalls,NR_syscalls+1
 919           .equ NR_syscalls,NR_syscalls+1
 920   .equ NR_syscalls,NR_syscalls+1
 921   .equ NR_syscalls,NR_syscalls+1
 922   .equ NR_syscalls,NR_syscalls+1
 923   .equ NR_syscalls,NR_syscalls+1
 924           .equ NR_syscalls,NR_syscalls+1
 925   .equ NR_syscalls,NR_syscalls+1
 926   .equ NR_syscalls,NR_syscalls+1
 927   .equ NR_syscalls,NR_syscalls+1
 928   .equ NR_syscalls,NR_syscalls+1
 929           .equ NR_syscalls,NR_syscalls+1
 930   .equ NR_syscalls,NR_syscalls+1
 931   .equ NR_syscalls,NR_syscalls+1
 932   .equ NR_syscalls,NR_syscalls+1
 933   .equ NR_syscalls,NR_syscalls+1
 934           .equ NR_syscalls,NR_syscalls+1
 935   .equ NR_syscalls,NR_syscalls+1
 936   .equ NR_syscalls,NR_syscalls+1
 937   .equ NR_syscalls,NR_syscalls+1
 938   .equ NR_syscalls,NR_syscalls+1
 939           .equ NR_syscalls,NR_syscalls+1
 940   .equ NR_syscalls,NR_syscalls+1
 941   .equ NR_syscalls,NR_syscalls+1
 942   .equ NR_syscalls,NR_syscalls+1
 943   .equ NR_syscalls,NR_syscalls+1
 944           .equ NR_syscalls,NR_syscalls+1
 945   .equ NR_syscalls,NR_syscalls+1
 946   .equ NR_syscalls,NR_syscalls+1
 947   .equ NR_syscalls,NR_syscalls+1
 948   .equ NR_syscalls,NR_syscalls+1
 949           .equ NR_syscalls,NR_syscalls+1
 950   .equ NR_syscalls,NR_syscalls+1
 951   .equ NR_syscalls,NR_syscalls+1
 952               .equ NR_syscalls,NR_syscalls+1
 953   .equ NR_syscalls,NR_syscalls+1
 954           .equ NR_syscalls,NR_syscalls+1
 955   .equ NR_syscalls,NR_syscalls+1
 956   .equ NR_syscalls,NR_syscalls+1
 957   .equ NR_syscalls,NR_syscalls+1
 958   .equ NR_syscalls,NR_syscalls+1
 959           .equ NR_syscalls,NR_syscalls+1
 960   .equ NR_syscalls,NR_syscalls+1
 961   .equ NR_syscalls,NR_syscalls+1
 962   .equ NR_syscalls,NR_syscalls+1
 963   .equ NR_syscalls,NR_syscalls+1
 964           .equ NR_syscalls,NR_syscalls+1
 965   .equ NR_syscalls,NR_syscalls+1
 966   .equ NR_syscalls,NR_syscalls+1
 967   .equ NR_syscalls,NR_syscalls+1
 968   .equ NR_syscalls,NR_syscalls+1
 969           .equ NR_syscalls,NR_syscalls+1
 970   .equ NR_syscalls,NR_syscalls+1
 971   .equ NR_syscalls,NR_syscalls+1
 972   .equ NR_syscalls,NR_syscalls+1
 973   .equ NR_syscalls,NR_syscalls+1
 974           .equ NR_syscalls,NR_syscalls+1
 975   .equ NR_syscalls,NR_syscalls+1
 976   .equ NR_syscalls,NR_syscalls+1
 977   .equ NR_syscalls,NR_syscalls+1
 978   .equ NR_syscalls,NR_syscalls+1
 979           .equ NR_syscalls,NR_syscalls+1
 980   .equ NR_syscalls,NR_syscalls+1
 981   .equ NR_syscalls,NR_syscalls+1
 982   .equ NR_syscalls,NR_syscalls+1
 983   .equ NR_syscalls,NR_syscalls+1
 984           .equ NR_syscalls,NR_syscalls+1
 985   .equ NR_syscalls,NR_syscalls+1
 986   .equ NR_syscalls,NR_syscalls+1
 987   .equ NR_syscalls,NR_syscalls+1
 988   .equ NR_syscalls,NR_syscalls+1
 989           .equ NR_syscalls,NR_syscalls+1
 990   .equ NR_syscalls,NR_syscalls+1
 991   .equ NR_syscalls,NR_syscalls+1
 992   .equ NR_syscalls,NR_syscalls+1
 993   .equ NR_syscalls,NR_syscalls+1
 994           .equ NR_syscalls,NR_syscalls+1
 995   .equ NR_syscalls,NR_syscalls+1
 996   .equ NR_syscalls,NR_syscalls+1
 997   .equ NR_syscalls,NR_syscalls+1
 998   .equ NR_syscalls,NR_syscalls+1
 999           .equ NR_syscalls,NR_syscalls+1
1000   .equ NR_syscalls,NR_syscalls+1
1001   .equ NR_syscalls,NR_syscalls+1
1002   .equ NR_syscalls,NR_syscalls+1
1003   .equ NR_syscalls,NR_syscalls+1
1004           .equ NR_syscalls,NR_syscalls+1
1005   .equ NR_syscalls,NR_syscalls+1
1006   .equ NR_syscalls,NR_syscalls+1
1007   .equ NR_syscalls,NR_syscalls+1
1008   .equ NR_syscalls,NR_syscalls+1
1009           .equ NR_syscalls,NR_syscalls+1
1010   .equ NR_syscalls,NR_syscalls+1
1011   .equ NR_syscalls,NR_syscalls+1
1012   .equ NR_syscalls,NR_syscalls+1
1013   .equ NR_syscalls,NR_syscalls+1
1014           .equ NR_syscalls,NR_syscalls+1
1015   .equ NR_syscalls,NR_syscalls+1
1016   .equ NR_syscalls,NR_syscalls+1
1017   .equ NR_syscalls,NR_syscalls+1
1018   .equ NR_syscalls,NR_syscalls+1
1019           .equ NR_syscalls,NR_syscalls+1
1020   .equ NR_syscalls,NR_syscalls+1
1021   .equ NR_syscalls,NR_syscalls+1
1022   .equ NR_syscalls,NR_syscalls+1
1023   .equ NR_syscalls,NR_syscalls+1
1024           .equ NR_syscalls,NR_syscalls+1
1025   .equ NR_syscalls,NR_syscalls+1
1026   .equ NR_syscalls,NR_syscalls+1
1027   .equ NR_syscalls,NR_syscalls+1
1028 
1029 .equ syscalls_padding, ((NR_syscalls + 3) & ~3) - NR_syscalls
1030 
1031 
1032 .rept syscalls_padding
1033   .equ NR_syscalls,NR_syscalls+1
1034 .endr
1035 # 102 "arch/arm/kernel/entry-common.S" 2
1036 
1042 
1043 .ifne NR_syscalls - (388)
1044 .error "__NR_syscalls is not equal to the size of the syscall table"
1045 .endif
1046 # 122 "arch/arm/kernel/entry-common.S"
1047  .align 5
1048 .globl vector_swi ; .align 0 ; vector_swi:
1049 

cpp.sh

1 arm-linux-gnueabihf-gcc -Wp,-MD,arch/arm/kernel/.entry-common.o.d  -nostdinc -isystem /media/work/altera/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.9.2/include -I./arch/arm/include -Iarch/arm/include/generated/uapi -Iarch/arm/include/generated  -Iinclude -I./arch/arm/include/uapi -Iarch/arm/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -D__KERNEL__ -mlittle-endian -Iarch/arm/mach-bcm2709/include   -D__ASSEMBLY__ -mabi=aapcs-linux -mno-thumb-interwork -mfpu=vfp -funwind-tables -marm -D__LINUX_ARM_ARCH__=7 -march=armv7-a  -include asm/unified.h -msoft-float  -DCC_HAVE_ASM_GOTO            -E arch/arm/kernel/entry-common.S > /tmp/a.c

2 arm-linux-gnueabihf-as -c /tmp/a.c
3 arch/arm/kernel/entry-common.S: Assembler messages:
4 arch/arm/kernel/entry-common.S:110: Error: __NR_syscalls is not equal to the size of the syscall table

add_arm_linux_new_syscall.patch L113 我用了可以傳入一個參數的 macro, 可以參考其它的 system call 看看如何使用不同參數的版本, ex: SYSCALL_DEFINE3。system call 並不是表面上看的那麼單純, 我認為只有實作過才真的能理解它, 否則你只是停留在理論的理解。

test_my_syscall.c 用來呼叫這個 new system call, 這是把 mmu ttbr0 印出來, 我本來想解開 linux mmu 位址轉換的祕密, 很可惜失敗了。

test_my_syscall.c

 1 #include <stdio.h>
 2 #include <stdlib.h>
 3 #include <unistd.h>
 4 #include <errno.h>
 5 typedef unsigned int u32;
 6 
 7 int my_syscall(int reg_type)
 8 {     
 9   printf("user mode ## reg_type : %d\n", reg_type);
10   return (long) syscall(388, reg_type);
11 } 
12 
13 int main(int argc, char *argv[])
14 {
15   int reg_type=0;
16 
17   if (argc >= 2)
18     reg_type = strtol(argv[1], NULL, 10);
19 
20   printf("page size: %d\n", getpagesize());
21 
22   u32 ttbr0 = my_syscall(reg_type);
23   if (ttbr0 == -1)
24   {
25     perror("err");
26   }
27   printf("reg: %x\n", ttbr0);
28   return 0;
29 }