[系统运维]ubuntu 20.04 qemu linux6.0.1 制作ext4根文件系统

前言

• Linux 内核移植后，直接运行，会发现死机，提示没有根文件系统，没有shell 控制台

• 上一篇 ubuntu 20.04 qemu linux6.0.1 开发环境搭建 搭好了开发环境，这一篇，开始制作 根文件系统，让Linux 跑起来

开发环境

• win10 64位 VMware Workstation Pro 16

• ubuntu 20.04

• qemu （虚拟ARM开发板），vexpress-a9 平台

• Linux 6.0.1

下载并解压busybox

• 当前使用 busybox，可以快速制作Linux 根文件系统需要的常用命令，所以这里使用当前最新版本的busybox：busybox-1.35.0.tar.bz2

• busybox 官方下载地址：https://busybox.net/downloads/busybox-1.35.0.tar.bz2，当前可以直接到官方 https://busybox.net/downloads/ 下载最新的版本

• 把下载好的 busybox 放在 ubuntu 中，解压缩

zhangsz@zhangsz:~/linux/busybox$ ls
busybox-1.35.0.tar.bz2

zhangsz@zhangsz:~/linux/busybox$ tar xjvf busybox-1.35.0.tar.bz2

编译busybox

• 这里强调一点，如果需要一次性编译通过，需要使用 linux gcc 交叉编译工具链，而不是 arm-none-eabi- 这样的工具链，因为 如：arm-linux-gnueabihf- 这样的工具链中，集成了依赖Linux 相关的头文件

• 编译busybox-1.35.0，推荐的linux gcc 为： gcc-linaro-13.0.0-2022.10-x86_64_arm-linux-gnueabihf

• 因为这里是qemu 的模拟器，所以直接编译即可，暂时不需要 menuconfig 手动配置

• 编译命令如下：

/* menuconfig 配置，直接保存即可 */
$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig

/* 创建 busybox 输出的路径 */
$ mkdir /home/zhangsz/linux/rootfs/1016

/* 编译， install，这样就可以把 busybox 的编译产物放在CONFIG_PREFIX的路径下  */
$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_PREFIX=/home/zhangsz/linux/rootfs/1016/ install

• busybox 编译的产物：

zhangsz@zhangsz:~/linux/busybox/busybox-1.35.0$ cd /home/zhangsz/linux/rootfs/1016/
zhangsz@zhangsz:~/linux/rootfs/1016$ ls
bin  linuxrc  sbin  usr

• 注意busybox 直接编译的产物，还是确实lib库，确实Linux 内核启动控制台相关的一些东西，需要再补充

制作根文件系统目录

• 制作Linux 根文件系统目录的过程，各个平台其实都是大同小异的。

• 这里使用脚本，快速的制作一个根文件系统目录模板，然后再把 busybox 的编译产物、gcc 交叉编译环境下的 lib 拷贝过来，就可以打包成制定的文件系统镜像了

• 制作根文件系统目录模板打包脚本：mk_fs.sh

#!/bin/bash
echo "------Create rootfs start...--------"

rootfs=$1
rm -rf $rootfs
mkdir $rootfs
cd $rootfs

echo "--------Create root,dev....----------"  
mkdir root dev etc boot tmp var sys proc lib mnt home usr
mkdir etc/init.d etc/rc.d

echo "make node in dev/console dev/null"
sudo mknod -m 600 dev/console c 5 1
sudo mknod -m 600 dev/null  c 1 3

# create etc config /etc/inittab
echo -e "::sysinit:/etc/init.d/rcS " >etc/inittab
echo -e "::askfirst:-/bin/sh " >>etc/inittab
echo -e "::ctrlaltdel:/sbin/reboot " >>etc/inittab
echo -e "::shutdown:/bin/umount -a -r " >>etc/inittab

# create etc config /etc/init.d/rcs
echo -e "#! /bin/sh " >etc/init.d/rcS
echo -e "mount -t sysfs none /sys " >>etc/init.d/rcS
echo -e "mount -t proc none /proc " >>etc/init.d/rcS
echo -e "mount -t tmpfs tmpfs /tmp" >>etc/init.d/rcS
echo -e "mdev -s " >>etc/init.d/rcS
chmod +x etc/init.d/rcS

# create etc config /etc/fstab
echo -e "proc       /proc       proc        defaults 0 0 " >etc/fstab
echo -e "sysfs      /sys        sysfs       defaults 0 0 " >>etc/fstab
echo -e "devtmpfs   /dev        devtmpfs    defaults 0 0 " >>etc/fstab
echo -e "tmpfs      /tmp        tmpfs       defaults 0 0 " >>etc/fstab
echo -e "tmpfs      /var        tmpfs       defaults 0 0 " >>etc/fstab

echo "-------make rootfs done---------"

• 制作根文件目录方法

/* 新建 mk_fs.sh，复制一下脚本内容，保存 */
zhangsz@zhangsz:~/linux/rootfs$ vim mk_fs.sh


zhangsz@zhangsz:~/linux/rootfs$ ls
1016  mk_fs.sh

/* 脚本的执行权限 */
zhangsz@zhangsz:~/linux/rootfs$ chmod +x mk_fs.sh 

/* 这个脚本参数为 输出的目录 */
$ sudo ./mk_fs.sh ext4_rootfs

/* 根文件系统目录 模板 如下 */
zhangsz@zhangsz:~/linux/rootfs/ext4_rootfs$ ls
boot  dev  etc  home  lib  mnt  proc  root  sys  tmp  usr  var

• busybox 编译产物复制到 根文件系统模板目录下：

$ sudo cp -r /home/zhangsz/linux/rootfs/1016/* ext4_rootfs/

/* 查看发现 增加了  linuxrc 等busybox 的编译产物，还有一些是命令，都来自 busybox */
zhangsz@zhangsz:~/linux/rootfs$ ls ext4_rootfs/
bin  boot  dev  etc  home  lib  linuxrc  mnt  proc  root  sbin  sys  tmp  usr  var

• linux gcc 交叉编译工具链下的lib，具体如：gcc-linaro-13.0.0-2022.10-x86_64_arm-linux-gnueabihf/arm-linux-gnueabihf/libc/lib/，也复制到 根文件系统目录下的lib 文件夹下，因为默认busybox 是动态链接的，qemu 推荐使用 默认的配置，一些依赖库直接从工具链复制过来即可，当前有点大。

$ sudo cp -r /home/zhangsz/tools/gcc-linaro-13.0.0-2022.10-x86_64_arm-linux-gnueabihf/arm-linux-gnueabihf/libc/lib/* ext4_rootfs/lib/

rootfs.ext4.img 制作

• 直接使用 mkfs.ext4 好像不能把目录制作成img文件，这里分为两步：先制作一个 512MB 大小的文件，然后格式化为 ext4 ，这里的512MB 大小可以根据实际情况更改，如1024MB，当前要大于当前的根文件系统目录总大小

zhangsz@zhangsz:~/linux/img$ dd if=/dev/zero of=rootfs.ext4.img bs=1M count=512
512+0 records in
512+0 records out

/* mkfs.ext4 格式化 */
zhangsz@zhangsz:~/linux/img$ mkfs.ext4 rootfs.ext4.img 
mke2fs 1.45.5 (07-Jan-2020)
Discarding device blocks: done                            
Creating filesystem with 131072 4k blocks and 32768 inodes
Filesystem UUID: f35501cc-b8fa-4e70-b369-0229213b96e7
Superblock backups stored on blocks: 
	32768, 98304

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

• 复制 根文件系统目录下的所有文件到 rootfs.ext4.img 镜像中，文件系统镜像先mount 挂载，这样就可以复制文件进去了

zhangsz@zhangsz:~/linux/img$ sudo mkdir /mnt/rootfs
zhangsz@zhangsz:~/linux/img$ sudo mount rootfs.ext4.img /mnt/rootfs/
zhangsz@zhangsz:~/linux/img$ sudo cp -r /home/zhangsz/linux/rootfs/ext4_rootfs/* /mnt/rootfs/
zhangsz@zhangsz:~/linux/img$ sudo umount /mnt/rootfs

• 当前的img 目录下的东西：

zhangsz@zhangsz:~/linux/img$ ls -la
total 182848
drwxrwxr-x 2 zhangsz zhangsz      4096 10月 16 10:25 .
drwxrwxr-x 6 zhangsz zhangsz      4096 10月 16 09:47 ..
-rw-rw-r-- 1 zhangsz zhangsz 536870912 10月 16 10:27 rootfs.ext4.img
-rw-rw-r-- 1 zhangsz zhangsz     14081 10月 14 21:12 vexpress-v2p-ca9.dtb
-rwxrwxr-x 1 zhangsz zhangsz   5208104 10月 14 21:12 zImage

• 自此，初步qemu Linux 启动需要的根文件系统，就制作完成了

启动qemu

• 启动的命令如下：注意运行的路径：

$ cd /home/zhangsz/linux/img/
zhangsz@zhangsz:~/linux/img$ ls
rootfs.ext4.img  vexpress-v2p-ca9.dtb  zImage


zhangsz@zhangsz:~/linux/img$ qemu-system-arm -M vexpress-a9 -m 512M -kernel zImage -dtb vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0 rw console=ttyAMA0 init=linuxrc" -sd rootfs.ext4.img

启动信息

• qemu Linux 6.0.1 增加根文件系统后的启动信息如下：

zhangsz@zhangsz:~$ cd /home/zhangsz/linux/img/
zhangsz@zhangsz:~/linux/img$ ls
rootfs.ext4.img  vexpress-v2p-ca9.dtb  zImage
zhangsz@zhangsz:~/linux/img$ qemu-system-arm -M vexpress-a9 -m 512M -kernel zImage -dtb vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0 rw console=ttyAMA0 init=linuxrc" -sd rootfs.ext4.img
WARNING: Image format was not specified for 'rootfs.ext4.img' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
Booting Linux on physical CPU 0x0
Linux version 6.0.1 (zhangsz@zhangsz) (arm-linux-gnueabihf-gcc (GCC) 13.0.0 20221001 (experimental) [master revision 5299155bb80e90df822e1eebc9f9a0c8e4505a46], GNU ld (Linaro_Binutils-2022.10) 2.39.50.20221001) #1 SMP Fri Oct 14 21:05:09 CST 2022
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: V2P-CA9
Memory policy: Data cache writeback
Reserved memory: created DMA memory pool at 0x4c000000, size 8 MiB
OF: reserved mem: initialized node vram@4c000000, compatible id shared-dma-pool
cma: Reserved 16 MiB at 0x7f000000
Zone ranges:
  Normal   [mem 0x0000000060000000-0x000000007fffffff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000060000000-0x000000007fffffff]
Initmem setup node 0 [mem 0x0000000060000000-0x000000007fffffff]
CPU: All CPU(s) started in SVC mode.
percpu: Embedded 15 pages/cpu s30356 r8192 d22892 u61440
Built 1 zonelists, mobility grouping on.  Total pages: 130048
Kernel command line: root=/dev/mmcblk0 rw console=ttyAMA0 init=linuxrc
printk: log_buf_len individual max cpu contribution: 4096 bytes
printk: log_buf_len total cpu_extra contributions: 12288 bytes
printk: log_buf_len min size: 16384 bytes
printk: log_buf_len: 32768 bytes
printk: early log buf free: 14880(90%)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes, linear)
mem auto-init: stack:all(zero), heap alloc:off, heap free:off
Memory: 490808K/524288K available (8192K kernel code, 638K rwdata, 1844K rodata, 1024K init, 180K bss, 17096K reserved, 16384K cma-reserved)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
trace event string verifier disabled
rcu: Hierarchical RCU implementation.
rcu: 	RCU event tracing is enabled.
rcu: 	RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
GIC CPU mask not found - kernel will fail to boot.
GIC CPU mask not found - kernel will fail to boot.
L2C: platform modifies aux control register: 0x02020000 -> 0x02420000
L2C: DT/platform modifies aux control register: 0x02020000 -> 0x02420000
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 dynamic clock gating disabled, standby mode disabled
L2C-310 cache controller enabled, 8 ways, 128 kB
L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x46420001
rcu: srcu_init: Setting srcu_struct sizes based on contention.
sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
clocksource: arm,sp804: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns
smp_twd: clock not found -2
Console: colour dummy device 80x30
Calibrating local timer... 99.10MHz.
Calibrating delay loop... 2138.11 BogoMIPS (lpj=10690560)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
CPU: Testing write buffer coherency: ok
CPU0: Spectre v2: using BPIALL workaround
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x60100000 - 0x60100060
rcu: Hierarchical SRCU implementation.
rcu: 	Max phase no-delay instances is 1000.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
SMP: Total of 1 processors activated (2138.11 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 0
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
NET: Registered PF_NETLINK/PF_ROUTE protocol family
DMA: preallocated 256 KiB pool for atomic coherent allocations
cpuidle: using governor ladder
hw-breakpoint: debug architecture 0x4 unsupported.
Serial: AMBA PL011 UART driver
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
i2c 0-0039: Fixing up cyclic dependency with 1001f000.clcd
i2c 0-0039: Fixing up cyclic dependency with 10020000.clcd
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
Advanced Linux Sound Architecture Driver Initialized.
clocksource: Switched to clocksource arm,sp804
NET: Registered PF_INET protocol family
IP idents hash table entries: 8192 (order: 4, 65536 bytes, linear)
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 4096 bytes, linear)
Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
TCP established hash table entries: 4096 (order: 2, 16384 bytes, linear)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes, linear)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered PF_UNIX/PF_LOCAL protocol family
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
hw perfevents: enabled with armv7_cortex_a9 PMU driver, 5 counters available
workingset: timestamp_bits=30 max_order=17 bucket_order=0
squashfs: version 4.0 (2009/01/31) Phillip Lougher
jffs2: version 2.2. (NAND) ? 2001-2006 Red Hat, Inc.
9p: Installing v9fs 9p2000 file system support
io scheduler mq-deadline registered
io scheduler kyber registered
OF: graph: no port node found in /bus@40000000/motherboard-bus@40000000/iofpga@7,00000000/i2c@16000/dvi-transmitter@60
sii902x 0-0060: supply iovcc not found, using dummy regulator
sii902x 0-0060: supply cvcc12 not found, using dummy regulator
physmap-flash 40000000.flash: physmap platform flash device: [mem 0x40000000-0x43ffffff]
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
physmap-flash 40000000.flash: physmap platform flash device: [mem 0x44000000-0x47ffffff]
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
Concatenating MTD devices:
(0): "40000000.flash"
(1): "40000000.flash"
into device "40000000.flash"
physmap-flash 48000000.psram: physmap platform flash device: [mem 0x48000000-0x49ffffff]
smsc911x 4e000000.ethernet eth0: MAC Address: 52:54:00:12:34:56
isp1760 4f000000.usb: isp1760 bus width: 32, oc: digital
isp1760 4f000000.usb: NXP ISP1760 USB Host Controller
isp1760 4f000000.usb: new USB bus registered, assigned bus number 1
isp1760 4f000000.usb: Scratch test failed. 0x00000000
isp1760 4f000000.usb: can't setup: -19
isp1760 4f000000.usb: USB bus 1 deregistered
usbcore: registered new interface driver usb-storage
rtc-pl031 10017000.rtc: registered as rtc0
rtc-pl031 10017000.rtc: setting system clock to 2022-10-16T02:56:32 UTC (1665888992)
mmci-pl18x 10005000.mmci: Got CD GPIO
mmci-pl18x 10005000.mmci: Got WP GPIO
mmci-pl18x 10005000.mmci: mmc0: PL181 manf 41 rev0 at 0x10005000 irq 35,36 (pio)
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
aaci-pl041 10004000.aaci: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 37
aaci-pl041 10004000.aaci: FIFO 512 entries
NET: Registered PF_PACKET protocol family
9pnet: Installing 9P2000 support
Registering SWP/SWPB emulation handler
10009000.uart: ttyAMA0 at MMIO 0x10009000 (irq = 38, base_baud = 0) is a PL011 rev1
printk: console [ttyAMA0] enabled
1000a000.uart: ttyAMA1 at MMIO 0x1000a000 (irq = 39, base_baud = 0) is a PL011 rev1
1000b000.uart: ttyAMA2 at MMIO 0x1000b000 (irq = 40, base_baud = 0) is a PL011 rev1
1000c000.uart: ttyAMA3 at MMIO 0x1000c000 (irq = 41, base_baud = 0) is a PL011 rev1
drm-clcd-pl111 1001f000.clcd: assigned reserved memory node vram@4c000000
drm-clcd-pl111 1001f000.clcd: using device-specific reserved memory
drm-clcd-pl111 1001f000.clcd: core tile graphics present
drm-clcd-pl111 1001f000.clcd: this device will be deactivated
drm-clcd-pl111 1001f000.clcd: Versatile Express init failed - -19
drm-clcd-pl111 10020000.clcd: DVI muxed to daughterboard 1 (core tile) CLCD
input: AT Raw Set 2 keyboard as /devices/platform/bus@40000000/bus@40000000:motherboard-bus@40000000/bus@40000000:motherboard-bus@40000000:iofpga@7,00000000/10006000.kmi/serio0/input/input0
mmc0: new SD card at address 4567
drm-clcd-pl111 10020000.clcd: initializing Versatile Express PL111
drm-clcd-pl111 10020000.clcd: DVI muxed to daughterboard 1 (core tile) CLCD
mmcblk0: mmc0:4567 QEMU! 512 MiB 
drm-clcd-pl111 10020000.clcd: initializing Versatile Express PL111
drm-clcd-pl111 10020000.clcd: DVI muxed to daughterboard 1 (core tile) CLCD
drm-clcd-pl111 10020000.clcd: initializing Versatile Express PL111
drm-clcd-pl111 10020000.clcd: DVI muxed to daughterboard 1 (core tile) CLCD
drm-clcd-pl111 10020000.clcd: initializing Versatile Express PL111
ALSA device list:
  #0: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 37
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/bus@40000000/bus@40000000:motherboard-bus@40000000/bus@40000000:motherboard-bus@40000000:iofpga@7,00000000/10007000.kmi/serio1/input/input2
drm-clcd-pl111 10020000.clcd: DVI muxed to daughterboard 1 (core tile) CLCD
drm-clcd-pl111 10020000.clcd: initializing Versatile Express PL111
EXT4-fs (mmcblk0): recovery complete
EXT4-fs (mmcblk0): mounted filesystem with ordered data mode. Quota mode: disabled.
VFS: Mounted root (ext4 filesystem) on device 179:0.
Freeing unused kernel image (initmem) memory: 1024K
Run linuxrc as init process
random: crng init done

Please press Enter to activate this console. 
/ # ls
bin         etc         linuxrc     proc        sys         var
boot        home        lost+found  root        tmp
dev         lib         mnt         sbin        usr
/ # drm-clcd-pl111 10020000.clcd: DVI muxed to daughterboard 1 (core tile) CLCD
drm-clcd-pl111 10020000.clcd: initializing Versatile Express PL111
amba 100e0000.memory-controller: deferred probe pending
amba 100e1000.memory-controller: deferred probe pending
amba 100e5000.watchdog: deferred probe pending
i2c 0-0039: deferred probe pending
amba 1000f000.wdt: deferred probe pending
amba 10020000.clcd: deferred probe pending

/ # 

• linux 的shell 控制台起来了，说明 qemu Linux 运行起来了

小结

• 本篇主要记录 基于 qemu 的 Linux 根文件系统制作的流程

• qemu Linux 环境，当前为ARM的环境，搭建的目的，就是为了学习嵌入式Linux 开发，当前最终需要实际的电路板验证

• 使用qemu 最大的好处就是不依赖于硬件，前期快速开发与验证一些平台无关的Linux驱动开发、应用开发，加快开发调试的效率。