526互联

device descriptor read/64, error -11

发布时间 2023-12-15 14:29:40作者: 闹闹爸爸

硬件平台：CM4 8G RAM核心板

Hardware        : BCM2711
Revision        : d03141
Serial          : 100000002297c853
Model           : Raspberry Pi Compute Module 4 Rev 1.1

系统：树莓派32位OS

问题背景：核心板是8G RAM，由于默认的PAGE_OFFSET是3G，所以内核分配的内存+DMA总大小只有1G，其余的都是High Memory。32位系统支持8G RAM需要打开kernel PAE，这样一来需要更多的low Memory来存储页表信息，所以真正给内核使用的low Memory只有不到600M了，这个时候，如果有的程序占用内存较大(瞬间物理内存占用超过200M)，会触发kernel的OOM-killer，把程序杀掉。

为了解决上述问题，所以调整了PAGE_OFFSET的大小分配，给low Memory + DMA分配了2G，方法如下：

修改内核.config

 # CONFIG_VMSPLIT_3G is not set                                                  
CONFIG_VMSPLIT_2G=y                                                             
# CONFIG_VMSPLIT_1G is not set

内核和模块编译完成后，使用新编译的内核和模块，开机后发现 USB 2.0 Hub [MTT] 不能正常工作，报“device descriptor read/64, error -11”的错误。

解决办法：

由于给low Memory + DMA分配了2G内存，所以DMA区域大小 dma_zone_size 也要进行相应的调整：

diff --git a/arch/arm/mach-bcm/bcm2711.c b/arch/arm/mach-bcm/bcm2711.c
index fa0300d8c79d..5f78beafda55 100644
--- a/arch/arm/mach-bcm/bcm2711.c
+++ b/arch/arm/mach-bcm/bcm2711.c
@@ -18,7 +18,7 @@ static const char * const bcm2711_compat[] = {
 
 DT_MACHINE_START(BCM2711, "BCM2711")
 #ifdef CONFIG_ZONE_DMA
-       .dma_zone_size  = SZ_1G,
+       .dma_zone_size  = SZ_2G,
 #endif
        .dt_compat = bcm2711_compat,
        .smp = smp_ops(bcm2836_smp_ops),
diff --git a/arch/arm/mach-bcm/board_bcm2835.c b/arch/arm/mach-bcm/board_bcm2835.c
index 91a758c61f48..fba645cdf73b 100644
--- a/arch/arm/mach-bcm/board_bcm2835.c
+++ b/arch/arm/mach-bcm/board_bcm2835.c
@@ -129,7 +129,7 @@ static const char * const bcm2711_compat[] = {
 
 DT_MACHINE_START(BCM2711, "BCM2711")
 #if defined(CONFIG_ZONE_DMA) && defined(CONFIG_ARM_LPAE)
-       .dma_zone_size  = SZ_1G,
+       .dma_zone_size  = SZ_2G,
 #endif
        .map_io = bcm2835_map_io,
        .init_machine = bcm2835_init,

这样一来，USB 2.0 Hub也工作正常了，low Memory也够用了，不会触发内核的Out Of Memory。

上述修改虽然解决了USB 2.0 Hub不能正常工作的问题，但是DMA还是有一个遗留问题，如下：

[    0.157492] WARNING: CPU: 0 PID: 1 at kernel/dma/swiotlb.c:683 swiotlb_map+0x384/0x42c
[    0.157531] bcm2835-dma fe007000.dma: swiotlb addr 0x00000ffffffff000+4096 overflow (mask ffffffff, bus limit ffffffff).
[    0.157569] Modules linked in:
[    0.157614] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.10.32-v7l+ #2
[    0.157637] Hardware name: BCM2711
[    0.157657] Backtrace: 
[    0.157692] [<8020c414>] (dump_backtrace) from [<8020c760>] (show_stack+0x20/0x24)
[    0.157724]  r7:ffffffff r6:00000000 r5:60000013 r4:812e6abc
[    0.157757] [<8020c740>] (show_stack) from [<80b8e7c0>] (dump_stack+0xcc/0xf8)
[    0.157793] [<80b8e6f4>] (dump_stack) from [<802215a0>] (__warn+0x110/0x114)
[    0.157821]  r10:81205048 r9:802a8b68 r8:000002ab r7:00000009 r6:00000000 r5:80e27c94
[    0.157849]  r4:8191fba4 r3:81205094
[    0.157875] [<80221490>] (__warn) from [<80221628>] (warn_slowpath_fmt+0x84/0xc0)
[    0.157906]  r9:00000009 r8:802a8b68 r7:000002ab r6:80e27c94 r5:80e27c50 r4:81205048
[    0.157939] [<802215a8>] (warn_slowpath_fmt) from [<802a8b68>] (swiotlb_map+0x384/0x42c)
[    0.157970]  r9:00000000 r8:65d07000 r7:819be180 r6:819be010 r5:00000000 r4:ffffffff
[    0.158006] [<802a87e4>] (swiotlb_map) from [<802a37e4>] (dma_map_page_attrs+0x284/0x394)
[    0.158037]  r10:00000001 r9:819be180 r8:00001000 r7:81205048 r6:819be010 r5:00000000
[    0.158063]  r4:ffffffff
[    0.158093] [<802a3560>] (dma_map_page_attrs) from [<807ec02c>] (bcm2835_dma_probe+0x280/0x618)
[    0.158125]  r10:0000000b r9:80c43d60 r8:00000000 r7:00000001 r6:81b17c40 r5:819be010
[    0.158151]  r4:00000000
[    0.158181] [<807ebdac>] (bcm2835_dma_probe) from [<80849788>] (platform_drv_probe+0x58/0xa8)
[    0.158212]  r10:00000000 r9:00000000 r8:812e9ef4 r7:00000000 r6:812e9ef4 r5:00000000
[    0.158238]  r4:819be010
[    0.158264] [<80849730>] (platform_drv_probe) from [<80847504>] (really_probe+0x100/0x3c4)
[    0.158293]  r7:00000000 r6:81400f4c r5:81400f44 r4:819be010
[    0.158321] [<80847404>] (really_probe) from [<808479b4>] (driver_probe_device+0x6c/0xc4)
[    0.158352]  r10:8105484c r9:00000005 r8:8180f800 r7:81205048 r6:812e9ef4 r5:812e9ef4
[    0.158379]  r4:819be010 r3:00000000
[    0.158405] [<80847948>] (driver_probe_device) from [<80847c04>] (device_driver_attach+0x68/0x70)
[    0.158433]  r5:00000000 r4:819be010
[    0.158459] [<80847b9c>] (device_driver_attach) from [<80847c9c>] (__driver_attach+0x90/0xcc)
[    0.158489]  r7:81205048 r6:819be010 r5:812e9ef4 r4:00000000
[    0.158517] [<80847c0c>] (__driver_attach) from [<80845528>] (bus_for_each_dev+0x84/0xc4)
[    0.158546]  r7:81205048 r6:80847c0c r5:812e9ef4 r4:00000000
[    0.158573] [<808454a4>] (bus_for_each_dev) from [<80846e70>] (driver_attach+0x2c/0x30)
[    0.158602]  r7:00000000 r6:812f0ea8 r5:81a80000 r4:812e9ef4
[    0.158629] [<80846e44>] (driver_attach) from [<80846740>] (bus_add_driver+0x174/0x1f8)
[    0.158663] [<808465cc>] (bus_add_driver) from [<80848350>] (driver_register+0x8c/0x124)
[    0.158692]  r7:00000000 r6:8108a614 r5:807eb81c r4:812e9ef4
[    0.158721] [<808482c4>] (driver_register) from [<808496d8>] (__platform_driver_register+0x50/0x58)
[    0.158750]  r5:807eb81c r4:812f0ea8
[    0.158778] [<80849688>] (__platform_driver_register) from [<807eb840>] (bcm2835_dma_init+0x24/0x28)
[    0.158806]  r5:807eb81c r4:81205048
[    0.158833] [<807eb81c>] (bcm2835_dma_init) from [<80202428>] (do_one_initcall+0x50/0x280)
[    0.158870] [<802023d8>] (do_one_initcall) from [<81001394>] (kernel_init_freeable+0x26c/0x2b8)
[    0.158901]  r8:8180f800 r7:8105482c r6:8108a614 r5:81356000 r4:80f1a040
[    0.158930] [<81001128>] (kernel_init_freeable) from [<80b91360>] (kernel_init+0x18/0x134)
[    0.158960]  r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:80b91348
[    0.158986]  r4:00000000
[    0.159009] [<80b91348>] (kernel_init) from [<802000ec>] (ret_from_fork+0x14/0x28)
[    0.159036] Exception stack(0x8191ffb0 to 0x8191fff8)
[    0.159059] ffa0:                                     00000000 00000000 00000000 00000000
[    0.159089] ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[    0.159118] ffe0: 00000000 00000000 00000000 00000000 00000013 00000000
[    0.159142]  r5:80b91348 r4:00000000
[    0.159171] ---[ end trace 5d2ccb3172321502 ]---
[    0.159204] bcm2835-dma fe007000.dma: Failed to map zero page
[    0.159256] bcm2835-dma: probe of fe007000.dma failed with error -12

DMA功能不能用啊，真是一大遗憾。

解决办法：

　　用树莓派64位 OS，完美适配8G RAM。