526互联

mongodb报错Sort exceeded memory limit of 104857600 bytes

发布时间 2023-07-14 17:07:49作者: abce

mongodb运行过程中，遇到错误信息：

2023-07-14T09:29:33.853 ERR Failed to QueryBsPoolUnivStat error="(QueryExceededMemoryLimitNoDiskUseAllowed) Executor error during find command :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting."
2023-07-14T09:29:33.853 ERR Failed to query LoadAllStockpoolUnivStatHistory  perf history error="(QueryExceededMemoryLimitNoDiskUseAllowed) Executor error during find command :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting."
panic: Load  all LoadAllStockpoolUnivStatHistory: (QueryExceededMemoryLimitNoDiskUseAllowed) Executor error during find command :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting.

goroutine 1 [running]:
abc/srv/abc/stock/data.(*DataClient).LoadMongoCache(0xc000042600)
        /test/srv/abc/stock/data/data.go:455 +0x9a5
abc/srv/abc/stock.NewDataServer(0xc000042600, {0xc0008b02e0, 0x4})
        /test/srv/abc/stock/provider.go:64 +0x2f4
main.setupRouter({0x7f6690308800, 0xc0003dd520}, 0x32?, {0xc0008b02e0, 0x4})
        /test/cmd/abce/main.go:83 +0x265
main.main()
        /test/cmd/abce/main.go:65 +0x685


mongodb的版本：5.0.13

报错产生的原因：
因为查询排序数据量太大，导致报错。
详细内容可以参看官方文档。地址如下：

https://www.mongodb.com/docs/manual/reference/operator/aggregation/sort/?_ga=2.90473562.1126669746.1689308968-1238152925.1673317434#sort-operator-and-memory

 

具体说明：

Starting in MongoDB 6.0, pipeline stages that require more than 100 megabytes of memory to execute write temporary files to disk by default. In earlier verisons of MongoDB, you must pass { allowDiskUse: true } to individual find and aggregate commands to enable this behavior.
Individual find and aggregate commands may override the allowDiskUseByDefault parameter by either: Using { allowDiskUse: true } to allow writing temporary files out to disk when allowDiskUseByDefault is set to false Using { allowDiskUse: false } to prohibit writing temporary files out to disk when allowDiskUseByDefault is set to true

 

解决方案：
根据文档，可以在查询中添加{ allowDiskUse: true }

不过本次没有修改代码并添加{ allowDiskUse: true }，因为发现集合中缺少索引，创建索引后，问题得到解决。

网上有人说可以修改参数：internalQueryExecMaxBlockingSortBytes。比如，执行以下命令：

db.adminCommand({setParameter: 1, internalQueryExecMaxBlockingSortBytes: 335544320})

但是，我查了一下，没有这个参数：

> use admin
switched to db admin
> db.runCommand( { getParameter : 1, "internalQueryExecMaxBlockingSortBytes" : 1 } );
{ "ok" : 0, "errmsg" : "no option found to get" }
>


其它
获取所有配置参数的值：

db.runCommand( { getParameter : '*' } )