背景： 在jupyter 中加载baichuan大模型时报错
报错一： from peft import PeftModel 报错
报错提示：python -m bitsandbytes
执行 python -m bitsandbytes 时又报错，报错内容为：
Traceback (most recent call last):
File "/home/miniconda3/envs/vllm/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/miniconda3/envs/vllm/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/miniconda3/envs/vllm/lib/python3.10/site-packages/bitsandbytes/__main__.py", line 101, in <module>
generate_bug_report_information()
File "/home/miniconda3/envs/vllm/lib/python3.10/site-packages/bitsandbytes/__main__.py", line 59, in generate_bug_report_information
paths = find_file_recursive('/usr/local', '*cuda*so')
File "/home/miniconda3/envs/vllm/lib/python3.10/site-packages/bitsandbytes/__main__.py", line 42, in find_file_recursive
raise RuntimeError('Something when wrong when trying to find file. Maybe you do not have a linux system?')
RuntimeError: Something when wrong when trying to find file. Maybe you do not have a linux system?
问题分析：报错原因，执行文件查找命令时，部分目录存在权限问题，把该目录排除出查找范围即可
vim /home/miniconda3/envs/vllm/lib/python3.10/site-packages/bitsandbytes/__main__.py +42
def find_file_recursive(folder, filename): #cmd = f'find {folder} -name {filename}' cmd = f'find {folder} -path "/usr/local/xxxxxxxx" -prune -o -name {filename}' print (cmd) out, err = execute_and_return(cmd) if len(err) > 0: raise RuntimeError('Something when wrong when trying to find file. Maybe you do not have a linux system?') return out
报错二: CUDA SETUP: Loading binary /home/miniconda3/envs/vllm/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117_nocublaslt.so...
/lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/miniconda3/envs/vllm/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117_nocublaslt.so)
经分析,bitandbytes代码中cuda设置这部分报错
from bitsandbytes.cuda_setup.main import CUDASetup setup = CUDASetup.get_instance() setup.run_cuda_setup() print (setup.lib) #None #该部分报错原因为加载驱动时报错，该部分代码直接在bash中执行时正常，在jupyter中执行结果为None import ctypes as ct path = '/home/miniconda3/envs/vllm/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117_nocublaslt.so' lib=ct.cdll.LoadLibrary(path) print (lib)
根据博客：https://www.cnblogs.com/jimlau/p/13263052.html 中方案二解决方案
cp /home/miniconda3/lib/libstdc++.so.6.0.29 /usr/lib64/
rm -f libstdc++.so.6
ln -s libstdc++.so.6.0.29 libstdc++.so.6
至此问题已解决
