异构算力集群硬件巡检:主流 GPU/NPU PCI 识别与驱动检测指南
·
前言
在信创混合算力平台、AI 推理训练集群交付运维场景中,异构算力卡的环境校验分为两大环节:硬件 PCI 识别校验、驱动与容器运行时校验。 lspci 命令用于确认服务器 BIOS、系统底层识别到算力硬件,仅代表物理设备可被系统发现;smi/gmi 系列工具验证内核驱动与固件加载正常;容器运行时检测,用于确认容器环境可透传算力设备,是容器化 AI 业务上线前必做检查。 本文汇总 NVIDIA、AMD、昇腾、海光 DCU、沐曦、摩尔线程、天数智芯、寒武纪、平头哥主流算力芯片的 PCI 厂商 ID、设备 ID、硬件识别命令、驱动与容器运行时检测命令,附带完整可直接运行的 Shell 巡检脚本。
一、各算力卡 PCI 识别 + 驱动 / 容器运行时检测
命令说明:
lspci -nn可同时打印厂商 ID: 设备 ID,是硬件定位首选;lspci -d 厂商ID:精准过滤对应厂商所有算力设备,不受系统字符集、设备名称本地化影响,适合自动化巡检脚本。
NVIDIA GPU
- 厂商 ID:
10de - PCI 硬件识别
# 通用检索
lspci | grep -i nvidia
# 按厂商ID精准过滤(推荐脚本使用)
lspci -d 10de:
适用:A100/A800/H100/H200/T4/V100 等全系列 NVIDIA GPU。
- 驱动与 NVIDIA Container Toolkit 检测
nvidia-smi >/dev/null 2>&1 && echo "NVIDIA driver OK" || (echo "NVIDIA driver issue"; exit 1) && sudo docker info 2>/dev/null | grep -q "Runtime.*nvidia" && echo "NVIDIA Container Toolkit OK" || (echo "NVIDIA Container Toolkit not configured"; exit 1)
AMD GPU(ROCm)
- 厂商 ID:
1002 - PCI 硬件识别
lspci -d 1002: | grep -E "Vega|MI25|MI50|MI100|MI200"
适用:AMD MI 系列计算卡。
- 驱动与 AMD 容器工具检测
amd-smi static >/dev/null 2>&1 && echo "AMD driver OK" || (echo "AMD driver issue"; exit 1) && sudo docker info 2>/dev/null | grep -q "amd" && echo "AMD Container Toolkit OK" || (echo "AMD Container Toolkit not configured"; exit 1)
昇腾 Ascend NPU
- 厂商 ID:
19e5- Ascend 310P:设备 ID
d500 - Ascend 910B:设备 ID
d801 - Ascend 910Pro/910B4:设备 ID
d802
- Ascend 310P:设备 ID
- PCI 硬件识别
# 单独查询310P
lspci -d 19e5:d500
# 查询所有昇腾NPU
lspci -d 19e5:
- 驱动与 Ascend 容器运行时检测
npu-smi info >/dev/null 2>&1 && echo "Ascend driver OK" || (echo "Ascend driver issue"; exit 1) && sudo docker info 2>/dev/null | grep -q "ascend" && echo "Ascend Container Toolkit OK" || (echo "Ascend Container Toolkit not configured"; exit 1)
海光 Hygon DCU
- 厂商 ID:
1d22- DCU Z100:设备 ID
2001,设备名称含co-关键字
- DCU Z100:设备 ID
- PCI 硬件识别
lspci -d 1d22:
lspci | grep -i "co-"
lspci | grep -i haiguang
- 驱动检测
hy-smi >/dev/null 2>&1 && echo "Hygon driver OK" || (echo "Hygon driver issue"; exit 1)
沐曦 Metax GPU
- 厂商 ID:
9999- C500 / K100:设备 ID
9999
- C500 / K100:设备 ID
- PCI 硬件识别
lspci -d 9999:
lspci | grep -i 9999
- 驱动检测
mx-smi >/dev/null 2>&1 && echo "Metax driver OK" || (echo "Metax driver issue"; exit 1)
摩尔线程 Moore Threads
- 厂商 ID:
1ef3- MTT S30 / MTT S80
- PCI 硬件识别
lspci -d 1ef3:
lspci | grep -i "moore"
- 驱动与容器运行时检测
mthreads-gmi >/dev/null 2>&1 && echo "Moore Threads driver OK" || (echo "Moore Threads driver issue"; exit 1) && sudo docker info 2>/dev/null | grep -q "mthreads" && echo "Moore Threads Container Toolkit OK" || (echo "Moore Threads Container Toolkit not configured"; exit 1)
天数智芯 Iluvatar GPU
- 厂商 ID:
1e3e- G100、BI 系列
- PCI 硬件识别
lspci -d 1e3e:
lspci | grep -i 1e3e
- 驱动与容器运行时检测
ixsmi >/dev/null 2>&1 && echo "Iluvatar driver OK" || (echo "Iluvatar driver issue"; exit 1) && sudo docker info 2>/dev/null | grep -q "iluvatar" && echo "Iluvatar Container Toolkit OK" || (echo "Iluvatar Container Toolkit not configured"; exit 1)
寒武纪 Cambricon
- 厂商 ID:
1bc0- MLU270 / MLU370 / MLU490
- PCI 硬件识别
lspci -d 1bc0:
lspci | grep -i cambricon
- 驱动检测
cnmon info >/dev/null 2>&1 && echo "Cambricon driver OK" || (echo "Cambricon driver issue"; exit 1)
平头哥 T-Head PPU
- 厂商 ID:
1ef2 - PCI 硬件识别
lspci -d 1ef2:
lspci | grep -i "t-head\|ppu"
- 驱动检测
ppu-smi >/dev/null 2>&1 && echo "T-Head PPU driver OK" || (echo "T-Head PPU driver issue"; exit 1)
二、完整一键巡检脚本 run.sh
#!/bin/bash
echo "====================================="
echo " 异构算力巡检工具 "
echo "====================================="
echo -e "\n===== 1. PCI硬件识别检测 ====="
echo "[NVIDIA]"
lspci -d 10de:
echo "[AMD GPU]"
lspci -d 1002:
echo "[Ascend NPU]"
lspci -d 19e5:
echo "[Hygon DCU]"
lspci -d 1d22:
echo "[沐曦 Metax]"
lspci -d 9999:
echo "[摩尔线程]"
lspci -d 1ef3:
echo "[天数智芯 Iluvatar]"
lspci -d 1e3e:
echo "[寒武纪 Cambricon]"
lspci -d 1bc0:
echo "[平头哥 T-Head PPU]"
lspci -d 1ef2:
echo -e "\n===== 2. 驱动&容器运行时检测 ====="
# NVIDIA
nvidia-smi >/dev/null 2>&1 && echo "✅ NVIDIA driver OK" || echo "❌ NVIDIA driver issue"
sudo docker info 2>/dev/null | grep -q "Runtime.*nvidia" && echo "✅ NVIDIA Container Toolkit OK" || echo "❌ NVIDIA Container Toolkit not configured"
# AMD
amd-smi static >/dev/null 2>&1 && echo "✅ AMD driver OK" || echo "❌ AMD driver issue"
sudo docker info 2>/dev/null | grep -q "amd" && echo "✅ AMD Container Toolkit OK" || echo "❌ AMD Container Toolkit not configured"
# Ascend
npu-smi info >/dev/null 2>&1 && echo "✅ Ascend driver OK" || echo "❌ Ascend driver issue"
sudo docker info 2>/dev/null | grep -q "ascend" && echo "✅ Ascend Container Toolkit OK" || echo "❌ Ascend Container Toolkit not configured"
# Hygon DCU
hy-smi >/dev/null 2>&1 && echo "✅ Hygon driver OK" || echo "❌ Hygon driver issue"
# Metax
mx-smi >/dev/null 2>&1 && echo "✅ Metax driver OK" || echo "❌ Metax driver issue"
# Moore Threads
mthreads-gmi >/dev/null 2>&1 && echo "✅ Moore Threads driver OK" || echo "❌ Moore Threads driver issue"
sudo docker info 2>/dev/null | grep -q "mthreads" && echo "✅ Moore Threads Container Toolkit OK" || echo "❌ Moore Threads Container Toolkit not configured"
# Iluvatar
ixsmi >/dev/null 2>&1 && echo "✅ Iluvatar driver OK" || echo "❌ Iluvatar driver issue"
sudo docker info 2>/dev/null | grep -q "iluvatar" && echo "✅ Iluvatar Container Toolkit OK" || echo "❌ Iluvatar Container Toolkit not configured"
# Cambricon
cnmon info >/dev/null 2>&1 && echo "✅ Cambricon driver OK" || echo "❌ Cambricon driver issue"
# T-Head PPU
ppu-smi >/dev/null 2>&1 && echo "✅ T-Head PPU driver OK" || echo "❌ T-Head PPU driver issue"
echo -e "\n===== 巡检完成 ====="
脚本使用方式:
chmod +x run.sh
./run.sh
三、运维排查要点
- lspci 看不到算力卡:优先排查 PCIe 插槽、供电、BIOS 开启 SR-IOV / 设备开关、虚拟机 PCI 直通配置。
- lspci 可识别设备,但 smi 工具报错:内核驱动版本不匹配、固件缺失、未加载对应内核模块。
- 容器运行时检测失败:对应容器工具包未安装,docker daemon 未重启加载 runtime。
- 部分国产算力当前暂无官方容器 runtime,可跳过该段校验。
四、小结
混合异构算力集群交付,硬件识别只是第一步,完整巡检需要 PCI 硬件识别、驱动可用性、容器运行时三层校验。本文脚本可集成到服务器装机自动化流程、CI/CD 流水线、定期运维巡检任务,快速定位异构算力环境问题,降低集群上线故障排查成本。
鲲鹏昇腾开发者社区是面向全社会开放的“联接全球计算开发者,聚合华为+生态”的社区,内容涵盖鲲鹏、昇腾资源,帮助开发者快速获取所需的知识、经验、软件、工具、算力,支撑开发者易学、好用、成功,成为核心开发者。
更多推荐

所有评论(0)