Ascend 910B 快速部署Qwen3-VL-30B-A3B-Instruct

环境：8张32GB显存大910B

下载模型权重

pip install modelscope
modelscope download --model Qwen/Qwen3-VL-30B-A3B-Instruct

安装

运行Docker容器：

# Update the vllm-ascend image
export IMAGE=quay.io/ascend/vllm-ascend:v0.18.0

docker run --rm \
--name vllm-ascend \
--shm-size=1g \
--net=host \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /root/.cache:/root/.cache \
-v /data:/data \
-v <path/to/your/media>:/media \
-it $IMAGE bash

设置环境变量：

# Load model from ModelScope to speed up download
export VLLM_USE_MODELSCOPE=True

# Set `max_split_size_mb` to reduce memory fragmentation and avoid out of memory
export PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256

启动服务：

vllm serve Qwen/Qwen3-VL-30B-A3B-Instruct \
--tensor-parallel-size 2 \
--enable-expert-parallel \
--limit-mm-per-prompt.video 0 \
--max-model-len 128000

通过接口访问服务：

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "Qwen/Qwen3-VL-30B-A3B-Instruct",
    "messages": [
        {"role": "system", "content": "你是一个图片识别助手."},
        {"role": "user", "content": [
            {"type": "image_url", "image_url": "base64图片"},
            {"type": "text", "text": "请帮我分析这张图片中是什么?"}
        ]}
    ],
    "max_completion_tokens": 100
    }'

可以通过这个网站来查看更详细的使用方法。
https://docs.vllm.ai/projects/ascend/zh-cn/v0.18.0/index.html#