Atlas 300i duo 部署bge-m3笔记

模型下载

https://modelscope.cn/models/BAAI/bge-m3

pip install modelscope
modelscope download --model  BAAI/bge-m3 --local_dir /data/alg/bge-m3

mis-tei镜像下载

https://www.hiascend.com/developer/ascendhub/detail/07a016975cc341f3a5ae131f2b52399d
docker pull --platform=arm64 swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:26.0.0-310p-openeuler24.03-py3.11

export MODEL_DIR=/data/alg

docker run \
  --privileged \
  -u root \
  --net host \
  --name tei-m3 \
  -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
  -v /usr/local/dcmi:/usr/local/dcmi:ro \
  -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi:ro \
  -v /usr/local/Ascend/firmware:/usr/local/Ascend/firmware:ro \
  -v /usr/local/sbin:/usr/local/sbin:ro \
  -v /etc/ascend_install.info:/etc/ascend_install.info:ro \
  -e MODEL_DIR=${MODEL_DIR} \
  -e ASCEND_VISIBLE_DEVICES=4,5 \
  -e ASCEND_RT_VISIBLE_DEVICES=4,5 \
  -v ${MODEL_DIR}:${MODEL_DIR} \
  -itd \
  swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:26.0.0-310p-ubuntu22.04-py3.11 \
  --model-id ${MODEL_DIR}/bge-m3 \
  --pooling cls \
  --hostname 0.0.0.0 \
  --port 29516 \
  --prometheus-port 19000

输出以上日志及部署成功

测试

curl -s 10.0.88.53:29516/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"bge-m3","input":["你好"]}' \
  | python3 -c "import sys,json; d=json.load(sys.stdin); print(len(d['data'][0]['embedding']))"

成功部署！
参考连接：https://www.hiascend.com/developer/ascendhub/detail/07a016975cc341f3a5ae131f2b52399d