命令如下

Browse Source

docker run -dit --name gpt-120b \
-e NVIDIA_VISIBLE_DEVICES=0,1  \
--shm-size 64g \
--network=host -v ./:/data/models \
--restart=always \
lmsysorg/sglang:dev  python3 -m sglang.launch_server --model /data/models --host 0.0.0.0 --port 40002 --trust-remote --tp 2 --served-model-name gpt-120b --tool-call-parser gpt-oss
自己替换下模型ID，针对模型部署和评测详细感兴趣的可以微信搜索 AI云枢 有详细讲解和其他模型的部署评测教程

...

This commit is contained in:

liguoyu3564

2025-08-15 01:14:46 +00:00

parent 6d460b1f00

commit ffb4de7642

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

Diff Content Not Available