diff --git a/README.md b/README.md index d5d9715..9bf8465 100644 --- a/README.md +++ b/README.md @@ -11,11 +11,11 @@ library_name: transformers

- ━━━━━━━━━━━━━━━━━━━━━━━━━ + ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Unlocking the Reasoning Potential of Language Model
From Pretraining to Posttraining
- ━━━━━━━━━━━━━━━━━━━━━━━━━ + ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

@@ -35,7 +35,41 @@ library_name: transformers
-> This model repository is licensed under the MIT License. +--- + +## Updates + +[2025.05.30] We scaled the SFT dataset from approximately 500K to 6M instances and continuously expanding the RL training window size from 32K to 48K, the performance of [MiMo-7B-RL-0530](https://huggingface.co/XiaomiMiMo/MiMo-7B-RL-0530) on AIME24 can be continuously improved and eventually surpass that of DeepSeek R1 (79.8). + + + + + + + + + + + + +

+

+

+ + + + + + + + + + + + +
BenchmarkMiMo-7B-RLMiMo-7B-RL-0530
Mathematics
MATH500
(Pass@1)
95.897.2
AIME 2024
(Pass@1)
68.280.1
AIME 2025
(Pass@1)
55.470.2
Code
LiveCodeBench v5
(Pass@1)
57.860.9
LiveCodeBench v6
(Pass@1)
49.352.2
STEM
GPQA-Diamond
(Pass@1)
54.460.6
General
Alignbench1.1
(Evaluated by GPT4.1)
6.97.4
+ +--- ## I. Introduction @@ -122,7 +156,7 @@ MiMo-7B series ### SGLang Inference -Thanks to the [contribution](https://github.com/sgl-project/sglang/pull/5921) from the SGLang team, we supported MiMo in SGLang mainstream within 24h with MTP coming soon. +Thanks to the [MiMo model support](https://github.com/sgl-project/sglang/pull/5921) and [MTP](https://github.com/sgl-project/sglang/pull/6059) from the SGLang team, we supported MiMo in SGLang mainstream. Example Script @@ -132,9 +166,14 @@ python3 -m uv pip install "sglang[all] @ git+https://github.com/sgl-project/sgla # Launch SGLang Server python3 -m sglang.launch_server --model-path XiaomiMiMo/MiMo-7B-RL --host 0.0.0.0 --trust-remote-code + +# Launch MTP Server +python3 -m sglang.launch_server --model-path XiaomiMiMo/MiMo-7B-RL --trust-remote-code \ +--speculative-algorithm EAGLE --speculative-num-steps 1 --speculative-eagle-topk 1 \ +--speculative-num-draft-tokens 2 --mem-fraction 0.5 ``` -Detailed usage can be found in [SGLang documents](https://docs.sglang.ai/backend/send_request.html). MTP will also be supported in 24h. +Detailed usage can be found in [SGLang documents](https://docs.sglang.ai/backend/send_request.html). ### vLLM inference @@ -223,7 +262,7 @@ print(tokenizer.decode(output.tolist()[0])) ```bibtex @misc{coreteam2025mimounlockingreasoningpotential, title={MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining}, - author={{Xiaomi LLM-Core Team}}, + author={LLM-Core-Team Xiaomi}, year={2025}, eprint={2505.07608}, archivePrefix={arXiv},