diff --git a/README.md b/README.md index 04420e2..deffcb5 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,79 @@ +--- +license: other +license_name: bilibili +license_link: LICENSE +--- +
-

- Index-1.9B-32K -

+

+ Index-1.9B-32K +

-[切换到中文](https://modelscope.cn/models/IndexTeam/Index-1.9B-32K/file/view/master?fileName=README_zh.md&status=1) +[切换到中文](https://huggingface.co/IndexTeam/Index-1.9B-32K/blob/main/README_zh.md)
+ + ## Model Overview Index-1.9B-32K is a language model with only 1.9 billion parameters, yet it supports a context length of 32K (meaning this extremely small model can read documents of over 35,000 words in one go). The model has undergone Continue Pre-Training and Supervised Fine-Tuning (SFT) specifically for texts longer than 32K tokens, based on carefully curated long-text training data and self-built long-text instruction sets. The model is now open-source on both Hugging Face and ModelScope. -**Despite its small size (about 2% of models like GPT-4), Index-1.9B-32K demonstrates excellent long-text processing capabilities**. Below are comparison results with GPT-4 and GPT-3.5-turbo-16k: +Despite its small size (about 2% of models like GPT-4), Index-1.9B-32K demonstrates excellent long-text processing capabilities. +As shown in the figure below, our 1.9B-sized model's score even surpasses that of the 7B-sized model. Below is a comparison with models like GPT-4 and Qwen2: +

+

Comparison of Index-1.9B-32K with GPT-4, Qwen2, and other models in Long Context capability +

+ +In a 32K-length needle-in-a-haystack test, Index-1.9B-32K achieved excellent results, as shown in the figure below. The only exception was a small yellow spot (91.08 points) in the region of (32K length, 10% depth), with all other areas performing excellently in mostly green zones. +
+ +

NeedleBench Evaluation

+
## Index-1.9B-32K Model Download, Usage, and Technical Report: For details on downloading, usage, and the technical report for Index-1.9B-32K, see: - +
+ Index-1.9B-32K Long Context Technical Report.md +
--- --- +--- + +## Usage：Long Text Translation and Summary（Index-1.9B-32K） +- Clone the code repository for model execution and evaluation: +```shell +git clone https://github.com/bilibili/Index-1.9B +cd Index-1.9B +``` +- Download the model files to your local machine. + +- Use pip to install the required environment: + +```shell +pip install -r requirements.txt +``` + +- Run the interactive tool for long text: **demo/cli_long_text_demo.py** +- The model will, by default, read this file: data/user_long_text.txt and summarize the text in Chinese. +- You can open a new window and modify the file content in real-time, and the model will read the updated file and summarize it. + +```shell +cd demo/ +CUDA_VISIBLE_DEVICES=0 python cli_long_text_demo.py --model_path '/path/to/model/' --input_file_path data/user_long_text.txt +``` +- Run & Interaction Example (Translation and summarization of the Bilibili financial report released on 2024.8.22 in English --- [Original English report here](https://github.com/bilibili/Index-1.9B/tree/main/demo/data/user_long_text.txt))： +
+ +

Translation and Summary (Bilibili financial report released on 2024.8.22)

+
+ + ## Limitations and Disclaimer Index-1.9B-32K may generate inaccurate, biased, or otherwise objectionable content in some cases. The model cannot understand or express personal opinions or value judgments when generating content, and its output does not represent the views or stance of the model developers. Therefore, please use the generated content with caution. Users are responsible for evaluating and verifying the generated content and should refrain from spreading harmful content. Before deploying any related applications, developers should conduct safety tests and fine-tune the model based on specific use cases. diff --git a/README_zh.md b/README_zh.md index ea653b1..294542d 100644 --- a/README_zh.md +++ b/README_zh.md @@ -1,30 +1,77 @@ +
-

- Index-1.9B-32K -

- -[Switch to English](https://modelscope.cn/models/IndexTeam/Index-1.9B-32K/file/view/master?fileName=README.md&status=1) +

+ Index-1.9B-32K +

+ +[Switch to English](https://huggingface.co/IndexTeam/Index-1.9B-32K/blob/main/README.md)
+ + --- ## 模型简介 -Index-1.9B-32K 是一个仅有 1.9B 参数、却具备 32K 上下文长度的语言模型（这意味着，这个超小精灵可以一次性读完 3.5 万字的文档）。该模型专门针对 32K 以上的长文本进行了持续预训练（Continue Pre-Train）和监督微调（SFT），主要基于我们精心清洗的长文本预训练语料、自建的长文本指令集进行训练。目前，我们已在 Hugging Face 和 ModelScope 上同步开源。 +Index-1.9B-32K 是一个仅有 1.9B 参数、却具备 32K 上下文长度的语言模型（这意味着，这个超小精灵可以一次性读完 3.5 万字以上的文档）。该模型专门针对 32K 以上的长文本进行了持续预训练（Continue Pre-Train）和监督微调（SFT），主要基于我们精心清洗的长文本预训练语料、自建的长文本指令集进行训练。目前，我们已在 Hugging Face 和 ModelScope 上同步开源。 -Index-1.9B-32K **以极小的模型体积（体积约为GPT-4等模型的2%）实现了出色的长文本处理能力**。以下为与 GPT-4、GPT-3.5-turbo-16k 的对比评测结果： +Index-1.9B-32K 以极小的模型体积（约为 GPT-4 等模型的 2%），实现了出色的长文本处理能力。如下图，我们1.9B尺寸的模型分数甚至远超7B尺寸的模型。以下为与 GPT-4、Qwen2等模型的对比： +

+ +

+

Index-1.9B-32K与GPT-4、Qwen2等模型长文本能力对比

+Index-1.9B-32K在32K长度的大海捞针测试下，评测结果优异，如下图，评测结果只在（32K 长度，%10 深度）区域有一处黄斑（91.08分），其他范围表现优异，几乎全绿。 +
+ +

大海捞针评测

+
+ ## Index-1.9B-32K模型下载、使用、技术报告： Index-1.9B-32K模型下载、使用方法、技术报告详见： - +
+ Index-1.9B-32K长上下文技术报告.md +
--- --- +--- + + +## 使用：长文本翻译&总结（Index-1.9B-32K） +- 下载仓库： +```shell +git clone https://github.com/bilibili/Index-1.9B +cd Index-1.9B +``` +- 下载模型到本地. + +- 使用 pip 安装依赖: + +```shell +pip install -r requirements.txt +``` + +- 运行长文本专用的交互工具：demo/cli_long_text_demo.py +- 模型默认会读取该文件：data/user_long_text.txt，将对文本内容进行中文总结。 +- 可以新建一个窗口，实时修改文件内容，模型会读取最新的文件内容并总结。 + +```shell +cd demo/ +CUDA_VISIBLE_DEVICES=0 python cli_long_text_demo.py --model_path '/path/to/model/' --input_file_path data/user_long_text.txt +``` +- 运行&交互效果（翻译并总结哔哩哔哩公司于2024.8.22发布的英文财报 --- [英文财报原文在这里](https://github.com/bilibili/Index-1.9B/tree/main/demo/data/user_long_text.txt))： +
+ +

翻译总结（哔哩哔哩公司于2024.8.22发布的英文财报）

+
+ + ## 局限性与免责申明 Index-1.9B-32K在某些情况下可能会产生不准确、有偏见或其他令人反感的内容。模型生成内容时无法理解、表达个人观点或价值判断，其输出内容不代表模型开发者的观点和立场。因此，请谨慎使用模型生成的内容，用户在使用时应自行负责对其进行评估和验证，请勿将生成的有害内容进行传播，且在部署任何相关应用之前，开发人员应根据具体应用对模型进行安全测试和调优。 @@ -38,10 +85,12 @@ Index-1.9B-32K的模型权重则需要遵循[[模型许可协议]{.underline}](h Index-1.9B-32K模型权重对学术研究**完全开放**，并且支持**免费商用**。 + ## 引用 如果你觉得我们的工作对你有帮助，欢迎引用！ + ```shell @article{Index-1.9B-32K, title={Index-1.9B-32K Long Context Technical Report}, diff --git a/modeling_index.py b/modeling_index.py index c374a55..c6d7461 100644 --- a/modeling_index.py +++ b/modeling_index.py @@ -111,8 +111,8 @@ class IndexRotaryEmbedding(torch.nn.Module): self._set_cos_sin_cache( seq_len=self.max_position_embeddings, device=self.inv_freq.device, dtype=torch.get_default_dtype() ) - print("rope cache value:") - self._show_rope_cache_info() + #print("rope cache value:") + #self._show_rope_cache_info() def _show_rope_cache_info( self ): np.set_printoptions(precision=6, threshold=10, edgeitems=5, linewidth=200, suppress=True) diff --git a/z-attach-pic-needle-bench-en.png b/z-attach-pic-needle-bench-en.png new file mode 100644 index 0000000..9bfa6c0 Binary files /dev/null and b/z-attach-pic-needle-bench-en.png differ diff --git a/z-attach-pic-pk-all.png b/z-attach-pic-pk-all.png new file mode 100644 index 0000000..ba88cd7 Binary files /dev/null and b/z-attach-pic-pk-all.png differ diff --git a/z-attach-pic-qa-mark.png b/z-attach-pic-qa-mark.png new file mode 100644 index 0000000..f3830b3 Binary files /dev/null and b/z-attach-pic-qa-mark.png differ