Add memory view

.github/workflows/deploy_sphinx_docs.yml

@@ -0,0 +1,88 @@
+name: deploy-sphinx-documentation-to-pages
+
+on:
+  pull_request:
+    types: [opened, synchronize]
+    paths:
+      - 'docs/sphinx_doc/**/*'
+  push:
+    branches:
+      - main
+
+jobs:
+  pages:
+    runs-on: ubuntu-20.04
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v4
+    - name: Setup Python
+      uses: actions/setup-python@master
+      with:
+        python-version: '3.10'
+    - name: Choose Pandoc
+      shell: bash
+      run: |
+        case $RUNNER_OS in
+          "Linux")
+              printf 'INSTALLER_SUFFIX=1-amd64.deb' >> $GITHUB_ENV
+              ;;
+           "macOS")
+              printf 'INSTALLER_SUFFIX=macOS.pkg' >> $GITHUB_ENV
+              ;;
+            *)
+              printf 'Do not know how to install pandoc on %s\n' "$RUNNER_OS"
+              exit 1
+              ;;
+        esac
+    - name: Download Pandoc
+      shell: bash
+      env:
+        GITHUB_TOKEN: ${{ github.token }}
+        REPO: jgm/pandoc
+        DOWNLOAD_URL: 'https://github.com/jgm/pandoc/releases/download/'
+      run: |
+        gh release download ${{ inputs.version }} \
+            --repo "$REPO" \
+            --pattern '*'${{ env.INSTALLER_SUFFIX }}
+        printf 'INSTALLER_VERSION=%s' \
+               "$(ls pandoc-*-${{ env.INSTALLER_SUFFIX }} | \
+                      sed 's/pandoc-\([0-9.]*\)-.*/\1/')" \
+               >> $GITHUB_ENV
+    - name: Install Pandoc
+      shell: bash
+      env:
+        INSTALLER: pandoc-${{ env.INSTALLER_VERSION }}-${{ env.INSTALLER_SUFFIX }}
+      run: |
+        case $RUNNER_OS in
+          "Linux")
+              sudo apt install ./$INSTALLER
+              ;;
+           "macOS")
+              sudo installer -pkg ./$INSTALLER -target '/'
+              ;;
+            *)
+              echo "$RUNNER_OS not supported"
+              exit 1
+              ;;
+        esac
+        rm $INSTALLER
+    - name: Install Sphinx Dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+        pip install -r docs/sphinx_doc/requirements.txt
+    - name: Build Documentation
+      run: |
+        cd docs/sphinx_doc
+        bash build_sphinx_doc.sh
+    - name: Upload Documentation
+      uses: actions/upload-artifact@v3
+      with:
+        name: SphinxDoc
+        path: 'docs/sphinx_doc/build/html'
+    - name: Push Pages
+      uses: peaceiris/actions-gh-pages@v3
+      if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
+      with:
+        github_token: ${{ secrets.GITHUB_TOKEN }}
+        publish_dir: 'docs/sphinx_doc/build/html'

Dockerfile

@@ -11,9 +11,9 @@
 #    sudo docker build --network=host -t memoryscope .
 
 # To run docker image: 
-#    sudo docker run -it --rm --net=host memoryscope
+#    sudo docker run -it --rm --memory=4G --net=host memoryscope
 # To run docker image with arguments (refer to memoryscope/core/config/arguments.py): 
-#    sudo docker run -it --rm --net=host -e "OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" -e "language=en" -e "human_name=superman" -e "generation_backend=openai_generation" -e "generation_model=gpt-4o" -e "embedding_backend=openai_embedding" -e "embedding_model=text-embedding-3-small" -e "enable_ranker=False" memoryscope
+#    sudo docker run -it --rm --memory=4G --net=host -e "OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" -e "language=en" -e "human_name=superman" -e "generation_backend=openai_generation" -e "generation_model=gpt-4o" -e "embedding_backend=openai_embedding" -e "embedding_model=text-embedding-3-small" -e "enable_ranker=False" memoryscope
 
 FROM python:3.11
 

README.md

@@ -1,33 +1,37 @@
 English | [**中文**](./README_ZH.md)
 
 # MemoryScope
-
-![MemoryScope Logo](./docs/images/logo.png)
-
+<p align="center">
+ <img src="./docs/images/logo.png" alt="MemoryScopeLogo" width="75%">
+</p>
 Equip your LLM chatbot with a powerful and flexible long term memory system.
 
 [![](https://img.shields.io/badge/python-3.10+-blue)](https://pypi.org/project/memoryscope/)
-[![](https://img.shields.io/badge/pypi-v0.1.1-blue?logo=pypi)](https://pypi.org/project/memoryscope/)
+[![](https://img.shields.io/badge/pypi-v0.1.1.0-blue?logo=pypi)](https://pypi.org/project/memoryscope/)
 [![](https://img.shields.io/badge/license-Apache--2.0-black)](./LICENSE)
-[![](https://img.shields.io/badge/Docs-English%7C%E4%B8%AD%E6%96%87-blue?logo=markdown)](https://modelscope.github.io/memoryscope/#welcome-to-memoryscope-tutorial-hub)
-[![](https://img.shields.io/badge/Docs-API_Reference-blue?logo=markdown)](https://modelscope.github.io/memoryscope/)
-[![](https://img.shields.io/badge/Contribute-Welcome-green)](https://modelscope.github.io/memoryscope/tutorial/contribute.html)
+[![](https://img.shields.io/badge/Docs-English%7C%E4%B8%AD%E6%96%87-blue?logo=markdown)](https://modelscope.github.io/MemoryScope/en/index.html#welcome-to-memoryscope-tutorial)
+[![](https://img.shields.io/badge/Docs-API_Reference-blue?logo=markdown)](https://modelscope.github.io/MemoryScope/en/docs/api.html)
+[![](https://img.shields.io/badge/Contribute-Welcome-green)](https://modelscope.github.io/MemoryScope/en/docs/contribution.html)
 
 ----
 ## 📰 News
 
-- **[2024-09-10]** We release MemoryScope v0.1.1 now, which is also available in [PyPI](https://pypi.org/simple/memoryscope/)!
+- **[2024-09-10]** We release MemoryScope v0.1.1.0 now, which is also available in [PyPI](https://pypi.org/simple/memoryscope/)!
 ----
 ## 🌟 What is MemoryScope？
 MemoryScope provides LLM chatbots with powerful and flexible long-term memory capabilities, offering a framework for building such abilities. 
 It can be applied to scenarios like personal assistants and emotional companions, continuously learning through long-term memory to remember users' basic information as well as various habits and preferences. 
 This allows users to gradually experience a sense of "understanding" when using the LLM.
 
 ### Demo
-![english-gif](https://github.com/user-attachments/assets/1754c814-1342-4288-a8a3-74d0b40f59a6)
+<p align="center">
+ <img src="https://github.com/user-attachments/assets/1754c814-1342-4288-a8a3-74d0b40f59a6" alt="en_demo" width="75%">
+</p>
 
 ### Framework
-![Framework](./docs/images/framework.png)
+<p align="center">
+ <img src="./docs/images/framework.png" alt="Framework" width="75%">
+</p>
 
 💾 Memory Database: MemoryScope is equipped with a vector database (default is *ElasticSearch*) to store all memory fragments recorded in the system.
 

README_ZH.md

@@ -1,17 +1,17 @@
 [**English**](./README.md) | 中文
 
 # MemoryScope
-
-![MemoryScope Logo](./docs/images/logo.png)
-
+<p align="center">
+ <img src="./docs/images/logo.png" alt="MemoryScopeLogo" width="75%">
+</p>
 为您的大语言模型聊天机器人配备强大且灵活的长期记忆系统。
 
 [![](https://img.shields.io/badge/python-3.10+-blue)](https://pypi.org/project/memoryscope/)
 [![](https://img.shields.io/badge/pypi-v0.1.1-blue?logo=pypi)](https://pypi.org/project/memoryscope/)
 [![](https://img.shields.io/badge/license-Apache--2.0-black)](./LICENSE)
-[![](https://img.shields.io/badge/Docs-English%7C%E4%B8%AD%E6%96%87-blue?logo=markdown)](https://modelscope.github.io/memoryscope/#welcome-to-memoryscope-tutorial-hub)
-[![](https://img.shields.io/badge/Docs-API_Reference-blue?logo=markdown)](https://modelscope.github.io/memoryscope/)
-[![](https://img.shields.io/badge/Contribute-Welcome-green)](https://modelscope.github.io/memoryscope/tutorial/contribute.html)
+[![](https://img.shields.io/badge/Docs-English%7C%E4%B8%AD%E6%96%87-blue?logo=markdown)](https://modelscope.github.io/MemoryScope/zh/index.html#id1)
+[![](https://img.shields.io/badge/Docs-API_Reference-blue?logo=markdown)](https://modelscope.github.io/MemoryScope/zh/docs/api.html)
+[![](https://img.shields.io/badge/Contribute-Welcome-green)](https://modelscope.github.io/MemoryScope/zh/docs/contribution.html)
 
 <!-- 创空间
 [![](https://img.shields.io/badge/ModelScope-Demos-4e29ff.svg?logo=data:image/svg+xml;base64,PHN2ZyB2aWV3Qm94PSIwIDAgMjI0IDEyMS4zMyIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KCTxwYXRoIGQ9Im0wIDQ3Ljg0aDI1LjY1djI1LjY1aC0yNS42NXoiIGZpbGw9IiM2MjRhZmYiIC8+Cgk8cGF0aCBkPSJtOTkuMTQgNzMuNDloMjUuNjV2MjUuNjVoLTI1LjY1eiIgZmlsbD0iIzYyNGFmZiIgLz4KCTxwYXRoIGQ9Im0xNzYuMDkgOTkuMTRoLTI1LjY1djIyLjE5aDQ3Ljg0di00Ny44NGgtMjIuMTl6IiBmaWxsPSIjNjI0YWZmIiAvPgoJPHBhdGggZD0ibTEyNC43OSA0Ny44NGgyNS42NXYyNS42NWgtMjUuNjV6IiBmaWxsPSIjMzZjZmQxIiAvPgoJPHBhdGggZD0ibTAgMjIuMTloMjUuNjV2MjUuNjVoLTI1LjY1eiIgZmlsbD0iIzM2Y2ZkMSIgLz4KCTxwYXRoIGQ9Im0xOTguMjggNDcuODRoMjUuNjV2MjUuNjVoLTI1LjY1eiIgZmlsbD0iIzYyNGFmZiIgLz4KCTxwYXRoIGQ9Im0xOTguMjggMjIuMTloMjUuNjV2MjUuNjVoLTI1LjY1eiIgZmlsbD0iIzM2Y2ZkMSIgLz4KCTxwYXRoIGQ9Im0xNTAuNDQgMHYyMi4xOWgyNS42NXYyNS42NWgyMi4xOXYtNDcuODR6IiBmaWxsPSIjNjI0YWZmIiAvPgoJPHBhdGggZD0ibTczLjQ5IDQ3Ljg0aDI1LjY1djI1LjY1aC0yNS42NXoiIGZpbGw9IiMzNmNmZDEiIC8+Cgk8cGF0aCBkPSJtNDcuODQgMjIuMTloMjUuNjV2LTIyLjE5aC00Ny44NHY0Ny44NGgyMi4xOXoiIGZpbGw9IiM2MjRhZmYiIC8+Cgk8cGF0aCBkPSJtNDcuODQgNzMuNDloLTIyLjE5djQ3Ljg0aDQ3Ljg0di0yMi4xOWgtMjUuNjV6IiBmaWxsPSIjNjI0YWZmIiAvPgo8L3N2Zz4K)](https://modelscope.cn/studios?name=memoryscope&page=1&sort=latest)
@@ -21,29 +21,33 @@
 ----
 ## 📰 新闻
 
-- **[2024-09-10]** 我们现在发布了 MemoryScope v0.1.1，该版本也可以在 [PyPI](https://pypi.org/simple/memoryscope/) 上获取！
+- **[2024-09-10]** 我们现在发布了 MemoryScope v0.1.1.0，该版本也可以在 [PyPI](https://pypi.org/simple/memoryscope/) 上获取！
 ----
 
 ## 🌟 什么是MemoryScope？
 MemoryScope可以为LLM聊天机器人提供强大且灵活的长期记忆能力，并提供了构建长期记忆能力的框架。
 MemoryScope可以用于个人助理、情感陪伴等记忆场景，通过长期记忆能力来不断学习，记得用户的基础信息以及各种习惯和喜好，使得用户在使用LLM时逐渐感受到一种“默契”。
 
 ### Demo
-![chinese-gif](https://github.com/user-attachments/assets/57519274-8c01-4d88-bcd2-0ebce3551e5d)
+<p align="center">
+ <img src="https://github.com/user-attachments/assets/57519274-8c01-4d88-bcd2-0ebce3551e5d" alt="zh_demo" width="75%">
+</p>
+
 
 ### 核心框架：
-![Framework](./docs/images/framework.png)
+<p align="center">
+ <img src="./docs/images/framework.png" alt="Framework" width="75%">
+</p>
 
 💾 记忆数据库: MemoryScope配备了向量数据库(默认是*ElasticSearch*)，用于存储系统中记录的所有记忆片段。
 
 🔧 核心worker库: MemoryScope将长期记忆的能力原子化，抽象成单独的worker，包括query信息过滤，observation抽取，insight更新等20+worker。
 
-🛠️ 核心Op库: 并基于worker的pipeline构建了memory服务的核心operation，实现了记忆检索，记忆巩固等核心能力。
+🛠️ 核心Operation库: MemoryScope将workers组织为工作流(workflow)，构建了处理记忆的完善Operation库，实现了记忆检索，记忆巩固等核心能力。
 
 - 记忆检索：当用户输入对话，此操作返回语义相关的记忆片段。如果输入对话包含对时间的指涉，则同时返回相应时间中的记忆片段。
 - 记忆巩固：此操作接收一批用户的输入对话，并从对话中提取重要的用户信息，将其作为 *observation* 形式的记忆片段存储在记忆数据库中。
-- 反思与再巩固：每隔一段时间，此操作对新记录的 *observations* 进行反思，以形成和更新 *insight*
-   形式的记忆片段。然后执行记忆再巩固，以确保记忆片段之间的矛盾和重复得到妥善处理。
+- 反思与再巩固：每隔一段时间，此操作对新记录的 *observations* 进行反思，以形成和更新 *insight* 形式的记忆片段。然后执行记忆再巩固，以确保记忆片段之间的矛盾和重复得到妥善处理。
 
 ⚙️ 最佳实践:
 
@@ -62,7 +66,7 @@ MemoryScope可以用于个人助理、情感陪伴等记忆场景，通过长期
 - 过滤掉用户输入的虚构内容，以避免LLM产生幻觉。
 
 ⏰ 时间敏感性:
-- T系统在执行记忆检索和记忆巩固时具备时间敏感性，因此在输入对话包含对时间的指涉时，可以检索到准确的相关信息。
+- 在执行记忆检索和记忆巩固时具备时间敏感性，因此在输入对话包含对时间的指涉时，可以检索到准确的相关信息。
 
 ----
 

docker-compose.yml

@@ -1,22 +1,13 @@
-version: '3.8'
-
 services:
   memory_scope_main:
     build:
-      context: ./
+      context: ghcr.io/modelscope/memoryscope:main
     environment:
       DASHSCOPE_API_KEY: "sk-0000000000"
       # OPENAI_API_KEY: "sk-0000000000"
     volumes:
       - ./memoryscope/core/config:/memory_scope_project/memoryscope/memoryscope/core/config
-    depends_on:
-      - elasticsearch
-
-  elasticsearch:
-    image: docker.elastic.co/elasticsearch/elasticsearch:8.13.2
-    ports:
-      - "9200:9200"
-    environment:
-      discovery.type: "single-node"
-      xpack.security.enabled: "false"
-      xpack.license.self_generated.type: "trial"
+    deploy:
+      resources:
+        limits:
+          memory: 4G

docs/installation.md

@@ -1,6 +1,6 @@
 # Installing MemoryScope
 
-## I. Install with docker [Recommended]
+## I. Install with docker [Recommended] [x86_64]
 
 1. Clone the repository and edit settings
     ```bash
@@ -16,13 +16,13 @@
     sudo docker build --network=host -t memoryscope .
     ```
 
-3. Launch Docker container
+3. Launch Docker container (ARM architecture not yet supported)
     ```bash
     sudo docker run -it --rm --net=host memoryscope
     ```
 
 
-## II. Install with docker compose [Recommended]
+## II. Install with docker compose [Recommended] [x86_64]
 
 1. Clone the repository and edit settings
     ```bash
@@ -38,7 +38,7 @@
     OPENAI_API_KEY: "sk-0000000000"
     ```
 
-3. Run `docker-compose up` to build and launch the memory-scope cli interface.
+3. Run `docker-compose run memory_scope_main` to build and launch the memory-scope cli interface. (ARM architecture not yet supported)
 
 
 ## III. Install from PyPI

docs/installation_zh.md

@@ -1,6 +1,6 @@
 # MemoryScope 安装指南
 
-## 一、使用 Docker 安装 [推荐]
+## 一、使用 Docker 安装 [推荐] [x86_64]
 
 1. 克隆仓库并编辑配置
     ```bash
@@ -16,13 +16,14 @@
     sudo docker build --network=host -t memoryscope .
     ```
 
-3. 启动 Docker 容器
+3. 启动 Docker 容器 (备注：ARM架构暂不支持)
     ```bash
     sudo docker run -it --rm --net=host memoryscope
     ```
 
 
-## 二、使用 Docker Compose 安装 [推荐]
+
+## 二、使用 Docker Compose 安装 [推荐] [x86_64]
 
 1. 克隆仓库并编辑配置
     ```bash
@@ -38,7 +39,7 @@
     DASHSCOPE_API_KEY: "sk-0000000000"
     ```
 
-3. 运行 `docker-compose up` 命令来构建并启动 MemoryScope CLI 界面。
+3. 运行 `docker-compose run memory_scope_main` 命令来构建并启动 MemoryScope CLI 界面。 (备注：ARM架构暂不支持)
 
 
 ## 三、通过 PYPI 安装

docs/sphinx_doc/requirements.txt

@@ -11,4 +11,3 @@ sphinxcontrib-mermaid
 myst-parser
 autodoc_pydantic
 nbsphinx
-Pandoc

examples/cli/CLI_README.md

@@ -53,3 +53,12 @@ Here are the available options that can be set through either method:
 - `--enable_ranker`: A boolean indicating whether to use a dummy ranker (default is `False`).
 - `--rank_backend`: The backend used for ranking responses.
 - `--rank_model`: The model used for ranking responses.
+
+### 3. View Memory
+You can open two command line windows following the method in the second step. 
+In one command line window, you can have a conversation with the AI, while in the other, you can check the AI's long-term memory about the user.
+Use /help to open the command line help, and find the command /list_memory along with the corresponding auto-refresh instruction.
+```
+/list_memory refresh_time=5
+```
+Then you can enjoy a pleasant conversation with the AI!

examples/cli/CLI_README_ZH.md

@@ -53,3 +53,12 @@ memoryscope --language="en" \
 - `--enable_ranker`: 一个布尔值，指示是否使用排名器（默认为 False）。
 - `--rank_backend`: 用于排名回复的后端。
 - `--rank_model`: 用于排名回复的模型。
+
+### 3. 查看记忆
+按照第二步的方式可以打开两个命令行的窗口。
+其中一个命令行窗口可以和AI进行对话，另一个命令行窗口可以查看AI关于用户的长期记忆
+使用/help打开命令行帮助，找到/list_memory的命令和对应自动刷新的指令。
+```
+/list_memory refresh_time=5
+```
+接下来就可以和AI进行愉快地交流啦。

memoryscope/__init__.py

@@ -1,5 +1,5 @@
 """ Version of MemoryScope."""
-__version__ = "0.1.0.10"
+__version__ = "0.1.1.0"
 import fire
 
 from memoryscope.core.config.arguments import Arguments  # noqa: F401

setup.py

@@ -54,7 +54,7 @@ def package_files(directory):
 
 setuptools.setup(
     name="memoryscope",
-    version="0.1.0.10",
+    version="0.1.1.0",
     author=', '.join([author['name'] for author in authors]),
     author_email=', '.join([author['email'] for author in authors]),
     description="MemoryScope is a powerful and flexible long term memory system for LLM chatbots. It consists of a "