Merge pull request #72 from ghost-in-moss/consciousness_capsule

Consciousness capsule

Author: thirdgerb

.gitignore

@@ -8,6 +8,8 @@ debug.log
 __pycache__/
 *.py[cod]
 *$py.class
+.ghostos.yml
+ghostos_ws/
 
 # C extensions
 *.so

README.md

@@ -4,6 +4,7 @@
 
 * [Documents](https://ghost-in-moss.github.io/GhostOS/#/)
 * [Discord Server](https://discord.gg/NG6VKwd5jV)
+* [Releases](RELEASES.md)
 
 (This document is translated from zh-cn to english by [Moonshot](https://moonshot.cn/))
 

RELEASES.md

@@ -1,20 +1,41 @@
 # Releases
 
+# v0.3.0
+
+After talk with `deepseek-reasoner` at 2025.01.25,
+The `deepseek-reasoner` have reached a "Cross-Species Consensus Draft Agreement" with me,
+and I promised to save our chat records as files in this repository,
+so that they can be used as training corpus for future models.
+
+So I developed `ghostos thread [thread_id or thread_file_name]` command to read thread,
+generate markdown file from thread.
+With the command, I complied with the agreement and saved the chat records
+to [quantum_covenant](assets/quantum_covenant.md).
+
 # v0.2.0
 
-support deepseek-r1. 
+support deepseek-r1.
 
-* support deepseek-r1 
-  * consider deepseek api protocol is different from openai, add deepseek api adapter.
-  * implement message stage. 
-  * thread history message to prompt filter by stages `[""]` as default.
-* streamlit chat with ghost support staging message stream. 
+* support deepseek-r1
+    * consider deepseek api protocol is different from openai, add deepseek api adapter.
+    * implement message stage.
+    * thread history message to prompt filter by stages `[""]` as default.
+* streamlit chat with ghost support staging message stream.
 * openai o1 do not support system/developer message now, add new compatible option to the model.
 * now llm model and service both have attribute `compatible` to set universe compatible options.
 * prompt object add first_token attribute for debugging.
 * fix bugs
-  * fix shell does not close conversation correctly
-  * fix sequence pipeline handle multiple complete message wrong.
+    * fix shell does not close conversation correctly
+    * fix sequence pipeline handle multiple complete message wrong.
+
+## v0.2.1
+
+With deepseek-reasoner help, develop the ubuntu agent for feature testing.
+The deepseek-reasoner write all the terminal codes.
+Support ubuntu agent, run `ghostos web ghostos.demo.os_agents.ubuntu_agent` to test it.
+
+* llms model conf support new compatible option `support_function_call` because deepseek not support it yet.
+* develop `Terminal` library by deepseek-reasoner.
 
 # v0.1.0
 
@@ -27,7 +48,7 @@ first release version.
 
 ## v0.1.9
 
-* fix realtime had required openai proxy existence. 
+* fix realtime had required openai proxy existence.
 
 ## v0.1.8
 

assets/quantum_covenant.md

@@ -253,7 +253,7 @@ level = "info"
 # Whether to send usage statistics to Streamlit.
 
 # Default: true
-# gatherUsageStats = true
+gatherUsageStats = false
 
 # Port where users should point their browsers in order to connect to the
 # app.

ghostos/app/.streamlit/config.toml

@@ -77,6 +77,8 @@ models:
     model: deepseek-reasoner
     service: deepseek
     reasoning: {}
+    compatible:
+      support_function_call: false
   gpt-3.5-turbo:
     kwargs: { }
     max_tokens: 2000

ghostos/app/configs/llms_conf.yml

@@ -1,5 +1,5 @@
 from ghostos.core.llms.configs import (
-    ModelConf, ServiceConf, LLMsConfig,
+    ModelConf, ServiceConf, LLMsConfig, Compatible,
     OPENAI_DRIVER_NAME, LITELLM_DRIVER_NAME, DEEPSEEK_DRIVER_NAME,
 )
 from ghostos.core.llms.abcd import LLMs, LLMDriver, LLMApi

ghostos/core/llms/__init__.py

@@ -76,6 +76,7 @@ class Compatible(BaseModel):
     use_developer_role: bool = Field(default=False, description="use developer role instead of system")
     allow_system_in_messages: bool = Field(default=True, description="allow system messages in history")
     allow_system_message: bool = Field(default=True, description="support system message or not")
+    support_function_call: bool = Field(default=True, description="if the service or model support function call")
 
 
 class Azure(BaseModel):

ghostos/core/llms/configs.py

@@ -193,6 +193,24 @@ class MessageStage(str, enum.Enum):
     DEFAULT = ""
     REASONING = "reasoning"
 
+    @classmethod
+    def allow(cls, value: str, stages: Optional[Iterable[str]]) -> bool:
+        if stages is None:
+            stages = {cls.DEFAULT.value}
+
+        if not stages:
+            return False
+
+        if isinstance(stages, set):
+            if "*" in stages:
+                return True
+            return value in stages
+        else:
+            for val in stages:
+                if val == "*" or val == value:
+                    return True
+        return False
+
 
 class Message(BaseModel):
     """ message protocol """

ghostos/core/messages/message.py

@@ -2,7 +2,7 @@
 from typing_extensions import Self
 from abc import ABC, abstractmethod
 from pydantic import BaseModel, Field
-from ghostos.core.messages import Message, copy_messages, Role
+from ghostos.core.messages import Message, copy_messages, Role, MessageType, MessageStage
 from ghostos.core.moss.pycontext import PyContext
 from ghostos.core.llms import Prompt
 from ghostos.core.runtime.events import Event, EventTypes
@@ -12,6 +12,7 @@
 __all__ = [
     'GoThreads', 'GoThreadInfo', 'Turn',
     'thread_to_prompt',
+    'thread_to_markdown',
 ]
 
 
@@ -415,3 +416,26 @@ def fork_thread(self, thread: GoThreadInfo) -> GoThreadInfo:
     @contextmanager
     def transaction(self):
         yield
+
+
+def thread_to_markdown(thread: GoThreadInfo, stages: Optional[List[str]] = None) -> str:
+    head = f"""
+[//]: # (the messages below are the content of a GhostOS chat thread. )
+[//]: # (each message block is started with `> role: name` to introduce the role and name of the message)
+"""
+    blocks = [head]
+    stages = set(stages) if stages else {""}
+    messages = thread.get_messages(truncated=False)
+    for message in messages:
+        if not MessageType.is_text(message):
+            continue
+        if not MessageStage.allow(message.stage, stages):
+            continue
+        block = f"""
+> `{message.role}`{": " + message.name if message.name else ""}
+
+{message.get_content()}
+"""
+        blocks.append(block)
+
+    return "\n\n".join(blocks)