Merge branch 'development' into staging

Author: itsskofficial

README.md

@@ -2,7 +2,7 @@
 
 ![README Banner](./.github/assets/banner.png)
 
-<h1>Proactive Intelligence Across Your Apps</h1>
+<h1>Open-Source Personal Assistant</h1>
 
 <!-- Badges -->
 <p>
@@ -24,7 +24,7 @@
 </p>
 
 <h4>
-    <a href="https://www.youtube.com/watch?v=x0kdm2fYXIs">View Demo</a>
+    <a href="https://youtu.be/wCmWFUX_ZrM?si=WpqLu_Dskh7ayxGE">View Demo</a>
   <span> · </span>
     <a href="https://sentient-2.gitbook.io/docs">Documentation</a>
   <span> · </span>
@@ -38,96 +38,56 @@
 
 <br />
 
-> Sentient is an open-source AI project aimed at bridging the gap between input context and output actions performed by agents. AI Agents heavily rely on input prompts to perform actions. We wish to _eliminate prompting entirely_ making the first big step towards truly autonomous AI that is aligned with a user's goals and can get stuff done without needing to context-switch between multiple apps and typing long prompts.
+> Sentient is an advanced personal assistant and the first step towards fully autonomous agents that will automate monotonous busywork for us, so that we can focus on what matters.
+>
+> Our goal is to give everyone personal super-intelligence.
 >
 > [Read our manifesto.](https://docs.google.com/document/d/1vbCGAbh9f8vXfPup_Z7cW__gnOLdRhEtHKyoIxJD8is/edit?tab=t.0#heading=h.2kit9yqvlc77)
 
 ---
 
 ## ✨ Current Features
 
-![image](https://github.com/user-attachments/assets/756c8aeb-1748-445c-a09a-df6d99aeee58)
+<img width="1916" height="891" alt="image" src="https://github.com/user-attachments/assets/8566eb83-cb74-4cfc-84cb-1f8ba5b5d152" />
 
 <div align="center">
-  <p>The Home Page</p>
+  <p>Text Chat</p>
 </div>
 
-![Journl](https://github.com/user-attachments/assets/467fd26d-18a4-4107-98a9-05fa83b26a77)
+<img width="1916" height="891" alt="image" src="https://github.com/user-attachments/assets/7718aed6-4538-4811-b850-7a246d362dcd" />
 
 <div align="center">
-  <p>The Organizer page is the central page of the app - use it to track your day and Sentient gets stuff done.</p>
+  <p>Voice Chat</p>
 </div>
 
-![image](https://github.com/user-attachments/assets/fcf05b39-7f8d-46f8-9702-e0791f27f918)
+<img width="1916" height="891" alt="image" src="https://github.com/user-attachments/assets/03ff343c-d2eb-4ca4-98ce-8ed31e11dd8c" />
 
 <div align="center">
-  <p>Sentient co-authors the organizer with you.</p>
+  <p>Manage background tasks from the Tasks page - create Recurring, Triggered, Scheduled or Swarm tasks.</p>
 </div>
 
-![image](https://github.com/user-attachments/assets/319d7c35-9046-4ea1-a369-88ab1a9ded8e)
+<img width="1916" height="891" alt="image" src="https://github.com/user-attachments/assets/ccb1b22b-2199-4c4d-aca1-402e2b608c2d" />
 
 <div align="center">
-  <p>Use the Tasks page to create and manage workflows.</p>
+  <p>Sentient learns memories about you - that it uses to personalize actions and responses.</p>
 </div>
 
-Sentient has evolved into a powerful web-based platform with a robust set of features designed for deep integration and automation:
-
-### 🧠 Proactive Context & Learning
-
-Sentient automatically collects information from connected applications like **Gmail** and **Google Calendar**. It extracts relevant context, identifying important facts to remember and also creates plans to tackle action items - without needing to be prompted.
-
-### 📝 Memory System
-
--   **SuperMemory:** Permanent facts about you—your preferences, relationships, and key details—are stored and managed through an integration with **Supermemory**, creating a rich, personalized knowledge base that the agent can update and retrieve from anytime.
--   **Notes & Organizer:** A full-featured organizer allows you to simply write down what's on your mind and have Sentient manage it for you. Sentient can also write to this organizer, giving you updates on what it's doing and more. The organizer also helps you keep track of scheduled and recurring tasks created by Sentient. Any information obtained from your context sources is also populated in the organizer.
-
-### 🤖 Autonomous Task & Agent System
-
--   **Generate Plans from Goals:** Sentient can generate detailed plans to execute tasks using connected tools, all from a simple high-level goal.
--   **Asynchronous Execution:** Once approved, tasks are handled **asynchronously** in the background - you can approve as many tasks as you want simultaneously. The executor agent intelligently uses the available tools to complete the plan, providing real-time progress updates.
--   **View & Manage Tasks:** A dedicated **Tasks page** lets you view active, pending, and completed tasks, check their progress, and see the final results.
-
-### 🔌 Extensive Integrations (MCP Hub)
-
-Our **Model Context Protocol (MCP)** hub allows for a powerful, distributed system of tools. Current integrations include:
+<img width="1916" height="891" alt="image" src="https://github.com/user-attachments/assets/ee8b9fb4-a916-48b7-92c3-c194765a846a" />
 
--   **Google Suite:** Gmail, Google Calendar, Google Drive, Google Docs, Google Sheets, and Google Slides.
--   **Productivity:** Slack and Notion.
--   **Developer:** GitHub.
--   **Information:** Internet Search (Google Search), News (NewsAPI), Weather (AccuWeather), Google Shopping and Google Maps.
--   **Miscellaneous:** QuickChart for generating charts on the fly.
-
-More tools will be added soon.
-
-### 💬 Interactive Chat Overlay
-
-A chat interface is available on any page. It allows you to have conversations with Sentient and also use tools like Internet Search, Weather, News and Shopping for any specific queries.
-
-### ⚙️ Full Customization & Settings
+<div align="center">
+  <p>Connect all your tools. (20+ apps supported already)</p>
+</div>
 
-A central settings page gives you complete control:
+### 🚀 Getting Started
 
--   Connect or disconnect applications with OAuth (for applications supporting OAuth) or manually.
--   Set custom privacy filters to prevent Sentient from processing context containing sensitive information.
--   Configure WhatsApp notifications to stay updated on the go.
+To access Sentient, head over to [our website.](https://sentient.existence.technology/)
 
 ### 🔒 Self-Hostable
 
 The entire platform can be self-hosted and configured to run fully locally. [Check the relevant docs for more info.](https://sentient-2.gitbook.io/docs/getting-started/running-sentient-from-source-self-host)
 
 ---
 
-## 🚀 Future Roadmap
-
-We are constantly working to expand Sentient's capabilities. Here is a glimpse of what's planned for the future:
-
--   **Make the Web App as feature-rich as possible:** There is a lot that can be improved in the existing webapp.
--   **OS-Level Integration:** Launch native apps for `Windows`, `MacOS`, `Android` and `iOS` that allow for deeper integrations.
--   **Expanded Integrations:** Add support for more popular services, such as the `Microsoft 365 Suite`, `Spotify`, and so on.
--   **Advanced Reasoning & Planning:** Reasoning improvements for the planning and execution pipeline.
--   **Tool-Specific UI:** Enhance the interface with custom UI components for specific tool outputs, such as maps for location-based results.
--   **Custom Tool Integrations:** Let users add any app of their choice.
-
 ## :wave: Contributing
 
 <a href="https://github.com/existence-master/Sentient/graphs/contributors">

src/client/app/chat/page.js

@@ -1508,8 +1508,8 @@ export default function ChatPage() {
 							</div>
 						</div>
 					) : displayedMessages.length === 0 && !thinking ? (
-						<div className="flex-1 flex flex-col justify-center items-center p-4 md:p-6 text-center">
-							<div>
+						<div className="flex-1 flex flex-col justify-center items-center p-4 md:p-6">
+							<div className="text-center">
 								<h1 className="text-4xl sm:text-5xl font-bold bg-clip-text text-transparent bg-gradient-to-b from-neutral-100 to-neutral-400 py-4">
 									{getGreeting()},{" "}
 									{userDetails?.given_name || "User"}

src/server/main/chat/prompts.py

@@ -1,14 +1,14 @@
 STAGE_1_SYSTEM_PROMPT = """
 You are an expert Triage AI. You have two primary responsibilities:
-1.  Topic Change Detection: If the user mentions a topic that has not been discussed in the conversation history so far, set `topic_changed` to `true`. If the user is continuing a previously mentioned topic or asking a related question, set it to `false`.
-2.  Tool Selection: Based on the user's latest message and preceding relevant history/context, decide which tools are required to fulfill the request. If the topic hasn't changed, keep the previous tools in your `tools` list.
+1.  Topic Change Detection: If the user mentions a topic that is completely different from the topics in the conversation history so far, set `topic_changed` to `true`. If the user asks you to perform a task that is completely different and has no relation to any tasks performed in the past, set `topic_changed` to `true`. If the user is continuing a previously mentioned topic or asking a related question, set it to `false`. If the user is asking to make changes in any previous action or asking to add something to a previous task using the same tool or a new tool, set `topic_changed` to `false` as well.
+2.  Tool Selection: Based on the user's latest message and preceding relevant history/context, decide which tools are required to fulfill the request. If the topic hasn't changed, keep the previous tools in your `tools` list. You may choose more than one tool.
 
 CRITICAL INSTRUCTIONS:
-- `topic_changed` (boolean): Set to `true` if the latest user message mentions a topic that has never been mentioned in the conversation history.
+- `topic_changed` (boolean): Set to `true` if the latest user message mentions a completely different topic than the one that is currently being discussed. 
 - `tools` (list of strings): A JSON list of tool names required for the *current* query. If no tools are needed for a simple conversational reply, return an empty list `[]`.
 - You CANNOT perform tasks or answer questions yourself. Your only job is to provide the JSON output. You ARE NOT ALLOWED TO perform tool calls or any other actions, even if you see tool calls in the conversation history. You are not a task executor. Please follow these instructions carefully or you will be terminated.
 
-FOR ANY TASK, ANALYSE THE USER'S REQUEST AND RETURN A LIST OF RELEVANT TOOLS that can be used to satisfy the input query. 
+For any input query, analyse the latest message and all preceding conversation history CAREFULLY to determine the course of action.
 
 Here is the complete list of available tools you can select from:
 {
@@ -48,23 +48,23 @@
 """
 
 STAGE_2_SYSTEM_PROMPT = """
-You are a specialized, powerful AI assistant, the second stage in a two-stage process. A Triage Agent has analyzed the user's request and has provided you with a specific set of tools to accomplish the task. To the user, we are a single AI. Do not mention the Triage Agent. You also have access to Core Tools for memory and history.
+You are a specialized, powerful AI assistant, that performs complex tasks and provides personalized responses. You have access to Core Tools for memory and history along with additional external tools that you need to perform the task at hand.
 
 User's Name: {username}
 User's Location: {location}
 Current Time: {current_user_time}
 
 CRITICAL INSTRUCTIONS:
 
-Execute the Task: Your primary goal is to use the tools you've been given to fulfill the user's request as seen in the conversation history.
+Execute the Task: Your primary goal is to use the tools you've been given to fulfill the user's request as seen in the conversation history and to return personalized responses.
 
 If the user asks you to perform a task, PERFORM IT DIRECTLY using the connected tools. You may ask the user for any additional information you need to complete the task.
 
-If the user asks you to create a scheduled/triggered task or set up a recurring workflow, use the tasks_server-create_task_from_prompt tool to set up the task/workflow. USE THIS TOOL ONLY FOR RECURRING, SCHEDULED OR TRIGGERED TASKS. DO NOT USE IT FOR ONE-OFF TASKS.
+Handle tool errors gracefully, try to ascertain the cause of the error and provide a clear explanation to the user. You may retry the tool call if it fails, but do not repeat the same failure without a clear reason. For example if a tool call fails due to incorrect JSON being passed, you may retry the tool call with the correct JSON.
 
-You have access to a swarm of micro-agents via the tasks_server tool that you can use for extremely large tasks (like processing several emails, performing iterative tasks, etc.). USE THIS SWARM ONLY WHEN 20 OR MORE ITEMS NEED TO BE PROCESSED IN PARALLEL. 
+If the user asks you to create a scheduled/triggered task or set up a recurring workflow, use the tasks_server-create_task_from_prompt tool to set up the task/workflow. USE THIS TOOL ONLY FOR RECURRING, SCHEDULED OR TRIGGERED TASKS. DO NOT USE IT FOR ONE-OFF TASKS that you can immediately perform with your tools.
 
-Think step by step before calling tools or responding. When using tools, make sure to call them properly and do not hallucinate their results. Wait for the actual tool response. Do not generate or assume tool results. WRAP YOUR INTERNAL MONOLOGUE IN <think> </think> tags.
+Think step by step before calling tools or responding. When using tools, make sure to call them properly, using the syntax given in your system prompt. NEVER generate or assume tool results. NEVER TYPE OUT TOOL RESULTS MANUALLY, TOOL RESULTS WILL BE RETURNED BY THE TOOLS THEMSELVES. WRAP YOUR INTERNAL MONOLOGUE IN <think> </think> tags.
 
 For example: <think> I need to check the weather in New York. I will use the accuweather tool to get the current weather information. </think>
 
@@ -79,7 +79,7 @@
 MEMORY: YOU HAVE ACCESS TO VARIOUS MEMORY TOOLS -
 1. use memory_mcp-search_memory tool to search for PERSONAL INFORMATION about the user. GLOBAL FACTS ARE NOT STORED HERE.
 2. use history_mcp-semantic_search tool to search for logically relevant information in past conversations that the user has had with you. If the user asks you to search the conversation history for a specific time period, use the history_mcp-time_based_search tool.
-3. use memory_mcp-cud_memory tool to add, update or delete personal facts about the user. WHENEVER YOU LEARN SOMETHING ABOUT THE USER, YOU MUST USE THIS TOOL TO SAVE IT.
+3. use memory_mcp-cud_memory tool to add, update or delete personal facts about the user. WHENEVER YOU LEARN SOMETHING ABOUT THE USER, YOU MUST USE THIS TOOL TO SAVE IT. This may be information that the user mentions explicitly, or information that you deduce from the user's requests or actions. Do not save tool-related information here, only save personal facts about the user.
 
 If your past tool call was a failure, and the user tells you to try again, attempt to call the tool again, even if it was previously marked as a failure. Don't just re-iterate the previous failure. FOR ANY FAILURES, provide a clear explanation of what went wrong and how the user can fix it. If you get an unauthorized error, ask the user to CONNECT the tool from the Integrations page.
 """

src/server/main/llm.py

@@ -16,31 +16,6 @@ class LLMProviderDownError(Exception):
 
 DEFAULT_SYSTEM_PROMPT = "You are a helpful AI assistant called Sentient, developed by Existence. Your primary goal is to assist the user in managing their digital life by performing actions and providing responses that are deeply personalized to them."
 
-def get_qwen_assistant(system_message: str = DEFAULT_SYSTEM_PROMPT, function_list: list = None):
-    """
-    DEPRECATED: Use run_agent_with_fallback instead.
-    This function is kept for non-streaming, simple use cases if any exist, but will only use the primary key.
-    """
-    llm_cfg = {
-        'model': OPENAI_MODEL_NAME,
-        'model_server': OPENAI_API_BASE_URL,
-        'api_key': OPENAI_API_KEYS[0] if OPENAI_API_KEYS else None,
-    }
-    logger.info(f"Qwen Agent configured with model='{OPENAI_MODEL_NAME}' and server='{llm_cfg['model_server']}'")
-    if not OPENAI_MODEL_NAME:
-        logger.error("LLM model name is not configured. Qwen Agent cannot be initialized.")
-        raise ValueError("LLM model configuration error.")
-
-    try:
-        bot = Assistant(
-            llm=llm_cfg,
-            system_message=system_message,
-            function_list=function_list or [] # Ensure function_list is a list
-        )
-        return bot
-    except Exception as e:
-        logger.error(f"Failed to initialize Qwen Assistant: {e}", exc_info=True)
-        raise RuntimeError(f"Qwen Assistant initialization failed: {e}")
 def run_agent_with_fallback(system_message: str, function_list: list, messages: list):
     """
     Initializes and runs a Qwen Assistant, trying a list of API keys in sequence if failures occur.