fix: Optimizing AI agent creation and deleting garbage threads

[Pavan-Microsoft]

Purpose

This pull request introduces significant enhancements to the backend architecture, focusing on improving the management of Azure AI agents, optimizing caching, and enhancing logging practices. Key updates include the creation of a singleton factory for Azure AI agents, restructuring the FastAPI application lifecycle, and implementing a custom cache for managing agent threads.

Backend Architecture Enhancements:

• Singleton Azure AI Agent Factory: Introduced AgentFactory in src/api/agents/agent_factory.py to manage a singleton instance of AzureAIAgent. This ensures centralized agent creation and deletion, with support for plugins and Azure AI configuration.

• FastAPI Lifespan Management: Refactored src/api/app.py to integrate AgentFactory into the FastAPI application lifecycle. The lifespan context manager initializes the agent on startup and cleans up resources during shutdown. [1] [2]

Caching Improvements:

• Custom TTLCache for Agent Threads: Added ExpCache in src/api/services/chat_service.py to handle thread eviction and expiration. This ensures Azure AI agent threads are properly deleted when no longer needed, reducing resource leakage.

Logging and Error Handling:

• Enhanced Logging: Improved logging consistency across multiple files (src/api/services/chat_service.py, src/api/plugins/chat_with_data_plugin.py) by replacing f-strings with parameterized logging for better performance and security. [1] [2] [3] [4]

Minor Updates:

• Constructor Update for ChatService: Modified ChatService to accept a Request object, enabling access to the application state for agent management. [1] [2]
• Agent Instructions Update: Adjusted system instructions for generating T-SQL queries in src/api/plugins/chat_with_data_plugin.py.

These changes collectively improve resource management, scalability, and maintainability of the backend system.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

A single shared agent will be used for handling requests.
Threads will be deleted automatically from the cache 1 hour after conversation expiration.

[github-actions]

🎉 This PR is included in version 3.3.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀