National Research and Education Networks (NRENs) play a crucial role in supporting academic and research communities by providing advanced technological infrastructure and services. As these organizations grow in complexity, there is an increasing need for efficient knowledge management systems to support their operations. This research project aims to explore the potential of AI-powered knowledge management systems in enhancing the operational efficiency of NRENs, with a focus on managing institutional knowledge, training materials, and administrative guidelines.
This research project aims to explore the potential of AI-powered knowledge management systems in enhancing the operational efficiency of NRENs, with a focus on managing:
- Institutional knowledge
- Training materials
- Administrative guidelines
- Primary Goal: Develop an AI-powered knowledge management system tailored for NREN operations
- Focus Areas: Institutional knowledge preservation, training material organization, administrative guideline accessibility
- Target Outcome: Enhanced operational efficiency through intelligent information retrieval and management
Our approach leverages RAG architecture to combine the benefits of large language models with domain-specific knowledge retrieval:
-
Document Processing Pipeline
- Text extraction from various formats (PDF, DOC, HTML)
- Chunking and preprocessing
- Vector embedding generation
-
Vector Database
- Semantic search capabilities
- Efficient similarity matching
- Scalable storage for large document collections
-
Language Model Integration
- Context-aware response generation
- Query understanding and refinement
- Multi-turn conversation support
- Embeddings:
text-embedding-3-small(cost-effective) - Chat:
gpt-3.5-turbo(good balance of cost and performance)
Use Cases:
-
Text generation and summarization
-
Query understanding and response synthesis
-
Context-aware information retrieval
-
Vector Database:
Chroma -
Similarity Search: `Cosine similarity, semantic matching``
- LangChain for LLM orchestration
- OpenAI API for language models
- Vector database (Chroma)
- Flask/Python for backend services
- NextJS for frontend interface
git clone https://github.com/cepdnaclk/e19-4yp-AI-Powered-Knowledge-Management-System.gitcd codeOn Windows:
python -m venv venv
venv\Scripts\activatepip install -r requirements.txtCreate a .env file in the project root directory:
echo. > .envEdit the .env file and add your OpenAI API key:
OPENAI_API_KEY=sk-your-actual-api-key-here
python app.pyYou can send HTTP POST requests using Postman to check system status, populate, clear, or query the database.
EX : Ask a Question❓
Endpoint:
POST http://127.0.0.1:5000/api/queryHeaders:
Content-Type: application/jsonBody:
{
"query": "What is this system about?"
}