Skip to content

Commit ff8859d

Browse files
author
Timothy Dodd
committed
Update README.md for clarity and feature enhancements
Significantly revise the README.md to improve clarity and detail about the LogMk log monitoring solution. Key updates include a rewritten introduction, expanded features section with new functionalities, a restructured architecture overview, and refined quick start and development setup instructions. Additional configuration examples and updated tech stack and contributing guidelines have been added. The document is also formatted for better presentation, including the use of emojis for visual appeal.
1 parent 87f21b3 commit ff8859d

1 file changed

Lines changed: 184 additions & 63 deletions

File tree

‎README.md‎

Lines changed: 184 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,98 +1,219 @@
11

22
![logmk-128](https://github.com/user-attachments/assets/15f1c5b9-f636-4135-b9c2-1cdeb1274bf6)
33

4-
54
# LogMk
6-
LogMk is a lightweight log monitoring solution for Kubernetes environments. It monitors pod logs, saves them in a central MySQL database, and provides a web portal for viewing and filtering log activity. Designed for simplicity, LogMk may not be robust enough for large enterprise-scale environments but is perfect for smaller setups or development environments.
7-
8-
## Features
9-
- Real-Time Log Monitoring: View logs in real-time using the web portal.
10-
- Centralized Storage: All logs are stored in a MySQL database.
11-
- Filter and Search: Easily filter and search logs to find specific entries.
12-
- Simple Deployment: Deploy a .NET 9 worker agent on all Kubernetes nodes to start collecting logs.
135

6+
LogMk is a lightweight log monitoring solution for Kubernetes environments. It consists of three main components that work together to collect, store, and visualize pod logs in real-time. Designed for simplicity and ease of deployment, LogMk is perfect for smaller setups, development environments, and teams that need straightforward log monitoring without enterprise complexity.
147

15-
![image](https://github.com/user-attachments/assets/b77821c6-9caf-4db9-b481-af058a6476fe)
8+
## ✨ Features
169

10+
- **Real-time Log Streaming**: View logs in real-time with SignalR WebSocket integration
11+
- **Centralized Storage**: All logs stored in MySQL with configurable retention policies
12+
- **Advanced Filtering**: Filter by namespace, pod, container, log level, and time ranges
13+
- **Modern Web Interface**: Angular 20 SPA with dark theme and responsive design
14+
- **Kubernetes Native**: DaemonSet deployment for automatic log collection across all nodes
15+
- **JWT Authentication**: Secure access with token-based authentication and refresh tokens
16+
- **Docker Ready**: Containerized components for easy deployment
1717

18+
![LogMk Dashboard](https://github.com/user-attachments/assets/b77821c6-9caf-4db9-b481-af058a6476fe)
1819

20+
## 🏗️ Architecture
1921

20-
## Components
2122
LogMk consists of three main components:
2223

23-
1. .NET 9 Worker Agent
24+
### 1. **LogMkAgent** (DaemonSet)
25+
- **Purpose**: Log collector deployed on every Kubernetes node
26+
- **Technology**: .NET 9 background service
27+
- **Function**: Monitors `/var/log/pods` directory and streams logs to the API
28+
- **Features**: Batch processing, file watching, configurable log paths
2429

25-
Deployed on all Kubernetes nodes.
26-
Collects pod logs and sends them to the central API.
30+
### 2. **LogMkApi** (Central Server)
31+
- **Purpose**: REST API server and SignalR hub
32+
- **Technology**: .NET 9 with ServiceStack.OrmLite and SignalR
33+
- **Function**: Receives logs from agents, stores in MySQL, broadcasts to web clients
34+
- **Features**: JWT authentication, automatic schema migration, log retention management
2735

28-
2. Angular 18 Web Portal
36+
### 3. **LogMkWeb** (Web Interface)
37+
- **Purpose**: User interface for log viewing and filtering
38+
- **Technology**: Angular 20 with TypeScript and Bootstrap 5
39+
- **Function**: Real-time log visualization with advanced filtering capabilities
40+
- **Features**: Dark theme, responsive design, SignalR integration, date range picker
2941

30-
Provides a user-friendly interface for viewing and filtering logs.
31-
Utilizes SignalR for real-time log updates.
42+
## 🚀 Quick Start
3243

33-
3. .NET 9 API
44+
### Prerequisites
45+
- **Kubernetes Cluster** (v1.20+)
46+
- **MySQL Database** (v8.0+)
47+
- **.NET 9 SDK** (for building from source)
48+
- **Node.js 20+** and **Angular CLI** (for frontend development)
49+
- **Docker** (for containerized deployment)
3450

35-
- Receives log data from the worker agents.
36-
- Exposes endpoints for the web portal to access log data.
37-
- Implements SignalR for real-time communication with the web portal.
51+
### Development Setup
3852

39-
## Installation
40-
### Prerequisites
41-
- Kubernetes Cluster
42-
- .NET 9 SDK (for building)
43-
- MySQL Database
44-
- Angular CLI (for web portal development)
45-
### Setup
46-
1. Deploy Worker Agent
53+
#### 1. **Clone and Build**
54+
```bash
55+
git clone <repository-url>
56+
cd LogMk2/src
4757

48-
Build and deploy the .NET 9 worker agent on all Kubernetes nodes.
58+
# Build entire solution
59+
dotnet build LogMk.sln
60+
```
61+
62+
#### 2. **Database Setup**
63+
```bash
64+
# Create MySQL database
65+
mysql -u root -p -e "CREATE DATABASE logmk;"
66+
67+
# Update connection string in src/LogMkApi/appsettings.json
68+
{
69+
"ConnectionStrings": {
70+
"DefaultConnection": "Server=localhost;Database=logmk;Uid=root;Pwd=yourpassword;"
71+
}
72+
}
73+
```
4974

50-
``` bash
51-
# Example command to build the worker agent
52-
dotnet build src/LogMkWorkerAgent
75+
#### 3. **Run API Server**
76+
```bash
77+
# Navigate to API project
78+
cd src/LogMkApi
5379

54-
# Example command to deploy the agent
55-
kubectl apply -f deployment.yaml
80+
# Run with hot reload
81+
dotnet run
82+
# API available at: https://localhost:7001
5683
```
5784

58-
2. Set Up MySQL Database
85+
#### 4. **Run Web Application**
86+
```bash
87+
# Navigate to web project
88+
cd src/LogMkWeb
5989

60-
Create a MySQL database to store the logs. Update the connection string in the API configuration.
90+
# Install dependencies
91+
npm install
6192

62-
3. Deploy API
93+
# Start development server
94+
npm start
95+
# Web app available at: http://localhost:6200
96+
```
6397

64-
Build and deploy the .NET 9 API.
98+
#### 5. **Deploy Agent (Kubernetes)**
99+
```bash
100+
# Build and deploy agent
101+
docker build -f LogMkAgent/Dockerfile -t logmk-agent .
102+
kubectl apply -f k8s/agent-daemonset.yaml
103+
```
65104

66-
``` bash
105+
### Production Deployment
67106

68-
dotnet build src/LogMkApi
69-
dotnet publish src/LogMkApi -o /publish
107+
#### Docker Build Commands
108+
```bash
109+
# From src/ directory
110+
docker build -f LogMkApi/Dockerfile -t logmk-api .
111+
docker build -f LogMkAgent/Dockerfile -t logmk-agent .
70112
```
71113

72-
4. Set Up Web Portal
114+
#### Kubernetes Deployment
115+
```bash
116+
# Deploy all components
117+
kubectl apply -f k8s/
118+
```
73119

74-
Navigate to the web portal directory and install dependencies.
120+
## 🎯 Usage
121+
122+
1. **Access the Web Interface**: Navigate to `http://localhost:6200` (development) or your deployed URL
123+
2. **Authentication**: Log in with your configured credentials
124+
3. **Real-time Monitoring**: Logs stream automatically via SignalR connection
125+
4. **Filtering**: Use the sidebar filters to narrow down logs by:
126+
- Namespace
127+
- Pod name
128+
- Container
129+
- Log level
130+
- Time range
131+
5. **Search**: Use the search bar for full-text log content searching
132+
133+
## 📋 Configuration
134+
135+
### API Configuration (`src/LogMkApi/appsettings.json`)
136+
```json
137+
{
138+
"ConnectionStrings": {
139+
"DefaultConnection": "Server=localhost;Database=logmk;Uid=root;Pwd=password;"
140+
},
141+
"JwtSettings": {
142+
"SecretKey": "your-secret-key",
143+
"ExpiryMinutes": 60,
144+
"RefreshTokenExpiryDays": 7
145+
},
146+
"LogRetention": {
147+
"RetentionDays": 30,
148+
"CleanupIntervalHours": 24
149+
}
150+
}
151+
```
75152

76-
``` bash
77-
cd src/LogMkWebPortal
78-
npm install
153+
### Agent Configuration (`src/LogMkAgent/appsettings.json`)
154+
```json
155+
{
156+
"ApiSettings": {
157+
"BaseUrl": "https://logmk-api:7001",
158+
"BatchSize": 100,
159+
"BatchIntervalSeconds": 5
160+
},
161+
"LogPaths": [
162+
"/var/log/pods"
163+
]
164+
}
165+
```
166+
167+
## 🛠️ Development
168+
169+
### Frontend Development
170+
```bash
171+
cd src/LogMkWeb
172+
173+
# Generate new component
174+
ng generate component _components/my-component
175+
176+
# Run tests
177+
npm test
178+
179+
# Build for production
180+
npm run prod
79181
```
80-
Build and serve the web portal.
81182

82-
``` bash
83-
ng build --prod
84-
ng serve
183+
### Backend Development
184+
```bash
185+
# Build solution
186+
dotnet build src/LogMk.sln
187+
188+
# Run specific project
189+
dotnet run --project src/LogMkApi
190+
dotnet run --project src/LogMkAgent
85191
```
86-
## Usage
87-
Access the web portal via your browser.
88-
Use the filter and search functionalities to find specific logs.
89-
Monitor logs in real-time through the SignalR integration.
90-
The Api also hosts the web project
91-
## Contributing
92-
Contributions are welcome! Please fork the repository and submit a pull request.
93-
94-
## License
95-
MIT License
96-
97-
## Acknowledgements
98-
The .NET and Angular communities for their excellent frameworks.
192+
193+
## 📝 Tech Stack
194+
195+
- **Backend**: .NET 9, C#, ServiceStack.OrmLite, SignalR
196+
- **Frontend**: Angular 20, TypeScript, Bootstrap 5, ng2-charts
197+
- **Database**: MySQL 8.0+
198+
- **Authentication**: JWT with refresh tokens
199+
- **Real-time**: SignalR WebSockets
200+
- **Infrastructure**: Docker, Kubernetes, GitHub Actions
201+
202+
## 🤝 Contributing
203+
204+
1. Fork the repository
205+
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
206+
3. Commit your changes (`git commit -m 'Add amazing feature'`)
207+
4. Push to the branch (`git push origin feature/amazing-feature`)
208+
5. Open a Pull Request
209+
210+
## 📄 License
211+
212+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
213+
214+
## 🙏 Acknowledgments
215+
216+
- The .NET and Angular communities for their excellent frameworks
217+
- SignalR team for real-time communication capabilities
218+
- Bootstrap team for the responsive UI framework
219+
- Kubernetes community for container orchestration

0 commit comments

Comments
 (0)