Skip to content

Commit a057f98

Browse files
omrozowicz-splunksrv-rr-github-tokenajasnosz
authored
feat: release 1.2.0 (#95)
* fix: add celery integration with redis replication * fix: update Dockerfile * fix: update env variables * fix: update env variables * fix: update env variables * fix: update celery_start.sh script * fix: redis version * fix: get rid off loggers * chore(release): 1.1.2-beta.1 ## [1.1.2-beta.1](v1.1.1...v1.1.2-beta.1) (2025-11-18) ### Bug Fixes * add celery integration with redis replication ([3feb790](3feb790)) * get rid off loggers ([bc025f3](bc025f3)) * redis version ([37445e6](37445e6)) * update celery_start.sh script ([c7f5e1a](c7f5e1a)) * update Dockerfile ([9a6622c](9a6622c)) * update env variables ([7c6b11c](7c6b11c)) * update env variables ([212f8a1](212f8a1)) * update env variables ([ea659f6](ea659f6)) * work with redis ha ([be7fabc](be7fabc)) * fix: add mongodb ha env variables and remove waits * fix: add wait for mongodb * fix: add wait for mongodb * fix: refactor construct script * chore(release): 1.1.2-beta.2 ## [1.1.2-beta.2](v1.1.2-beta.1...v1.1.2-beta.2) (2025-12-09) ### Bug Fixes * add mongodb ha env variables and remove waits ([c8f1ac3](c8f1ac3)) * add wait for mongodb ([e66bf29](e66bf29)) * add wait for mongodb ([7865bfd](7865bfd)) * refactor construct script ([8a009ea](8a009ea)) * update script with mongodb connection string ([1f8b015](1f8b015)) * fix: add fallback * fix: if condition * fix: add loggin level * fix: fix logging issues * fix: update changelog * chore(release): 1.1.2-beta.3 ## [1.1.2-beta.3](v1.1.2-beta.2...v1.1.2-beta.3) (2025-12-22) ### Bug Fixes * add fallback ([db6aa3a](db6aa3a)) * add fallback for connection strings ([1011453](1011453)) * add loggin level ([31eba9c](31eba9c)) * fix logging issues ([11da9c9](11da9c9)) * if condition ([4c479ff](4c479ff)) * update changelog ([4e6296a](4e6296a)) * feat: add authorisation (#100) * feat: add authorisation * feat: remove hash_password.ppy * test: upgrade Flask version * test: upgrade python to 3.10 * test: update test * chore(release): 1.2.0-beta.1 # [1.2.0-beta.1](v1.1.2-beta.3...v1.2.0-beta.1) (2026-04-29) ### Features * add authorisation ([#100](#100)) ([0aa5181](0aa5181)) * fix: dependency update (#101) * fix: dependency update * fix: upgrade node * chore: update ci * chore: test fix * chore: bump pytest * fix: change dockerfile permissions * chore(release): 1.2.0-beta.2 # [1.2.0-beta.2](v1.2.0-beta.1...v1.2.0-beta.2) (2026-04-29) ### Bug Fixes * dependency update ([#101](#101)) ([6d0c579](6d0c579)) --------- Co-authored-by: srv-rr-github-token <94607705+srv-rr-github-token@users.noreply.github.com> Co-authored-by: Agnieszka Jasnosz <139114006+ajasnosz@users.noreply.github.com>
1 parent d35a82a commit a057f98

46 files changed

Lines changed: 30979 additions & 5591 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci-build.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ jobs:
7878
cache-to: type=inline
7979
- uses: actions/setup-node@v4.0.2
8080
with:
81-
node-version: "20.12"
81+
node-version: "22"
8282

8383
build-backend:
8484
name: build-backend
@@ -128,4 +128,4 @@ jobs:
128128
cache-to: type=inline
129129
- uses: actions/setup-node@v4.0.2
130130
with:
131-
node-version: "20.12"
131+
node-version: "22"

.github/workflows/ci-main.yaml

Lines changed: 15 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -37,13 +37,15 @@ jobs:
3737
strategy:
3838
matrix:
3939
python-version:
40-
- 3.9
40+
- "3.12"
4141
steps:
4242
- uses: actions/checkout@v4
4343
- name: Setup python
44-
uses: actions/setup-python@v4
44+
uses: actions/setup-python@v5
4545
with:
4646
python-version: ${{ matrix.python-version }}
47+
cache: 'pip'
48+
cache-dependency-path: backend/requirements.txt
4749
- name: Install packages
4850
working-directory: ./backend
4951
run: pip install -r ./requirements.txt
@@ -56,25 +58,21 @@ jobs:
5658
strategy:
5759
matrix:
5860
node-version:
59-
- 20.12
61+
- 22
6062
steps:
6163
- uses: actions/checkout@v4
6264
- name: Set Node.js ${{ matrix.node-version }}
6365
uses: actions/setup-node@v4.0.2
6466
with:
6567
node-version: ${{ matrix.node-version }}
66-
- name: Run install
67-
uses: borales/actions-yarn@v4
68-
with:
69-
cmd: install
70-
dir: 'frontend'
68+
cache: 'yarn'
69+
cache-dependency-path: frontend/yarn.lock
70+
- name: Install dependencies
71+
working-directory: ./frontend
72+
run: yarn install --frozen-lockfile
7173
- name: Build
72-
uses: borales/actions-yarn@v4
73-
with:
74-
cmd: build
75-
dir: 'frontend'
76-
- name: Run test in sub-folder
77-
uses: borales/actions-yarn@v4
78-
with:
79-
cmd: test
80-
dir: 'frontend'
74+
working-directory: ./frontend
75+
run: yarn build
76+
- name: Run tests
77+
working-directory: ./frontend
78+
run: yarn test

.github/workflows/ci-release.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ jobs:
7474
cache-to: type=inline
7575
- uses: actions/setup-node@v4.0.2
7676
with:
77-
node-version: "20.12"
77+
node-version: "22"
7878

7979
build-backend:
8080
name: build-backend
@@ -125,7 +125,7 @@ jobs:
125125
cache-to: type=inline
126126
- uses: actions/setup-node@v4.0.2
127127
with:
128-
node-version: "20.12"
128+
node-version: "22"
129129
release:
130130
name: Release
131131
needs: [build-frontend, build-backend]
@@ -147,7 +147,7 @@ jobs:
147147
owner: ${{ github.repository_owner }}
148148
- uses: actions/setup-node@v4.0.2
149149
with:
150-
node-version: "20.12"
150+
node-version: "22"
151151
- name: Semantic Release
152152
id: version
153153
uses: splunk/semantic-release-action@v1.3.4

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# Changelog
22

33
### Changed
4+
- add fallback for MONGODB and REDIS connection strings, fix logging
5+
- create MONGODB connection string from environment variables instead of full MONGO_URI variable
46
- create REDIS connection string from environment variables instead of full REDIS_URL variable
57

68
## 1.1.0

backend/Dockerfile

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,23 @@
1-
FROM python:3.9
1+
FROM python:3.12-slim
22
WORKDIR /app
33

4-
COPY requirements.txt app.py ./
5-
COPY SC4SNMP_UI_backend ./SC4SNMP_UI_backend
6-
RUN pip install -r ./requirements.txt
7-
ENV FLASK_DEBUG production
4+
COPY --chown=10000:10000 requirements.txt app.py ./
5+
COPY --chown=10000:10000 SC4SNMP_UI_backend ./SC4SNMP_UI_backend
6+
RUN pip install --no-cache-dir -r ./requirements.txt
7+
ENV FLASK_DEBUG=production
88

9+
# Copy scripts (executable, owned by non-root user)
10+
COPY --chown=10000:10000 ./flask_start.sh /flask_start.sh
11+
COPY --chown=10000:10000 ./celery_start.sh /celery_start.sh
12+
COPY --chown=10000:10000 construct-connection-strings.sh /app/construct-connection-strings.sh
913

10-
COPY ./flask_start.sh /flask_start.sh
11-
RUN chmod +x /flask_start.sh
12-
13-
COPY ./celery_start.sh /celery_start.sh
14-
RUN chmod +x /celery_start.sh
14+
# Set executable bit BEFORE USER switch
15+
RUN chmod 0755 /flask_start.sh /celery_start.sh /app/construct-connection-strings.sh
1516

17+
# Switch to non-root
1618
USER 10000:10000
1719

1820
EXPOSE 5000
19-
CMD ["gunicorn", "-b", ":5000", "app:flask_app", "--log-level", "INFO"]
21+
22+
# Use flask_start.sh which sources construct-connection-strings.sh
23+
CMD ["/flask_start.sh"]

backend/SC4SNMP_UI_backend/__init__.py

Lines changed: 163 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,13 @@
1-
from flask import Flask
1+
import re
2+
import sys
3+
import time
4+
5+
from flask import Flask, g
6+
from flask_cors import CORS
7+
from flask_limiter import Limiter
8+
from flask_limiter.util import get_remote_address
29
from pymongo import MongoClient
10+
from pymongo.errors import ConnectionFailure, ServerSelectionTimeoutError
311
import os
412
import logging
513
from celery import Celery
@@ -8,63 +16,198 @@
816

917
load_dotenv()
1018

11-
__version__ = "1.1.1"
19+
__version__ = "1.2.0-beta.2"
1220

1321
MONGO_URI = os.getenv("MONGO_URI")
22+
log = logging.getLogger('gunicorn.error')
23+
log.setLevel(logging.INFO)
24+
25+
def wait_for_mongodb_replicaset(logger, mongo_uri, max_retries=120, retry_interval=5):
26+
"""
27+
Wait for MongoDB to be ready before starting the application.
28+
For replica sets, waits for PRIMARY to be elected.
29+
"""
30+
mongo_mode = os.getenv("MONGODB_MODE", "standalone").lower()
31+
if mongo_mode == "standalone":
32+
logger.info("MongoDB is in standalone mode, skipping ReplicaSet wait")
33+
return
34+
35+
if not mongo_uri:
36+
logger.warning("MONGO_URI not set, exiting application")
37+
sys.exit(1)
38+
39+
logger.info(f"Waiting for MongoDB ReplicaSet to be ready and elect the primary...")
40+
41+
for attempt in range(1, max_retries + 1):
42+
if attempt != 1:
43+
time.sleep(retry_interval)
44+
try:
45+
# Try to connect
46+
client = MongoClient(
47+
mongo_uri, serverSelectionTimeoutMS=5000, connectTimeoutMS=5000
48+
)
49+
50+
# Execute a simple operation to verify PRIMARY exists
51+
client.admin.command("ping")
52+
53+
# For replica sets, verify PRIMARY exists
54+
if "replicaSet=" in mongo_uri:
55+
if client.primary is None:
56+
continue
57+
logger.info(f"PRIMARY found: {client.primary}")
58+
59+
client.close()
60+
logger.info("MongoDB is ready")
61+
return
62+
63+
except (ServerSelectionTimeoutError, ConnectionFailure, Exception) as e:
64+
if attempt >= max_retries:
65+
logger.info(f"MongoDB not ready after {max_retries * retry_interval}s")
66+
logger.info(f" Error: {e}")
67+
sys.exit(1)
68+
69+
logger.info(f" Still waiting... ({attempt}/{max_retries})")
70+
71+
wait_for_mongodb_replicaset(log, MONGO_URI)
1472
mongo_client = MongoClient(MONGO_URI)
1573

1674
VALUES_DIRECTORY = os.getenv("VALUES_DIRECTORY", "")
1775
KEEP_TEMP_FILES = os.getenv("KEEP_TEMP_FILES", "false")
1876

19-
REDIS_HOST = os.getenv("REDIS_HOST", "snmp-redis")
20-
REDIS_PORT = os.getenv("REDIS_PORT", "6379")
21-
REDIS_PASSWORD = os.getenv("REDIS_PASSWORD", "")
22-
REDIS_DB = os.getenv("REDIS_DB", "1")
23-
CELERY_DB = os.getenv("CELERY_DB", "0")
77+
REDBEAT_URL = os.getenv("REDIS_URL", "redis://snmp-redis-headless:6379")
78+
CELERY_BROKER_URL = os.getenv("CELERY_BROKER_URL", "sentinel://snmp-redis-sentinel:26379")
79+
REDIS_SENTINEL_SERVICE = os.getenv("REDIS_SENTINEL_SERVICE", "snmp-redis-sentinel")
80+
REDIS_MODE = os.getenv("REDIS_MODE", "standalone")
2481

25-
if REDIS_PASSWORD:
26-
redis_base = f"redis://:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}"
27-
else:
28-
redis_base = f"redis://{REDIS_HOST}:{REDIS_PORT}"
2982

30-
# fallback
31-
REDBEAT_URL = os.getenv("REDIS_URL", f"{redis_base}/{REDIS_DB}")
32-
CELERY_BROKER_URL = os.getenv("CELERY_BROKER_URL", f"{redis_base}/{CELERY_DB}")
83+
class NoValuesDirectoryException(Exception):
84+
pass
3385

3486

35-
class NoValuesDirectoryException(Exception):
87+
class AuthNotConfiguredException(Exception):
3688
pass
3789

90+
91+
limiter = Limiter(key_func=get_remote_address, default_limits=[])
92+
93+
3894
def create_app():
3995
if len(VALUES_DIRECTORY) == 0:
4096
raise NoValuesDirectoryException
4197

4298
app = Flask(__name__)
4399

100+
auth_enabled = os.getenv("AUTH_ENABLED", "true").lower() == "true"
101+
if auth_enabled:
102+
missing = []
103+
for var in ("AUTH_USERNAME", "AUTH_PASSWORD_HASH", "JWT_SECRET"):
104+
if not os.getenv(var):
105+
missing.append(var)
106+
if missing:
107+
raise AuthNotConfiguredException(
108+
f"AUTH_ENABLED=true but {', '.join(missing)} not set. "
109+
"Set these env vars or set AUTH_ENABLED=false to disable authentication."
110+
)
111+
else:
112+
log.warning(
113+
"SECURITY: AUTH_ENABLED=false. All endpoints are accessible without authentication. "
114+
"Do NOT use this configuration in production or on any network-exposed deployment. "
115+
"Restrict access via ClusterIP/NetworkPolicy and use only for local development."
116+
)
117+
118+
allowed_origins_env = os.getenv("ALLOWED_ORIGINS", "").strip()
119+
if allowed_origins_env == "*":
120+
# Reflect any origin. Intended for local development only.
121+
log.warning(
122+
"SECURITY: ALLOWED_ORIGINS=* reflects any browser Origin. "
123+
"Do NOT use in production; set an explicit allow-list."
124+
)
125+
cors_origins = [re.compile(r".*")]
126+
elif allowed_origins_env:
127+
cors_origins = [o.strip() for o in allowed_origins_env.split(",") if o.strip()]
128+
elif not auth_enabled:
129+
# When auth is disabled (dev mode), reflect any origin so the UI works
130+
# out of the box on NodePort setups. Security warning already logged above.
131+
cors_origins = [re.compile(r".*")]
132+
else:
133+
cors_origins = ["http://localhost:8080"]
134+
135+
CORS(app, origins=cors_origins, supports_credentials=True)
136+
137+
limiter.init_app(app)
138+
limiter.storage_uri = REDBEAT_URL
139+
140+
if REDIS_MODE == "replication":
141+
broker_transport_options = {
142+
"priority_steps": list(range(10)),
143+
"sep": ":",
144+
"queue_order_strategy": "priority",
145+
"service_name": "mymaster",
146+
"master_name": "mymaster",
147+
"socket_timeout": 5,
148+
"retry_policy": {
149+
"max_retries": 100,
150+
"interval_start": 0,
151+
"interval_step": 2,
152+
"interval_max": 5,
153+
},
154+
"db": 1,
155+
"sentinels": [(REDIS_SENTINEL_SERVICE, 26379)],
156+
"password": os.getenv("REDIS_PASSWORD", None),
157+
}
158+
else:
159+
broker_transport_options = {
160+
"priority_steps": list(range(10)),
161+
"sep": ":",
162+
"queue_order_strategy": "priority"
163+
}
164+
44165
app.config.from_mapping(
45166
CELERY=dict(
46167
task_default_queue="apply_changes",
47168
broker_url=CELERY_BROKER_URL,
48169
beat_scheduler="redbeat.RedBeatScheduler",
49170
redbeat_redis_url = REDBEAT_URL,
50-
broker_transport_options={
51-
"priority_steps": list(range(10)),
52-
"sep": ":",
53-
"queue_order_strategy": "priority",
54-
},
171+
broker_transport_options=broker_transport_options,
55172
task_ignore_result=True,
56173
redbeat_lock_key=None,
57174
),
58175
)
59176
celery_init_app(app)
177+
178+
from SC4SNMP_UI_backend.auth.routes import auth_blueprint
60179
from SC4SNMP_UI_backend.profiles.routes import profiles_blueprint
61180
from SC4SNMP_UI_backend.groups.routes import groups_blueprint
62181
from SC4SNMP_UI_backend.inventory.routes import inventory_blueprint
63182
from SC4SNMP_UI_backend.apply_changes.routes import apply_changes_blueprint
183+
app.register_blueprint(auth_blueprint)
64184
app.register_blueprint(profiles_blueprint)
65185
app.register_blueprint(groups_blueprint)
66186
app.register_blueprint(inventory_blueprint)
67187
app.register_blueprint(apply_changes_blueprint)
188+
189+
from SC4SNMP_UI_backend.auth.utils import (
190+
AUTH_ENABLED as _auth_on,
191+
refresh_token_payload,
192+
make_cookie_kwargs,
193+
COOKIE_NAME,
194+
JWT_EXPIRY_HOURS as _jwt_hours,
195+
)
196+
197+
@app.after_request
198+
def refresh_idle_token(response):
199+
if not _auth_on:
200+
return response
201+
try:
202+
should_refresh = g.get("refresh_token", False)
203+
payload = g.get("token_payload")
204+
except RuntimeError:
205+
return response
206+
if should_refresh and payload:
207+
new_token = refresh_token_payload(payload)
208+
response.set_cookie(**make_cookie_kwargs(new_token, max_age=_jwt_hours * 3600))
209+
return response
210+
68211
gunicorn_logger = logging.getLogger('gunicorn.error')
69212
app.logger.handlers = gunicorn_logger.handlers
70213
app.logger.setLevel(gunicorn_logger.level)

0 commit comments

Comments
 (0)