Add Logrotate Support for FM and Sites by iamimmanuelraj · Pull Request #399 · rtCamp/Frappe-Manager

iamimmanuelraj · 2026-04-15T12:56:41Z

This pull request introduces system-level log rotation for Frappe Manager by updating supervisor log file management and adding a new logrotate configuration and command. It also includes a migration to apply these changes to existing installations. The most important changes are grouped below.

Log rotation improvements:

Added stdout_logfile_maxbytes and stdout_logfile_backups (10MB, 5 backups) to all relevant [program:…] sections in supervisor.conf and its template, ensuring supervisor-managed logs are rotated and limited in size. [1] [2] [3] [4] [5] [6] [7] [8]
Added a new Jinja2 template logrotate.tmpl and logic to generate a comprehensive logrotate.conf covering all major Frappe Manager logs (bench, nginx, MariaDB, CLI), supporting daily rotation, compression, and safe log handling.

CLI and command enhancements:

Introduced a new fm self logrotate command to generate and optionally install the logrotate config system-wide, with platform-specific instructions and error handling for unsupported operations. [1] [2] [3]
Marked self logrotate as a "docker-free" command, allowing it to run even if Docker is not running, and updated the CLI logic to support this. [1] [2] [3]

Migration for existing installations:

Added a migration (migrate_0_20_0.py) to back up old supervisor configs, regenerate them with log rotation settings, and generate the new logrotate configuration for all users upgrading to v0.20.0.

Version bump:

Bumped the application version to 0.20.0.dev0 to reflect these changes.

Signed-off-by: Immanuel Raj <iamimmanuelraj@gmail.com>

…isor and system-level log rotation Signed-off-by: Immanuel Raj <iamimmanuelraj@gmail.com>

Copilot

Pull request overview

Adds system-level log rotation support to Frappe Manager by configuring Supervisor-managed log limits, introducing a logrotate config generator/installer command, and shipping a migration to apply these changes on upgrade.

Changes:

Add Supervisor stdout/stderr rotation limits (max bytes + backups) across Frappe services and supervisord itself.
Introduce a logrotate.tmpl plus fm self logrotate to generate (and optionally install) a logrotate config.
Add a v0.20.0 migration to back up/regenerate Supervisor configs and write the logrotate config for existing installations.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
frappe_manager/templates/supervisor.conf.tmpl	Adds Supervisor log size/backups limits for per-program stdout/stderr logs.
frappe_manager/templates/logrotate.tmpl	New logrotate configuration template covering bench/nginx/mariadb/FM CLI logs.
frappe_manager/migration_manager/migrations/migrate_0_20_0.py	New v0.20.0 migration to back up and regenerate Supervisor configs and write logrotate.conf.
frappe_manager/migration_manager/migration_constants.py	Adds `self logrotate` to migration whitelist and defines `DOCKER_FREE_COMMANDS`.
frappe_manager/commands/self/logrotate.py	New CLI command to generate/install logrotate config from the template.
frappe_manager/commands/self/init.py	Registers the new `self logrotate` command.
frappe_manager/commands/init.py	Adds docker-free command detection to skip Docker daemon checks / first-install image pull.
frappe_manager/about.py	Bumps version to `0.20.0.dev0`.
Docker/frappe/supervisord.conf	Adds supervisord logfile rotation limits.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-15T13:02:35Z

+
+                    current_version = Version(get_current_fm_version())
+                    fm_config_manager.version = current_version
+                    fm_config_manager.export_to_toml()


This introduces new behavior that affects command startup (skipping Docker checks based on DOCKER_FREE_COMMANDS), but there doesn’t appear to be test coverage asserting that a docker-free command like self logrotate can run without Docker and without initializing global services. Adding a unit test around app_callback / command dispatch (similar to existing CLI flag tests) would help prevent regressions.

Suggested change

fm_config_manager.export_to_toml()

fm_config_manager.export_to_toml()

else:

# Docker-free commands must also skip the remaining startup initialization

# (migration/config/bootstrap checks) so they can run without Docker and

# without touching global services.

return

Copilot · 2026-04-15T13:02:36Z

+            is_docker_free = full_command in DOCKER_FREE_COMMANDS
+
+            if not is_docker_free:
+                if not DockerClient().server_running():
+                    output.exit("Docker daemon not running. Please start docker service")
+
+                if not CLI_FM_CONFIG_PATH.exists():


DOCKER_FREE_COMMANDS currently only skips the initial Docker daemon check and first-install image pull, but ServicesManager.init() / entrypoint_checks() still run later in app_callback for all commands. That will still instantiate DockerClient and attempt to create/pull/start global services, so fm self logrotate will still fail when Docker is unavailable. Consider short-circuiting the ServicesManager initialization/entrypoint checks when is_docker_free is true (and only populate ctx.obj with what the command actually needs).

Copilot · 2026-04-15T13:02:36Z

+    output.print(f":white_check_mark: Logrotate config written to [blue]{dest}[/blue]")
+
+    if install:
+        system_dest = Path("/etc/logrotate.d/frappe-manager")
+        try:
+            result = subprocess.run(
+                ["sudo", "cp", str(dest), str(system_dest)],
+                check=True,
+                capture_output=True,
+                text=True,
+            )
+            output.print(
+                f":white_check_mark: Installed to [blue]{system_dest}[/blue]. "
+                "Logs will rotate daily."
+            )


The success messages embed :white_check_mark: inside the message string instead of using the emoji_code parameter that the OutputHandler API expects. This will likely render the literal text :white_check_mark: in Rich output and JSON logs rather than an emoji code. Pass the emoji via emoji_code=... and keep the message text free of emoji codes (consistent with other commands).

Copilot · 2026-04-15T13:02:36Z

+    current_system = platform.system()
+    if install and current_system == "Darwin":
+        output.error(
+            "System-wide logrotate installation is not supported on macOS. "
+            "Use newsyslog or install logrotate via Homebrew and manage manually."
+        )
+        raise typer.Exit(1)
+


--install is documented as Linux-only, but the current platform guard only blocks macOS (Darwin). On Windows/other platforms this will still try to run sudo cp ... /etc/logrotate.d/... and then print macOS-specific instructions in the non-install path. Please explicitly require platform.system() == 'Linux' for --install, and tailor the non-Linux guidance/error message per platform (or exit with an unsupported-platform error).

Copilot · 2026-04-15T13:02:37Z

+    compress
+    delaycompress
+    missingok
+    notifempty
+    copytruncate
+    su {{ user }} {{ user }}
+}
+
+# ── Per-site nginx logs ─────────────────────────────────────────────────────────
+{{ frappe_dir }}/sites/*/configs/nginx/logs/*.log {
+    daily
+    rotate 14
+    compress
+    delaycompress
+    missingok
+    notifempty
+    copytruncate
+    su {{ user }} {{ user }}
+}
+
+# ── Global nginx-proxy logs ─────────────────────────────────────────────────────
+{{ frappe_dir }}/services/nginx-proxy/logs/*.log {
+    daily
+    rotate 14
+    compress
+    delaycompress
+    missingok
+    notifempty
+    copytruncate
+    su {{ user }} {{ user }}
+}
+
+# ── Global MariaDB logs ─────────────────────────────────────────────────────────
+{{ frappe_dir }}/services/mariadb/logs/*.log {
+    daily
+    rotate 14
+    compress
+    delaycompress
+    missingok
+    notifempty
+    copytruncate
+    su {{ user }} {{ user }}
+}
+
+# ── FM CLI logs ─────────────────────────────────────────────────────────────────
+# Note: fm.log is also rotated automatically by Python's RotatingFileHandler
+# (10 MB per file, 3 backups). This entry provides an additional daily sweep.
+{{ frappe_dir }}/logs/fm.log {
+    daily
+    rotate 14
+    compress
+    delaycompress
+    missingok
+    notifempty
+    copytruncate
+    su {{ user }} {{ user }}
+}


This logrotate config rotates the active *.log files daily without dateext, but many of these logs are already rotated by other mechanisms (Supervisor *_logfile_maxbytes produces .log.1/.log.2..., and fm.log is rotated by Python RotatingFileHandler using the same suffix scheme). Running logrotate on the same base filenames risks collisions/overwrites and log loss. Consider adding dateext (and an explicit dateformat) or excluding logs that are already size-rotated, especially {{ frappe_dir }}/logs/fm.log.

Also, su {{ user }} {{ user }} assumes the primary group name equals the username, which isn’t always true; it’s safer to render the user’s primary group explicitly or omit su when rotating files under a user-owned directory.

Suggested change

compress

delaycompress

missingok

notifempty

copytruncate

su {{ user }} {{ user }}

}

# ── Per-site nginx logs ─────────────────────────────────────────────────────────

{{ frappe_dir }}/sites/*/configs/nginx/logs/*.log {

daily

rotate 14

compress

delaycompress

missingok

notifempty

copytruncate

su {{ user }} {{ user }}

}

# ── Global nginx-proxy logs ─────────────────────────────────────────────────────

{{ frappe_dir }}/services/nginx-proxy/logs/*.log {

daily

rotate 14

compress

delaycompress

missingok

notifempty

copytruncate

su {{ user }} {{ user }}

}

# ── Global MariaDB logs ─────────────────────────────────────────────────────────

{{ frappe_dir }}/services/mariadb/logs/*.log {

daily

rotate 14

compress

delaycompress

missingok

notifempty

copytruncate

su {{ user }} {{ user }}

}

# ── FM CLI logs ─────────────────────────────────────────────────────────────────

# Note: fm.log is also rotated automatically by Python's RotatingFileHandler

# (10 MB per file, 3 backups). This entry provides an additional daily sweep.

{{ frappe_dir }}/logs/fm.log {

daily

rotate 14

compress

delaycompress

missingok

notifempty

copytruncate

su {{ user }} {{ user }}

}

dateext

dateformat -%Y%m%d

compress

delaycompress

missingok

notifempty

copytruncate

}

# ── Per-site nginx logs ─────────────────────────────────────────────────────────

{{ frappe_dir }}/sites/*/configs/nginx/logs/*.log {

daily

rotate 14

dateext

dateformat -%Y%m%d

compress

delaycompress

missingok

notifempty

copytruncate

}

# ── Global nginx-proxy logs ─────────────────────────────────────────────────────

{{ frappe_dir }}/services/nginx-proxy/logs/*.log {

daily

rotate 14

dateext

dateformat -%Y%m%d

compress

delaycompress

missingok

notifempty

copytruncate

}

# ── Global MariaDB logs ─────────────────────────────────────────────────────────

{{ frappe_dir }}/services/mariadb/logs/*.log {

daily

rotate 14

dateext

dateformat -%Y%m%d

compress

delaycompress

missingok

notifempty

copytruncate

}

# ── FM CLI logs ─────────────────────────────────────────────────────────────────

# Note: fm.log is rotated automatically by Python's RotatingFileHandler

# (10 MB per file, 3 backups), so it is intentionally excluded here to avoid

# suffix collisions and accidental overwrite of rotated files.

Copilot · 2026-04-15T13:02:37Z

+import multiprocessing
+import os
+import platform
+from pathlib import Path


Path is imported but not used in this migration module. If you want to keep lint clean and avoid dead imports, please remove the unused import (or start using it explicitly).

Suggested change

from pathlib import Path

iamimmanuelraj added 2 commits April 15, 2026 15:23

feat: add logrotate command and configuration for Frappe Manager logs

c3b0d74

Signed-off-by: Immanuel Raj <iamimmanuelraj@gmail.com>

feat: update version to 0.20.0 and add logrotate migration for superv…

605bae3

…isor and system-level log rotation Signed-off-by: Immanuel Raj <iamimmanuelraj@gmail.com>

Copilot AI review requested due to automatic review settings April 15, 2026 12:56

Copilot started reviewing on behalf of iamimmanuelraj April 15, 2026 12:57 View session

Copilot AI reviewed Apr 15, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Logrotate Support for FM and Sites#399

Add Logrotate Support for FM and Sites#399
iamimmanuelraj wants to merge 2 commits into
developfrom
logrotate

iamimmanuelraj commented Apr 15, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 15, 2026

Uh oh!

Copilot AI Apr 15, 2026

Uh oh!

Copilot AI Apr 15, 2026

Uh oh!

Copilot AI Apr 15, 2026

Uh oh!

Copilot AI Apr 15, 2026

Uh oh!

Copilot AI Apr 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-                    fm_config_manager.export_to_toml()
+                    fm_config_manager.export_to_toml()
+            else:
+                # Docker-free commands must also skip the remaining startup initialization
+                # (migration/config/bootstrap checks) so they can run without Docker and
+                # without touching global services.
+                return

Conversation

iamimmanuelraj commented Apr 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants