v1.2.7: Fix Paper install (#15), Link-folder button (#15), Linux memory leak (#4)

Issue #15 (cant install paper server):
- download_server_jar now lowercases server_type; mcutils returns HTTP 500
  for capitalized types (e.g. 'Paper'), so versions loaded but download
  failed with a generic 'Failed to download Server JAR.' error.
- download_file_from_url: retry with backoff on transient CDN/Cloudflare
  5xx, browser User-Agent to avoid bot-blocking, and surface the real
  failure reason (URL + status) to the UI.
- install_server dispatch + saved profile type normalized to lowercase.
- Fixed 'Link Project' button doing nothing: api.importServer was missing
  from the frontend API client (silent TypeError). It now detects the
  engine/version from the folder, registers the profile, and shows
  loading/error feedback in the Setup Wizard.

Issue #4 (Massive RAM leak on Linux - 32GB gone in 50s):
- Truncate log lines to 8KB before storing in the in-memory deques
  (log_history 1000, app_log_history 500) and async log queue (2000).
  A force-flushed 1MB newline-less chunk could otherwise be retained per
  entry, allowing gigabytes to pile up.
- Reuse the binary read buffer (del/clear) instead of allocating a new
  bytearray per flushed line, reducing allocator churn and glibc arena
  growth that kept RSS high after a server stopped.
- malloc_trim(0) on Linux after server stop/kill so freed heap is
  returned to the OS instead of held in the malloc arena.
- Stop calling logging.basicConfig() in api_client.py and java_manager.py
  at import time (ran before api_server configured the root logger, so
  the backend_debug.log FileHandler was never installed). Root logger
  level lowered DEBUG->INFO, removing huge per-chunk debug log garbage.
- Cap Server List Ping JSON payload at 1MB and bound the Varint reader.

Author: CalaKuad1

README.md

@@ -28,7 +28,23 @@
 
 ---
 
-## What's New in v1.2.5
+## What's New in v1.2.7
+
+### Fixed Server Installation (Paper/Spigot/Fabric)
+- **Paper install fixed** — Installing Paper (and Spigot/Fabric) no longer fails with "Failed to download Server JAR." The server type is now lowercased before hitting the mcutils API, which returned HTTP 500 for capitalized types.
+- **Reliable downloads** — Server JAR downloads now retry on transient CDN/Cloudflare errors with backoff, and the real failure reason (URL + status) is shown in the UI instead of a generic message.
+
+### Fixed "Link to folder"
+- **Import existing server works again** — The "Link Project" button in the Setup Wizard was silently broken (missing API method). It now detects the engine/version from your folder and registers the server, with loading and error feedback.
+
+### Linux Memory Leak Fix
+- **Massive RAM leak resolved** — On Linux, `api_server` could hold gigabytes after a server stopped. Log lines are now truncated before being stored in memory, the read buffer is reused, freed heap is returned to the OS via `malloc_trim`, and the backend no longer generates huge volumes of per-chunk debug log garbage.
+- **Real log file now works** — `backend_debug.log` is now actually written (a logging setup bug previously sent everything to stderr).
+
+<details>
+<summary><strong>Earlier versions</strong></summary>
+
+### What's New in v1.2.5
 
 ### Fixed Server Addresses (DNS)
 - **Permanent domain** — Your server gets a permanent address like `survival.play.tudominio.app` that never changes
@@ -48,9 +64,6 @@
 - **Auto-Tunnel** — Tunnel starts automatically with the server (toggle in Advanced)
 - **Boot from library** — "Boot" button on server cards in the library
 
-<details>
-<summary><strong>Earlier versions</strong></summary>
-
 ### Auto-Restart on Crash
 - Server automatically restarts after unexpected shutdowns — toggle it on/off from the Dashboard header
 - Up to 3 restart attempts with a 3-second delay between retries
@@ -81,6 +94,8 @@
 
 </details>
 
+</details>
+
 ---
 
 ## Download & Install

backend/api_server.py

@@ -52,7 +52,7 @@
 
     logging.basicConfig(
         filename=os.path.join(log_dir, "backend_debug.log"),
-        level=logging.DEBUG,
+        level=logging.INFO,
         format="%(asctime)s - %(levelname)s - %(message)s",
     )
     logging.info("Backend starting up...")
@@ -74,10 +74,17 @@
     get_forge_versions,
     get_neoforge_versions,
     download_server_jar,
+    download_file_from_url,
 )
 from utils.mods_manager import ModsManager
 
 
+# Maximum length of a single log line retained in memory (deques / async queue).
+# Matches server_handler.MAX_LOG_LINE_LEN — kept duplicated here to avoid a
+# circular import. Bounds retained memory when server output has no newlines.
+MAX_LOG_LINE_LEN = 8000
+
+
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     global state
@@ -372,6 +379,15 @@ def broadcast_log_sync(self, message, level="normal", server_id=None):
                     message = message.replace("\r", "")
                 msg_obj = {"message": message, "level": level, "server_id": server_id}
 
+            # Bound memory: truncate huge messages (e.g. a 1MB force-flushed log
+            # chunk) before they enter app_log_history (deque 500) or the async
+            # log queue (maxsize 2000). Without this, a chatty/idle server with
+            # newline-less output could retain gigabytes in these structures.
+            msg_text_check = msg_obj.get("message") if isinstance(msg_obj, dict) else None
+            if isinstance(msg_text_check, str) and len(msg_text_check) > MAX_LOG_LINE_LEN:
+                msg_obj = dict(msg_obj)
+                msg_obj["message"] = msg_text_check[:MAX_LOG_LINE_LEN] + " …[truncated]"
+
             # Store in app-level history for Dashboard polling
             # Skip verbose installer logs (recipe files, etc.) to avoid spam
             msg_text = (
@@ -883,6 +899,12 @@ def install_server(req: InstallRequest):
     def run_install():
         logging.info("Installation thread started")
         try:
+            # Normalize server type to lowercase so the mcutils download URL and
+            # the dispatch below behave consistently regardless of the casing the
+            # frontend sends (e.g. "Paper" vs "paper"). mcutils returns HTTP 500
+            # for capitalized types, which previously broke Paper installs.
+            server_type = (req.server_type or "").lower()
+
             # Helper to send structured progress
             def send_progress(pct, msg, **kwargs):
                 state.install_progress = pct
@@ -951,7 +973,7 @@ def progress_callback(p):
                     send_progress(scaled, "Downloading server files...")
 
             # 3. Instalación del Servidor
-            if req.server_type.lower() == "forge":
+            if server_type == "forge":
                 # Crear un handler temporal con la ruta de Java CORRECTA explícita
                 temp_handler = ServerHandler(
                     install_path,
@@ -994,7 +1016,7 @@ def forge_progress(p):
                     forge_ver, req.version, forge_progress
                 )
 
-            elif req.server_type.lower() == "neoforge":
+            elif server_type == "neoforge":
                 # NeoForge logic
                 temp_handler = ServerHandler(
                     install_path,
@@ -1035,13 +1057,18 @@ def neoforge_progress(p):
                 temp_handler.install_neoforge_server(neoforge_ver, neoforge_progress)
 
             else:
-                # Vanilla / Paper / Fabric logic
+                # Vanilla / Paper / Spigot / Fabric logic
                 jar_path = os.path.join(install_path, "server.jar")
                 success = download_server_jar(
-                    req.server_type, req.version, jar_path, progress_callback
+                    server_type, req.version, jar_path, progress_callback
                 )
                 if not success:
-                    raise Exception("Failed to download Server JAR.")
+                    # Surface the real failure reason (URL/status/exception) instead
+                    # of a generic message so users can diagnose Paper/Spigot issues.
+                    reason = getattr(
+                        download_file_from_url, "last_error", None
+                    ) or "Unknown error"
+                    raise Exception(f"Failed to download Server JAR: {reason}")
 
             # 4. Configuración Final
             send_progress(95, "Finalizing configuration...")
@@ -1050,7 +1077,7 @@ def neoforge_progress(p):
             new_server_data = {
                 "name": req.folder_name,
                 "path": install_path,
-                "type": req.server_type,
+                "type": server_type,
                 "version": req.version,
                 "ram_min": req.ram_min,
                 "ram_max": req.ram_max,

backend/server/server_handler.py

@@ -19,6 +19,31 @@
 import time
 from typing import Optional
 
+# Cap the length of a single log line kept in memory. Server output that contains
+# no newlines (e.g. progress bars, binary-ish blobs) can be force-flushed as a
+# single ~1MB chunk; without truncation each bounded deque would retain megabytes
+# per entry (1000 + 500 + 2000 entries => multiple GB). 8KB is plenty for any
+# real log line while keeping worst-case retained memory small.
+MAX_LOG_LINE_LEN = 8000
+
+
+def _malloc_trim():
+    """Return freed heap memory to the OS on Linux (no-op elsewhere).
+
+    glibc's malloc keeps freed allocations in its arena for reuse instead of
+    releasing them back to the kernel, so RSS can stay high (and even appear to
+    "leak") after a server stops. Calling malloc_trim(0) after a server exits
+    makes the backend give back that memory.
+    """
+    if sys.platform == "linux":
+        try:
+            import ctypes
+            libc = ctypes.CDLL("libc.so.6", use_errno=True)
+            if hasattr(libc, "malloc_trim"):
+                libc.malloc_trim(0)
+        except Exception:
+            pass
+
 
 class ServerHandler:
     def __init__(
@@ -100,6 +125,10 @@ def _log(self, message, level="normal"):
             message = message.rstrip()
             if not message:
                 return
+            # Bound memory: truncate force-flushed chunks (up to 1MB with no
+            # newlines) so the bounded deques don't retain megabytes per entry.
+            if len(message) > MAX_LOG_LINE_LEN:
+                message = message[:MAX_LOG_LINE_LEN] + " …[truncated]"
             msg_obj = {"message": message, "level": level, "server_id": self.server_id}
         else:
             msg_obj = message
@@ -954,6 +983,11 @@ def _run_server(self, command, env):
                     }
                 )
 
+            # Return freed memory to the OS on Linux (glibc arena trimming).
+            # This is what makes the backend's RSS actually drop after a server
+            # stops instead of holding onto peak usage indefinitely.
+            _malloc_trim()
+
     def _process_log_line(
         self,
         line,
@@ -1126,11 +1160,11 @@ def _read_output(self, pipe, level):
                         list_inline_pattern,
                         list_header_pattern,
                     )
-                    buffer = bytearray()
+                    # Reuse the same bytearray object (clear() may free the
+                    # underlying storage) instead of allocating a new one every
+                    # flush — reduces allocator churn and glibc arena growth.
+                    buffer.clear()
                     continue
-                logging.debug(
-                    f"Handler: Received chunk of {len(chunk)} bytes from {level}"
-                )
                 while b"\n" in buffer or b"\r" in buffer:
                     # Find the first line break
                     idx_n = buffer.find(b"\n")
@@ -1150,7 +1184,9 @@ def _read_output(self, pipe, level):
                             sep_len = 2  # Handle \r\n
 
                     line_bytes = buffer[:idx]
-                    buffer = bytearray(buffer[idx + sep_len :])
+                    # Remove the consumed line in-place instead of creating a new
+                    # bytearray each iteration (avoids per-line allocation).
+                    del buffer[: idx + sep_len]
 
                     line = line_bytes.decode("utf-8", errors="replace")
                     self._process_log_line(
@@ -1250,6 +1286,7 @@ def _kill_process_tree(self):
                         "server_id": self.server_id,
                     }
                 )
+            _malloc_trim()
 
     def wait_for_stop(self, timeout=30):
         """Blocks until the server process has exited or the timeout is reached."""

backend/utils/api_client.py

@@ -4,13 +4,18 @@
 import logging
 import json
 import os
+import time
 import xml.etree.ElementTree as ET
 from collections import defaultdict
 import re
 import zipfile
 
-# Configure logging
-logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
+# NOTE: Do NOT call logging.basicConfig() here. This module is imported by
+# api_server.py BEFORE api_server configures the root logger, and basicConfig is
+# a no-op once the root logger already has handlers. Configuring the root logger
+# here would prevent api_server's FileHandler from ever being installed (logs
+# would silently go to stderr instead of backend_debug.log). Let api_server own
+# the root logger configuration; module-level logging.* calls propagate to it.
 
 def get_server_versions(server_type):
     """Fetches available server versions for a given type from mcutils.com API."""
@@ -186,48 +191,84 @@ def fetch_username_from_uuid(uuid_str):
 
 def download_server_jar(server_type, server_version, save_path, progress_callback):
     """Downloads the server.jar file for a given type and version with progress."""
+    # CRITICAL: mcutils.com API is case-sensitive and returns HTTP 500 for
+    # capitalized server types (e.g. "Paper"). Always normalize to lowercase.
+    server_type = (server_type or "").lower()
     download_url = f"https://mcutils.com/api/server-jars/{server_type}/{server_version}/download"
     return download_file_from_url(download_url, save_path, progress_callback)
 
-def download_file_from_url(download_url, save_path, progress_callback):
-    """Downloads a file from a specific URL with progress."""
+# A browser-like User-Agent avoids bot-blocking by Cloudflare-fronted CDNs
+# (mcutils.com, fill-data.papermc.io, piston-data.mojang.com).
+_DOWNLOAD_HEADERS = {
+    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
+    "(KHTML, like Gecko) Chrome/124.0 Safari/537.36"
+}
+
+def download_file_from_url(download_url, save_path, progress_callback, retries=3):
+    """Downloads a file from a specific URL with progress.
+
+    Retries on transient failures (Cloudflare 500/429, connection resets) and
+    returns False on failure. The last error is logged with the URL + status so
+    callers can surface a useful message instead of a generic one.
+    """
     os.makedirs(os.path.dirname(save_path), exist_ok=True)
-    try:
-        with requests.get(download_url, stream=True, timeout=30) as r:
-            r.raise_for_status()
-            total_size_raw = r.headers.get('content-length')
-            total_size = int(total_size_raw) if total_size_raw else 0
-            bytes_downloaded = 0
-            last_reported_mb = 0
-            
-            logging.info(f"Downloading {download_url} - Total size: {total_size if total_size > 0 else 'unknown'}")
-            
-            with open(save_path, 'wb') as f:
-                for chunk in r.iter_content(chunk_size=65536): # 64KB chunks for better throughput
-                    if not chunk:
-                        continue
-                    f.write(chunk)
-                    bytes_downloaded += len(chunk)
-                    
-                    if total_size > 0:
-                        progress = (bytes_downloaded / total_size) * 100
-                        progress_callback(progress)
-                    else:
-                        # Fallback: Report "activity" every 1MB
-                        current_mb = bytes_downloaded // (1024 * 1024)
-                        if current_mb > last_reported_mb:
-                            last_reported_mb = current_mb
-                            # Send a small incremental progress or just trigger the callback
-                            # to keep the UI alive. We'll send a "mock" slow progress
-                            mock_progress = min(current_mb * 5, 95) # Caps at 95 until real finish
-                            progress_callback(mock_progress)
-                            logging.debug(f"Downloaded {current_mb}MB (unknown total size)")
-                        
-        progress_callback(100)
-        return True
-    except Exception as e:
-        logging.error(f"Failed to download file: {e}")
-        return False
+    last_error = None
+    for attempt in range(1, retries + 1):
+        try:
+            with requests.get(
+                download_url, stream=True, timeout=30, headers=_DOWNLOAD_HEADERS
+            ) as r:
+                if r.status_code >= 500:
+                    # Transient server/CDN error — retry with backoff
+                    last_error = f"HTTP {r.status_code} from {download_url}"
+                    logging.warning(
+                        f"Download attempt {attempt}/{retries} failed: {last_error}"
+                    )
+                    if attempt < retries:
+                        time.sleep(2 * attempt)
+                    continue
+                r.raise_for_status()
+                total_size_raw = r.headers.get('content-length')
+                total_size = int(total_size_raw) if total_size_raw else 0
+                bytes_downloaded = 0
+                last_reported_mb = 0
+
+                logging.info(f"Downloading {download_url} - Total size: {total_size if total_size > 0 else 'unknown'}")
+
+                with open(save_path, 'wb') as f:
+                    for chunk in r.iter_content(chunk_size=65536):  # 64KB chunks
+                        if not chunk:
+                            continue
+                        f.write(chunk)
+                        bytes_downloaded += len(chunk)
+
+                        if total_size > 0:
+                            progress = (bytes_downloaded / total_size) * 100
+                            progress_callback(progress)
+                        else:
+                            # Fallback: Report "activity" every 1MB
+                            current_mb = bytes_downloaded // (1024 * 1024)
+                            if current_mb > last_reported_mb:
+                                last_reported_mb = current_mb
+                                mock_progress = min(current_mb * 5, 95)
+                                progress_callback(mock_progress)
+                                logging.debug(f"Downloaded {current_mb}MB (unknown total size)")
+
+            progress_callback(100)
+            return True
+        except requests.RequestException as e:
+            last_error = f"{e} ({download_url})"
+            logging.warning(f"Download attempt {attempt}/{retries} failed: {last_error}")
+            if attempt < retries:
+                time.sleep(2 * attempt)
+        except Exception as e:
+            last_error = f"{e} ({download_url})"
+            logging.error(f"Failed to download file: {last_error}")
+            return False
+    logging.error(f"Download failed after {retries} attempts: {last_error}")
+    # Stash the last error on the function so callers can surface a useful message
+    download_file_from_url.last_error = last_error
+    return False
 
 def download_and_extract_zip(url, extract_to_dir, progress_callback, contains_single_folder=True):
     """

backend/utils/java_manager.py

@@ -11,7 +11,11 @@
 import logging
 from typing import Optional, Dict, Tuple, Callable
 
-logging.basicConfig(level=logging.INFO)
+# NOTE: Do NOT call logging.basicConfig() here — this module is imported by
+# api_server.py before api_server configures the root logger, and basicConfig is
+# a no-op once the root logger already has handlers. Configuring root here would
+# prevent api_server's FileHandler (backend_debug.log) from being installed.
+# Use a module logger; it propagates to the root logger api_server sets up.
 logger = logging.getLogger(__name__)