Skip to content

Commit a349ebb

Browse files
JE-ChenJE-Chen
authored
Merge pull request #36 from Integration-Automation/dev
refactor: enforce SonarQube/Codacy linter compliance
2 parents da994b6 + 45e03bf commit a349ebb

11 files changed

Lines changed: 434 additions & 169 deletions

File tree

‎CLAUDE.md‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
# CLAUDE.md — MailThunder
2+
3+
## Project Overview
4+
5+
MailThunder (`je_mail_thunder`) is a Python email automation library wrapping SMTP and IMAP4 protocols. It provides JSON-based scripting, project templates, and a socket server for remote execution.
6+
7+
- **Language**: Python 3.9+
8+
- **Package**: `je_mail_thunder` (PyPI: `je-mail-thunder`)
9+
- **License**: MIT
10+
- **Entry point**: `je_mail_thunder/__main__.py`
11+
12+
## Build & Test
13+
14+
```bash
15+
pip install -e . # Install in dev mode
16+
pip install -r dev_requirements.txt
17+
pytest # Run tests (testpaths = test/)
18+
```
19+
20+
## Architecture
21+
22+
```
23+
je_mail_thunder/
24+
smtp/smtp_wrapper.py # SMTPWrapper (extends SMTP_SSL)
25+
imap/imap_wrapper.py # IMAPWrapper (extends IMAP4_SSL)
26+
utils/
27+
executor/ # Command pattern — JSON action executor
28+
socket_server/ # TCP socket server for remote command execution
29+
save_mail_user_content/ # Credential storage (JSON file / env vars)
30+
project/template/ # Template method pattern for project scaffolding
31+
package_manager/ # Dynamic package loading
32+
json/ # JSON file I/O
33+
json_format/ # JSON processing
34+
file_process/ # Directory file listing
35+
logging/ # Centralized logger instance
36+
exception/ # Custom exception hierarchy
37+
```
38+
39+
## Design Patterns & Software Engineering Principles
40+
41+
### Required Patterns
42+
43+
- **Wrapper / Adapter Pattern**: `SMTPWrapper` and `IMAPWrapper` extend stdlib classes to add logging, auto-login, and context manager support. New protocol wrappers must follow this pattern.
44+
- **Command Pattern**: The `Executor` class maps string command names to callable actions. All new executable features must register through `event_dict`.
45+
- **Template Method Pattern**: Project scaffolding uses `template_executor.py` / `template_keyword.py`. Extend templates by adding keyword handlers, not by modifying the base flow.
46+
- **Singleton-like Module Instances**: `smtp_instance`, `imap_instance`, `executor`, `package_manager` are module-level singletons. Do not create duplicate global instances.
47+
- **Context Manager Protocol**: All wrappers implement `__enter__` / `__exit__`. New resource-holding classes must do the same.
48+
49+
### Engineering Principles
50+
51+
- **Single Responsibility**: Each module under `utils/` handles one concern. Do not merge unrelated logic into a single module.
52+
- **Open/Closed**: Extend behavior by adding new commands to `Executor.event_dict` or new template keywords — not by modifying existing method signatures.
53+
- **DRY**: The login logic (`try_to_login_with_env_or_content`) is shared across SMTP/IMAP. If adding new auth sources, update the shared credential flow in `save_mail_user_content/`.
54+
- **Fail Fast with Logging**: All public methods catch exceptions, log via `mail_thunder_logger`, and avoid silent failures. Follow this pattern for any new code.
55+
56+
## Performance Guidelines
57+
58+
- **Lazy Initialization**: `smtp_instance` and `imap_instance` are created at import time with try/except fallback to `None`. Use `later_init()` for deferred login — do not block module import with network calls.
59+
- **Avoid Redundant I/O**: When processing multiple emails, prefer batch operations. Do not open/close connections per email.
60+
- **Minimize Memory Allocation**: Use generators or iterators for large mailbox operations instead of building full lists in memory.
61+
- **Connection Reuse**: Reuse `SMTPWrapper` / `IMAPWrapper` instances within a session. Do not create new connections for each send/receive operation.
62+
- **File I/O**: Use context managers (`with` statements) for all file operations to ensure prompt resource release.
63+
64+
## Dead Code Policy
65+
66+
- **Remove unused imports, variables, functions, and classes** before committing. Do not leave commented-out code blocks.
67+
- **No placeholder or stub code** unless explicitly required for an interface contract.
68+
- **No backwards-compatibility shims** — if something is unused, delete it completely.
69+
- Run a linter check before committing to catch unreferenced symbols.
70+
71+
## Security Requirements (Mandatory)
72+
73+
### Credential Handling
74+
- **Never hardcode credentials** in source code. Credentials must come from `mail_thunder_content.json` (local, gitignored) or environment variables only.
75+
- **Never log credentials**. Sanitize all log messages — ensure `user`, `password`, and token values are never written to `mail_thunder_logger` or stdout.
76+
- **Never commit** `.env` files, `mail_thunder_content.json`, or any file containing secrets.
77+
78+
### Input Validation
79+
- **Validate all external input** at system boundaries: JSON action files, socket server commands, CLI arguments, email headers.
80+
- **Sanitize file paths** — use `os.path.basename()` and reject path traversal patterns (`..`, absolute paths) in user-supplied filenames, especially in `output_all_mail_as_file` and attachment handling.
81+
- **Limit socket recv buffer** and validate JSON payloads before execution to prevent injection or denial-of-service.
82+
83+
### Command Execution Safety
84+
- The `Executor` registers all Python builtins into `event_dict`. Be aware that this allows arbitrary builtin calls via JSON commands. Any new command registration via `add_command_to_executor` must validate that only `types.MethodType` or `types.FunctionType` are accepted (already enforced).
85+
- **Never use `eval()` or `exec()`** on untrusted input.
86+
- **Never use `subprocess.shell=True`** with user-provided strings.
87+
88+
### Network Security
89+
- SMTP uses `SMTP_SSL` (port 465) — always use SSL/TLS. Do not downgrade to plain SMTP.
90+
- IMAP uses `IMAP4_SSL` — always use SSL/TLS. Do not downgrade to plain IMAP.
91+
- Socket server binds to `localhost` by default. Do not change the default bind address to `0.0.0.0` without explicit user configuration.
92+
93+
### Dependency Security
94+
- Keep dependencies minimal (`requirements.txt` is intentionally small).
95+
- Audit new dependencies before adding. Prefer stdlib solutions.
96+
97+
## Commit Convention
98+
99+
- Write concise commit messages that describe the "why", not just the "what".
100+
- **Do not mention any AI assistant, model name, or tool name** (including but not limited to Claude, GPT, Copilot, etc.) in commit messages, PR descriptions, or code comments.
101+
- **Do not include `Co-Authored-By` headers referencing AI tools.**
102+
- Format: `<type>: <description>` (e.g., `fix: prevent path traversal in mail export`, `feat: add OAuth2 support for IMAP login`).
103+
- Types: `feat`, `fix`, `refactor`, `test`, `docs`, `chore`, `perf`, `security`.
104+
105+
## Code Style
106+
107+
- Follow existing project conventions — no type annotations on code you didn't write unless fixing a bug there.
108+
- Use `mail_thunder_logger` for all logging. No `print()` in library code (only in CLI/socket server output).
109+
- Exception hierarchy rooted at `MailThunderException`. New exceptions must subclass it.
110+
- All public methods need docstrings following the existing `:param` / `:return:` style.
111+
112+
## Linter Compliance (SonarQube / Codacy / Pylint / Flake8)
113+
114+
All code must pass static analysis from SonarQube, Codacy, Pylint, and Flake8. The rules below encode the most common quality-gate failures for this codebase — follow them proactively rather than waiting for a linter report.
115+
116+
### Complexity & Size Limits
117+
- **Cognitive Complexity ≤ 15** per function (SonarQube `python:S3776`). Refactor deeply nested conditionals into early-returns or helper functions.
118+
- **Cyclomatic Complexity ≤ 10** per function (Pylint `R0912`). Split branchy logic.
119+
- **Function length ≤ 50 lines**, **class length ≤ 300 lines**, **module length ≤ 750 lines** (SonarQube defaults). Decompose longer units.
120+
- **Parameters ≤ 7** per function (Pylint `R0913`). Group related arguments into dataclasses or dicts.
121+
- **Max line length: 120 characters** (Flake8 `E501`, configured project-wide).
122+
- **Max nesting depth ≤ 4** (SonarQube `python:S134`).
123+
124+
### Naming (PEP 8 / Pylint `C0103`)
125+
- `snake_case` for functions, methods, variables, modules; `PascalCase` for classes; `UPPER_SNAKE_CASE` for module-level constants.
126+
- No single-letter names except loop counters (`i`, `j`, `k`) or well-known math conventions.
127+
- Avoid shadowing builtins (`list`, `dict`, `id`, `type`, `input`, `file`) — SonarQube `python:S5806`.
128+
129+
### Exception Handling (SonarQube / Bandit)
130+
- **Never use bare `except:`** — always catch specific exceptions (SonarQube `python:S5754`, Bandit `B110`).
131+
- **Do not swallow exceptions silently**. Log via `mail_thunder_logger.error(...)` and re-raise or convert to a `MailThunderException` subclass.
132+
- **Do not use `except Exception as e: pass`** — Codacy `PyLint-W0702/W0703`.
133+
- Chain exceptions with `raise NewError(...) from original_error` to preserve traceback (SonarQube `python:S5708`).
134+
135+
### Duplication & Dead Code
136+
- **No duplicated blocks ≥ 3 lines** (SonarQube `python:S4144` / `common-py:DuplicatedBlocks`). Extract shared logic into helpers.
137+
- **No unused imports / variables / parameters / private functions** (Pylint `W0611`, `W0612`, `W0613`, `W0238`).
138+
- **No unreachable code** after `return` / `raise` / `break` (SonarQube `python:S1763`).
139+
- **No commented-out code** (SonarQube `python:S125`).
140+
- **No `TODO` / `FIXME` without an issue reference** (SonarQube `python:S1135`). Either fix it or file a ticket and reference it.
141+
142+
### Comparison & Logic Correctness
143+
- Use `is None` / `is not None` rather than `== None` (Pylint `C0121`, SonarQube `python:S5727`).
144+
- Use `isinstance(x, T)` instead of `type(x) == T` (Pylint `C0123`).
145+
- Do not compare boolean literals with `==` (`if flag:` not `if flag == True:`) — SonarQube `python:S1125`.
146+
- No constant conditions in `if` / `while` (SonarQube `python:S1145`).
147+
- No identical expressions on both sides of binary operators (SonarQube `python:S1764`).
148+
149+
### Mutable Defaults & Side Effects
150+
- **Never use mutable default arguments** (`def f(x=[])`) — Pylint `W0102`, SonarQube `python:S5717`. Use `None` and initialize inside the function.
151+
- No side effects at import time beyond logger setup and module-level singleton construction that already exists in this project.
152+
153+
### Security Hotspots (Bandit / SonarQube)
154+
- **No hardcoded credentials / tokens / IPs** (Bandit `B105`-`B107`, SonarQube `python:S2068`).
155+
- **No `assert` for runtime validation** — asserts are stripped in optimized mode (Bandit `B101`).
156+
- **No `pickle` / `marshal` / `shelve` on untrusted data** (Bandit `B301`).
157+
- **No `yaml.load` without `SafeLoader`** (Bandit `B506`).
158+
- **No weak hashing** (`md5`, `sha1`) for security purposes (Bandit `B303`, `B324`).
159+
- **No `random` module for security tokens** — use `secrets` (Bandit `B311`).
160+
- **No `tempfile.mktemp`** — use `NamedTemporaryFile` (Bandit `B306`).
161+
- **No binding to `0.0.0.0`** without explicit user opt-in (Bandit `B104`).
162+
- **No SSL context disabling cert verification** (Bandit `B501`).
163+
- **No XML parsing with `xml.etree` / `xml.sax` / `minidom`** on untrusted input — use `defusedxml` (Bandit `B314`-`B320`).
164+
165+
### Imports & Structure
166+
- No wildcard imports (`from x import *`) outside `__init__.py` re-export (Pylint `W0401`).
167+
- No relative imports beyond one level (`from ..x`). Prefer absolute (`from je_mail_thunder.x`).
168+
- Imports ordered: stdlib, third-party, local — separated by blank lines (Flake8 `isort`).
169+
- No circular imports (Pylint `R0401`).
170+
171+
### Formatting
172+
- 4-space indentation, no tabs (Flake8 `W191`).
173+
- Two blank lines between top-level defs, one blank line between methods (PEP 8 / Flake8 `E302`/`E303`).
174+
- No trailing whitespace (Flake8 `W291`), files end with a single newline (Flake8 `W292`).
175+
- No multiple statements on one line (Flake8 `E701`/`E702`).
176+
177+
### Documentation
178+
- Every public module, class, and function has a docstring (Pylint `C0111` / `missing-docstring`). Use `:param` / `:return:` / `:raises:` style already in use.
179+
- No misleading docstrings — update them when behavior changes.
180+
181+
### Enforcement Workflow
182+
- Before committing: run `pip install pylint flake8 bandit` and locally execute `pylint je_mail_thunder`, `flake8 je_mail_thunder`, `bandit -r je_mail_thunder`.
183+
- Treat any new SonarQube / Codacy finding on changed lines as a blocker. Do not suppress rules (`# noqa`, `# pylint: disable=`) without a comment explaining why and which specific rule is being suppressed.

‎je_mail_thunder/imap/imap_wrapper.py‎

Lines changed: 58 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import os
2+
import re
13
from email import message_from_bytes
24
from email import policy
35
from email.header import decode_header
@@ -33,24 +35,33 @@ def later_init(self):
3335
except Exception as error:
3436
mail_thunder_logger.error(f"imap_later_init, failed: {repr(error)}")
3537

38+
@staticmethod
39+
def _resolve_credentials():
40+
user_info = read_output_content()
41+
if isinstance(user_info, dict):
42+
user = user_info.get("user")
43+
password = user_info.get("password")
44+
if user is not None and password is not None:
45+
return user, password
46+
env_info = get_mail_thunder_os_environ()
47+
if isinstance(env_info, dict):
48+
user = env_info.get("mail_thunder_user")
49+
password = env_info.get("mail_thunder_user_password")
50+
if user is not None and password is not None:
51+
return user, password
52+
return None
53+
3654
def try_to_login_with_env_or_content(self):
3755
"""
3856
Try to find user and password on cwd /mail_thunder_content.json or env var
3957
:return: None
4058
"""
4159
mail_thunder_logger.info("imap_try_to_login_with_env_or_content")
4260
try:
43-
user_info = read_output_content()
44-
if user_info is not None and isinstance(user_info, dict):
45-
if user_info.get("user", None) is not None and user_info.get("password", None) is not None:
46-
self.login(user_info.get("user"), user_info.get("password"))
47-
else:
48-
user_info = get_mail_thunder_os_environ()
49-
if user_info is not None and isinstance(user_info, dict):
50-
if user_info.get("mail_thunder_user", None) is not None and user_info.get(
51-
"mail_thunder_user_password", None) is not None:
52-
self.login(user_info.get("mail_thunder_user"), user_info.get("mail_thunder_user_password"))
53-
except Exception as error:
61+
credentials = self._resolve_credentials()
62+
if credentials is not None:
63+
self.login(*credentials)
64+
except OSError as error:
5465
mail_thunder_logger.info(
5566
f"imap_try_to_login_with_env_or_content, "
5667
f"failed: {repr(error) + ' ' + mail_thunder_content_login_failed}")
@@ -64,7 +75,7 @@ def select_mailbox(self, mailbox: str = "INBOX", readonly: bool = False):
6475
mail_thunder_logger.info(f"imap_select_mailbox, mailbox: {mailbox}, readonly: {readonly}")
6576
try:
6677
select_status = self.select(mailbox=mailbox, readonly=readonly)
67-
return True if select_status[0] == "OK" else False
78+
return select_status[0] == "OK"
6879
except Exception as error:
6980
mail_thunder_logger.error(
7081
f"imap_select_mailbox, mailbox: {mailbox}, readonly: {readonly}, failed: {repr(error)}")
@@ -93,13 +104,13 @@ def search_mailbox(self, search_str: [str, list] = "ALL", charset: str = None) -
93104

94105
def mail_content_list(
95106
self, search_str: [str, list] = "ALL", charset: str = None) -> List[Dict[str, Union[str, bytes]]]:
96-
mail_thunder_logger.info(f"imap_mail_content_list, search_str: {search_str}, charset: {charset}")
97107
"""
98108
Get all mail content as list
99109
:param search_str: Search pattern
100-
:param charset: Charset pattern
110+
:param charset: Charset pattern
101111
:return: All mail content as list [{"SUBJECT": "mail_subject", "FROM": "mail_from", "TO": "mail_to"}]
102112
"""
113+
mail_thunder_logger.info(f"imap_mail_content_list, search_str: {search_str}, charset: {charset}")
103114
try:
104115
mail_list = self.search_mailbox(search_str, charset)
105116
mail_content_dict = dict()
@@ -124,39 +135,61 @@ def mail_content_list(
124135
mail_thunder_logger.error(
125136
f"imap_mail_content_list, search_str: {search_str}, charset: {charset}, failed: {repr(error)}")
126137

138+
@staticmethod
139+
def _sanitize_subject_as_filename(subject) -> str:
140+
"""
141+
Derive a safe filename from a mail SUBJECT header.
142+
Strips directory components and any separator / traversal token.
143+
Falls back to "mail" when the sanitized result is empty.
144+
"""
145+
if subject is None:
146+
return "mail"
147+
name = os.path.basename(str(subject))
148+
name = name.replace("\x00", "")
149+
name = re.sub(r"[\\/\r\n\t]", "_", name)
150+
while ".." in name:
151+
name = name.replace("..", "_")
152+
name = name.strip(" .")
153+
return name if name else "mail"
154+
127155
def output_all_mail_as_file(
128156
self, search_str: [str, list] = "ALL", charset: str = None) -> List[Dict[str, Union[str, bytes]]]:
129-
mail_thunder_logger.info(f"imap_mail_content_list, search_str: {search_str}, charset: {charset}")
130157
"""
131158
Get all mail content data and output as file
132159
:param search_str: Search pattern
133-
:param charset: Charset pattern
160+
:param charset: Charset pattern
134161
:return: All mail content as list [{"SUBJECT": "mail_subject", "FROM": "mail_from", "TO": "mail_to"}]
135162
"""
163+
mail_thunder_logger.info(f"imap_output_all_mail_as_file, search_str: {search_str}, charset: {charset}")
136164
try:
137165
all_mail = self.mail_content_list(search_str=search_str, charset=charset)
138166
same_name_dict: Dict[str, int] = dict()
167+
cwd = os.path.abspath(os.getcwd())
139168
for mail in all_mail:
140-
if same_name_dict.get((mail.get("SUBJECT"))) is None:
141-
same_name_dict.update({mail.get("SUBJECT"): 0})
142-
else:
143-
same_name_dict.update({mail.get("SUBJECT"): same_name_dict.get(mail.get("SUBJECT")) + 1})
144-
with open(mail.get("SUBJECT") + str(same_name_dict.get(mail.get("SUBJECT"))), "w+") as file:
169+
safe_name = self._sanitize_subject_as_filename(mail.get("SUBJECT"))
170+
count = same_name_dict.get(safe_name, -1) + 1
171+
same_name_dict[safe_name] = count
172+
target_path = os.path.abspath(os.path.join(cwd, safe_name + str(count)))
173+
if os.path.commonpath([cwd, target_path]) != cwd:
174+
mail_thunder_logger.error(
175+
f"imap_output_all_mail_as_file, rejected path traversal: {target_path}")
176+
continue
177+
with open(target_path, "w+") as file:
145178
if isinstance(mail.get("BODY"), bytes):
146179
file.write(mail.get("BODY").decode("utf-8"))
147180
else:
148181
file.write(mail.get("BODY"))
149182
return all_mail
150183
except Exception as error:
151184
mail_thunder_logger.error(
152-
f"imap_mail_content_list, search_str: {search_str}, charset: {charset}, failed: {repr(error)}")
185+
f"imap_output_all_mail_as_file, search_str: {search_str}, charset: {charset}, failed: {repr(error)}")
153186

154187
def quit(self):
155188
"""
156189
Quit service and close connect
157190
:return: None
158191
"""
159-
mail_thunder_logger.info(f"MT_imap_quit")
192+
mail_thunder_logger.info("MT_imap_quit")
160193
try:
161194
self.close()
162195
self.logout()
@@ -166,5 +199,6 @@ def quit(self):
166199

167200
try:
168201
imap_instance = IMAPWrapper()
169-
except Exception:
202+
except OSError as _imap_init_error:
203+
mail_thunder_logger.error(f"imap_instance init failed: {repr(_imap_init_error)}")
170204
imap_instance = None

0 commit comments

Comments
 (0)