Skip to content

Commit da177d5

Browse files
MaxLogicMaxLogic
committed
update README.md
1 parent 3909b57 commit da177d5

1 file changed

Lines changed: 146 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 146 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,147 @@
11
# EncodingFixTool
2-
encoding fix tool tailored to delphi pas files
2+
3+
A fast, parallel, command-line fixer for Delphi source file encodings.
4+
It scans `.pas`, `.dpr`, …; detects UTF-8/ASCII vs. legacy single-byte encodings; repairs mixed lines (Windows-1250/1252/ANSI), and writes clean UTF-8 with an optional BOM — without breaking your line endings or folder structure.
5+
6+
---
7+
8+
## Why?
9+
10+
* Old codebases often mix ANSI and UTF-8 (sometimes within a single file).
11+
* Delphi 12+ works best when sources are in UTF-8 (usually with BOM).
12+
* Manual converting is slow and risky. This tool automates it safely and fast.
13+
14+
---
15+
16+
## Quick start
17+
18+
```
19+
EncodingFixTool [params]
20+
```
21+
22+
Examples:
23+
24+
```bat
25+
:: Dry run, current folder, defaults (.pas,.dpr, recursive, BOM=yes)
26+
EncodingFixTool dry
27+
28+
:: Fix a project tree verbosely and back it up before changes
29+
EncodingFixTool path=C:\Projs\MyApp v bkp-dir=C:\backup\myapp
30+
31+
:: Only scan .pas files in ./src (not recursive), remove BOM if present
32+
EncodingFixTool path=.\src recursive=n ext=pas utf8-bom=n
33+
34+
:: Multiple ext forms are OK; quoted lists work
35+
EncodingFixTool ext="*.pas,*.dpr, .dfm"
36+
```
37+
38+
> Tip: Parameters accept `key=value` or `key:value`. You can prefix flags with `-` (and `/` on Windows), e.g. `-v`, `-dry`.
39+
40+
---
41+
42+
## Parameters
43+
44+
| Param | Aliases | Values | Default | Meaning |
45+
| ------------- | ------- | -------------------------- | ----------- | ---------------------------------------------------------------------------------------------- |
46+
| `help` | `-h` | — | — | Prints extended help and exits. |
47+
| `dry` | — | — | off | Dry run: analyze and report what **would** change; no writes. |
48+
| `s`/`silent` | — | — | off | No console output. (Overrides `verbose`.) |
49+
| `v`/`verbose` | — | — | off | More output: “OK” lines etc. (Ignored if `silent`.) |
50+
| `path` | — | dir | current dir | Directory to scan. |
51+
| `recursive` | — | `y`/`n`/`yes`/`no`/`1`/`0` | `y` | Recurse into subfolders. |
52+
| `ext` | — | CSV list | `pas,dpr` | File extensions to include. Smart parsing: accepts `pas`, `.pas`, `*.pas`. Quoted lists OK. |
53+
| `utf8-bom` | — | `y`/`n` | `y` | Whether to **save with** UTF-8 BOM. **Pure US-ASCII files are always left without a BOM**. |
54+
| `bkp-dir` | — | dir | empty | If set, backs up every file **before** overwriting, preserving the relative path below `path`. |
55+
56+
### Backup path example
57+
58+
If `path=C:\tmp\` and `bkp-dir=C:\bkp` and a processed file is
59+
`C:\tmp\src\foo\bar\Main.pas` → backup is written to:
60+
61+
```
62+
C:\bkp\src\foo\bar\Main.pas
63+
```
64+
65+
---
66+
67+
## What it does (algorithm)
68+
69+
1. **Gather files**
70+
Walk the `path` (recursively by default), matching the configured extensions.
71+
72+
2. **Process in parallel**
73+
Uses `TParallel.For` to utilize multiple cores. Console output is synchronized, and the “files changed” counter is atomic.
74+
75+
3. **Detect encoding per file** (fast pre-check)
76+
77+
* Uses `System.WideStrUtils.DetectUTF8Encoding` on the raw bytes.
78+
* If **US-ASCII** → file is left as-is (never add a BOM).
79+
* If **UTF-8** → ensure BOM matches `utf8-bom` option; adjust if needed (optionally backing up first).
80+
* Else (treated as **ANSI / unknown**) → go to step 4.
81+
82+
4. **Repair mixed/legacy encodings per line**
83+
84+
* Split the original bytes by raw CR/LF/CRLF (no decoding yet).
85+
* For each line, try:
86+
87+
* **UTF-8 (strict)** — if it round-trips, use it.
88+
* **Windows-1250** (Central Europe/PL), **Windows-1252** (Western/DE), and **ANSI** — decode and **score**:
89+
90+
* +2 for valid Polish/German diacritics,
91+
* +1 for typical source characters (letters/digits/whitespace/common punctuation),
92+
* −2 for control chars (except tab),
93+
* −1 for U+FFFD replacements.
94+
* Pick the **highest-scoring** decode for that line.
95+
* Reassemble the file:
96+
97+
* Preserve the **dominant original EOL style** (CRLF/LF/CR).
98+
* Preserve whether the file ended **with a trailing EOL**.
99+
* Save as **UTF-8** (BOM per `utf8-bom`), optionally to backup first.
100+
101+
5. **Summary**
102+
At the end prints elapsed time and number of files changed.
103+
104+
---
105+
106+
## Output examples
107+
108+
Dry run:
109+
110+
```
111+
Would fix: src\Utils\StrTools.pas (mixed bytes; would save UTF-8 (BOM=Y, EOL=CRLF) (dry-run))
112+
OK : src\Main.dpr (UTF-8 OK)
113+
Done in 00:12.384. Files changed: 0
114+
```
115+
116+
Actual run:
117+
118+
```
119+
Fixed: src\Utils\StrTools.pas (detected Windows-1250; saved UTF-8 (BOM=Y, EOL=CRLF))
120+
Fixed: src\Forms\About.pas (detected UTF-8; Added UTF-8 BOM)
121+
Done in 00:08.972. Files changed: 2
122+
```
123+
124+
> When `silent` is enabled, only the summary is suppressed too. In dry-run mode, change messages are phrased as “Would fix”.
125+
126+
---
127+
128+
## Behavior details
129+
130+
* **BOM policy**
131+
132+
* Default is **BOM = yes** (Delphi IDE/compiler are happiest this way).
133+
* **US-ASCII** files are **never** given a BOM, even if `utf8-bom=y`.
134+
* Existing UTF-8 files are re-saved only if the BOM policy differs.
135+
136+
* **Line endings**
137+
Dominant EOL (CRLF/LF/CR) is detected from raw bytes and **preserved**; trailing newline presence is preserved.
138+
139+
* **Relative reporting**
140+
Paths in logs are shown **relative to** the scanned `path`, for readability.
141+
142+
---
143+
144+
## Exit codes
145+
146+
* `0` – completed (even if some files failed; failures are printed).
147+
* `2` – the `path` argument didn’t exist.

0 commit comments

Comments
 (0)