Skip to content

Commit 433ade9

Browse files
Add Truncate methods (#13)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 73af5d8 commit 433ade9

9 files changed

Lines changed: 656 additions & 48 deletions

File tree

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
.DS_Store
2+
*.out

README.md

Lines changed: 26 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -133,33 +133,39 @@ goarch: arm64
133133
pkg: github.com/clipperhouse/displaywidth/comparison
134134
cpu: Apple M2
135135
136-
BenchmarkString_Mixed/clipperhouse/displaywidth-8 10326 ns/op 163.37 MB/s 0 B/op 0 allocs/op
137-
BenchmarkString_Mixed/mattn/go-runewidth-8 14415 ns/op 117.03 MB/s 0 B/op 0 allocs/op
138-
BenchmarkString_Mixed/rivo/uniseg-8 19343 ns/op 87.21 MB/s 0 B/op 0 allocs/op
136+
BenchmarkString_Mixed/clipperhouse/displaywidth-8 10400 ns/op 162.21 MB/s 0 B/op 0 allocs/op
137+
BenchmarkString_Mixed/mattn/go-runewidth-8 14296 ns/op 118.00 MB/s 0 B/op 0 allocs/op
138+
BenchmarkString_Mixed/rivo/uniseg-8 19770 ns/op 85.33 MB/s 0 B/op 0 allocs/op
139139
140-
BenchmarkString_EastAsian/clipperhouse/displaywidth-8 10561 ns/op 159.74 MB/s 0 B/op 0 allocs/op
141-
BenchmarkString_EastAsian/mattn/go-runewidth-8 23790 ns/op 70.91 MB/s 0 B/op 0 allocs/op
142-
BenchmarkString_EastAsian/rivo/uniseg-8 19322 ns/op 87.31 MB/s 0 B/op 0 allocs/op
140+
BenchmarkString_EastAsian/clipperhouse/displaywidth-8 10593 ns/op 159.26 MB/s 0 B/op 0 allocs/op
141+
BenchmarkString_EastAsian/mattn/go-runewidth-8 23980 ns/op 70.35 MB/s 0 B/op 0 allocs/op
142+
BenchmarkString_EastAsian/rivo/uniseg-8 19777 ns/op 85.30 MB/s 0 B/op 0 allocs/op
143143
144-
BenchmarkString_ASCII/clipperhouse/displaywidth-8 1033 ns/op 123.88 MB/s 0 B/op 0 allocs/op
145-
BenchmarkString_ASCII/mattn/go-runewidth-8 1168 ns/op 109.59 MB/s 0 B/op 0 allocs/op
146-
BenchmarkString_ASCII/rivo/uniseg-8 1585 ns/op 80.74 MB/s 0 B/op 0 allocs/op
144+
BenchmarkString_ASCII/clipperhouse/displaywidth-8 1032 ns/op 124.09 MB/s 0 B/op 0 allocs/op
145+
BenchmarkString_ASCII/mattn/go-runewidth-8 1162 ns/op 110.16 MB/s 0 B/op 0 allocs/op
146+
BenchmarkString_ASCII/rivo/uniseg-8 1586 ns/op 80.69 MB/s 0 B/op 0 allocs/op
147147
148-
BenchmarkString_Emoji/clipperhouse/displaywidth-8 3034 ns/op 238.61 MB/s 0 B/op 0 allocs/op
149-
BenchmarkString_Emoji/mattn/go-runewidth-8 4797 ns/op 150.94 MB/s 0 B/op 0 allocs/op
150-
BenchmarkString_Emoji/rivo/uniseg-8 6612 ns/op 109.50 MB/s 0 B/op 0 allocs/op
148+
BenchmarkString_Emoji/clipperhouse/displaywidth-8 3017 ns/op 240.01 MB/s 0 B/op 0 allocs/op
149+
BenchmarkString_Emoji/mattn/go-runewidth-8 4745 ns/op 152.58 MB/s 0 B/op 0 allocs/op
150+
BenchmarkString_Emoji/rivo/uniseg-8 6745 ns/op 107.34 MB/s 0 B/op 0 allocs/op
151151
152-
BenchmarkRune_Mixed/clipperhouse/displaywidth-8 3343 ns/op 504.67 MB/s 0 B/op 0 allocs/op
153-
BenchmarkRune_Mixed/mattn/go-runewidth-8 5414 ns/op 311.62 MB/s 0 B/op 0 allocs/op
152+
BenchmarkRune_Mixed/clipperhouse/displaywidth-8 3381 ns/op 498.90 MB/s 0 B/op 0 allocs/op
153+
BenchmarkRune_Mixed/mattn/go-runewidth-8 5383 ns/op 313.41 MB/s 0 B/op 0 allocs/op
154154
155-
BenchmarkRune_EastAsian/clipperhouse/displaywidth-8 3393 ns/op 497.17 MB/s 0 B/op 0 allocs/op
156-
BenchmarkRune_EastAsian/mattn/go-runewidth-8 15312 ns/op 110.17 MB/s 0 B/op 0 allocs/op
155+
BenchmarkRune_EastAsian/clipperhouse/displaywidth-8 3395 ns/op 496.96 MB/s 0 B/op 0 allocs/op
156+
BenchmarkRune_EastAsian/mattn/go-runewidth-8 15645 ns/op 107.83 MB/s 0 B/op 0 allocs/op
157157
158-
BenchmarkRune_ASCII/clipperhouse/displaywidth-8 256.9 ns/op 498.32 MB/s 0 B/op 0 allocs/op
159-
BenchmarkRune_ASCII/mattn/go-runewidth-8 265.7 ns/op 481.75 MB/s 0 B/op 0 allocs/op
158+
BenchmarkRune_ASCII/clipperhouse/displaywidth-8 257.8 ns/op 496.57 MB/s 0 B/op 0 allocs/op
159+
BenchmarkRune_ASCII/mattn/go-runewidth-8 267.3 ns/op 478.89 MB/s 0 B/op 0 allocs/op
160160
161-
BenchmarkRune_Emoji/clipperhouse/displaywidth-8 1336 ns/op 541.96 MB/s 0 B/op 0 allocs/op
162-
BenchmarkRune_Emoji/mattn/go-runewidth-8 2304 ns/op 314.23 MB/s 0 B/op 0 allocs/op
161+
BenchmarkRune_Emoji/clipperhouse/displaywidth-8 1338 ns/op 541.24 MB/s 0 B/op 0 allocs/op
162+
BenchmarkRune_Emoji/mattn/go-runewidth-8 2287 ns/op 316.58 MB/s 0 B/op 0 allocs/op
163+
164+
BenchmarkTruncateWithTail/clipperhouse/displaywidth-8 3689 ns/op 47.98 MB/s 192 B/op 14 allocs/op
165+
BenchmarkTruncateWithTail/mattn/go-runewidth-8 8069 ns/op 21.93 MB/s 192 B/op 14 allocs/op
166+
167+
BenchmarkTruncateWithoutTail/clipperhouse/displaywidth-8 3457 ns/op 66.24 MB/s 0 B/op 0 allocs/op
168+
BenchmarkTruncateWithoutTail/mattn/go-runewidth-8 10441 ns/op 21.93 MB/s 0 B/op 0 allocs/op
163169
```
164170

165171
Here are some notes on [how to make Unicode things fast](https://clipperhouse.com/go-unicode/).

comparison/COMPATIBILITY_ANALYSIS.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,20 @@ Regional indicator pairs (flags like 🇺🇸) are composed of two Regional Indi
4646
- go-runewidth: 3 columns (1+1+1)
4747
- uniseg: 6 columns (2+2+2)
4848

49+
I (@clipperhouse) believe that 2 is the correct width, they are emojis. Ghostty and iTerm display
50+
regional flags as width 2, as does VS Code. Mac Terminal (Tahoe macOS 26) displays them as 1. Sigh.
51+
52+
To repro in your terminal of choice:
53+
54+
```
55+
echo "🇺🇸🇯🇵🇬🇧abc\n123456"
56+
```
57+
58+
I have considered detecting the terminal (like a user agent) and using width 1 for
59+
Mac Terminal as a special case. I kinda hate that, because if Apple corrects it, then
60+
the behavior changes. OTOH, I assume Mac Terminal is the most popular terminal and
61+
so it might be better for end-users.
62+
4963
## Variation Selectors
5064

5165
VS15 and VS16 from [Unicode TR51](https://unicode.org/reports/tr51/#Emoji_Variation_Sequences)

comparison/README.md

Lines changed: 26 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -20,31 +20,37 @@ goarch: arm64
2020
pkg: github.com/clipperhouse/displaywidth/comparison
2121
cpu: Apple M2
2222
23-
BenchmarkString_Mixed/clipperhouse/displaywidth-8 10326 ns/op 163.37 MB/s 0 B/op 0 allocs/op
24-
BenchmarkString_Mixed/mattn/go-runewidth-8 14415 ns/op 117.03 MB/s 0 B/op 0 allocs/op
25-
BenchmarkString_Mixed/rivo/uniseg-8 19343 ns/op 87.21 MB/s 0 B/op 0 allocs/op
23+
BenchmarkString_Mixed/clipperhouse/displaywidth-8 10400 ns/op 162.21 MB/s 0 B/op 0 allocs/op
24+
BenchmarkString_Mixed/mattn/go-runewidth-8 14296 ns/op 118.00 MB/s 0 B/op 0 allocs/op
25+
BenchmarkString_Mixed/rivo/uniseg-8 19770 ns/op 85.33 MB/s 0 B/op 0 allocs/op
2626
27-
BenchmarkString_EastAsian/clipperhouse/displaywidth-8 10561 ns/op 159.74 MB/s 0 B/op 0 allocs/op
28-
BenchmarkString_EastAsian/mattn/go-runewidth-8 23790 ns/op 70.91 MB/s 0 B/op 0 allocs/op
29-
BenchmarkString_EastAsian/rivo/uniseg-8 19322 ns/op 87.31 MB/s 0 B/op 0 allocs/op
27+
BenchmarkString_EastAsian/clipperhouse/displaywidth-8 10593 ns/op 159.26 MB/s 0 B/op 0 allocs/op
28+
BenchmarkString_EastAsian/mattn/go-runewidth-8 23980 ns/op 70.35 MB/s 0 B/op 0 allocs/op
29+
BenchmarkString_EastAsian/rivo/uniseg-8 19777 ns/op 85.30 MB/s 0 B/op 0 allocs/op
3030
31-
BenchmarkString_ASCII/clipperhouse/displaywidth-8 1033 ns/op 123.88 MB/s 0 B/op 0 allocs/op
32-
BenchmarkString_ASCII/mattn/go-runewidth-8 1168 ns/op 109.59 MB/s 0 B/op 0 allocs/op
33-
BenchmarkString_ASCII/rivo/uniseg-8 1585 ns/op 80.74 MB/s 0 B/op 0 allocs/op
31+
BenchmarkString_ASCII/clipperhouse/displaywidth-8 1032 ns/op 124.09 MB/s 0 B/op 0 allocs/op
32+
BenchmarkString_ASCII/mattn/go-runewidth-8 1162 ns/op 110.16 MB/s 0 B/op 0 allocs/op
33+
BenchmarkString_ASCII/rivo/uniseg-8 1586 ns/op 80.69 MB/s 0 B/op 0 allocs/op
3434
35-
BenchmarkString_Emoji/clipperhouse/displaywidth-8 3034 ns/op 238.61 MB/s 0 B/op 0 allocs/op
36-
BenchmarkString_Emoji/mattn/go-runewidth-8 4797 ns/op 150.94 MB/s 0 B/op 0 allocs/op
37-
BenchmarkString_Emoji/rivo/uniseg-8 6612 ns/op 109.50 MB/s 0 B/op 0 allocs/op
35+
BenchmarkString_Emoji/clipperhouse/displaywidth-8 3017 ns/op 240.01 MB/s 0 B/op 0 allocs/op
36+
BenchmarkString_Emoji/mattn/go-runewidth-8 4745 ns/op 152.58 MB/s 0 B/op 0 allocs/op
37+
BenchmarkString_Emoji/rivo/uniseg-8 6745 ns/op 107.34 MB/s 0 B/op 0 allocs/op
3838
39-
BenchmarkRune_Mixed/clipperhouse/displaywidth-8 3343 ns/op 504.67 MB/s 0 B/op 0 allocs/op
40-
BenchmarkRune_Mixed/mattn/go-runewidth-8 5414 ns/op 311.62 MB/s 0 B/op 0 allocs/op
39+
BenchmarkRune_Mixed/clipperhouse/displaywidth-8 3381 ns/op 498.90 MB/s 0 B/op 0 allocs/op
40+
BenchmarkRune_Mixed/mattn/go-runewidth-8 5383 ns/op 313.41 MB/s 0 B/op 0 allocs/op
4141
42-
BenchmarkRune_EastAsian/clipperhouse/displaywidth-8 3393 ns/op 497.17 MB/s 0 B/op 0 allocs/op
43-
BenchmarkRune_EastAsian/mattn/go-runewidth-8 15312 ns/op 110.17 MB/s 0 B/op 0 allocs/op
42+
BenchmarkRune_EastAsian/clipperhouse/displaywidth-8 3395 ns/op 496.96 MB/s 0 B/op 0 allocs/op
43+
BenchmarkRune_EastAsian/mattn/go-runewidth-8 15645 ns/op 107.83 MB/s 0 B/op 0 allocs/op
4444
45-
BenchmarkRune_ASCII/clipperhouse/displaywidth-8 256.9 ns/op 498.32 MB/s 0 B/op 0 allocs/op
46-
BenchmarkRune_ASCII/mattn/go-runewidth-8 265.7 ns/op 481.75 MB/s 0 B/op 0 allocs/op
45+
BenchmarkRune_ASCII/clipperhouse/displaywidth-8 257.8 ns/op 496.57 MB/s 0 B/op 0 allocs/op
46+
BenchmarkRune_ASCII/mattn/go-runewidth-8 267.3 ns/op 478.89 MB/s 0 B/op 0 allocs/op
4747
48-
BenchmarkRune_Emoji/clipperhouse/displaywidth-8 1336 ns/op 541.96 MB/s 0 B/op 0 allocs/op
49-
BenchmarkRune_Emoji/mattn/go-runewidth-8 2304 ns/op 314.23 MB/s 0 B/op 0 allocs/op
48+
BenchmarkRune_Emoji/clipperhouse/displaywidth-8 1338 ns/op 541.24 MB/s 0 B/op 0 allocs/op
49+
BenchmarkRune_Emoji/mattn/go-runewidth-8 2287 ns/op 316.58 MB/s 0 B/op 0 allocs/op
50+
51+
BenchmarkTruncateWithTail/clipperhouse/displaywidth-8 3689 ns/op 47.98 MB/s 192 B/op 14 allocs/op
52+
BenchmarkTruncateWithTail/mattn/go-runewidth-8 8069 ns/op 21.93 MB/s 192 B/op 14 allocs/op
53+
54+
BenchmarkTruncateWithoutTail/clipperhouse/displaywidth-8 3457 ns/op 66.24 MB/s 0 B/op 0 allocs/op
55+
BenchmarkTruncateWithoutTail/mattn/go-runewidth-8 10441 ns/op 21.93 MB/s 0 B/op 0 allocs/op
5056
```

comparison/behavior_test.go

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,9 @@ func TestLibraryBehaviorComparison(t *testing.T) {
7878
},
7979

8080
// Regional Indicator Pairs (flags) - the key difference
81+
// TODO: 2 is the correct width, that's what Ghostty and iTerm do.
82+
// Sadly, Mac Terminal displays width 1. Perhaps we should special-case
83+
// it, not sure.
8184
{
8285
name: "Flags",
8386
input: "🇺🇸🇯🇵🇬🇧",
@@ -255,3 +258,153 @@ func TestFlagBehaviorDetailed(t *testing.T) {
255258
t.Logf("%s | %d | %d | %d | %d | %d", flag, displaywidthDefault, goRunewidthDefault, goRunewidthStrictFalse, goRunewidthStrictTrue, unisegDefault)
256259
}
257260
}
261+
262+
func TestTruncateComparison(t *testing.T) {
263+
testCases := []struct {
264+
name string
265+
input string
266+
maxWidth int
267+
tail string
268+
}{
269+
{
270+
name: "ASCII truncation",
271+
input: "Hello World",
272+
maxWidth: 5,
273+
tail: "...",
274+
},
275+
{
276+
name: "CJK truncation",
277+
input: "中文测试",
278+
maxWidth: 4,
279+
tail: "...",
280+
},
281+
{
282+
name: "Emoji truncation",
283+
input: "😀🚀🎉",
284+
maxWidth: 4,
285+
tail: "...",
286+
},
287+
{
288+
name: "Flags truncation",
289+
input: "🇺🇸🇯🇵🇬🇧", // known difference
290+
maxWidth: 4,
291+
tail: "...",
292+
},
293+
{
294+
name: "Mixed content truncation",
295+
input: "Hello 世界! 😀🇺🇸",
296+
maxWidth: 10,
297+
tail: "...",
298+
},
299+
{
300+
name: "No truncation needed",
301+
input: "Hi",
302+
maxWidth: 10,
303+
tail: "...",
304+
},
305+
{
306+
name: "Empty tail",
307+
input: "Hello World",
308+
maxWidth: 5,
309+
tail: "",
310+
},
311+
{
312+
name: "Width exactly equal to string width",
313+
input: "Hello",
314+
maxWidth: 5,
315+
tail: "...",
316+
},
317+
{
318+
name: "Width exactly equal to string width with emoji",
319+
input: "😀🚀",
320+
maxWidth: 4,
321+
tail: "...",
322+
},
323+
{
324+
name: "Width exactly equal to string width with CJK",
325+
input: "中文",
326+
maxWidth: 4,
327+
tail: "...",
328+
},
329+
{
330+
name: "MaxWidth is 0",
331+
input: "Hello",
332+
maxWidth: 0,
333+
tail: "...",
334+
},
335+
{
336+
name: "MaxWidth is 1",
337+
input: "Hello",
338+
maxWidth: 1,
339+
tail: "...",
340+
},
341+
{
342+
name: "MaxWidth is 2",
343+
input: "Hello",
344+
maxWidth: 2,
345+
tail: "...",
346+
},
347+
{
348+
name: "Empty string input",
349+
input: "",
350+
maxWidth: 5,
351+
tail: "...",
352+
},
353+
{
354+
name: "Tail wider than maxWidth",
355+
input: "Hello",
356+
maxWidth: 2,
357+
tail: "中文", // width 4, wider than maxWidth
358+
},
359+
{
360+
name: "Tail with emoji",
361+
input: "Hello",
362+
maxWidth: 5,
363+
tail: "😀",
364+
},
365+
{
366+
name: "MaxWidth exactly equal to tail width",
367+
input: "Hello World",
368+
maxWidth: 3, // exactly width of "..."
369+
tail: "...",
370+
},
371+
{
372+
name: "Input with control characters",
373+
input: "hello\nworld",
374+
maxWidth: 8,
375+
tail: "...",
376+
},
377+
{
378+
name: "Single wide character truncation",
379+
input: "中",
380+
maxWidth: 1,
381+
tail: "...",
382+
},
383+
{
384+
name: "Single emoji truncation",
385+
input: "😀",
386+
maxWidth: 1,
387+
tail: "...",
388+
},
389+
}
390+
391+
for _, tc := range testCases {
392+
t.Run(tc.name, func(t *testing.T) {
393+
// displaywidth
394+
displaywidthResult := displaywidth.TruncateString(tc.input, tc.maxWidth, tc.tail)
395+
displaywidthWidth := displaywidth.String(displaywidthResult)
396+
397+
// go-runewidth
398+
goRunewidthResult := runewidth.Truncate(tc.input, tc.maxWidth, tc.tail)
399+
goRunewidthWidth := runewidth.StringWidth(goRunewidthResult)
400+
401+
if displaywidthWidth != goRunewidthWidth {
402+
t.Logf("displaywidth and go-runewidth results differ for %q: %d != %d", tc.input, displaywidthWidth, goRunewidthWidth)
403+
}
404+
405+
if displaywidthResult != goRunewidthResult {
406+
t.Logf("displaywidth and go-runewidth results differ for %s : %s != %s", tc.input, displaywidthResult, goRunewidthResult)
407+
}
408+
})
409+
}
410+
}

0 commit comments

Comments
 (0)