Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion test_unstructured/cleaners/test_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ def test_clean_ordered_bullets(text, expected):
("She had a flower in her hair.", "She had a flower in her hair."),
("The coffin was placed in the grave.", "The coffin was placed in the grave."),
("The buffle zone was clearly marked.", "The buffle zone was clearly marked."),
("The craſtsman worked with dedication.", "The craftsman worked with dedication."),
("The laſt chapter was the best.", "The last chapter was the best."),
("The symbol ʪ is very rare.", "The symbol ls is very rare."),
("The word 'cœur' means 'heart' in French.", "The word 'coeur' means 'heart' in French."),
("The word 'Œuvre' refers to the works", "The word 'OEuvre' refers to the works"),
Expand Down
2 changes: 1 addition & 1 deletion unstructured/cleaners/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ def clean_ligatures(text) -> str:
"fl": "fl",
"ffi": "ffi",
"ffl": "ffl",
"ſt": "ft",
"ſt": "st", # U+FB05 LATIN SMALL LIGATURE LONG S T -> "st" (per Unicode NFKC)
"ʪ": "ls",
"œ": "oe",
"Œ": "OE",
Expand Down