Skip to content

fix: guard han_solo list_cut[-1] and narrow longest.py exception catch - #1382

Merged
bact merged 2 commits into
PyThaiNLP:devfrom
phoneee:fix/tokenize-defensive-guards
Mar 30, 2026
Merged

fix: guard han_solo list_cut[-1] and narrow longest.py exception catch#1382
bact merged 2 commits into
PyThaiNLP:devfrom
phoneee:fix/tokenize-defensive-guards

Conversation

@phoneee

@phoneee phoneee commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

What do these changes do

  • Guard list_cut[-1] in han_solo when list is empty

  • Narrow except BaseException to except IndexError in longest

  • Passed code styles and structures

  • Passed code linting checks and unit test

han_solo: if CRF predicts "0" for first char, list_cut is empty and
list_cut[-1] raises IndexError. Add fallback to append when empty.

longest: except BaseException catches KeyboardInterrupt and SystemExit.
Narrow to except IndexError which is the only expected exception.
@bact bact added this to PyThaiNLP Mar 29, 2026
@bact bact added the bug bugs in the library label Mar 29, 2026

@bact bact left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bact

bact commented Mar 30, 2026

Copy link
Copy Markdown
Member

Btw, have to put type annotation for list_cut to pass mypy

Comment thread pythainlp/tokenize/han_solo.py Outdated

@bact bact left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the type annotation

@sonarqubecloud

Copy link
Copy Markdown

@bact
bact merged commit 9bc9bb5 into PyThaiNLP:dev Mar 30, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug bugs in the library

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants