Skip to content

feat: Add token decoding for multilingual Whisper #139

Closed
chmjkb wants to merge 38 commits intov0.4.0-rc1from
@chmjkb/multilingual-whisper-tokenizer-fix
Closed

feat: Add token decoding for multilingual Whisper #139
chmjkb wants to merge 38 commits intov0.4.0-rc1from
@chmjkb/multilingual-whisper-tokenizer-fix

Conversation

@chmjkb
Copy link
Copy Markdown
Collaborator

@chmjkb chmjkb commented Mar 18, 2025

Description

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update (improves or adds clarity to existing documentation)

Tested on

  • iOS
  • Android

Testing instructions

Screenshots

Related issues

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

@chmjkb chmjkb marked this pull request as draft March 18, 2025 13:37
mkopcins and others added 29 commits March 24, 2025 14:39
## Description

<!-- Provide a concise and descriptive summary of the changes
implemented in this PR. -->

### Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update (improves or adds clarity to existing
documentation)

### Tested on

- [ ] iOS
- [ ] Android

### Testing instructions

<!-- Provide step-by-step instructions on how to test your changes.
Include setup details if necessary. -->

### Screenshots

<!-- Add screenshots here, if applicable -->

### Related issues

<!-- Link related issues here using #issue-number -->

### Checklist

- [ ] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have updated the documentation accordingly
- [ ] My changes generate no new warnings

### Additional notes

<!-- Include any additional information, assumptions, or context that
reviewers might need to understand this PR. -->

Co-authored-by: Mateusz Kopciński <mateusz.kopcinski@swmansnion.com>
## Description

Bump version in all example apps to 0.3.2

### Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update (improves or adds clarity to existing
documentation)

### Tested on

- [x] iOS
- [x] Android

### Testing instructions

<!-- Provide step-by-step instructions on how to test your changes.
Include setup details if necessary. -->

### Screenshots

<!-- Add screenshots here, if applicable -->

### Related issues

<!-- Link related issues here using #issue-number -->

### Checklist

- [ ] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have updated the documentation accordingly
- [ ] My changes generate no new warnings

### Additional notes

<!-- Include any additional information, assumptions, or context that
reviewers might need to understand this PR. -->

Co-authored-by: Mateusz Kopciński <mateusz.kopcinski@swmansnion.com>
## Description

<!-- Provide a concise and descriptive summary of the changes
implemented in this PR. -->

### Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [x] Documentation update (improves or adds clarity to existing
documentation)

### Tested on

- [ ] iOS
- [ ] Android

### Testing instructions

<!-- Provide step-by-step instructions on how to test your changes.
Include setup details if necessary. -->

### Screenshots

<!-- Add screenshots here, if applicable -->

### Related issues

<!-- Link related issues here using #issue-number -->

### Checklist

- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have updated the documentation accordingly
- [ ] My changes generate no new warnings

### Additional notes

<!-- Include any additional information, assumptions, or context that
reviewers might need to understand this PR. -->

Co-authored-by: Maciej Rys <maciej.rys@swmansioncom>
## Description

<!-- Provide a concise and descriptive summary of the changes
implemented in this PR. -->

### Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update (improves or adds clarity to existing
documentation)

### Tested on

- [ ] iOS
- [ ] Android

### Testing instructions

<!-- Provide step-by-step instructions on how to test your changes.
Include setup details if necessary. -->

### Screenshots

<!-- Add screenshots here, if applicable -->

### Related issues

<!-- Link related issues here using #issue-number -->

### Checklist

- [ ] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have updated the documentation accordingly
- [ ] My changes generate no new warnings

### Additional notes

<!-- Include any additional information, assumptions, or context that
reviewers might need to understand this PR. -->
@chmjkb chmjkb force-pushed the @chmjkb/multilingual-whisper-tokenizer-fix branch from e66f7c5 to a68a97f Compare March 31, 2025 08:08
@chmjkb chmjkb changed the base branch from main to v0.4.0-rc1 March 31, 2025 08:08
@chmjkb chmjkb changed the base branch from v0.4.0-rc1 to @nk/skip-special-tokens March 31, 2025 08:08
@chmjkb chmjkb changed the base branch from @nk/skip-special-tokens to v0.4.0-rc1 March 31, 2025 08:11
@chmjkb chmjkb marked this pull request as ready for review March 31, 2025 09:18
@chmjkb chmjkb requested a review from mkopcins March 31, 2025 09:18
@chmjkb chmjkb closed this Mar 31, 2025
@mkopcins mkopcins deleted the @chmjkb/multilingual-whisper-tokenizer-fix branch October 14, 2025 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants