add featherless provider (access to 7900+ open source models)#1138
Conversation
|
DescriptionSummary By MatterAI
🔄 What ChangedThis Pull Request introduces Featherless.ai as a new Large Language Model (LLM) provider.
🔍 Impact of the ChangeThis change significantly expands the application's LLM capabilities by integrating Featherless.ai, which provides access to over 7,900+ open-source models from Hugging Face. This enhances the flexibility and choice of models available for chat and text completion tasks within the application. The integration follows existing architectural patterns for adding new providers, ensuring maintainability and scalability. 📁 Total Files Changed4 files were changed in this pull request:
🧪 Test AddedNo specific tests were mentioned or added in the provided patch. It is highly recommended to add unit and integration tests to verify the correct functionality of the new Featherless.ai provider, including API call correctness, response parsing, and robust error handling for various scenarios. 🔒Security VulnerabilitiesNone detected in the provided patch. The API key handling via a Bearer token is a standard and secure practice. MotivationAdds Featherless.ai as an LLM provider giving access to over 7,900+ models and counting on Hugging Face Type of Change
How Has This Been Tested?
Screenshots (if applicable)N/A Checklist
Related Issues. Tip Quality Recommendations
Sequence DiagramsequenceDiagram
participant App as Application
participant ProvidersModule as src/providers/index.ts
participant FeatherlessAIConfigModule as src/providers/featherless-ai/index.ts
participant FeatherlessAIAPI as src/providers/featherless-ai/api.ts
participant FeatherlessAIService as Featherless.ai API
App->>ProvidersModule: Request LLM service (e.g., chatComplete, complete)
ProvidersModule->>FeatherlessAIConfigModule: Route request to FeatherlessAIConfig
FeatherlessAIConfigModule->>FeatherlessAIAPI: Call API configuration (featherlessAIAPIConfig)
FeatherlessAIAPI->>FeatherlessAIAPI: Get Base URL (https://api.featherless.ai/v1)
FeatherlessAIAPI->>FeatherlessAIAPI: Get Headers (Authorization: Bearer apiKey)
FeatherlessAIAPI->>FeatherlessAIAPI: Get Endpoint (fn: chatComplete -> /chat/completions; fn: complete -> /completions)
FeatherlessAIAPI->>FeatherlessAIService: HTTP Request (POST /chat/completions or /completions)
FeatherlessAIService-->>FeatherlessAIAPI: API Response
FeatherlessAIAPI-->>FeatherlessAIConfigModule: Processed API Response
FeatherlessAIConfigModule-->>ProvidersModule: Transformed Response
ProvidersModule-->>App: Final LLM Response
|
![matterai-app[bot]](https://avatars.githubusercontent.com/in/474549?s=60&v=4){kind=small}
| return ''; | ||
| } | ||
| }, | ||
| }; No newline at end of file |
There was a problem hiding this comment.
🛠️ Code Refactor
Issue: The API implementation is missing a newline at the end of the file.
Fix: Add a newline at the end of the file to follow best practices.
Impact: Improves code consistency and prevents potential issues with some tools that expect files to end with a newline.
| }; | |
| }; | |
| chatComplete: true, | ||
| complete: true, | ||
| }), | ||
| }; No newline at end of file |
There was a problem hiding this comment.
🛠️ Code Refactor
Issue: The file is missing a newline at the end of the file.
Fix: Add a newline at the end of the file to follow best practices.
Impact: Improves code consistency and prevents potential issues with some tools that expect files to end with a newline.
| }; | |
| }; | |
| import { ProviderAPIConfig } from '../types'; | ||
|
|
||
| export const featherlessAIAPIConfig: ProviderAPIConfig = { | ||
| getBaseURL: () => 'https://api.featherless.ai/v1', | ||
| headers({ providerOptions }) { | ||
| const { apiKey } = providerOptions; | ||
| return { Authorization: `Bearer ${apiKey}` }; | ||
| }, | ||
| getEndpoint({ fn }) { | ||
| switch (fn) { | ||
| case 'chatComplete': | ||
| return `/chat/completions`; | ||
| case 'complete': | ||
| return '/completions'; | ||
| default: | ||
| return ''; | ||
| } | ||
| }, | ||
| }; No newline at end of file |
There was a problem hiding this comment.
💡 Optional Recommendation
Issue: Error handling for API failures is not explicitly implemented.
Fix: Consider adding error handling for API failures.
Impact: Improves robustness of the application by properly handling API errors.
| import { ProviderAPIConfig } from '../types'; | |
| export const featherlessAIAPIConfig: ProviderAPIConfig = { | |
| getBaseURL: () => 'https://api.featherless.ai/v1', | |
| headers({ providerOptions }) { | |
| const { apiKey } = providerOptions; | |
| return { Authorization: `Bearer ${apiKey}` }; | |
| }, | |
| getEndpoint({ fn }) { | |
| switch (fn) { | |
| case 'chatComplete': | |
| return `/chat/completions`; | |
| case 'complete': | |
| return '/completions'; | |
| default: | |
| return ''; | |
| } | |
| }, | |
| }; | |
| import { ProviderAPIConfig } from '../types'; | |
| export const featherlessAIAPIConfig: ProviderAPIConfig = { | |
| getBaseURL: () => 'https://api.featherless.ai/v1', | |
| headers({ providerOptions }) { | |
| const { apiKey } = providerOptions; | |
| if (!apiKey) { | |
| throw new Error('Featherless AI API key is required'); | |
| } | |
| return { Authorization: `Bearer ${apiKey}` }; | |
| }, | |
| getEndpoint({ fn }) { | |
| switch (fn) { | |
| case 'chatComplete': | |
| return `/chat/completions`; | |
| case 'complete': | |
| return '/completions'; | |
| default: | |
| return ''; | |
| } | |
| }, | |
| }; |
| import { FEATHERLESS_AI } from '../../globals'; | ||
| import { | ||
| chatCompleteParams, | ||
| completeParams, | ||
| responseTransformers, | ||
| } from '../open-ai-base'; | ||
| import { ProviderConfigs } from '../types'; | ||
| import { featherlessAIAPIConfig } from './api'; | ||
|
|
||
| export const FeatherlessAIConfig: ProviderConfigs = { | ||
| chatComplete: chatCompleteParams([], { | ||
| model: 'mistralai/Magistral-Small-2506', | ||
| }), | ||
| complete: completeParams([], { model: 'mistralai/Magistral-Small-2506' }), | ||
| api: featherlessAIAPIConfig, | ||
| responseTransforms: responseTransformers(FEATHERLESS_AI, { | ||
| chatComplete: true, | ||
| complete: true, | ||
| }), | ||
| }; No newline at end of file |
There was a problem hiding this comment.
💡 Optional Recommendation
Issue: The default model is hardcoded without any documentation about available models.
Fix: Add a comment explaining the default model choice and possibly other available models.
Impact: Improves code documentation and helps users understand model options.
| import { FEATHERLESS_AI } from '../../globals'; | |
| import { | |
| chatCompleteParams, | |
| completeParams, | |
| responseTransformers, | |
| } from '../open-ai-base'; | |
| import { ProviderConfigs } from '../types'; | |
| import { featherlessAIAPIConfig } from './api'; | |
| export const FeatherlessAIConfig: ProviderConfigs = { | |
| chatComplete: chatCompleteParams([], { | |
| model: 'mistralai/Magistral-Small-2506', | |
| }), | |
| complete: completeParams([], { model: 'mistralai/Magistral-Small-2506' }), | |
| api: featherlessAIAPIConfig, | |
| responseTransforms: responseTransformers(FEATHERLESS_AI, { | |
| chatComplete: true, | |
| complete: true, | |
| }), | |
| }; | |
| import { FEATHERLESS_AI } from '../../globals'; | |
| import { | |
| chatCompleteParams, | |
| completeParams, | |
| responseTransformers, | |
| } from '../open-ai-base'; | |
| import { ProviderConfigs } from '../types'; | |
| import { featherlessAIAPIConfig } from './api'; | |
| // Featherless AI provides access to 7900+ open source models | |
| // Default model is set to Magistral-Small-2506, but many others are available | |
| // See https://featherless.ai for the full list of supported models | |
| export const FeatherlessAIConfig: ProviderConfigs = { | |
| chatComplete: chatCompleteParams([], { | |
| model: 'mistralai/Magistral-Small-2506', | |
| }), | |
| complete: completeParams([], { model: 'mistralai/Magistral-Small-2506' }), | |
| api: featherlessAIAPIConfig, | |
| responseTransforms: responseTransformers(FEATHERLESS_AI, { | |
| chatComplete: true, | |
| complete: true, | |
| }), | |
| }; |
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
|
Hi @DarinVerheijke , can you please fix formatting with |
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
@VisargD done, thank you! |
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
3 participants
Description
Adds Featherless.ai as an LLM provider giving access to over 7,900+ models and counting on Hugging Face
Motivation
Featherless.ai was recently announced as Hugging Face largest LLM inference provider and this gives Portkey-AI users access to most LLMs on Hugging Face
Type of Change
How Has This Been Tested?
Screenshots (if applicable)
Checklist
Related Issues