Skip to content

feat: add cloud-based LLM support and robust error recovery#41

Open
amaynez wants to merge 1 commit into
nikmcfly:mainfrom
amaynez:feat/cloud-llm-support
Open

feat: add cloud-based LLM support and robust error recovery#41
amaynez wants to merge 1 commit into
nikmcfly:mainfrom
amaynez:feat/cloud-llm-support

Conversation

@amaynez
Copy link
Copy Markdown

@amaynez amaynez commented May 3, 2026

This pull request introduces comprehensive support for cloud-based LLM providers like OpenRouter, significantly expanding the accessibility of MiroFish-Offline.

The primary objective of these changes is to democratize access to the platform. By allowing users to offload the heavy lifting of model inference to cloud providers, we enable participation from a much broader user base. This includes individuals who lack the high performance hardware typically required for sufficiently intelligent local inference.

Key Enhancements:

  1. Cloud Provider Integration: Users can now choose between local Ollama instances and cloud-based OpenAI-compatible endpoints. This flexibility ensures that the system remains functional even on resource-constrained devices.
  2. Robust JSON Repair: A new multi-phase JSON repair logic has been implemented. This ensures that the system can recover from truncated or malformed responses, which are common when dealing with complex structured extractions or low-latency cloud endpoints.
  3. Modular LLM Configuration: The internal architecture now supports separate configurations for different tasks. For example, users can use a lighter model for general interactions while utilizing a more powerful model specifically for graph extraction and ontology generation.
  4. Optimized Deployment: Docker Compose configurations have been updated to make the Ollama service optional. This simplifies the initial setup for cloud-centric users while maintaining full offline capabilities for those with the necessary hardware.
  5. Improved Error Visibility: The frontend now extracts and displays backend error messages more effectively, providing a better debugging experience for developers and a clearer interface for users.

By incorporating these changes, the project transitions from a hardware-dependent tool to a versatile platform that adapts to the user's environment. This strategy not only increases the potential user base but also improves the overall reliability and professional polish of the codebase.

nocode-ananas added a commit to nocode-ananas/polylop that referenced this pull request Jun 6, 2026
…kept FAST patch, kept 24h timeout, kept ollama out of compose)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant