🧠 OpenCluely

Core is working; improvements are shipping daily.

---

OpenCluely is a revolutionary AI-powered desktop application that provides invisible, real-time assistance during technical rounds.

🎬 Demo Video

OopenCluelyDemo.mp4

🌟 Why OpenCluely?

🥷 100% Stealth Mode Invisible to Screen Sharing: Zoom, Teams, Meet, Discord Process Disguise: Appears as normal system process (Terminal, Activity Monitor, Settings) Click-Through Windows: Transparent overlay technology Draggable UI: Move windows anywhere on screen Zero Detection: Bypasses all recording software

🚀 AI-Powered Intelligence Direct Image Analysis: Screenshots are analyzed by Gemini (no Tesseract OCR) Voice Commands: Optional Azure Speech or local OpenAI Whisper Context Memory: Remembers entire interview conversation Multi-Language Support: C++, Python, Java, JavaScript, C Smart Response Window: Draggable with close button

🖼️ Modern UI Features

📱 Interactive Windows

• Floating Overlay Bar: Compact command center with camera, mic, and skill selector
• Draggable Answer Window: Move and resize AI response window anywhere
• Close Button: Clean × button to close answer window when needed
• Auto-Hide Mic: Microphone button appears only when a speech provider is available
• Interactive Chat: Full conversation window with markdown support

🎨 Visual Design

• Glass Morphism: Beautiful blur effects and transparency
• Adaptive Layout: UI adjusts based on available services
• Smart Resizing: Windows resize automatically to fit content
• Professional Look: Mimics system applications for perfect stealth

---

🎯 ctional Overview

📋 Core Components

🖱️ Main Overlay Floating command bar Screenshot capture (⌘⇧S) Microphone toggle (Optional) Skill selector (DSA) Language picker Status indicator

💬 Interactive Chat Real-time transcription AI conversation Markdown formatting Session memory Listening animations Auto-scroll messages

📊 Answer Window Draggable interface Close button (×) Split layout for code Full markdown support Syntax highlighting Smart content sizing

---

✅ To-Do List & Development Status

🎯 Core Features (Completed)

• Stealth overlay with draggable command bar and click‑through toggle
• Screenshot capture with direct Gemini analysis (no OCR step)
• AI response window with markdown and code highlighting
• Global shortcuts (capture, visibility, interaction, chat, settings)
• Session memory and chat UI
• Language picker and DSA skill prompt
• Optional Azure Speech / local Whisper integration with auto‑hide mic
• Multi‑monitor and area capture APIs
• Window binding and positioning system
• Settings management with app icon/stealth modes

🚧 Planned Features (In Development)

• Hidden during screen share (auto‑hide all windows while screen is being shared)
• Multi‑model support (OpenAI/Anthropic/Local backends alongside Gemini)
• Auto‑typer for code snippets (paste or simulate typing into editors/IDEs)
• Export conversation history (save sessions as markdown/PDF)
• Performance optimizations (faster startup, reduced memory usage)
• Enhanced stealth modes (process name randomization, deeper OS integration)

---

⚙️ Configuration

The setup script automatically handles configuration. You only need:

# Required: Google Gemini API Key (setup script will ask for this)
GEMINI_API_KEY=your_gemini_api_key_here

# Optional: Speech Recognition (pick one provider)
SPEECH_PROVIDER=whisper

# Azure option
AZURE_SPEECH_KEY=your_azure_speech_key
AZURE_SPEECH_REGION=your_region

# Local Whisper option
WHISPER_COMMAND=whisper
WHISPER_MODEL_DIR=.whisper-models
WHISPER_MODEL=turbo
WHISPER_LANGUAGE=en
WHISPER_SEGMENT_MS=4000

Note: Speech recognition is completely optional. If no configured provider is available, the microphone button will be automatically hidden from all interfaces.

📦 Download Pre-Built Installers

Don't want to clone and build? Download a pre-built installer for your platform from the Releases page.

Platform	File	Notes
Windows	OpenCluely-Setup-*.exe	NSIS installer; auto-creates Start Menu shortcut
Windows	OpenCluely-*-portable.exe	Portable, no install required
macOS (Apple Silicon)	OpenCluely-*-arm64.dmg	M1 / M2 / M3 / M4 Macs
macOS (Intel)	OpenCluely-*-x64.dmg	Older Intel Macs
Linux (Debian/Ubuntu)	OpenCluely-*.deb	Auto-pulls system deps: Python 3.10+, ffmpeg, GTK, NSS
Linux (Universal)	OpenCluely-*.AppImage	No install — chmod +x then run

Every release is built automatically by GitHub Actions across Windows, macOS, and Linux runners in parallel and uploaded with SHA-256 checksums.

🚀 Quick Start & Installation

⚡ Three Simple Steps (All Operating Systems)

1. Clone the repository (skip if you downloaded a pre-built installer above)

   git clone https://github.com/TechyCSR/OpenCluely.git
   cd OpenCluely

2. Run the setup script (One command does everything!)

   ./setup.sh

   The setup script will:

   • Install all Node dependencies automatically
   • Create your .env file from env.example if needed (with safe defaults)
   • Set up a local Whisper virtualenv in .venv-whisper (optional, 3 GB)
   • Configure .env to use local Whisper by default
   • Launch OpenCluely

3. First-run onboarding

   • On first launch, if no Gemini API key is configured, the app automatically opens the Settings window and walks you through entering it.
   • You can paste the key in the Settings UI, or edit .env directly — both work.
   • Get a free key from Google AI Studio.

   Note: Setup will not hard-block if GEMINI_API_KEY is missing — the app launches either way and prompts you when needed.

💻 Platform-Specific Notes

• Windows: Use Git Bash (comes with Git for Windows), WSL, or any bash environment
• macOS/Linux: Use your regular terminal
• All platforms: No manual npm commands needed - the setup script handles everything
• Windows Whisper path: setup.sh now writes WHISPER_COMMAND=.venv-whisper/Scripts/whisper.exe
• macOS/Linux Whisper path: setup.sh writes WHISPER_COMMAND=.venv-whisper/bin/whisper

🎛️ Setup Script Options

./setup.sh --build          # Build distributable for your OS
./setup.sh --ci             # Use npm ci instead of npm install
./setup.sh --no-run         # Setup only, don't launch the app
./setup.sh --install-system-deps  # Install sox for microphone (optional)
./setup.sh --skip-whisper  # Skip the local Whisper bootstrap

🔧 Optional: Speech Setup (For Voice Features)

Voice recognition is optional. You can use either Azure Speech or local OpenAI Whisper.

For the local Whisper path, ./setup.sh now handles the full repo-local setup:

1. Creates .venv-whisper

2. Installs openai-whisper

3. Points .env at .venv-whisper/bin/whisper

4. Creates .whisper-models

5. Runs npm run test-speech

6. For Azure Speech:

   • Visit Azure Portal
   • Create a Speech Service
   • Copy your key and region

7. For local Whisper:

   • Run ./setup.sh --install-system-deps
   • Or install required audio tools such as ffmpeg and sox yourself
   • On Windows, install audio tooling separately and prefer Git Bash or WSL for setup.sh

8. Add one provider to your .env file:

   GEMINI_API_KEY=your_gemini_api_key_here
   SPEECH_PROVIDER=azure
   AZURE_SPEECH_KEY=your_azure_speech_key
   AZURE_SPEECH_REGION=your_region

   GEMINI_API_KEY=your_gemini_api_key_here
   SPEECH_PROVIDER=whisper
   WHISPER_COMMAND=whisper
   WHISPER_MODEL_DIR=.whisper-models
   WHISPER_MODEL=turbo
   WHISPER_LANGUAGE=en
   WHISPER_SEGMENT_MS=4000

9. The app picks up changes immediately — no restart needed. The microphone buttons appear as soon as the config is valid.

🎮 How to Use

🖱️ Main Controls

Action	Shortcut	Description
Screenshot Capture	⌘⇧S	Capture screen and analyze via Gemini (image understanding)
Toggle Speech	Alt+R	Start/stop voice recognition (if configured)
Toggle Visibility	⌘⇧V	Show/hide all windows
Toggle Interaction	⌘⇧I or Alt+A	Enable/disable window interaction
Switch to Chat	⌘⇧C	Open interactive chat window
Settings	⌘,	Open settings panel

🎯 Workflow

1. Start OpenCluely → App appears as system process (Terminal/Activity Monitor)
2. Position Windows → Drag overlay and answer windows to preferred locations
3. Capture Questions → Use screenshot (⌘⇧S) or voice commands
4. Get AI Answers → Instant responses in draggable answer window
5. Interactive Chat → Type or speak for detailed conversations
6. Stay Stealth → All operations invisible to screen recording

🔧 Advanced Features

🎨 Window Management

• Draggable Interface: Click and drag any window to reposition
• Auto-resize: Windows automatically adjust to content
• Close Button: Click × to close answer window
• Always on Top: Windows stay above all applications

🧠 AI Intelligence

• Context Awareness: Remembers entire conversation
• Code Detection: Automatically formats code blocks
• Language Specific: Tailored responses for selected programming language
• Session Memory: Maintains context across multiple questions
• Image Understanding: DSA prompt is applied only for new image-based queries; chat messages don’t include the full prompt
• Multi-monitor & Area Capture: Programmatic APIs allow targeting a display and optional rectangular crop for focused analysis

🔊 Optional Voice Features (Azure Speech / Local Whisper)

• Chunked Local Transcription: Local Whisper transcribes short recorded segments on your machine
• Real-time Transcription: Azure Speech supports live interim recognition
• Listening Animation: Visual feedback during recording
• Interim Results: Available with Azure Speech
• Auto-processing: Instant AI responses to voice input ]

---

🧩 Troubleshooting

Setup Issues

• setup.sh not found or won't run

  • Make sure you're in the OpenCluely directory: cd OpenCluely
  • Make the script executable: chmod +x setup.sh
  • On Windows, use Git Bash (comes with Git for Windows)

• Setup script stops with exit code 130

  • This means you pressed Ctrl+C. Just run ./setup.sh again

• Node or npm not found

  • Install Node.js 18+ from nodejs.org
  • Restart your terminal and try again

App Issues

• Electron won't start or shows blank window (Linux)

  • Try: npm run dev
  • Ensure X11/XWayland is available if running in headless environments

• macOS screen capture doesn't work

  • Grant "Screen Recording" permission in System Settings → Privacy & Security → Screen Recording
  • Quit and relaunch the app after granting permission

• Windows SmartScreen blocks the app

  • Click "More info" → "Run anyway" or use npm start during development

• Microphone/voice not working

  • Voice is optional - ignore related warnings if you don't need it
  • Azure mode: add valid Azure keys to .env
  • Whisper mode: install openai-whisper, ffmpeg, and sox, then set SPEECH_PROVIDER=whisper

⚖️ Legal & Ethics

📋 Disclaimer

OpenCluely is provided for educational and research purposes. Users are responsible for:

• Complying with interview guidelines
• Respecting company policies
• Understanding legal implications
• Using ethically and responsibly

🔒 Privacy

• No data collection or telemetry
• All processing happens locally
• API communications are encrypted
• Session data stays on your device

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

Website

🌐 opencluely.techycsr.dev

---

�� Acknowledgments

• Google Gemini: Powering AI intelligence

• Azure Speech / Whisper: Optional voice recognition

• Electron: Cross-platform desktop framework

• Community: Amazing contributors and feedback

• Vysper: UI and code structure inspiration — see Vysper by varun-singhh

---

⭐ Star this repo if OpenCluely helped you ace your interviews or you vibed with it!

Made with ❤️ by TechyCSR