Interactive AI Chatbot 🤖✨

An interactive 3D AI chatbot built with React Three Fiber, powered by Google's Gemini API and ElevenLabs for voice synthesis.

Features

🎭 Interactive 3D avatar with realistic animations
🧠 Powered by Google Gemini AI for intelligent conversations
🔊 High-quality voice synthesis using ElevenLabs API
🎮 Gamified chat experience with points and achievements
📱 Responsive design that works on desktop and mobile
🌈 Beautiful particle effects and animations
🎯 Multimodal support (text, voice, and visual inputs)

Tech Stack

Frontend: Next.js 14, React 18
3D Graphics: React Three Fiber (@react-three/fiber), Three.js
3D Utilities: Drei (@react-three/drei)
AI: Google Gemini API
Voice: ElevenLabs API
State Management: Zustand
Styling: Tailwind CSS
Animations: Framer Motion

Getting Started

Prerequisites

Node.js 18+
npm or yarn
Google AI Studio API key
ElevenLabs API key (see ELEVENLABS_SETUP.md for setup instructions)

Installation

Clone the repository

git clone https://github.com/yourusername/r3f-ai-chatbot.git
cd r3f-ai-chatbot

Install dependencies
```
npm install
# or
yarn install
```

Set up environment variables

Create a .env.local file in the root directory:

# Google Gemini AI
NEXT_PUBLIC_GEMINI_API_KEY=your_gemini_api_key_here

# ElevenLabs API
NEXT_PUBLIC_ELEVENLABS_API_KEY=your_elevenlabs_api_key_here
NEXT_PUBLIC_ELEVENLABS_VOICE_ID=pNInz6obpgDQGcFmaJgB

# Optional: Custom configuration
NEXT_PUBLIC_APP_NAME="AI Chatbot"
NEXT_PUBLIC_MAX_TOKENS=1000

Run the development server
```
npm run dev
# or
yarn dev
```
Open your browser

Navigate to http://localhost:3000

API Keys Setup

Google Gemini API

Go to Google AI Studio
Create a new API key
Copy the key to your .env.local file

ElevenLabs API

Sign up at ElevenLabs
Go to your profile settings
Copy your API key
Choose a voice ID from your available voices

Multimodal Features

Text Input

Standard chat interface
Real-time typing indicators
Message history

Voice Input

Click and hold to record
Real-time speech-to-text
Voice activity detection

Visual Input

Image upload support
Screenshot analysis
Camera integration (mobile)

Voice Output

Natural voice responses
Configurable voice settings
Lip-sync with 3D avatar

Gamification Features

Point System

Message Bonus: 10 points per message
Streak Bonus: Extra points for consecutive days
Discovery Bonus: Points for trying new features
Achievement Bonus: Special rewards for milestones

Achievements

🎯 First Chat: Send your first message
🔥 On Fire: 10 messages in a row
🎭 Voice Master: Use voice input 5 times
📸 Visionary: Upload and analyze an image
💎 Legendary: Reach 1000 points
🌟 Daily User: Chat for 7 consecutive days

Levels

Novice (0-100 points)
Explorer (101-300 points)
Conversationalist (301-600 points)
Expert (601-1000 points)
Master (1000+ points)

Interactive Features

3D Avatar Interactions

Idle Animations: Breathing, blinking, subtle movements
Speaking Animations: Mouth movement sync with voice
Gesture Recognition: Hand gestures for emphasis
Emotion Display: Facial expressions based on conversation tone

Environmental Interactions

Particle Systems: Responsive to conversation energy
Lighting Changes: Dynamic based on time/mood
Background Music: Ambient sounds that adapt
Physics Objects: Interactive elements in the scene

Playful Elements

Easter Eggs: Hidden interactions and responses
Mini Games: Simple games within the chat
Customization: Avatar appearance options
Themes: Different environmental settings

Making It More Interactive

1. Enhanced Avatar System

// Add more personality traits
const avatarPersonalities = {
  friendly: { animations: ["wave", "smile"], voice: "warm" },
  professional: { animations: ["nod", "thinking"], voice: "clear" },
  playful: { animations: ["dance", "jump"], voice: "energetic" },
};

2. Context-Aware Responses

// Implement mood detection
const conversationContext = {
  mood: "happy", // detected from user messages
  topic: "technology", // current conversation topic
  userPreferences: [], // learned over time
};

3. Interactive Elements

Clickable Objects: 3D items that trigger conversations
Environment Changes: User can modify the virtual space
Multi-user Support: Multiple avatars for group chats
AR Integration: Camera overlay features

Advanced Gamification

1. Quest System

const quests = [
  {
    id: "explorer",
    title: "Digital Explorer",
    description: "Ask about 5 different topics",
    reward: 100,
    progress: 0,
    target: 5,
  },
];

2. Social Features

Leaderboards: Compare scores with friends
Sharing: Share interesting conversations
Challenges: Daily/weekly challenges
Badges: Collectible achievement badges

3. Progression System

Skill Trees: Unlock conversation abilities
Customization Rewards: Avatar accessories and environments
Premium Features: Advanced voice options, effects

Project Structure

r3f-ai-chatbot/
├── components/
│   ├── 3d/
│   │   ├── Avatar.js
│   │   ├── Environment.js
│   │   ├── Particles.js
│   │   └── Scene.js
│   ├── ui/
│   │   ├── ChatInterface.js
│   │   ├── VoiceRecorder.js
│   │   ├── GameElements.js
│   │   └── Settings.js
│   └── layout/
│       ├── Header.js
│       └── Footer.js
├── lib/
│   ├── gemini.js
│   ├── elevenlabs.js
│   ├── gameStore.js
│   └── utils.js
├── pages/
│   ├── api/
│   │   ├── chat.js
│   │   └── voice.js
│   ├── _app.js
│   ├── _document.js
│   └── index.js
├── public/
│   ├── models/
│   ├── sounds/
│   └── textures/
├── styles/
│   └── globals.css
├── .env.local
├── next.config.js
├── tailwind.config.js
└── package.json

Deployment

Vercel (Recommended)

npm run build
vercel --prod

Other Platforms

Netlify: Connect your GitHub repository
Railway: Deploy with one click
Heroku: Use the Next.js buildpack

Performance Optimization

Lazy Loading: 3D models and textures load on demand
Level of Detail: Lower quality models for distant objects
Texture Compression: Optimized image formats
Audio Streaming: Progressive audio loading
Caching: API responses cached for better performance

Contributing

Fork the repository
Create your feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

React Three Fiber for the amazing 3D React renderer
Google AI for the Gemini API
ElevenLabs for realistic voice synthesis
Drei for helpful R3F utilities

Support

If you have any questions or need help, please:

📧 Open an issue on GitHub
💬 Join our Discord community
📚 Check out the documentation

Made with ❤️ and lots of ☕

FilesExpand file tree

readme.md

Latest commit

History