Product Requirements Document: YC Cofounder Matching Automation (v5 - Production Ready)

Overview

Python script that automates discovering, evaluating, and inviting YC Startup School cofounder candidates using Gemini API and browser automation with session persistence. Features interactive human-in-the-loop mode with rich terminal UI for message review and approval before sending.

Technical Stack

Language: Python 3.9+
Browser Automation: Playwright with stealth configuration
LLM: Google Gemini API (gemini-2.0-flash-exp)
HTML Parsing: BeautifulSoup4
Terminal UI: Rich (for colored, formatted console output)
Environment: venv
Dependencies: playwright, google-generativeai, python-dotenv, tenacity, beautifulsoup4, rich

Core Requirements

1. Environment Setup

Use venv for isolation
Store credentials in .env file (Gemini API key only)
Single requirements.txt file
Generate storage_state.json for session persistence
.gitignore includes: storage_state.json, logs/, .env

2. Configuration

# .env
GEMINI_API_KEY=your_key
EVALUATION_THRESHOLD=5
MAX_INVITES_PER_RUN=10          # Conservative default - start with 5
MIN_DELAY_SECONDS=3             # Minimum delay between actions
MAX_DELAY_SECONDS=8             # Maximum delay between actions
INTERACTIVE_TIMEOUT=60          # Seconds to wait for initial decision only

Configuration Notes:

config.py loads .env and merges with CLI argument overrides
All settings have sensible defaults
CLI args take precedence over .env values

3. Input Files

prompts/evaluation.txt - Concise criteria for scoring (expects JSON response)
prompts/invitation.txt - Template with placeholders: {name}, {skills}, {bio}
storage_state.json - Browser session (generated, auto-gitignored)

4. Core Functionality

4.1 Authentication & Session Management

First run: python main.py --auth
- Opens headed browser
- User manually logs into YC Startup School
- Script waits for user confirmation ("Press Enter when logged in")
- Saves session to storage_state.json
- Exits with success message
Subsequent runs: Load storage_state.json, skip login entirely
Session expiry: Detect and exit with clear re-auth instructions

4.2 Discovery Mode (Tab Strategy for State Preservation)

Navigation Architecture:

Feed Tab (Primary): Maintains the candidate list and scroll position
Profile Tab (Temporary): Opens, processes, then closes for each candidate

Flow:

Navigate to cofounder matching feed: startupschool.org/cofounder-matching
Locate all visible candidate cards: .candidate-card (or equivalent selector)
For each candidate:
- Extract card link URL from feed (don't click yet)
- Open link in new background tab using Playwright:
```
profile_page = context.new_page()
profile_page.goto(candidate_url)
```
- Switch context to profile tab
- Extract profile data via content_parser.py
- Perform evaluation and interaction
- Close profile tab: profile_page.close()
- Return focus to feed tab
- Random delay (3-8s) before next candidate
Stop when: MAX_INVITES_PER_RUN reached OR feed exhausted OR error threshold exceeded

Why Tab Strategy:

Preserves feed scroll position (no "back button" resets)
Mimics power-user browsing behavior (more authentic)
Reduces redundant page loads (Feed → Profile → Feed → Profile)
Prevents feed state loss in React/SPA applications

Note: candidate_identifier extracted from URL - may be UUID, slug, or numeric ID (handle as string)

4.3 Browser Automation (Stealth - Phase 1 Priority)

Configure Playwright browser launch:

browser = playwright.chromium.launch(
    headless=headless,
    args=['--disable-blink-features=AutomationControlled']  # Most effective stealth flag
)

Configure Playwright context:
- Random User-Agent rotation (5+ variants)
- Disable navigator.webdriver flag:
```
context.add_init_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
```
- Randomized viewport: 1920x1080 ±100px
- Enable timezone/locale matching user's system
Random delays between ALL actions:
- After page load: 3-8s
- Before click: 2-5s
- After form submit: 5-10s
Extract full profile via content_parser.py:
- Name, bio, skills (list), experience, location, interests
Interaction flow (non-interactive mode):
- Click "invite to connect" button
- Wait for message form to appear
- Fill invitation message
- Submit invitation
- Wait for confirmation
- Random delay before next candidate

4.4 Content Parsing & HTML Cleaning

content_parser.py responsibilities:

Use BeautifulSoup4 for robust HTML parsing
Strip all HTML tags, preserve text content
Handle malformed HTML gracefully (try-except, log warning)
Clean whitespace: multiple spaces → single space, trim newlines
Token management:
- Truncate bio/profile to 2,500 characters (generous context for better matching)
- Token estimation: len(text.split()) * 1.3 ≈ tokens
- Gemini Flash supports 1M tokens, so context is not limiting factor
- Log info if truncation occurs
Error handling: If parsing fails completely, skip candidate and log error

4.5 LLM Integration (Structured Output)

Evaluation Call:

{
  "score": 7,
  "reasoning": "Strong technical background in ML, aligned startup interests in climate tech"
}

Use Gemini's generation_config with response_mime_type="application/json"
Define response schema:

{
  "type": "object",
  "properties": {
    "score": {"type": "integer", "minimum": 1, "maximum": 10},
    "reasoning": {"type": "string", "maxLength": 300}
  },
  "required": ["score", "reasoning"]
}

Retry with exponential backoff (3 attempts, 2s/4s/8s) on API failures
Input token limit check: Ensure prompt + profile < 30k tokens (leave margin)

Invitation Call (only if score > threshold):

Inject scraped data into prompt: {name}, {skills}, {bio}
Response schema:

{
  "message": "Hi Sarah, I noticed your experience with..."
}

Validation:
- Message contains candidate's name (case-insensitive check)
- Message length: 50-300 characters
- Reject if generic phrases detected: "Dear candidate", "To whom it may concern"
- On validation failure: Log error, skip candidate

Cost Management:

Each candidate = 2 API calls (eval + invitation)
Gemini Flash: ~$0.10 per 1M input tokens
Estimated cost: $0.01-0.02 per 100 candidates
Note in README: Monitor usage at aistudio.google.com

4.6 State Management

Track invites sent in current run (in-memory counter)
Immediate logging after each candidate action
No sensitive data in logs (no API keys, passwords)
Log format allows audit trail for debugging

4.7 Interactive Mode (--interactive) with Rich UI

Purpose: Human-in-the-loop workflow with color-coded terminal UI for clarity.

Terminal UI (using Rich library):

from rich.console import Console
from rich.panel import Panel
from rich.prompt import Prompt
from rich import box

console = Console()

# Display formatted card
console.print(Panel(
    f"[bold cyan]Candidate #{count}: {name}[/bold cyan]\n"
    f"Profile: {url}\n\n"
    f"[bold]Score:[/bold] [{'green' if score >= 7 else 'yellow' if score >= 5 else 'red'}]{score}/10[/]\n"
    f"[bold]Reasoning:[/bold] {reasoning}\n\n"
    f"[bold]Generated Message:[/bold]\n[cyan]{message}[/cyan]",
    title="Review Candidate",
    box=box.DOUBLE
))

Color Coding:

Score ≥7: Green (strong match)
Score 5-6: Yellow (moderate match)
Score <5: Red (should not see in interactive mode due to threshold)
Message text: Cyan (stands out)
Candidate name: Bold cyan

Flow:

Script opens profile in new tab
Evaluate via Gemini API
If score ≤ threshold: Log as skipped, continue to next
If score > threshold:
- Generate invitation message via Gemini
- PAUSE execution
- Display formatted card (see above)
- Prompt: [Y] Send [N] Skip [E] Edit [Q] Quit >
- Start 60-second timeout for initial decision

Input Handling:

Y (or 'y', 'yes'):
- Submit message via browser automation
- Log as "sent"
- Close profile tab
- Continue to next
N (or 'n', 'no'):
- Skip candidate
- Log as "skipped-manual"
- Close profile tab
- Continue to next
E (or 'e', 'edit'):
- Disable timeout (critical: no timeout during editing)
- Prompt: Enter new message (press Enter when done):
- User types custom message (wait indefinitely)
- Validate length (50-300 chars)
- If valid: send custom message via browser
- If invalid: re-prompt
- Log as "sent-custom"
- Close profile tab
- Continue to next
Q (or 'q', 'quit'):
- Close profile tab
- Save current state
- Display summary statistics
- Graceful exit
Timeout (60s no input on initial prompt):
- Auto-skip candidate
- Log as "skipped-timeout"
- Close profile tab
- Continue to next

Timeout Logic (Critical):

Timeout only applies to initial [Y] [N] [E] [Q] decision prompt
Once user presses E (Edit), timeout is suspended indefinitely
User can take as long as needed to compose custom message
Prevents "rage quit" scenario where partially-typed message is lost

ADHD-Friendly Design:

Color-coded scores reduce cognitive load
Clear visual hierarchy with Rich panels
No timeout pressure during message composition
Can quit gracefully at any point

5. Script Structure

yc-matcher/
├── README.md
├── requirements.txt
├── .env.example
├── .gitignore
├── main.py                   # Entry point, CLI, orchestration
├── config.py                 # Load .env, merge CLI args
├── session_manager.py        # Auth, cookie handling, session validation
├── browser.py                # Playwright automation with stealth + tab mgmt
├── content_parser.py         # BeautifulSoup HTML parsing, token mgmt
├── evaluator.py              # Gemini API calls with retry logic
├── interactive.py            # Interactive mode UI with Rich formatting
├── logger.py                 # CSV logging, error logging
├── prompts/
│   ├── evaluation.txt
│   └── invitation.txt
├── storage_state.json        # Generated, gitignored
└── logs/
    ├── invitations.csv       # Generated, gitignored
    └── errors.log            # Generated, gitignored

6. CLI Interface

# First time setup (manual login)
python main.py --auth

# Interactive mode - review each candidate before sending (RECOMMENDED)
python main.py --interactive

# Batch modes
python main.py --dry-run              # Evaluate only, no messages
python main.py --auto                 # Fully automated (use with caution)

# Options
python main.py --interactive --headless false    # Show browser
python main.py --interactive --max-invites 5     # Limit invites
python main.py --start-fresh                     # Force re-auth

CLI Descriptions:

--auth: Open browser for manual login, save session, exit
--interactive: [RECOMMENDED] Review each candidate with rich UI, approve/edit/skip messages
--dry-run: Evaluate candidates, log scores/reasoning. No messages generated or sent.
--auto: Fully automated mode - generates and sends messages without human review (use cautiously)
--headless false: Show browser window (default: true)
--max-invites N: Limit invitations per run (default: 10)
--start-fresh: Delete existing session, force re-authentication

Default Mode: If no mode flag provided, defaults to --interactive for safety.

7. Error Handling & Retry Logic

API Calls (using tenacity):

@retry(stop=stop_after_attempt(3), 
       wait=wait_exponential(multiplier=2, min=2, max=8),
       retry=retry_if_exception_type((TimeoutError, ConnectionError)))

Specific Error Scenarios:

Session expired: Detect 401/redirect to login → Exit with message: "Session expired. Run: python main.py --auth"
Rate limit (429): Log warning, wait 60s, continue (count against retry budget)
Page load timeout: Retry 3x with 5s wait, then skip candidate
Tab management errors: Close orphaned tabs, log warning, continue
Malformed HTML: Log warning, attempt partial parse, skip if total failure
Invalid Gemini response: Retry API call, log if all attempts fail, skip candidate
Network errors: Retry with exponential backoff
Interactive timeout: Auto-skip after 60s of no input (initial prompt only)

Error Threshold:

If 5 consecutive candidates fail (any reason), exit with error summary
Prevents runaway script on platform changes

8. Output & Logging

Console Output (Interactive Mode with Rich):

╭─────────────────────────────────────────╮
│ YC Cofounder Matcher - Interactive Mode │
╰─────────────────────────────────────────╯

[12:34:56] Authenticating with saved session...
[12:34:58] ✓ Session valid
[12:35:00] Starting discovery...
[12:35:02] Found 25 candidates in feed

╔═══════════════════════════════════════════════════╗
║                Review Candidate                    ║
╠═══════════════════════════════════════════════════╣
║ Candidate #1: Alice Smith                          ║
║ Profile: startupschool.org/.../alice-smith        ║
║                                                    ║
║ Score: 8/10                                        ║
║ Reasoning: Strong Django experience, SaaS focus   ║
║                                                    ║
║ Generated Message:                                 ║
║ Hi Alice, I noticed your experience with Django   ║
║ and your interest in dev tools. I'm building a    ║
║ similar platform and would love to connect!       ║
╚═══════════════════════════════════════════════════╝

[Y] Send  [N] Skip  [E] Edit  [Q] Quit > y

[12:35:15] ✓ Invitation sent to Alice Smith
[12:35:20] Processing candidate 2/25...

Console Output (Auto Mode):

[12:34:56] Starting discovery in auto mode...
[12:34:58] Found 25 candidates in feed
[12:35:02] Processing alice-smith (1/25)...
[12:35:05]   Score: 7/10 - Strong ML background
[12:35:08]   ✓ Invitation sent
[12:35:15] Processing bob-jones (2/25)...
[12:35:18]   Score: 3/10 - Mismatch in interests
[12:35:18]   ✗ Skipped (below threshold)
...
[12:40:23] Complete: 10 invites sent, 15 evaluated

logs/invitations.csv:

timestamp,candidate_id,name,score,reasoning,action,message_preview,mode
2024-01-15 12:35:08,alice-smith,Alice Smith,8,"Strong Django experience",sent,"Hi Alice, I noticed your...",interactive
2024-01-15 12:35:18,bob-jones,Bob Jones,3,"Mismatch in interests",skipped-threshold,,interactive
2024-01-15 12:35:30,carol-wang,Carol Wang,7,"ML expertise",skipped-manual,,interactive
2024-01-15 12:35:45,dave-kim,Dave Kim,9,"Perfect match",sent-custom,"Hey Dave, let's chat about...",interactive
2024-01-15 12:35:55,eve-patel,Eve Patel,6,"Moderate fit",skipped-timeout,,interactive

Action Types:

sent: Message generated and sent (auto or interactive 'Y')
sent-custom: User edited message and sent (interactive 'E')
skipped-threshold: Score below threshold
skipped-manual: User chose to skip (interactive 'N')
skipped-timeout: Interactive timeout (no input on initial prompt)
error: Processing failed

logs/errors.log:

Timestamp, error type, stack trace, candidate_id (if applicable)
Tab management errors
Rotates at 10MB (use logging.handlers.RotatingFileHandler)

9. Safety, Rate Limiting & Mitigation Strategies

Default Safety Settings:

MAX_INVITES_PER_RUN=10 (recommend starting with 5)
MIN_DELAY=3s, MAX_DELAY=8s between actions
Random delays after form submissions: 5-10s
User-Agent rotation
Stealth configuration (disable webdriver flag + AutomationControlled)
Interactive mode as default for human oversight
Tab strategy for authentic browsing patterns

Mitigation Strategies (README Disclosure):

Bot Detection Risk: This tool attempts to mimic human behavior through:

Tab-based navigation (opens profiles in new tabs, preserves feed state)

Random delays (3-8s between actions)

Session persistence (no repeated logins)

Browser fingerprint randomization

Interactive mode for human oversight

However, these measures are not foolproof. YC may:

Introduce CAPTCHAs at any time

Flag accounts with suspicious patterns

Change platform structure, breaking the script

Recommendations:

Always start with --interactive mode

Begin with --max-invites 5 to test

Run no more than once per day

Monitor your account for unusual activity warnings

Stop immediately if you receive any platform warnings

10. README.md Content

# YC Cofounder Matcher

⚠️ **Legal & TOS Warning**
Automated interaction with YC Startup School may violate their Terms of Service. This tool is for educational purposes only. Use at your own risk. The authors assume no liability for account suspensions or bans.

**Bot Detection Risk**: This script mimics human behavior through tab-based navigation, random delays, and session persistence, but is not foolproof. YC may introduce CAPTCHAs, flag suspicious activity, or change their platform. **Always start with interactive mode** and monitor your account.

## Setup

### 1. Install Dependencies
```bash
git clone <repo> && cd yc-matcher
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
playwright install chromium

2. Configure

cp .env.example .env
# Edit .env and add your Gemini API key from aistudio.google.com

3. Customize Prompts

Edit prompts/evaluation.txt with your criteria
Edit prompts/invitation.txt with your template (use {name}, {skills}, {bio})

4. Authenticate

python main.py --auth

Browser will open
Manually log into YC Startup School
Press Enter in terminal when logged in
Session saved to storage_state.json

Usage

Recommended: Interactive Mode

# Review each candidate with rich color-coded UI
python main.py --interactive --max-invites 5

Interactive Controls:

Y - Send the generated message
N - Skip this candidate
E - Edit the message before sending (no timeout while editing)
Q - Quit and save progress

Features:

Color-coded scores (Green ≥7, Yellow 5-6)
Formatted panels for easy reading
No timeout pressure during message composition
ADHD-friendly visual hierarchy

Safe Testing

# Dry run (no messages generated or sent)
python main.py --dry-run --max-invites 5

Automated Mode (Use with Caution)

# Fully automated - no human review
python main.py --auto --max-invites 5

⚠️ Auto mode is NOT recommended for first-time use. Always test with --interactive first.

Re-authentication

# If session expires
python main.py --auth

Command Reference

Command	Description
`--auth`	Open browser for manual login, save session, exit
`--interactive`	[RECOMMENDED] Review each candidate with rich UI
`--dry-run`	Evaluate candidates only, no message generation
`--auto`	Fully automated mode (no human review)
`--max-invites N`	Limit invites per run (default: 10)
`--headless false`	Show browser window
`--start-fresh`	Force re-authentication

Default: If no mode specified, defaults to --interactive.

How It Works

Tab Strategy (Preserves Feed State)

Script stays on the feed page (main tab)
Opens each candidate profile in a new tab
Processes the profile in the background tab
Closes the tab and returns to feed
Moves to next candidate (no feed refresh needed)

This mimics power-user browsing and preserves scroll position.

Architecture Flow

Feed Tab (Persistent)
    ↓
New Tab → Scrape → Evaluate (Gemini) → Generate (Gemini)
    ↓
Interactive Review (Rich UI)
    ↓
Send/Skip → Close Tab → Return to Feed → Next Candidate

Output Files

logs/invitations.csv: All processed candidates with scores, actions, and outcomes
logs/errors.log: Detailed error information
Console: Real-time progress with rich formatting

Cost Estimation

Gemini API: ~$0.01-0.02 per 100 candidates (Flash model)
Monitor usage: https://aistudio.google.com/app/billing

Troubleshooting

Issue	Solution
"Session expired"	Run `python main.py --auth`
Rate limited	Reduce `--max-invites`, increase delays in `.env`
CAPTCHA appears	Platform detected automation - wait 24hrs, reduce frequency
HTML parsing errors	Platform structure changed - file GitHub issue
Tab errors	Script auto-closes orphaned tabs, check logs
Timeout during edit	Timeout only applies to initial prompt, not edit mode

Safety Recommendations

Start with interactive mode: Always use --interactive for first runs
Begin small: Use --max-invites 5 initially
Run infrequently: Once per day maximum
Monitor account: Check for YC warnings after each run
Review patterns: If many candidates score low, refine prompts/evaluation.txt
Stop if flagged: Immediately cease usage if account shows warnings
Use edit mode: Customize messages for high-value candidates (press E)

Technical Details

Stealth Configuration

--disable-blink-features=AutomationControlled flag
navigator.webdriver override
Randomized User-Agent (5+ variants)
Random delays (3-8s between actions)

Tab Management

Feed tab remains open (preserves state)
Profile tabs open/close per candidate
Prevents "back button" resets
Mimics authentic browsing patterns

Interactive Timeout

60s timeout only for initial decision ([Y] [N] [E] [Q])
No timeout during Edit mode - compose at your own pace
Auto-skip if no input to prevent hanging

License

MIT - Educational purposes only. See LICENSE file.


### 11. Implementation Priority (Revised)

**Phase 1: Foundation (Stealth + Tabs)**
- `config.py` - Environment loading
- `session_manager.py` - Authentication flow
- `browser.py` - Playwright with:
  - `args=['--disable-blink-features=AutomationControlled']`
  - User-Agent rotation
  - `navigator.webdriver` override
  - **Tab management** (new_page(), close())
- `--auth` CLI command working

**Phase 2: Discovery (Tab Strategy)**
- Feed scraping with `.candidate-card` selector
- Extract candidate URLs from feed
- Implement tab-based navigation:
  - Open profile in new tab
  - Process in background
  - Close tab after processing
  - Return to feed tab
- `content_parser.py` - BeautifulSoup HTML parsing

**Phase 3: Evaluation**
- `evaluator.py` - Gemini API with JSON schema
- Retry logic with tenacity
- `--dry-run` mode

**Phase 4: Message Generation**
- Invitation generation with personalization
- Validation logic (name check, length, generic phrase detection)

**Phase 5: Interactive Mode with Rich UI**
- Install `rich` library
- `interactive.py` - Console UI with:
  - `Panel` for formatted cards
  - Color-coded scores
  - Prompt with timeout
  - **Timeout suspension for Edit mode**
- `--interactive` CLI flag

**Phase 6: Automated Mode**
- Full invitation flow with browser interaction
- Rate limiting enforcement
- Random delays between actions
- `--auto` CLI flag

**Phase 7: Polish**
- Comprehensive logging with action types
- Tab cleanup on errors
- Error threshold logic
- CLI refinements
- README finalization
- Cost tracking notes

## Non-Requirements

- No GUI (terminal-based interactive mode only)
- No database (CSV logging only)
- No concurrent processing (sequential only)
- No unit tests (manual testing sufficient for utility script)
- No CI/CD pipeline
- No internationalization (English only)
- No batch CSV review workflow (interactive mode replaces this)

## Success Criteria

1. ✅ User logs in once via `--auth`, session persists across runs
2. ✅ Script uses tab strategy to preserve feed state (no back-button navigation)
3. ✅ Gemini returns valid JSON with score + reasoning (100% success rate with retries)
4. ✅ Invitations are personalized (name + specific profile details)
5. ✅ Interactive mode displays rich color-coded UI for clarity
6. ✅ Timeout only applies to initial decision, **not** during Edit mode
7. ✅ Random delays and stealth config prevent immediate bot detection
8. ✅ Conservative defaults (10 invites max, 3-8s delays, interactive mode)
9. ✅ `--dry-run` allows safe testing before live run
10. ✅ README enables complete setup in <10 minutes
11. ✅ Error handling prevents crashes on platform changes
12. ✅ Logs provide full audit trail with action types
13. ✅ User can edit messages without timeout pressure
14. ✅ Tab management is robust (closes orphaned tabs, handles errors)

## Risk Acknowledgment

This tool intentionally automates a platform that may not permit such automation. The PRD prioritizes:
1. **Safety**: Interactive mode as default, tab strategy, conservative limits, delays, session persistence
2. **Transparency**: Clear TOS warnings, cost disclosure
3. **Control**: Human-in-the-loop review, edit capability without timeout pressure, graceful quit
4. **Responsibility**: Immediate shutdown recommendations if flagged
5. **Authenticity**: Tab-based navigation mimics real power-user behavior
6. **Accessibility**: Rich UI with color coding reduces cognitive load for neurodivergent users

**Final Note**: This is not a "spam cannon" - it's a careful, human-supervised tool for targeted outreach. Tab strategy and interactive mode with rich UI ensure quality, reduce platform risk, and provide an excellent developer experience.

FilesExpand file tree

PRD.md

Latest commit

History