Skip to content

USER_GUIDE.md

Latest commit

 

History

History
695 lines (503 loc) · 19 KB

USER_GUIDE.md

File metadata and controls

695 lines (503 loc) · 19 KB

📖 Harwest - User Guide

Automated archiving of competitive programming submissions from Codeforces and AtCoder.

📋 Quick Setup

  1. Fork & clone this repository
  2. Install dependencies: pip install -r requirements.txt
  3. Run setup: python fresh_start.py
  4. Enable GitHub Actions in repository settings

Done! Harvesting runs automatically daily at 11:00 PM BDT.

🤖 Automated Harvesting

  • Daily schedule: 11:00 PM BDT (5:00 PM UTC)
  • Smart scanning: Checks last 30 days for efficiency
  • Manual trigger: Use Actions tab → "Run workflow"
  • Full scan option: For complete history or recovery

💻 Local Usage

Prerequisites

  • Python 3.7+
  • Git
  • Platform usernames

Basic Commands

# Quick harvest (last 30 days)
python -m harwest codeforces
python -m harwest atcoder

# Full scan (all submissions)
python -m harwest codeforces --full-scan
python -m harwest atcoder --full-scan

# Specific page
python -m harwest codeforces --start-page 5

Windows Control Panel

Easiest option: Double-click harwest_control_panel.bat

Menu options:

  • [1] Codeforces harvest
  • [2] AtCoder harvest
  • [3] Both platforms
  • [4] Reset & reconfigure
  • [5] Help

🔧 Configuration

File: config/users.json

{
  "codeforces": ["your_handle"],
  "atcoder": ["your_handle"]
}

Multiple accounts supported:

{
  "codeforces": ["handle1", "handle2"],
  "atcoder": ["handle1"]
}

🐛 Troubleshooting

Common Issues

  • "No module named 'harwest'" → Run pip install -r requirements.txt
  • "Config file not found" → Run python fresh_start.py
  • "Authentication failed" → Check repository permissions
  • Empty submissions → Check username spelling on platform

Git Issues

# Fix remote
git remote set-url origin https://github.com/YOUR_USERNAME/codeforces-atcoder-submissions.git

# Reset if needed
python fresh_start.py

📝 Command Reference

Command Description
python fresh_start.py Interactive setup/reset
harwest_control_panel.bat Windows GUI menu
python -m harwest PLATFORM Harvest recent submissions
python -m harwest PLATFORM --full-scan Harvest all submissions
python -m harwest PLATFORM --start-page N Start from page N

💡 Key Features

  • ✅ Zero maintenance - Fully automated
  • ✅ Multi-platform - Codeforces + AtCoder
  • ✅ Smart updates - Only recent submissions
  • ✅ Profile integration - GitHub contributions
  • ✅ Fallback links - Never lose submissions
  • ✅ Multi-account - Track multiple handles

❓ FAQ

Q: How does the 30-day check work?
A: Daily runs fetch last 30 days only (fast). Full scan gets everything.

Q: Can I track multiple accounts?
A: Yes, add multiple usernames in config/users.json arrays.

Q: Will this work in private repositories?
A: Yes, everything works the same.

Q: What if code fetch fails?
A: Automatic fallback links to platform submissions.

🎉 Happy harvesting! Automatically archive your competitive programming journey.

📑 Table of Contents

🚀 Quick Setup (5 Minutes)

Step 1: Fork & Clone

  1. Fork this repository: https://github.com/MishkatIT/codeforces-atcoder-submissions
  2. Clone your fork:
git clone https://github.com/YOUR_USERNAME/codeforces-atcoder-submissions.git
cd codeforces-atcoder-submissions

Step 2: Install Dependencies

pip install -r requirements.txt

Step 3: Run Fresh Start Setup

python fresh_start.py

What this script does:

  • ✅ Analyzes your repository structure
  • ✅ Clears previous owner's data (with confirmation)
  • ✅ Configures your Codeforces username
  • ✅ Configures your AtCoder username
  • ✅ Sets up git author information
  • ✅ Initializes fresh markdown files
  • ✅ Validates configuration

Interactive Example:

📊 ANALYZING REPOSITORY
Found existing data:
  • submissions.json           156 submissions
  • codeforces/               142 files
  • atcoder/                   14 files

⚠️  Do you want to DELETE all this data? (yes/no): yes

🧹 CLEARING SUBMISSION DATA
✅ Cleared: submissions.json, codeforces/, atcoder/

👤 CONFIGURE USERNAMES
Codeforces username: your_handle
AtCoder username: your_handle

✅ Username configuration saved!
✨ Setup complete!

Step 4: Commit & Push

git add .
git commit -m "Configure fork for personal use"
git push

Step 5: Enable GitHub Actions

  1. Go to your repository on GitHub
  2. Click "Actions" tab
  3. Click "I understand my workflows, go ahead and enable them"

✅ Setup Complete! Your repository will now harvest submissions automatically.

🤖 Automated Harvesting (GitHub Actions)

📅 Default Schedule

GitHub Actions runs automatically every day at 11:00 PM BDT (5:00 PM UTC).

What happens during each run
  • ✅ Checks last 30 days for new submissions (smart & efficient)
  • ✅ Fetches submission code with proper timestamps
  • ✅ Updates problem tags/ratings if they changed on platform
  • ✅ Generates/updates markdown tables (codeforces.md, atcoder.md)
  • ✅ Commits and pushes changes to your repository
  • ✅ Creates fallback links if code fetch fails

💡 Smart Scheduling: If repository is inactive >30 days, switches to monthly runs (1st of month) to save Actions minutes.

⚡ Manual Trigger

Want to run immediately or do a full scan?

  1. Navigate to Actions tab
  2. Click "Harwest Submissions" workflow
  3. Click "Run workflow" (top-right)
  4. Choose options:
    • Platform: all (default), codeforces, or atcoder
    • Full scan: ✅ Check for complete history | ⬜ Uncheck for last 30 days
  5. Click "Run workflow" button

When to use full scan:

  • ✅ First time setup
  • ✅ Complete re-sync needed
  • ✅ Recovering missing submissions
  • ❌ Regular daily updates (use default)

🔧 Customize Schedule

Want to change when harvesting runs?

Edit .github/workflows/harwest.yml:

schedule:
  - cron: '0 17 * * *'  # 11:00 PM BDT (5:00 PM UTC)

Examples:

  • 0 0 * * * - Midnight UTC daily
  • 0 12 * * * - Noon UTC daily
  • 0 0 * * 0 - Midnight UTC every Sunday

💻 Local Usage & Manual Harvesting

This section covers running Harwest manually on your local machine, including detailed setup, harvesting commands, and automation options.

📋 Prerequisites

  • Python 3.7 or higher
  • Git installed and configured
  • Your Codeforces/AtCoder usernames

🚀 Local Quick Start

1. Clone Your Repository

git clone https://github.com/YOUR_USERNAME/codeforces-atcoder-submissions.git
cd codeforces-atcoder-submissions

2. Install Dependencies

pip install -r requirements.txt

3. Configure Usernames & Git (Recommended)

Run the interactive setup:

python fresh_start.py

This script will:

  • Clear old data (if you want)
  • Ask for your Codeforces/AtCoder usernames
  • Set up git author info
  • Initialize config and markdown files

Manual alternative: Edit config/users.json:

{
  "codeforces": ["YourCodeforcesHandle"],
  "atcoder": ["YourAtCoderHandle"]
}

📥 Harvesting Submissions

Codeforces

# Harvest all submissions (full scan)
python -m harwest codeforces --full-scan
# Harvest only new submissions (incremental)
python -m harwest codeforces
# Harvest from a specific page
python -m harwest codeforces --start-page 5

AtCoder

# Harvest all submissions (full scan)
python -m harwest atcoder --full-scan
# Harvest only new submissions (incremental)
python -m harwest atcoder
# Harvest from a specific page
python -m harwest atcoder --start-page 3

Automation Mode (Non-Interactive)

For scripts or CI/CD:

python -m harwest codeforces --auto --full-scan
python -m harwest atcoder --auto --full-scan

📊 Markdown Output

  • codeforces.md — Codeforces submissions table
  • atcoder.md — AtCoder submissions table

These are auto-generated/updated after each harvest.

🔧 Advanced Usage

  • Multiple accounts: Add more usernames in config/users.json arrays
  • Custom submissions directory: Set the directory field in users.json
  • Language mapping: Edit harwest/lib/resources/language.json to add new language extensions

🧙‍♂️💡 Control Panel (Windows) — Highly Recommended!

✨ The easiest way to manage everything locally!

Just double-click or run:

harwest_control_panel.bat
  • 🟢 Menu-driven interface for harvesting, full setups, and more
  • 🟢 No need to remember commands
  • 🟢 Perfect for Windows users

Why use it:

  • 🟢 Menu-driven interface for harvesting, full setups, and quick actions
  • 🟢 No need to remember flags or commands
  • 🟢 Built-in options for Normal or Full scans and System Reset and Update User Info

Menu options (what it runs):

  • [1] Harvest Codeforces -> python -m harwest codeforces (or --full-scan when selected)
  • [2] Harvest AtCoder -> python -m harwest atcoder (or --full-scan when selected)
  • [3] Harvest Both Platforms -> runs Codeforces then AtCoder
  • [4] System Reset and Update User Info -> python fresh_start.py (interactive setup)
  • [5] Help -> shows help text
  • [0] Exit

Notes & tips:

  • Ensure python and git are available in your PATH before running the batch file.
  • Run the batch file from the repository root so paths resolve correctly.
  • You can schedule this .bat using Windows Task Scheduler for daily automation (see "Scheduled Tasks" later).

🔄 Common Workflows

📅 Daily Manual Update

# Harvest new submissions from both platforms
python -m harwest codeforces
python -m harwest atcoder

# Commit and push changes
git add .
git commit -m "Update submissions $(date +'%Y-%m-%d')"
git push

📁 Repository Structure

Click to expand repository structure

After first harvest, your repository will have this structure:

📦 codeforces-atcoder-submissions/
├── 📄 README.md                   # Repository overview
├── 📄 USER_GUIDE.md               # This comprehensive guide
├── 📄 codeforces.md               # Codeforces submissions table
├── 📄 atcoder.md                  # AtCoder submissions table
├── 📄 fresh_start.py              # Interactive setup script
├── 📄 harwest_control_panel.bat   # Windows control panel (menu-driven)
├── 📄 requirements.txt            # Python dependencies
├── 📄 setup.py                    # Package installer (for devs)
├── 📄 LICENSE                     # License file
├── 📄 MANIFEST.in                 # Package manifest
│
├── 📂 submissions/
│   ├── 📄 submissions.json        # Metadata database
│   ├── 📂 codeforces/
│   │   └── 📂 [contest_id]/       # e.g., "1234/"
│   │       └── 📄 [problem].[ext] # e.g., "A_Problem_Name.cpp"
│   └── 📂 atcoder/
│       └── 📂 [contest_id]/       # e.g., "abc123/"
│           └── 📄 [problem].[ext] # e.g., "abc123A.cpp"
│
├── 📂 config/
│   ├── 📄 README.md               # Configuration documentation
│   └── 📄 users.json              # Your platform usernames
│
├── 📂 harwest/                    # Main Python package
│   ├── 📄 __init__.py
│   ├── 📄 __main__.py
│   ├── 📄 harwest.py              # Core harvest logic
│   └── 📂 lib/                    # Platform implementations
│       ├── 📄 abstractworkflow.py
│       ├── 📂 codeforces/
│       ├── 📂 atcoder/
│       ├── 📂 resources/
│       └── 📂 utils/
│
└── 📂 .github/
    └── 📂 workflows/
        └── 📄 harwest.yml         # GitHub Actions automation

🐛 Troubleshooting

❌ No Submissions Appearing

Verify Configuration:

  • ✅ Check config/users.json has correct usernames (case-sensitive!)
  • ✅ Confirm workflow ran successfully (Actions tab → green ✓)
  • ✅ Ensure you have accepted submissions on the platform
  • ✅ Check username spelling matches platform exactly

Review Workflow Logs:

  1. Go to Actions tab
  2. Click the failed/completed workflow run
  3. Expand "Harvest [Platform] submissions" step
  4. Look for error messages or warnings

❌ GitHub Actions Workflow Failing

Error Message Solution
"No username configured" Edit config/users.json with valid usernames
"Config file not found" Ensure config/users.json exists and is committed
"Rate limiting" Wait 10-15 minutes; will retry automatically tomorrow
"Permission denied" Re-enable GitHub Actions in Settings → Actions
"Authentication failed" Check repository permissions

❌ Local Command Errors

"No module named 'harwest'":

cd codeforces-atcoder-submissions
pip install -r requirements.txt

"Config file not found":

# Create config/users.json
mkdir -p config
echo '{"codeforces": ["handle"], "atcoder": ["handle"]}' > config/users.json

"Git push failed"

# Verify remote is configured
git remote -v

# Add remote if missing
git remote add origin https://github.com/YOUR_USERNAME/codeforces-atcoder-submissions.git

# Or update existing remote
git remote set-url origin https://github.com/YOUR_USERNAME/codeforces-atcoder-submissions.git

"Unknown language warning":

  • Add the language to harwest/lib/resources/language.json

"Empty submissions.json":

  • This is normal for a new setup. Run a harvest command.

❌ Tags/Ratings Not Updated

Tags and ratings automatically update! Daily runs check last 30 days and update metadata if problems were re-rated.

❌ Old Owner's Data Still Present

python fresh_start.py

The script will clear everything and reconfigure.

📝 Command Reference

Main Commands

Command Description
python fresh_start.py Interactive setup/reset
harwest_control_panel.bat The one batch to rule them all! Launch this for a magical menu (and a few laughs) to harvest everything locally.
python -m harwest --help Show help
python -m harwest PLATFORM Check recent submissions
python -m harwest PLATFORM --full-scan Harvest all submissions
python -m harwest PLATFORM --start-page N Start from page N
python -m harwest PLATFORM --auto Non-interactive mode

Advanced Options (Automation/CI)

Command Description
python -m harwest PLATFORM --directory PATH Custom submissions directory (overrides config)
python -m harwest PLATFORM --author-name NAME Set git author name (overrides config)
python -m harwest PLATFORM --author-email EMAIL Set git author email (overrides config)
python -m harwest PLATFORM --remote-url URL Set git remote URL (overrides config)

Platforms

  • codeforces - Codeforces submissions
  • atcoder - AtCoder submissions

Note: Some options shown in --help may not be fully implemented. Use python fresh_start.py for initial setup.

💡 Pro Tips

  1. First Run: Always use --full-scan to get complete history
  2. Fast Updates: Daily auto-runs check only last 30 days
  3. Tag Updates: Tags/ratings update automatically when changed
  4. Multiple Accounts: Add multiple handles: ["handle1", "handle2"]
  5. Private Fork: Works perfectly in private repositories
  6. Failed Code Fetch: Fallback links to platform automatically created
  7. Fresh Start Anytime: Run python fresh_start.py to setup

🔄 Need to Setup?

Clear everything and start fresh:

python fresh_start.py

Works for:

  • ✅ New fork users (clears previous owner's data)
  • ✅ Existing users wanting fresh start
  • ✅ Reconfiguring usernames
  • ✅ Fixing broken setup

📚 Additional Resources

❓ FAQ

Q: What is setup.py for? A: It's for installing Harwest as a PyPI package (developers only). Fork users don't need it.

Q: Can I use this with private repository? A: Yes! Everything works the same in private repositories.

Q: How does the 30-day check work? A: Daily auto-runs fetch submissions from last 30 days only (fast). Full scan gets everything (slower).

Q: What if code fetch fails?
A: A fallback link to the submission on the platform is automatically created. No submission is lost.

Q: Can I track multiple accounts?
A: Yes! Add multiple usernames in config/users.json:

{
  "codeforces": ["handle1", "handle2", "handle3"],
  "atcoder": ["handle1", "handle2"]
}

Q: Will this work if my repository is inactive?
A: Yes! After 30 days of inactivity, auto-runs switch to monthly (1st of month) to save Actions minutes. Manual triggers always work.

Q: How do I change my username?
A: Edit config/users.json or run python fresh_start.py to reconfigure interactively.

Q: Does this consume my GitHub Actions minutes?
A: Yes, but efficiently. Regular runs are very quick (~1-2 minutes). Smart scheduling reduces usage when inactive.

🎉 You're All Set!

Your automated competitive programming archive is ready!

📈 What Happens Next

  1. 💻 You solve problems on Codeforces or AtCoder
  2. 🤖 Harwest runs daily at 11:00 PM BDT (or manually triggered)
  3. 📊 Markdown files update with new submissions and tags
  4. 📈 GitHub profile grows with your coding contributions

🆘 Need Help?

  • 📖 Review sections above for detailed guidance
  • 🔍 Check Actions logs for workflow details
  • 🐛 Open an issue if you encounter problems
  • ⭐ Star the original project if helpful!

Happy Coding! 🚀

Automatically archive your competitive programming journey

Last Updated: January 20, 2026