|
1 | | -# github-org-repos-sync [](https://github.com/xbglowx/github-org-repos-sync/actions/workflows/build-test.yaml) [](https://github.com/xbglowx/github-org-repos-sync/actions/workflows/golangci-lint.yml) [](https://github.com/xbglowx/github-org-repos-sync/actions/workflows/codeql-analysis.yml) |
2 | | -Sync GitHub organization repos |
| 1 | +# github-org-repos-sync |
3 | 2 |
|
4 | | -## Build |
5 | | -1. `git clone git@github.com:xbglowx/github-org-repos-sync.git` |
6 | | -1. `go get -d .` |
7 | | -1. `go build .` |
| 3 | +[](https://github.com/xbglowx/github-org-repos-sync/actions/workflows/build-test.yaml) [](https://github.com/xbglowx/github-org-repos-sync/actions/workflows/golangci-lint.yml) [](https://github.com/xbglowx/github-org-repos-sync/actions/workflows/codeql-analysis.yml) [](https://github.com/xbglowx/github-org-repos-sync/actions/workflows/create-release.yaml) |
8 | 4 |
|
9 | | -## Auth |
10 | | -1. Authenticated access to the GitHub API via personal access token with scope **repo** |
11 | | - * `export GITHUB_TOKEN=<token>` |
12 | | -1. `git` cli authenticated to GitHub |
| 5 | +A command-line tool to efficiently sync all repositories from a GitHub organization to your local machine. Perfect for developers who need to maintain local copies of multiple repositories for backup, analysis, or bulk operations. |
13 | 6 |
|
14 | | -## How It Works |
15 | | -1. Generates a list of repos the caller has access to within the specified organization |
16 | | -1. Clones each repo if it doesn't exist locally in the destionation path |
17 | | -1. If the repo already exists: |
18 | | - 1. Switches to the default branch, only if the current branch is clean; Stashes if dirty |
19 | | - 1. Updates |
| 7 | +## Features |
| 8 | + |
| 9 | +- **Bulk Repository Sync**: Clone or update all repositories from a GitHub organization in one command |
| 10 | +- **Smart Updates**: Automatically switches to default branch and pulls latest changes |
| 11 | +- **Dirty Repository Handling**: Safely stashes uncommitted changes before updating |
| 12 | +- **Parallel Processing**: Configurable concurrency for faster synchronization |
| 13 | +- **Repository Filtering**: Include/exclude specific repositories by name pattern |
| 14 | +- **Archived Repository Support**: Option to skip archived repositories |
| 15 | +- **Permission Aware**: Automatically skips repositories without pull permissions |
| 16 | +- **Empty Repository Handling**: Gracefully handles empty repositories without errors |
| 17 | + |
| 18 | +## Installation |
| 19 | + |
| 20 | +### Download Pre-built Binaries |
| 21 | + |
| 22 | +Download the latest release for your platform from the [releases page](https://github.com/xbglowx/github-org-repos-sync/releases). |
| 23 | + |
| 24 | +### Build from Source |
| 25 | + |
| 26 | +1. **Clone the repository**: |
| 27 | + ```bash |
| 28 | + git clone git@github.com:xbglowx/github-org-repos-sync.git |
| 29 | + cd github-org-repos-sync |
| 30 | + ``` |
| 31 | + |
| 32 | +2. **Install dependencies**: |
| 33 | + ```bash |
| 34 | + go mod download |
| 35 | + ``` |
| 36 | + |
| 37 | +3. **Build the binary**: |
| 38 | + ```bash |
| 39 | + go build . |
| 40 | + ``` |
| 41 | + |
| 42 | +## Prerequisites |
| 43 | + |
| 44 | +### Authentication |
| 45 | + |
| 46 | +1. **GitHub Personal Access Token**: Create a token with `repo` scope |
| 47 | + ```bash |
| 48 | + export GITHUB_TOKEN=<your-token> |
| 49 | + ``` |
| 50 | + |
| 51 | +2. **Git CLI Authentication**: Ensure `git` is authenticated with GitHub (SSH keys or credential helper) |
20 | 52 |
|
21 | 53 | ## Usage |
22 | | -1. `./github-org-repos-sync --help` |
| 54 | + |
| 55 | +### Basic Usage |
| 56 | + |
| 57 | +```bash |
| 58 | +# Sync all repositories from an organization |
| 59 | +./github-org-repos-sync <organization> -d <destination-path> |
| 60 | + |
| 61 | +# Example |
| 62 | +./github-org-repos-sync myorg -d ~/repos/myorg |
| 63 | +``` |
| 64 | + |
| 65 | +### Advanced Options |
| 66 | + |
| 67 | +```bash |
| 68 | +# Show all available options |
| 69 | +./github-org-repos-sync --help |
| 70 | + |
| 71 | +# Sync with custom parallelism |
| 72 | +./github-org-repos-sync myorg -d ~/repos -p 5 |
| 73 | + |
| 74 | +# Skip archived repositories |
| 75 | +./github-org-repos-sync myorg -d ~/repos --skip-archived |
| 76 | + |
| 77 | +# Filter repositories (include only those containing "api") |
| 78 | +./github-org-repos-sync myorg -d ~/repos --include "api" |
| 79 | + |
| 80 | +# Exclude specific repositories |
| 81 | +./github-org-repos-sync myorg -d ~/repos --exclude "test" |
| 82 | +``` |
| 83 | + |
| 84 | +## How It Works |
| 85 | + |
| 86 | +1. **Repository Discovery**: Fetches a list of all repositories in the specified organization that you have access to |
| 87 | +2. **Local Check**: Determines if each repository already exists locally |
| 88 | +3. **Clone or Update**: |
| 89 | + - **New repositories**: Clones them to the destination path |
| 90 | + - **Existing repositories**: |
| 91 | + - Fetches latest changes from remote |
| 92 | + - Stashes uncommitted changes if the working directory is dirty |
| 93 | + - Switches to the repository's default branch |
| 94 | + - Pulls latest changes with rebase |
| 95 | +4. **Parallel Processing**: Processes multiple repositories concurrently for improved performance |
| 96 | +5. **Error Handling**: Gracefully handles edge cases like empty repositories, missing branches, and permission issues |
| 97 | + |
| 98 | +## Command Line Options |
| 99 | + |
| 100 | +| Flag | Description | Default | |
| 101 | +|------|-------------|---------| |
| 102 | +| `-d, --dest` | Destination directory for repositories | Required | |
| 103 | +| `-p, --parallelism` | Number of concurrent operations | 10 | |
| 104 | +| `--skip-archived` | Skip archived repositories | false | |
| 105 | +| `--include` | Include only repositories containing this string | "" | |
| 106 | +| `--exclude` | Exclude repositories containing this string | "" | |
| 107 | +| `-h, --help` | Show help message | - | |
| 108 | + |
| 109 | +## Examples |
| 110 | + |
| 111 | +### Backup an entire organization |
| 112 | +```bash |
| 113 | +./github-org-repos-sync mycompany -d ~/backups/mycompany-repos |
| 114 | +``` |
| 115 | + |
| 116 | +### Sync only active repositories |
| 117 | +```bash |
| 118 | +./github-org-repos-sync mycompany -d ~/repos --skip-archived |
| 119 | +``` |
| 120 | + |
| 121 | +### Sync only service repositories |
| 122 | +```bash |
| 123 | +./github-org-repos-sync mycompany -d ~/services --include "service" |
| 124 | +``` |
| 125 | + |
| 126 | +### Exclude test repositories |
| 127 | +```bash |
| 128 | +./github-org-repos-sync mycompany -d ~/repos --exclude "test" |
| 129 | +``` |
| 130 | + |
| 131 | +## Troubleshooting |
| 132 | + |
| 133 | +### Permission Issues |
| 134 | +- Ensure your GitHub token has `repo` scope |
| 135 | +- Verify you have access to the organization |
| 136 | +- Check that git CLI is properly authenticated |
| 137 | + |
| 138 | +### Network Issues |
| 139 | +- The tool will retry failed operations |
| 140 | +- Use lower parallelism (`-p 1`) for unstable connections |
| 141 | + |
| 142 | +### Large Organizations |
| 143 | +- Consider using filters (`--include`/`--exclude`) for large organizations |
| 144 | +- Monitor disk space when syncing many repositories |
| 145 | + |
| 146 | +## Contributing |
| 147 | + |
| 148 | +1. Fork the repository |
| 149 | +2. Create a feature branch (`git checkout -b feature/amazing-feature`) |
| 150 | +3. Commit your changes (`git commit -m 'Add amazing feature'`) |
| 151 | +4. Push to the branch (`git push origin feature/amazing-feature`) |
| 152 | +5. Open a Pull Request |
| 153 | + |
| 154 | +## License |
| 155 | + |
| 156 | +This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details. |
0 commit comments