Scraping GitLab repositories is required if we want to test them on Crater (and we do). This issue tracks the implementation of the scraper. The API calls we would need to make to scrape are: * Get a bunch of repositories with [`/api/v4/projects`](https://docs.gitlab.com/ee/api/projects.html#list-all-projects), filtering only Rust ones * Call [`/api/v4/projects/{id}/repository/tree`](https://docs.gitlab.com/ee/api/repositories.html#list-repository-tree) to check if the cargo files exists This is currently blocked on: * [x] Extend the GitLab `/projects` API to support filtering only Rust repos ([GitLab issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/51759))
Scraping GitLab repositories is required if we want to test them on Crater (and we do). This issue tracks the implementation of the scraper.
The API calls we would need to make to scrape are:
/api/v4/projects, filtering only Rust ones/api/v4/projects/{id}/repository/treeto check if the cargo files existsThis is currently blocked on:
/projectsAPI to support filtering only Rust repos (GitLab issue)