WIP: Appropriate Token error

[Sundraiz-Shah]

What changed

Before this fix, when something went wrong (wrong token, expired token, wrong URL), the tool would silently fail with a generic or confusing message. Now it tells you exactly what the problem is.

Situation	Before	After
Wrong GitHub token	"nothing harvested" (no reason given)	"The provided GitHub token is invalid"
Expired GitHub token	"nothing harvested" (no reason given)	"The GitHub token has expired"
Wrong GitHub URL	"nothing harvested" (no reason given)	"GitHub repository not found"
Wrong GitLab token	"GitLab requires a valid personal access token"	"The provided GitLab token is invalid"
Expired GitLab token	"GitLab requires a valid personal access token"	"The GitLab token has expired"
Wrong GitLab URL	"GitLab requires a valid personal access token"	"GitLab repository not found"

In short: instead of a confusing dead end, you now get a clear message telling you what went wrong and where.

Issue #240 Appropriate error when token is expired (Solved)
Issue #209 Differentiate error between URL and token (Solved)

[Aidajafarbigloo]

Thanks for the changes. There are still cases where the system does not display an appropriate error message to the user. Please address this and ensure error handling is consistent across various relevant scenarios, and validate the fix across multiple repositories to cover different cases and conditions.

[Aidajafarbigloo]

returns also for gh repos

[Aidajafarbigloo]

Instead I received "Unexpected response from GitLab API (HTTP 400)."

[Sundraiz-Shah]

What We Changed and Why
Problem We Were Solving
The tool only knew how to handle github.com and gitlab.com. Any other repository host company GitLab servers, university Git platforms, GitHub Enterprise was either misidentified or silently sent to the wrong API, giving confusing errors.

Change 1 — Smarter URL Detection (classify_url)
Before: The code checked "github" in url — a simple text search across the entire URL. A URL like gitlab.example.com/owner/github-mirror would be mistakenly treated as a GitHub repo because the word "github" appeared in the path.

After: We now parse only the domain part of the URL (e.g. github.company.com) and classify it into one of five types: public GitHub, GitHub Enterprise, public GitLab, self-hosted GitLab, or unknown. This is much more accurate.

Change 2 — GitHub Enterprise Support (_build_github_api_url)
Before: The code built the API address by doing a simple text replacement: swapping github.com with api.github.com/repos. For a company GitHub server like github.company.com, this produced a broken address.

After: We build the API address from the parsed URL pieces (owner name, repo name, host). For company GitHub servers, the correct API address format (/api/v3/) is used automatically.

Change 3 — Unknown Hosts No Longer Treated as GitLab
Before: Any URL that wasn't GitHub was assumed to be GitLab — including Bitbucket, Forgejo, or any unrecognised platform. This sent requests to the wrong API and gave misleading error messages.

After: Unknown hosts are identified as "unsupported" and shown a clear message. As a bonus, we also probe the host silently to check if it might be a self-hosted GitLab that just doesn't have "gitlab" in its name — if so, we handle it correctly.

Change 4 — Consistent Response Format
Before: GitHub success responses were missing the instance_type field that GitLab responses included. This could cause crashes in code that expected the field to always be present.

After: All responses now consistently include a forge field (renamed from instance_type) telling the caller exactly what type of platform was detected.

Change 5 — Accurate GitHub Token Error Message
Before: The code tried to detect whether a GitHub token was "expired" vs "invalid" by looking at the error text. But GitHub's API returns the exact same message (Bad credentials) for both cases — so expired tokens were always shown as "invalid", which was confusing.

After: The message now says "invalid or has expired", which is honest about both possibilities and tells the user exactly what to do.

[Aidajafarbigloo]

For the scenario where the input URL points to a self-hosted repository, whether the token is valid but does not match the repository, or the token itself is invalid, similar error message as following is appropriate:
"The token is not usable for this type of repository. To extract data from this repository, provide a token that complies with your organization's requirements."