How to organize our docs on checking local links?

[mre]

Now that #140 is merged, how does it relate to https://lychee.cli.rs/recipes/root-dir/?
From a user's perspective, I could see how it can be confusing to find the right recipe.

I'm speculating here, but I believe that many users have similar questions in mind:

• "Does lychee support local files?"
• "Does it work out of the box?"
• "It doesn't work out of the box. How can I fix it?"
• "I have a very special case in mind and need some detailed information. Where can I find it?"

At the moment, the information is spread across multiple pages.
I wonder if we should unify that.

I see a few options:

• Do nothing; it's fine.
• Add links between recipes (/recipes/root-dir/ <-> /recipes/local-folder/)
• Provide an overview page for checking local links and link to the specific recipes
• Merge all recipes about "local link checking" into one page

There are potentially more options that I forgot.

Would love to hear everyone's thoughts.

@katrinafyi @jacobdalamb @thomas-zahner fyi

[katrinafyi]

Yeah, it's a really good question.

Links between recipes is kind of what we currently have between root-dir and base-url. I think this doesn't work so well because it's more of an afterthought rather than an entry point. It also scales poorly - as we add more pages, every page needs to link to more pages.

I also think that putting it all on one page would get too big, and not everything applies to every use case.

I think that it would be really good to make a new sidebar section for local link checking. This would let us have an overview page for the topic as a whole, then list all the other pages in a chosen order. Some other recipes could also move into this sidebar section - I'm thinking of Pretty URLs for one.

This should make it easy for users to find the answers they're looking for. Recent changes to the lychee program---namely, making InvalidBaswJoin into a proper error---also make it easier to write documentation. We can tell users to start with default lychee out of the box and guide them to specific pages based on what errors they see.

[mre]

If I could start over again, here's how I would propose to structure the docs.

lychee docs
 │
 ├─ Getting Started
 │  ├─ Overview
 │  ├─ Installation
 │  └─ Quick Start
 │
 ├─ Usage
 │  ├─ CLI Reference
 │  ├─ Configuration File
 │  ├─ Output Formats
 │  └─ File Preprocessing
 │
 ├─ Common Tasks
 │  ├─ Filtering Links (exclude patterns, paths)
 │  ├─ Handling Anchors
 │  ├─ Working with Local Files (root-dir, base-url)
 │  ├─ URL Remapping & Migration
 │  ├─ Pretty URLs
 │  ├─ Wikilinks
 │  └─ Caching Results
 │
 ├─ CI/CD Integration
 │  ├─ GitHub Actions Setup
 │  ├─ Check Pull Requests
 │  ├─ Repository Monitoring
 │  ├─ PR Comments
 │  ├─ Caching in CI
 │  └─ Archive Broken Links
 │
 ├─ Troubleshooting
 │  ├─ Rate Limits
 │  ├─ Network Errors
 │  ├─ Too Many Open Files
 │  ├─ Redirects
 │  ├─ Custom Headers
 │  ├─ Status Codes
 │  └─ Mail Addresses
 │
 └─ Advanced
    ├─ How It Works
    ├─ Library Integration
    ├─ Contributing
    ├─ Who Uses Lychee
    ├─ Sponsors
    └─ Credits

The idea is to group the pages in a "goal-oriented" way. Each section would revolve around a particular lychee use-cases:

• Getting Started --> What is lychee and how do I get it running?
• Usage --> Okay, it runs; what can I do with it?
• Common Tasks --> Common adjustments people make, roughly ordered by frequency
• CI/CD Integration --> Once people are happy, how do they run it regularly?
• Troubleshooting -- Typical issues people encounter
• Advanced --> For people who want to learn more or use lychee as a building block for their own projects

There could be sub-sections for specific topics like root-dir/base-url, so "Working with Local Files" could be an overview page.

I wonder if it's too late for this. In my opinion, there's still a chance, because we can always add redirects, but please feel free to propose alternatives. I doubt that this is the best we can do, and maybe we won't even need such a big overhaul.

[thomas-zahner]

@mre I really like this table of content proposal. 👍 We could work towards it gradually.

[mre]

Thanks. The most important part is that we don't break any links along the way. Fortunately, we got a tool for that. 😆 Oh, and to everyone else: feel free to suggest changes. It's probably still not perfect.

[katrinafyi]

I also like organising things around these goals. I think I'd still like some organisation around what I see as the two big "verticals" of lychee: checking local files and checking online websites. It's possible that someone only ever uses lychee to point at their live website, and so they'd have no need for things like root-dir or wikilinks.

This would look something like this, but I'm not super attached to it. I also think that a Recipes section is maybe worth keeping, it's just that the current recipe section has more than just recipes...

# Introduction
Getting Started [this page also has lots of info that could be split up]
Command-Line Flags
Configuration File
Library Usage

# Input, Output & Common Options
Caching
Including / Excluding Links
Including / Excluding Paths
Checking Mail Addresses
Checking Anchor Links (Fragment Identifiers)
File Preprocessing
Remapping One URL to Another
[not documented yet, but also --dump and --files-from]
[--default-extension too]

# Local Files and Markdown
Local File Checking with --root-dir
Testing Sites Not Served from Root with --base-url
Pretty URLs (Fallback extensions and Index files)
Markdown Wikilinks
Checking a Local Folder with URL Remapping

# HTTP, Networking, and Concurrency
Custom Headers [and User Agent]
Accepted Status Codes
Rate Limits
Network Errors
Too Many Open Files
Too Many Redirects

# Github Actions
[as is]

[mre]

I like the idea of grouping the docs by use-case. As you said, many users will only ever care about checking websites, while some might want to check local files (which is, in my opinion, a bit of an advanced use-case).

Most people refer to the docs while getting started or with a very special, niche problem at hand. That was the idea behind the "Common Tasks" section and the "Troubleshooting"/"Advanced" sections, but I'm beginning too think that the grouping is too broad and that splitting it up by network/offline makes more sense.

I also liked the recipes sections but they tend to grow into a big ball of mud over time. That's why I introduced the CI/CD section was to clearly separate some parts into their own section.

I've tried to integrate your feedback below. Let's keep iterating on it. Having that conversation is super helpful.

  lychee docs
  │
  ├─ Getting Started
  │  ├─ Installation
  │  └─ Quick Start
  │
  ├─ Usage
  │  ├─ CLI Reference
  │  ├─ Configuration File
  │  ├─ Output Formats
  │  └─ File Preprocessing
  │
  ├─ Common Options
  │  ├─ Filtering Links
  │  ├─ Filtering Paths
  │  └─ Checking Anchors
  │
  ├─ HTTP & Networking
  │  ├─ Caching Results
  │  ├─ URL Remapping
  │  ├─ Custom Headers & User Agent
  │  ├─ Accepted Status Codes
  │  ├─ Rate Limits
  │  ├─ Redirects
  │  ├─ Network Errors
  │  ├─ Too Many Open Files
  │  └─ Mail Addresses
  │
  ├─ Checking Local Files
  │  ├─ Root Directory (--root-dir)
  │  ├─ Base URL (--base-url)
  │  ├─ Pretty URLs
  │  ├─ Markdown Wikilinks
  │  └─ Local Folder with URL Remapping
  │
  ├─ GitHub Actions
  │  ├─ Basic Setup
  │  ├─ Checking Pull Requests
  │  ├─ Repository Monitoring
  │  ├─ Adding PR Comments
  │  ├─ Caching in CI
  │  └─ Archiving Broken Links
  │
  ├─ Developers
  │  ├─ Using as a Library
  │  ├─ How lychee Works
  │  └─ Contributing
  │
  └─ About
     ├─ Who Uses lychee?
     ├─ Sponsors
     ├─ Credits
     └─ Trivia

(Removed Overview from that list. I don't think that most people would care. We could move it into a Trivia section.)