Feature idea: force redirects

[WilliamDEdwards]

Expectation

Site A:

"Want to know XYZ? Look here! http://example.com/docs/look-here"

Documentation:

look-here moved to look-over-there.

look-here is redirected to look-over-there using mkdocs-redirects.

Reality

Site A:

"Want to know XYZ? Look here! http://example.com/docs/look-here"

Documentation:

look-here moved to look-over-there.

look-here is not redirected to look-over-there using mkdocs-redirects, because someone forgot.

Site A now has a dead link.

What aren't solutions?

• In a controlled environment, dead links can be detected on the linking side. But not when sites that you don't control link to your documentation. Also not when mkdocs sites link to each other: mkdocs doesn't support dead link detection natively (Add command to detect broken links mkdocs#2051)
• Relying on the reviewer to think "this link moved, do we have a redirect?". PR checklists are too invasive, as moving content is not common, but extremely frustrating for users/readers when it causes a dead link.

Practical

I have no ideas as to how this could be implemented in practice. After all, mkdocs has no state, so no way to know which links used to exist, and now don't, ergo which links need a redirect.

[pawamoy]

Hi @WilliamDEdwards 🙂

I'm not sure to understand. You're talking about links to external sites right? If the external site doesn't setup a redirect, then yes, you're doomed. You can detect that the link is now broken, for example with linkchecker, but you won't know how to update it.

Therefore I'm not sure why we're even discussing this here as mkdocs-redirect has absolutely no way of solving that 😅

[WilliamDEdwards]

Hi @pawamoy,

No, I'm talking about links to mkdocs sites. Sorry if I phrased things confusingly.

mkdocs-redirects does a great job at redirecting A to B. But if one moves A to B, and forgets to redirect A to B, there is no way to detect that - except for a review process, which is more error-prone by definition. Anything that links to A now has a dead link. From a UX perspective, dead links - especially in documentation - are extremely frustrating.

Therefore, it would be useful to enforce the existence of redirects of all links that existed on an mkdocs site - so that links to the mkdocs will never become dead.

As mentioned in the issue, I don't see a viable way to do so, as mkdocs has no way of knowing which links existed. However, I at least wanted to pitch the idea, as such a feature would be very useful in practice.

[pawamoy]

Aaaaaaaaah much clearer, thanks. And that's a really good idea, too! I think it's doable by implementing a single option: a link to a sitemap file, which we can download and use to check for missing URLs.

plugins:
  - redirects:
      check: https://mysite.com/some-project/sitemap
      redirect_maps:
        old.md: new.md

Example sitemap: https://pawamoy.github.io/markdown-exec/sitemap.

Example code to extract URLs from the sitemap:

from xml.etree import ElementTree
tree = ElementTree.parse("sitemap.xml")
urls = [elem.text for elem in tree.iter() if "loc" in elem.tag]
print("\n".join(urls))

[pawamoy]

Actually since we have the site_url in mkdocs.yml we don't even need to add a new option. I'll also note that this can totally be implemented in a separate plugin.

[WilliamDEdwards]

That approach assumes that wherever builds docs can access site_url. Which might not be the case in enterprisey environments. But it’s the only option AFAIK, and perfectly acceptable if documented, IMO.

Enforcing redirects would also be useful for anchors (e.g. if a header title is renamed), but that may be somewhat out of scope for now.

Thanks for thinking along! And if it helps speed things up, I can discuss donating for a bounty.

[pawamoy]

Yep, pages themselves might not be reachable from the developer's workstation / online IDE. In that case we could imagine letting the user provide a local path to a sitemap file, and they'd be responsible for downloading it to their workstation / uploading it to their online IDE / tracking it with Git / caching it somewhere.

Supporting anchors would require more work indeed. It wouldn't be that hard though, so maybe could be done at once. We would generate a file with URLs and their anchors, that we can download again (as explained previously) to check against current build.

def register_anchors(self, url, anchors: Sequence[AnchorLink]) -> None:
    for anchor in anchors:
        self._urls.append(f"{page.url}#{anchor.id}")
        self.register_anchors(url, anchor.children)

def on_page_content(self, html: str, page: Page, **kwargs: Any) -> str:
        self._urls.append(page.url)
        self.register_anchors(page.url, page.toc.items)
        return html