Adds enhanced search (#287)

* Add enhanced search

* Include search js in sphinx build

* Use /_/ proxy on RTD

* Use /_/ proxy on RTD

* Second attempt at using _ RTD api

* Fix bug with keyboard navigation on project tabs

* Add some default classes

* Tidy up code

A more compact version of the URL building. Remove/define some variables.

* Change button style

Switching to classes `sd-btn sd-bg-primary sd-bg-text-primary` from https://sunpy--287.org.readthedocs.build/projects/sunpy-sphinx-theme/287/web-components.html

* Add filter icon

* Add Python side config

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Extract search CSS and re-org some files

* Fix some search css

* Add an editorconfig file because indentation

* More dark theme fixes

* Keep the linter happy

* Fix a config bug

* Add documentation

---------

Co-authored-by: Stuart Mumford <stuart@cadair.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nabil Freij <nabil.freij@gmail.com>

Author: 4 people

docs/customizing.rst

@@ -1,3 +1,5 @@
+.. _customizing:
+
 Customizing the Theme
 =====================
 
@@ -9,6 +11,9 @@ Changing the Theme Settings
 As well as all the default configuration settings we inherit from the pydata-sphinx-theme the sunpy-sphinx-theme provides the following configuration options.
 All of these options should be set inside the ``html_theme_options`` dictionary in your ``conf.py`` file.
 
+General Configuration
+^^^^^^^^^^^^^^^^^^^^^
+
 ``footer_center``
 #################
 
@@ -50,6 +55,67 @@ It's possible to add dropdown menus to the topnav by setting document equal to a
 Footer links are a number of links which will get placed in the center of the footer.
 They are specified in the same format as the ``navbar_links``.
 
+Analytics with Goat Counter
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The theme supports using `Goat Counter <https://www.goatcounter.com/>`__ to monitor traffic. Goat Counter is an Open Source project with free hosting, so a good fit for many projects using this theme.
+GoatCounter support is enabled by default for the ``sunpy.org`` domain, so if your ``sst_site_root="https://sunpy.org"`` then it will be enabled with the default SunPy settings.
+To enable it for other sites you need to do configure the following options.
+
+``goatcounter_analytics_url``
+#############################
+
+This is the analytics URL for the GoatCounter project, for example ``"https://sunpy.goatcounter.com/count"``.
+
+``goatcounter_non_domain_endpoint``
+###################################
+
+This setting configures how analytics work for domains which are not ``sst_site_root`` or subdomains of the root.
+The default is ``False`` meaning that if the domain isn't ``sst_site_root`` then GoatCounter is disabled.
+If you set this to a GoatCounter endpoint (such as ``goatcounter_analytics_endpoint`` then that endpoint will be used for domains which are not ``sst_site_root``.
+
+
+Read the Docs Search
+^^^^^^^^^^^^^^^^^^^^
+
+The theme supports an enhanced version of the `Server Side Search <https://docs.readthedocs.com/platform/stable/server-side-search/index.html>`__ Read the Docs add-on.
+This adds a similar UI, but enables searching multiple Read the Docs projects simultaneously, not just the current RTD project and all subprojects as supported by the official add-on.
+This means that when configured this enables all RTD projects using the theme to search all others, i.e. if you have a parent project "sunpy" and a subproject "ndcube" this search allows users in the "ndcube" documentation to also search "sunpy", whereas the official one only allows users in "sunpy" to also search "ndcube".
+
+This is enabled by default, but will not work unless accessing the documentation via Read the Docs, if the enhanced search fails for any reason it will fallback to the standard sphinx search.
+To facilitate debugging of this extension, it supports proxying requests to the RTD search API through the `corsproxy.io <https://corsproxy.io>`__ service if the ``?debug`` query parameter is set on the URL, set this if testing the search on a local build.
+
+``rtd_search``
+##############
+
+Enable or disable the RTD search functionality. Defaults to ``True``.
+
+``rtd_search_projects``
+#######################
+
+This configuration option is the list of projects that the search will query the RTD API for results.
+By default this configuration uses all the links in ``navbar_links``.
+The format of this option is a list of dictionaries with ``"name"`` and ``"links"`` keys, i.e.:
+
+.. code-block:: python
+
+   rtd_search_projects = [
+       {"name": "sunpy", "link": "https://docs.sunpy.org"}
+   ]
+
+
+``rtd_search_load_more_label``
+##############################
+
+This is the label shown on the button to load more results from the RTD API.
+The default is ``"Load more results"``.
+
+``rtd_search_no_results_label``
+###############################
+
+This is the label shown when no results match the query.
+The default is ``"There are no results for this search"``.
+
 Adjusting the Styling
 ---------------------
 

docs/index.rst

@@ -4,18 +4,22 @@ sunpy-sphinx-theme
 The ``sunpy-sphinx-theme`` is a `sphinx` theme designed for the needs of the SunPy Project's main website and subproject documentation pages.
 It's primary goals are:
 
-* To provide a consistent look and feel over all the sunpy.org websites and subdomains.
-* To give all documentation pages maintained by the SunPy project a branded appearance.
+* To provide a consistent look and feel over all the separate documentation sites for the project.
+* To give all documentation pages maintained a consistent branded appearance.
 * To provide a global navigation bar over all these sites so that it is possible to navigate between the sunpy.org site and all the documentation.
 
 Version 2 of this package achieves this by being a sub-theme of the `pydata-sphinx-theme` with a heavily modified header bar.
 The core changes between this theme and the `pydata-sphinx-theme` are:
 
 * Remove the toctree navigation from the header bar and place it all in the sidebar (in the same manner as sphinx-book-theme).
-* Add links to the header bar which are specified as theme config variables, with defaults for sunpy.org.
+* Add links to the header bar which are specified as theme config variables, so they are the same over all sites.
 * Add a center element to the footer bar.
 * Restyled the theme for SunPy colors.
 * Added optional support for analytics though `goatcounter.com <https://www.goatcounter.com/>`__ which defaults to on for sunpy.org domains.
+* Added optional support for a multi-project Read the Docs search interface.
+
+The theme is highly configurable, although the defaults are for the SunPy project, this theme can be used for other organisations where they have multiple documentation sites and want a unified user experience.
+For information on how to configure this theme for your site see :ref:`customizing`.
 
 .. grid:: 1 2 2 2
    :gutter: 3

src/sunpy_sphinx_theme/__init__.py

@@ -2,9 +2,11 @@
 SunPy Sphinx Theme.
 """
 
+import json
 import os
 from functools import partial
 from pathlib import Path
+from textwrap import dedent, indent
 from urllib.parse import urljoin
 
 from pydata_sphinx_theme import utils
@@ -154,12 +156,54 @@ def update_html_context(app: Sphinx, pagename: str, templatename: str, context,
     context["sst_pathto"] = partial(sst_pathto, context)
 
 
+def generate_search_config(app):
+    """
+    This function parses the config for the "Documentation" section of the theme config.
+    """
+    theme_config = utils.get_theme_options_dict(app)
+    search_projects = theme_config.get("rtd_search_projects", None)
+    if search_projects is None:
+        navbar_links = theme_config["navbar_links"]
+        doc_links = next(section[1] for section in navbar_links if section[0] == "Documentation")
+
+        def filter_doc_links(links):
+            out_links = []
+            for link in links:
+                if isinstance(link[1], list):
+                    out_links += filter_doc_links(link[1])
+                elif isinstance(link[1], str) and link[1].startswith("http"):
+                    out_links.append({"name": link[0], "link": link[1]})
+                else:
+                    err = f"Unable to parse {link} in the nav tree. Try setting search_projects explicitly or fixing navbar_links."
+                    raise ValueError(err)
+            return out_links
+
+        search_projects = filter_doc_links(doc_links)
+
+    load_more_label = theme_config.get("rtd_search_load_more_label", "Load more results")
+    no_results_label = theme_config.get("rtd_search_no_results_label", "There are no results for this search")
+    script = dedent(f"""
+        const set_search_config = {{
+          "no-results":{{
+            "label": "{no_results_label}"
+          }},
+          "load-more":{{
+            "label": "{load_more_label}",
+            "class": "btn sd-btn sd-bg-primary sd-bg-text-primary"
+          }},
+          "projects":{indent(json.dumps(search_projects, indent=2), " " * 10, predicate=lambda line: line.strip() != "[")}
+        }};
+    """)
+    app.add_js_file(None, body=script)
+
+
 def setup(app: Sphinx):
     # Register theme
     theme_dir = get_html_theme_path()
     app.add_html_theme("sunpy", theme_dir)
     app.add_css_file("sunpy_style.css", priority=600)
-    app.connect("builder-inited", update_config)
+    app.connect("builder-inited", update_config, priority=100)
+    app.connect("builder-inited", generate_search_config, priority=500)
     app.connect("html-page-context", update_html_context)
     # Conditionally include goat counter js
     # We can't do this in update_config as that causes the scripts to be duplicated.
@@ -198,6 +242,14 @@ def setup(app: Sphinx):
             loading_method="async",
         )
 
+        if theme_options.get("rtd_search", True):
+            # Add project-wide search
+            app.add_css_file("css/rtd_enhanced_search.css")
+            app.add_js_file(
+                "js/rtd_enhanced_search.js",
+                loading_method="async",
+            )
+
     return {
         "parallel_read_safe": True,
         "parallel_write_safe": True,

src/sunpy_sphinx_theme/cards.py

@@ -109,7 +109,7 @@ def run(self):
 
 
 def setup(app: Sphinx):
-    app.add_css_file("sunpy_cards.css", priority=600)
+    app.add_css_file("css/sunpy_cards.css", priority=600)
     app.add_directive("custom-card", Card)
     app.add_node(_Card, html=(visit_card_node, depart_card_node))
     return {

src/sunpy_sphinx_theme/theme/sunpy/static/css/rtd_enhanced_search.css

@@ -0,0 +1,168 @@
+#pst-search-dialog[open] {
+  left: revert;
+  top: 2rem;
+  margin-top: 0;
+  transform: revert;
+  max-height: calc(100vh - 4rem);
+}
+
+#pst-search-dialog[open] form.bd-search {
+  flex-grow: 0;
+}
+
+.readthedocs-search {
+  background: var(--pst-color-background);
+  border-radius: 0.25rem;
+  width: 100%;
+  display: flex;
+  flex-direction: column;
+  overflow: hidden;
+}
+
+.readthedocs-search form {
+  margin: 0.5rem;
+  border: 0;
+}
+
+.readthedocs-search form:focus-within {
+  outline: 2px solid var(--pst-color-accent);
+  border: 0;
+}
+
+/* .readthedocs-search form.loading input { */
+/* } */
+
+.readthedocs-search form.loading .fa-magnifying-glass {
+  background-image: url("data:image/svg+xml,<svg width='24' height='24' viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cstyle%3E.spinner_6kVp%7Btransform-origin:center;animation:spinner_irSm .75s infinite linear%7D@keyframes spinner_irSm%7B100%25%7Btransform:rotate(360deg)%7D%7D%3C/style%3E%3Cpath d='M10.72,19.9a8,8,0,0,1-6.5-9.79A7.77,7.77,0,0,1,10.4,4.16a8,8,0,0,1,9.49,6.52A1.54,1.54,0,0,0,21.38,12h.13a1.37,1.37,0,0,0,1.38-1.54,11,11,0,1,0-12.7,12.39A1.54,1.54,0,0,0,12,21.34h0A1.47,1.47,0,0,0,10.72,19.9Z' class='spinner_6kVp'/%3E%3C/svg>");
+  background-position: left center;
+  background-repeat: no-repeat;
+  background-size: contain;
+}
+
+html[data-theme="dark"] .readthedocs-search form.loading .fa-magnifying-glass {
+  background-image: url("data:image/svg+xml,<svg fill='white' width='24' height='24' viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cstyle%3E.spinner_6kVp%7Btransform-origin:center;animation:spinner_irSm .75s infinite linear%7D@keyframes spinner_irSm%7B100%25%7Btransform:rotate(360deg)%7D%7D%3C/style%3E%3Cpath d='M10.72,19.9a8,8,0,0,1-6.5-9.79A7.77,7.77,0,0,1,10.4,4.16a8,8,0,0,1,9.49,6.52A1.54,1.54,0,0,0,21.38,12h.13a1.37,1.37,0,0,0,1.38-1.54,11,11,0,1,0-12.7,12.39A1.54,1.54,0,0,0,12,21.34h0A1.47,1.47,0,0,0,10.72,19.9Z' class='spinner_6kVp'/%3E%3C/svg>");
+}
+
+.readthedocs-search form.loading .fa-magnifying-glass path {
+  display: none;
+}
+
+.readthedocs-search .content {
+  max-height: calc(100% - 110px);
+  overflow-y: auto;
+}
+
+.readthedocs-search .tablist {
+  position: sticky;
+  display: flex;
+  overflow-x: auto;
+  top: 0;
+  outline: 0;
+  background: var(--pst-color-background);
+}
+
+.readthedocs-search .tablist button {
+  display: flex;
+  border: 0;
+  border-bottom: max(3px, 0.1875rem, 0.12em) solid transparent;
+  color: var(--pst-color-text-base);
+  background-color: var(--pst-color-surface);
+  padding: 0.5rem 1rem;
+  align-items: center;
+  white-space: nowrap;
+}
+
+.readthedocs-search .tablist button:focus {
+  color: var(--sst-lightest-color);
+  background-color: var(--sst-dark-color);
+}
+
+.readthedocs-search .tablist button:focus,
+.readthedocs-search .tablist button[aria-selected="true"] {
+  outline: 0;
+  border-bottom: max(3px, 0.1875rem, 0.12em) solid var(--pst-color-secondary);
+}
+
+.readthedocs-search .tablist button svg {
+  padding-right: 0.25rem;
+}
+
+.readthedocs-search .tablist button .n {
+  padding-left: 0.25rem;
+}
+
+.readthedocs-search .results .results-message > * {
+  margin: 1rem;
+}
+
+.readthedocs-search .results ul {
+  list-style: none;
+  padding: 0;
+  margin: 0;
+}
+
+.readthedocs-search .results ul li {
+  margin: 0;
+  padding: 1rem;
+}
+
+.readthedocs-search .results ul li:focus {
+  background: var(--pst-color-attention-bg);
+  outline: 0;
+}
+
+.readthedocs-search .results ul li:first-child {
+  border: 0;
+}
+
+/* .readthedocs-search .results ul li > *:last-child { */
+/* } */
+
+.readthedocs-search .results ul li a:hover,
+.readthedocs-search .results ul li a:focus,
+.readthedocs-search .results ul li.selected a {
+  color: var(--pst-color-text-base);
+  background-color: var(--sst-footer-background-color);
+}
+
+.readthedocs-search .footer {
+  padding: 10px;
+  border-top: 1px solid #e1e4e5;
+  display: flex;
+  gap: 1em;
+  justify-content: space-between;
+  align-items: center;
+  font-size: 12px;
+  color: var(--pst-color-text-base);
+  background-color: var(--sst-footer-background-color);
+}
+
+.readthedocs-search .help {
+  display: flex;
+  list-style: none;
+  margin: 0;
+  padding: 0;
+  gap: 20px;
+}
+
+.readthedocs-search .credits {
+  display: flex;
+  align-items: center;
+  gap: 5px;
+  margin-left: auto;
+  text-align: right;
+}
+.readthedocs-search .credits a {
+  color: inherit;
+}
+
+.readthedocs-search .credits svg {
+  height: 20px;
+}
+
+@media only screen and (max-width: 70rem) {
+  .readthedocs-search .footer,
+  .readthedocs-search .help {
+    display: block;
+  }
+}