Website search should be improved

[ksen0]

Increasing Access

Currently, the keyword search on the website often does not help users find what they look for. This is evident in usability testing, and in issue reports. To make the information on the site more accessible, search should be improved.

Most appropriate sub-area of p5.js?

Other (specify if possible)

Feature request details

• Here is the current search script: https://github.com/processing/p5.js-website/blob/main/src/scripts/builders/search.ts
• There is also a filter on reference: https://github.com/processing/p5.js-website/blob/main/src/components/ReferenceDirectoryWithFilter/index.tsx
• Commit introducing main portion of current search: 993e6c5
• This issue is for research for what's possible technically: comments and draft PRs that demonstrate an idea are welcome from different people! But it is not yet ready for focused work. But keep in mind if you make a PR it may be closed.

Related issues

• Allow searching through the descriptions of reference pages, and not just through their titles #493 search through descriptions
• Reintroduce search field suggestions dropdown #459 search field suggestions dropdown feature
• CodeEmbed/frame.tsx: p5.sound never loads in iframes + postMessage uses wildcard origin #1249 searchProvider bug

[ksen0]

This was discussed in the March 16, 2026 call in the Discord/#p5js-website working group (including @davepagurek @kunstewi @skyash-dev @aashu2006 @nbogie @doradocodes @Nwakaego-Ego). Below are some of the topics:

Categories are not searcheable/filtreable in Reference

Category titles are not included in the index, because they're not pages (maybe they should be pages, or maybe they should be included in index somehow else) so searching for "dom" will not get p5.dom.

A couple weeks ago in #help there was also a discussion of the Reference "filter" function behaving in the same unintuitive way:

In this example the filter function in the reference was used, same behavior as search on "dom"

Because this is not search but a filter, so I'm adding a link to the filtering code to the issue description.

Fallback/404 behavior and testing reference search with archive site

Talked about the404 pages problem when getting URLs in the style of the archive site. If search works well enough (for exact terms of reference, it seems to), then the 404 behavior could be to link (possibly redirect?) to search + the term extracted from URL. E.g., https://archive.p5js.org/reference/#/p5/describeElement works as is, but https://p5js.org/reference/#/p5/describeElement will go to 404. Then 404 can link to "search" with "describeElement" as the query. This will work.

The above approach also means that the Archive reference can be a test suite for the search: that all archive reference entries should fetch corresponding reference pages when searched on main or beta.

Client or server side search?

After the call, @limzykenneth also clarified about search client side (current approach) vs server side (possible in the future):

In the current architecture, only client side search can be done since the site is deployed as a fully static site (ie. there is no server). As I type I'm working on moving the site to Cloudflare Workers [see: https://github.com//pull/1243 and https://github.com//pull/1244] , by then we can implement server side functionalities.

Search from the outside

Indexing by search engines is not great either: example google search for "shape". See also: #865

From google search console, I see that many pages (such as the color wheel example) are not indexed on the beta for "duplicate page without canonical tag" reason

[aashu2006]

Been thinking about this, should we keep the expected direction here more about improving the current keyword search (like expanding the index with descriptions/categories), or should we be moving toward a more discovery oriented search overall that can handle conceptual queries as well? I feel like that even small index changes could already solve a big part of the problem, so just wondering how far we want to go with this.

[ksen0]

@aashu2006 that's a great point, below are just my thoughts, it's open to discussion.

I think small changes with big positive impact (and no bad side effects that make situation worse than current) seem completely worth doing, and soon. Eg, If there's a clear, focused PR to include category headings without additional work (eg making category pages), I would support that. Expending index with descriptions I do have follow-up questions about (eg, how much does it expand the list of results?) so that will need a little more discussion.

Bigger changes are possible but I think it's important to decide on one major refactor/redesign and then work toward that, and be realistic (e.g., iteratively* test, target release for the fall or even 2027). I've just posted the first report (summary, full report coming soon) on the parent issue; the second group was focused exclusively on the Reference so it really makes sense to wait for them to conclude their work to decide on a big design direction. However, technical research can be done already, eg what mechanism can help handle conceptual queries?

*This would really benefit from a staging environment that people can test that is decoupled from the code versions.

[whynotdeep7]

I am thinking of fixing the search issue by improving how items are indexed and matched. I will organize modules as categories and submodules as subcategories, include keywords from descriptions, and update the search logic to consider all of these fields. This should make searches like "DOM" return relevant results like createDiv and createElement instead of unrelated ones.

[ksen0]

Hi @deepiee , welcome! Please be sure to read all the notes (if you want to you can also follow on the discord.p5js.org #p5js-website channel) - this issue is in a "research" phase. I've added "Discussion" label to be more clear. You can make gists / draft PRs to show an idea, but these would likely not be merged. As I wrote above, it would be better to separate/isolate tasks, so that "small changes with big impact" can be implemented quickly; while we take needed time to consider bigger changes. For your suggestion I would also wonder if it would return too many results in some cases?

[whynotdeep7]

Thanks @ksen0! Totally understand. I'll keep it in research/discussion mode and break this into smaller pieces.

[aashu2006]

@ksen0 Actually, that makes sense especially about returning too many results.
Maybe descriptions could be included but with lower weight than titles/categories, so they help with discovery without overwhelming exact matches? So something like: title > category > description in ranking. That way conceptual queries benefit, but precise searches should still behave as expected.

For e.g we have

{
  title: 'lerpColor',
  category: 'Color',
  description: 'Blends two colors to find the third color between them'
}

searching lerpColor - exact title match will stay on the top
searching blend colors - lerpColor still shows up via description, but below exact matches ..

[Nwakaego-Ego]

@ksen0 After reading the usability study[1] and looking at src/scripts/builders/search.ts, I want to share some thoughts and a local test I ran.

The study showed that 5 out of 6 participants went straight to search before trying navigation. The main problem is a vocabulary mismatch, the index only stores function names, but beginners search by what they want to do, not what things are called.
I tested adding description = data.description to the reference case and rebuilt the index locally. Here is what I got. 'Draw a circle' returns circle(). 'Load a 3d' returns loadModel() and createModel(). 'Set the style' returns 6 very good results. Result counts stayed small and manageable.

But 'dom' still fails, it returns random(), noise(), and randomGaussian() because d-o-m appears inside the word 'random'. Actual DOM functions didn't appear because their descriptions don't use the word 'dom'. Descriptions alone can't fix category searches. The module field needs to be added to the index separately.

Also, when you change the search script, you won't see any difference in the browser until you run npm run build:search first. This is because the search index is a pre-built file that doesn't update automatically. I stand to be corrected, i dont think this step is documented anywhere currently, and it could save other contributors a lot of confusion if it were."

Something else that could help is adding beginner-friendly aliases alongside descriptions in the search index. Aliases would point to the right functions even when the search term doesn't appear in the description. For example, someone searching 'draw circle' should find ellipse() and circle(). Someone searching 'button' should find createButton(). We don't need to do all 400 functions at once. Starting small, say 20 most commonly confused ones, would already make a big difference, and it's something the community could help with over time.

Building on what @aashu2006 suggested, combining descriptions and category names with weighted ranking seems like the right direction."

[aashu2006]

That’s really helpful @Nwakaego-Ego , especially the local test results, nice catch on the dom case. Also it makes sense that descriptions alone won’t handle category type queries, so adding the category field separately seems necessary. The alias idea is interesting too, feels like that can help with more beginner style queries without needing a big change. Starting small there sounds like a good approach.

[ksen0]

After our chat yesterday, I think to make progress we will benefit from (1) baseline eval and (2) observable iterative improvement. These are just my thoughts, sense-check from @davepagurek @doradocodes is welcome!

Eval & testing

I believe we do need a test plan, as lightweight as possible. Could be a manual test plan in contributor docs if there's no better idea, starting with 5-12 terms, some of which work well and some of which don't, such as based on examples we've already discussed: dom, strands, x, min, shape, lerpColor, blend Colors.

Unit testing is a bonus / nice to have: the test suite is automated as unit tests (but only if it's straightforward to implement using playwright). The unit test version can be relaxed and just check if correct materials appears in the total list, rather than checking precise position.

Evaluation should be a script, but does not need to be in CI: to help to guide our improvements and discussions. This could look at # of results and position of correct result. So it's not quite "unit tests" (though also useful) but an evaluation.

For example: searching for "x" gets dozens of results (not good) but the correct one is on top (good, position 0). Searching "Dom" get 5 results (good) but the correct one is missing (bad, position -1). Searching "strands" is debatable: the top result is the contributor doc; tutorial is there when you scroll; reference is missing. Not too many results (good) and they're sort of relevant but not as helpful or complete as they should be.

search term	result count	correct position / relevance	overall
x	❌ too many	✅ position 0	⚠️ mixed
dom	✅ good	❌ missing	⚠️ mixed
strands	✅ good	⚠️ partial	✅ acceptable
shape	❌ too many	❌ missing	❌ poor

This is just to support how we talk about if a proposed change "improves search" or not - improvements to this evaluation scheme are welcome but not required. If we want to be more rigorous about it, I think we could make a light MRR-based metric, but this also requires a longer list of search terms.

Iterative improvements

Once minimal eval is in place, I think the below would be ready for work:

1. introduce a PR for including category in a minimal way to fix the dom/strands problem and would not be expected to have side effects
2. introduce a PR (probably draft at first) for including weights, ideally as an isolated proposal, but possible including the descriptions proposal; this could be then tested. Outcome of community testing and review would be at least an expanded test suite, and possibly enough tweaking of this PR to make it mergeable

[aashu2006]

This makes a lot of sense, especially having a small evaluation setup to ground the discussion. The example cases dom, shape, x, etc. already highlight very different failure modes, so having something like this to compare before or after changes feels really useful. Happy to help test or expand the list if needed once we start trying out the category changes.