You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Document stopword stripping behavior for = and fulltext()
Update README to reflect that both exact phrase (=) and tokenized
search (fulltext()) strip default Redis stopwords before sending
queries. Adds a dedicated Stopword Handling section with examples
and the STOPWORDS 0 workaround. Replaces outdated 'preserves
stopwords' language.
-- Exact phrase match (stopwords like "of" are stripped automatically)
196
196
SELECT*FROM products WHERE title ='bank of america'
197
+
-- Produces: @title:"bank america"
197
198
198
199
-- Fuzzy search for typos (Levenshtein distance 2)
199
200
SELECT*FROM products WHERE fuzzy(title, 'laptap', 2)
@@ -214,10 +215,23 @@ SELECT title, score() AS relevance FROM products WHERE fulltext(title, 'laptop')
214
215
SELECT*FROM products WHERE fulltext(title, 'laptop') OR fulltext(description, 'laptop')
215
216
```
216
217
218
+
**Stopword handling:**
219
+
220
+
Both `=` (exact phrase) and `fulltext()` (tokenized search) automatically strip [Redis default stopwords](https://redis.io/docs/latest/develop/ai/search-and-query/advanced-concepts/stopwords/) before sending queries to RediSearch. This is necessary because RediSearch does not index stopwords, so including them in queries causes syntax errors or failed matches. A `UserWarning` is emitted when stopwords are removed.
221
+
222
+
For example, `WHERE title = 'bank of america'` produces `@title:"bank america"` because "of" is a default stopword and is never stored in the inverted index. The stripped phrase still matches correctly because the indexer assigns consecutive token positions after dropping stopwords.
223
+
224
+
To include stopwords in your queries, create your index with `STOPWORDS 0`:
225
+
226
+
```
227
+
FT.CREATE myindex ON HASH PREFIX 1 doc: STOPWORDS 0 SCHEMA title TEXT
228
+
```
229
+
217
230
**Notes:**
218
-
-`=` on TEXT fields performs **exact phrase** matching (preserves stopwords)
219
-
-`fulltext()` performs **tokenized** search (stopwords are filtered with a warning)
220
-
-`fuzzy()` and `fulltext()` only work on TEXT fields — using them on TAG or NUMERIC raises `ValueError`
231
+
-`=` on TEXT fields performs **exact phrase** matching (double-quoted)
232
+
-`fulltext()` performs **tokenized** AND search (parenthesized)
233
+
- Both operators strip stopwords and emit a warning when they do
234
+
-`fuzzy()` and `fulltext()` only work on TEXT fields; using them on TAG or NUMERIC raises `ValueError`
221
235
- OR must be **uppercase**: `'laptop OR tablet'` triggers union; lowercase `'laptop or tablet'` is treated as a regular three-word AND search
222
236
- Special characters (`@`, `|`, `-`, `*`, `+`, etc.) in search terms are automatically escaped
0 commit comments