Lut 32413 : Sorting issue in FormResponse Lists#635
Conversation
f513800 to
f7a1e5c
Compare
lobtx
left a comment
lobtx
left a comment
There was a problem hiding this comment.
your PR is great, I just required some code improvements that are not linked to the US
There was a problem hiding this comment.
please use strAttributeName here
| @@ -261,11 +262,11 @@ private Sort buildLuceneSort( FormItemSortConfig sortConfig ) | |||
| return new Sort( new SortedNumericSortField( sortConfig.getSortAttributeName( ), SortField.Type.LONG, sortConfig.isAscSort( ) ) ); | |||
There was a problem hiding this comment.
please use strAttributeName here
There was a problem hiding this comment.
duplicate return new Sort( new SortedNumericSortField( sortConfig.getSortAttributeName( ), SortField.Type.LONG, sortConfig.isAscSort( ) ) ); in both cases. Not in the scope of the US but could be great to unify conditions with an OR operator
There was a problem hiding this comment.
Replace with SortField.Type.INT for better consistency with the field type
| { | ||
| doc.add( new StringField( fieldNameBuilder.toString( ) + FormResponseSearchItem.FIELD_SELECT_SUFFIX, response.getResponseValue( ), Field.Store.YES ) ); | ||
| doc.add( new SortedDocValuesField( fieldNameBuilder.toString( ) + FormResponseSearchItem.FIELD_SELECT_SUFFIX, new BytesRef( response.getResponseValue( ) ) ) ); | ||
| String sortIndexList = LuceneUtils.createLuceneEntryKey( strQuestionCode, response.getIterationNumber( ) ); |
There was a problem hiding this comment.
This line throws IllegalArgumentException: DocValuesField "entry_code__iter_0" appears more than once in this document whenever a checkbox question has more than one option ticked (or any multi-valued question at the same iterationNumber).
In that case getEntryResponse() returns several Response objects sharing the same question code and the same iterationNumber. Since sortIndexList (line 791) is built from createLuceneEntryKey(code, iteration) without the field discriminant, each of those responses adds a SortedDocValuesField under the same field name to the same document — which Lucene forbids for a single-valued docvalues field.
As a result the document is never indexed: the form indexer daemon fails on every run and the response stays permanently unindexed (invisible in the multiview / search).
Fix suggestion: add the sort docvalue only once per sortIndexList key (dedup, e.g. a Set guard), or switch to a SortedSetDocValuesField if multi-value sorting is required.
sample:
near line 730
// Sort docvalues key is built without the field discriminant : a multi-valued
// response (e.g. checkbox with several options ticked) would add several
// SortedDocValuesField under the same name, which Lucene forbids. Keep only
// the first value per sort key.
Set<String> setSortListFieldUsed = new HashSet<>( );
near line 792
String sortIndexList = LuceneUtils.createLuceneEntryKey( strQuestionCode, response.getIterationNumber( ) );
if ( setSortListFieldUsed.add( sortIndexList ) )
{
doc.add( new SortedDocValuesField( sortIndexList, new BytesRef( response.getResponseValue( ) ) ) );
}
There was a problem hiding this comment.
Thanks for your review, I hadn't thought of that case.
The change you suggested has been made.
b5c85ac to
481b0e0
Compare
LUT-32413 : Create sort utils and add unit tests LUT-32413 : unit tests correction LUT-32413 : Adding correct index to form numbers + new sort for string LUT-32413 : pr review LUT-32413 : pr review LUT-32413 : Indexation Sort
5b5cc58 to
281fcea
Compare
|
replaced by #642 |
3 participants
Add sorting for String fields with dynamic indexes in the form response view