Skip to content

Commit e1bc1d9

Browse files
committed
fix selqa's failing test - cc @Iwontbecreative
1 parent 9d40ea8 commit e1bc1d9

1 file changed

Lines changed: 10 additions & 5 deletions

File tree

promptsource/templates/selqa/answer_selection_analysis/templates.yaml

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,11 @@ templates:
1818
5354e98d-8aa2-49d0-a50b-fc72a503d7d4: !Template
1919
answer_choices: No ||| Yes
2020
id: 5354e98d-8aa2-49d0-a50b-fc72a503d7d4
21-
jinja: '{% set possible_indexes = [] %}{% for c in candidates %}{% if c|trim %}{% set _ = possible_indexes.append(loop.index0) %}{% endif %}{% endfor %}{% set rand_index = possible_indexes | choice %} Would it make
22-
sense to reply "{{ candidates[rand_index]|trim|trim(''.'') }}" to the question
23-
"{{ question }}"? ||| {% if rand_index in answers %}{{answer_choices[1]}}{%else %}{{answer_choices[0]}}{%endif%}'
21+
jinja: '{% set possible_indexes = [] %}{% for c in candidates %}{% if c|trim %}{{
22+
possible_indexes.append(loop.index0) | default("", True) }}{% endif %}{% endfor %}{% set
23+
rand_index = possible_indexes | choice %} Would it make sense to reply "{{ candidates[rand_index]|trim|trim(''.'')
24+
}}" to the question "{{ question }}"? ||| {% if rand_index in answers %}{{answer_choices[1]}}{%else
25+
%}{{answer_choices[0]}}{%endif%}'
2426
metadata: !TemplateMetadata
2527
choices_in_prompt: false
2628
metrics:
@@ -46,8 +48,11 @@ templates:
4648
9de0a553-63e7-4b67-a6c5-1a15ac0d5483: !Template
4749
answer_choices: No ||| Yes
4850
id: 9de0a553-63e7-4b67-a6c5-1a15ac0d5483
49-
jinja: '{% set possible_indexes = [] %}{% for c in candidates %}{% if c|trim %}{% set _ = possible_indexes.append(loop.index0) %}{% endif %}{% endfor %}{% set rand_index = possible_indexes | choice %}Someone asked me "{{ question }}" I replied "{{ candidates[rand_index] }}" Does
50-
my answer make sense? ||| {% if rand_index in answers %}{{answer_choices[1]}}{%else %}{{answer_choices[0]}}{%endif%}
51+
jinja: '{% set possible_indexes = [] %}{% for c in candidates %}{% if c|trim %}{{
52+
possible_indexes.append(loop.index0) | default("", True) }}{% endif %}{% endfor
53+
%}{% set rand_index = possible_indexes | choice %}Someone asked me "{{ question
54+
}}" I replied "{{ candidates[rand_index] }}" Does my answer make sense? |||
55+
{% if rand_index in answers %}{{answer_choices[1]}}{%else %}{{answer_choices[0]}}{%endif%}
5156
5257
'
5358
metadata: !TemplateMetadata

0 commit comments

Comments
 (0)