-
Notifications
You must be signed in to change notification settings - Fork 10
OpenMay 21, 2025
No due date
•Last updated 71% complete
List view
0 of 22 selected 0 issues of 22 selected
Multiple model parameter with same value result in multiple evaluations
bugSomething isn't workingSomething isn't workingquestionFurther information is requestedFurther information is requestedStatus: Open.Allow to store and load all configuration items with Docker / Kubernetes runners
enhancementNew feature or requestNew feature or requestStatus: Open.#307 In symflower/eval-dev-quality;Kubernetes job watching does not work when erroring
bugSomething isn't workingSomething isn't workingStatus: Open.Add new subcommand
verifythat just does repository verification and run that in CIbugSomething isn't workingSomething isn't workingStatus: Open.#330 In symflower/eval-dev-quality;Roadmap for v1.1.0
roadmapCollection of issues for a releaseCollection of issues for a releaseStatus: Open.#382 In symflower/eval-dev-quality;Features, bugs, research, ... that we are not actively working on
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededroadmapCollection of issues for a releaseCollection of issues for a releaseStatus: Open.Update README to align with latest changes
enhancementNew feature or requestNew feature or requestStatus: Open.Extend the evalatuion.csv with the current maximum values, and keep them updated in the repositories
enhancementNew feature or requestNew feature or requestStatus: Open.Redo options
tokensandurlsas it looks like we cannot separate multiple values with ","bugSomething isn't workingSomething isn't workingStatus: Open.Add an indicator in the results directory that an evaluation has completed
enhancementNew feature or requestNew feature or requestStatus: Open.Rework how we allow to set (query) attributes of models
enhancementNew feature or requestNew feature or requestStatus: Open.Query model list of "OpenAI API" provider
enhancementNew feature or requestNew feature or requestStatus: Open.Should we remove
*-symflower-templatetasks? e.g. write-tests-symflower-templateenhancementNew feature or requestNew feature or requestStatus: Open.Do "symflower-fix" task for "code-repair"
enhancementNew feature or requestNew feature or requestStatus: Open.Migration task and new attribute CLI arguments are not explained in README
documentationImprovements or additions to documentationImprovements or additions to documentationStatus: Open.#414 In symflower/eval-dev-quality;Smart unit test templates for Spring services don't have Spring test annotation, misleading models to write non-Spring tests
bugSomething isn't workingSomething isn't workingStatus: Open.#420 In symflower/eval-dev-quality;Rework the JUnit4->JUnit5 migration LLM prompt
bugSomething isn't workingSomething isn't workingStatus: Open.Do not copy assessments of base tasks
bugSomething isn't workingSomething isn't workingStatus: Open.#435 In symflower/eval-dev-quality;Make Symflower template generation a no-disk operation
enhancementNew feature or requestNew feature or requestStatus: Open.#434 In symflower/eval-dev-quality;Assessments should have a type based on their data
bugSomething isn't workingSomething isn't workingStatus: Open.Rethink language scoring of "transpilation"
enhancementNew feature or requestNew feature or requestStatus: Open.Validate option
enhancementNew feature or requestNew feature or requestStatus: Open (in progress).symflower/eval-dev-qualitynumber 457#457 In symflower/eval-dev-quality;