Skip to content

New version of form-processing script for updated schema#191

Open
Dvermetten wants to merge 7 commits into
mainfrom
FormUpdate
Open

New version of form-processing script for updated schema#191
Dvermetten wants to merge 7 commits into
mainfrom
FormUpdate

Conversation

@Dvermetten

Copy link
Copy Markdown
Contributor

New (generated) script for converting the form responses (csv file) to the updated schema.

@Dvermetten Dvermetten requested a review from kvdblom June 2, 2026 11:54
@Dvermetten

Copy link
Copy Markdown
Contributor Author

Now adresses #186

@kvdblom kvdblom left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Github action generate_html fails:

  • when running uv run yaml_to_html.py, because problems.yaml contains '?' where input should be 'yes', 'no', 'some' or 'unknown'.
  • Happens at least for: suite_morepo and suite_cec2015_dmoo
  • Maybe these are ouput of the form processing script? Otherwise we can just change them to unknown and that should fix it

Github action `check_new_problems fails:

  • it cannot find module pydantic_yaml
  • Maybe the problem is that: pydantic_yaml is in requirements.txt but not in utils/requirements.txt
  • When running with uv run utils/validate_yaml.py problems.yaml (rather than python utils/validate_yaml.py problems.yaml as in the action), the action fails with ImportError: cannot import name 'default_columns' from 'yaml_to_html'

Execution of formresponse_to_yaml.py fails:

  • When run with uv run formresponse_to_yaml.py, the error No such file or directory: 'responses_OPL.csv' comes up
  • Suggestion: Change the default file name to the correct one
  • Suggestion 2: Add instructions in a README (maybe in utils? Perhaps this entire script should go there?) on how thi should be executed.
  • When run with uv run formresponse_to_yaml.py --csv OPL_form.csv execution fails with 44 validation errors -- Seemingly about execution times not being in the format expected by pydantic (but maybe not limited to that)

@kvdblom

kvdblom commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Merging main back in did no resolve any of these issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Identify and implement missing steps for processing google form output to YAML on github

2 participants