Skip to content

fix(validation): skip upload-time Pandera validation for ES#271

Open
vishpillai123 wants to merge 1 commit into
developfrom
fix/skip-es-upload-pandera
Open

fix(validation): skip upload-time Pandera validation for ES#271
vishpillai123 wants to merge 1 commit into
developfrom
fix/skip-es-upload-pandera

Conversation

@vishpillai123

@vishpillai123 vishpillai123 commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • route ES uploads through the any-format CSV path instead of upload-time Pandera
  • retain CSV parsing and PII column-name protections
  • leave school-specific conversion and schema validation to the Databricks ES pipeline

Asana

https://app.asana.com/1/6325821815997/project/1208486153653829/task/1216606862971306?focus=true

Test plan

  • Run src/webapp/validation_test.py
  • Run src/webapp/validation_pdp_read_path_test.py
  • Run src/webapp/validation_pdp_edvise_test.py
  • Verify Black formatting on changed Python files

Made with Cursor

Allow ES uploads through CSV parsing and PII guards so school-specific converters and schema checks can run later in Databricks.

Co-authored-by: Cursor <cursoragent@cursor.com>
@vishpillai123 vishpillai123 changed the title fix(api): skip upload-time Pandera for ES fix(validation): skip upload-time Pandera validation for ES Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant