import religious character from GIAS and correct mapping by starswan · Pull Request #8751 · DFE-Digital/teaching-vacancies

starswan · 2026-04-24T11:54:49Z

Trello card URL

https://trello.com/c/wzqdp28P/2820-incorrect-mapping-of-religious-character-schools-for-both-cofe-and-roman-catholic-schools

Changes in this PR:

Map religious character from GIAS, and then filter correctly by the field in the DB.

Remove gias_data and gias_data_hash from database

github-actions · 2026-04-28T08:16:27Z

Review app https://teaching-vacancies-review-pr-8751.test.teacherservices.cloud was successfully deleted

Copilot

Pull request overview

This PR moves religious character (and a few other GIAS-derived fields) out of the gias_data JSON and into first-class organisations columns, then updates filtering and import logic to use those columns.

Changes:

Add new organisations columns (religious_character, number_of_pupils, school_capacity, trust_school_flag_code, trusts_code) and populate them via the GIAS schools import.
Update faith/special-school filtering and SAT sign-in school lookup to query the new columns instead of gias_data.
Remove/stop using gias_data/gias_data_hash across code/tests/fixtures, including removing the hash refresh job from the schedule.

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
spec/system/publishers/publishers_can_filter_vacancies_in_their_dashboard_spec.rb	Updates school factory setup to use `school_type` instead of `gias_data`.
spec/system/jobseekers/jobseekers_can_search_for_schools_spec.rb	Updates faith/non-faith setup to use `religious_character` column.
spec/system/jobseekers/jobseekers_can_search_for_jobs_spec.rb	Updates faith/non-faith setup to use `religious_character` column.
spec/services/search/school_search_spec.rb	Updates school search specs to use `religious_character` column.
spec/services/gias/import_trusts_spec.rb	Removes expectations around storing trust `gias_data`.
spec/services/gias/import_schools_and_local_authorities_spec.rb	Updates stubbed GIAS CSV headers/rows to include new fields; removes `gias_data` assertions.
spec/queries/vacancy_filter_query_spec.rb	Updates vacancy filtering specs to use `religious_character` column.
spec/presenters/vacancy_decorator_spec.rb	Updates decorator spec to build schools with `religious_character` column.
spec/models/school_spec.rb	Removes `School#religious_character` JSON-based behaviour tests; updates catholic/variant tests.
spec/models/school_group_spec.rb	Removes assertions that `SchoolGroup` has a `gias_data` JSON column.
spec/models/organisation_spec.rb	Removes `refresh_gias_data_hash` behaviour specs.
spec/fixtures/st_albans_schools.yml	Removes embedded `gias_data`/hash from fixtures; adds `religious_character`.
spec/fixtures/liverpool_schools.yml	Removes embedded `gias_data`/hash from fixtures; adds `religious_character`.
spec/fixtures/basildon_schools.yml	Removes embedded `gias_data`/hash from fixtures; adds `religious_character`.
spec/factories/schools.rb	Replaces `gias_data` defaults/traits with scalar columns (pupils/capacity/religious_character).
spec/factories/school_groups.rb	Removes `gias_data` factory attribute for school groups.
db/schema.rb	Reflects new scalar columns added to `organisations`.
db/migrate/20260423141519_add_religious_character_to_school.rb	Adds new scalar columns onto `organisations`.
config/schedule.yml	Removes scheduled `RefreshOrganisationsGiasDataHashJob`.
config/analytics.yml	Removes `gias_data` from analytics export allow-list.
config/analytics_blocklist.yml	Blocklists the new scalar columns in analytics.
app/views/publishers/organisations/_organisation.html.slim	Switches “size” display logic to use `number_of_pupils` column presence.
app/services/search/school_search.rb	Updates faith-school filter to query `religious_character` column.
app/services/gias/import_trusts.rb	Stops persisting `gias_data` for trusts.
app/services/gias/import_schools_and_local_authorities.rb	Maps new GIAS CSV fields into scalar columns (religious character, pupil count, capacity, trust codes).
app/models/school.rb	Removes JSON-derived `religious_character` method; adds religious character type lists + validation; updates faith/catholic logic.
app/models/organisation.rb	Updates NON_FAITH types, changes `only_faith_schools` scope to use column, and ignores `gias_data` columns.
app/jobs/refresh_organisations_gias_data_hash_job.rb	Deletes the job that recomputed `gias_data_hash`.
app/helpers/organisations_helper.rb	Switches school size helpers to use scalar columns; adds `:nocov:` markers.
app/controllers/omniauth_callbacks_controller.rb	Updates SAT contained-school lookup to use `trust_school_flag_code`/`trusts_code` columns.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

starswan · 2026-05-20T14:29:47Z

  def apply_school_type_filter(scope)
    return scope unless school_types.present?

    if school_types.include?("special_school") && school_types.include?("faith_school")
-      scope.where.not("gias_data ->> 'ReligiousCharacter (name)' IN (?)", Organisation::NON_FAITH_RELIGIOUS_CHARACTER_TYPES)
+      scope.where.not(religious_character: Organisation::NON_FAITH_RELIGIOUS_CHARACTER_TYPES)
           .or(scope.where(detailed_school_type: Organisation::SPECIAL_SCHOOL_TYPES))
    elsif school_types.include?("special_school")
      scope.where(detailed_school_type: Organisation::SPECIAL_SCHOOL_TYPES)
    elsif school_types.include?("faith_school")
-      scope.where.not("gias_data ->> 'ReligiousCharacter (name)' IN (?)", Organisation::NON_FAITH_RELIGIOUS_CHARACTER_TYPES)
+      scope.where.not(religious_character: Organisation::NON_FAITH_RELIGIOUS_CHARACTER_TYPES)
    else


default in table is now 'None'

+        trust_school_flag_code: 5, # "Supported by a single-academy trust"
+        trusts_code: uid,
      )



KyleMacPherson · 2026-05-20T14:16:36Z

+  def change
+    add_column :organisations, :religious_character, :string
+    add_column :organisations, :number_of_pupils, :integer
+    add_column :organisations, :school_capacity, :integer
+    add_column :organisations, :trust_school_flag_code, :integer
+    add_column :organisations, :trusts_code, :integer
+  end


Do we need to add a backfill task to populate the religious_character field? Otherwise it looks like all schools, including faith schools, will have it set to "None" and the vacancy filtering for faith schools will be broken until the import is run again at 10PM (I think)

scruti · 2026-05-11T10:33:27Z

@@ -81,6 +81,8 @@ class Organisation < ApplicationRecord
    through: 7,
  }

+  self.ignored_columns += %i[gias_data gias_data_hash]
+


Nah, this is our common approach to two-step DB fields removal. Bad copilot.

scruti · 2026-05-11T10:42:44Z

+  - religious_character
+  - number_of_pupils
+  - trust_school_flag_code
+  - trusts_code
+  - school_capacity


🙋‍♂️ I suspect the only reason gias_data_hash was blocked from being sent to analytics was to avoid duplication of this computed field vs the raw gias_data JSON being sent already to Analytics.

As we are now removing both gias_data and gias_data_hash, I would assume these new fields should be included in the analytics.yml rather than blocked.

(And this is going to be one we need to catch up on with our Perf. Analyst!)

scruti · 2026-05-11T10:49:14Z

+  # New GIAS team now have some data dictionary information available at
+  # https://dfedigital.atlassian.net/wiki/spaces/GTP/pages/6155337742/Data+Dictionary
+  # but it's not quite complete/definitive
+  # (e.g establishment status says open/closed/proposed when reality is 'open proposed to close' and 'proposed to open')


Nice context! Thanks!

scruti · 2026-05-11T11:26:49Z

  validates :email, email_address: true, if: -> { email_changed? } # Allows data created prior to validation to still be valid

-  alias_attribute :data, :gias_data
+  # alias_attribute :data, :gias_data


Suggested change

# alias_attribute :data, :gias_data

starswan force-pushed the validate-incoming-school-types branch from 2b92fe8 to 3923ab8 Compare April 24, 2026 12:28

starswan force-pushed the map-gias-religious-character branch from af06be8 to 2a658be Compare April 24, 2026 12:32

starswan added the deploy label Apr 24, 2026

starswan had a problem deploying to review April 24, 2026 14:08 — with GitHub Actions Failure

starswan force-pushed the validate-incoming-school-types branch from 3923ab8 to 9fc0f94 Compare April 27, 2026 08:23

starswan force-pushed the map-gias-religious-character branch from dbe3838 to 32540fd Compare April 27, 2026 09:03

starswan had a problem deploying to review April 27, 2026 09:08 — with GitHub Actions Failure

starswan force-pushed the validate-incoming-school-types branch from 9fc0f94 to 93f60fe Compare April 27, 2026 15:49

starswan force-pushed the map-gias-religious-character branch from 32540fd to 8f0a6d0 Compare April 28, 2026 07:46

starswan had a problem deploying to review April 28, 2026 07:49 — with GitHub Actions Failure

starswan removed the deploy label Apr 28, 2026

starswan temporarily deployed to review April 28, 2026 08:14 — with GitHub Actions Inactive

starswan force-pushed the map-gias-religious-character branch from 8f0a6d0 to 8ee4570 Compare April 28, 2026 10:39

starswan added the deploy label Apr 28, 2026

starswan temporarily deployed to review April 28, 2026 10:46 — with GitHub Actions Inactive

starswan marked this pull request as ready for review April 28, 2026 11:05

starswan changed the title ~~Map gias religious character~~ import religious character from GIAS and correct mapping Apr 28, 2026

Base automatically changed from validate-incoming-school-types to main May 5, 2026 07:16

starswan force-pushed the map-gias-religious-character branch from 8ee4570 to 5772b6d Compare May 8, 2026 12:39

starswan temporarily deployed to review May 8, 2026 12:43 — with GitHub Actions Inactive

starswan force-pushed the map-gias-religious-character branch from 5772b6d to f6b3d75 Compare May 8, 2026 12:44

starswan temporarily deployed to review May 8, 2026 12:50 — with GitHub Actions Inactive

starswan force-pushed the map-gias-religious-character branch from f6b3d75 to f22e975 Compare May 8, 2026 13:49

starswan temporarily deployed to review May 8, 2026 13:53 — with GitHub Actions Inactive

starswan force-pushed the map-gias-religious-character branch from f22e975 to 46acfd0 Compare May 11, 2026 09:05

starswan temporarily deployed to review May 11, 2026 09:20 — with GitHub Actions Inactive

starswan temporarily deployed to review May 11, 2026 09:37 — with GitHub Actions Inactive

scruti requested a review from Copilot May 11, 2026 10:23

Copilot started reviewing on behalf of scruti May 11, 2026 10:24 View session

Copilot AI reviewed May 11, 2026

View reviewed changes

scruti reviewed May 11, 2026

View reviewed changes

starswan force-pushed the map-gias-religious-character branch from 39c05ac to 40d3baa Compare May 11, 2026 13:08

starswan temporarily deployed to review May 11, 2026 13:20 — with GitHub Actions Inactive

starswan changed the base branch from main to religious-character-part-1 May 11, 2026 15:26

starswan force-pushed the religious-character-part-1 branch from 1284f02 to 20fce7f Compare May 13, 2026 12:20

starswan force-pushed the map-gias-religious-character branch from 40d3baa to 999b3a3 Compare May 13, 2026 12:55

starswan temporarily deployed to review May 13, 2026 13:01 — with GitHub Actions Inactive

starswan force-pushed the religious-character-part-1 branch from 3015cbd to 476d48e Compare May 18, 2026 14:23

starswan force-pushed the map-gias-religious-character branch from 999b3a3 to c0c2eb5 Compare May 18, 2026 14:30

starswan had a problem deploying to review May 18, 2026 14:35 — with GitHub Actions Failure

starswan temporarily deployed to review May 18, 2026 14:59 — with GitHub Actions Inactive

starswan force-pushed the religious-character-part-1 branch from 476d48e to d16f0a5 Compare May 19, 2026 07:17

Base automatically changed from religious-character-part-1 to main May 19, 2026 11:58

KyleMacPherson approved these changes May 20, 2026

View reviewed changes

map religious_character from GIAS

b6af1f3

starswan force-pushed the map-gias-religious-character branch from c0c2eb5 to b6af1f3 Compare May 20, 2026 14:28

starswan temporarily deployed to review May 20, 2026 14:32 — with GitHub Actions Inactive

starswan merged commit d0fd34c into main May 20, 2026
20 of 21 checks passed

starswan deleted the map-gias-religious-character branch May 20, 2026 14:54

starswan temporarily deployed to review May 20, 2026 14:54 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

import religious character from GIAS and correct mapping #8751

import religious character from GIAS and correct mapping #8751
starswan merged 1 commit into
mainfrom
map-gias-religious-character

starswan commented Apr 24, 2026

Uh oh!

github-actions Bot commented Apr 28, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

starswan May 20, 2026

Uh oh!

Uh oh!

KyleMacPherson May 20, 2026

Uh oh!

Uh oh!

scruti May 11, 2026

Uh oh!

scruti May 11, 2026

Uh oh!

scruti May 11, 2026

Uh oh!

scruti May 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

starswan commented Apr 24, 2026

Trello card URL

Changes in this PR:

Uh oh!

github-actions Bot commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

starswan May 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

KyleMacPherson May 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

scruti May 11, 2026

Choose a reason for hiding this comment

Uh oh!

scruti May 11, 2026

Choose a reason for hiding this comment

Uh oh!

scruti May 11, 2026

Choose a reason for hiding this comment

Uh oh!

scruti May 11, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions Bot commented Apr 28, 2026 •

edited

Loading