Guard Salesforce::RoleSyncJob against missing parent records (#850)

## Problem

Heroku Connect was leaving `Contact_Editor_Affiliation__c` rows
permanently `FAILED` in the mirror with errors like:

```
Foreign key external ID: <school-uuid> not found for field EditorUUID__c in entity Editor__c
Foreign key external ID: <user-uuid>   not found for field Pi_Accounts_Unique_ID__c in entity Contact
```

This happens because `School` and `Role` `after_commit` callbacks both
fire in the same transaction and enqueue independent jobs
(`SchoolSyncJob` and `RoleSyncJob`) on the `salesforce_sync` queue. The
jobs use **different** concurrency keys, so the role write to the Heroku
Connect mirror can be pushed to Salesforce **before** the Editor record
lands. When that happens the lookup-by-external-id fails, Salesforce
rejects the INSERT, and Heroku Connect does **not** auto-retry
foreign-key resolution failures — the mirror row stays `FAILED` forever.

The same pattern happens against `Contact` when a role is created before
the upstream Pi Accounts → Salesforce Contact pipeline has materialised
the user.

### Evidence from production

Investigating Editors in Salesforce with no
`Contact_Editor_Affiliation__c`:

- 1,114 Editors with no CEA, of which 1,098 are rejected schools
(working as designed).
- 16 active residual cases. After classifying via `_hc_err`:
- 9 schools had no mirror row at all (one-off backfill recovered them).
- 1 school (Gurnard, created 3 days ago) had `_hc_err = "Foreign key
external ID … not found for field EditorUUID__c in entity Editor__c"`.
The parent `Editor__c` had since synced (`sfid` populated, `_hc_lastop =
SYNCED`), so the affiliation INSERT would now succeed — but Connect
never retries.
- 3 schools failed with the equivalent `Contact` error and have no
`Salesforce::Contact` mirror row at all (separate upstream issue, not in
scope).
- A new occurrence of the Editor race appeared in the Heroku Connect log
at `00:01:04 UTC` today for school `6a228ff8-…`, confirming the race is
still actively breaking new affiliations.

The 10 recoverable cases were re-synced manually; the SOQL count for
not-rejected Editors with no CEA dropped 16 → 6 (3 missing-Contact + 3
by-design schools with only student / no roles).

## Fix

Before writing to the affiliation mirror, verify the parent records
exist in the local Heroku Connect mirror **with `sfid` populated** (i.e.
Salesforce has acknowledged them). If either parent isn't ready, raise
`SalesforceRecordNotFound`.

`SalesforceSyncJob` already declares:

```ruby
retry_on SalesforceRecordNotFound, wait: :polynomially_longer, attempts: 10
```

…so this slots into the existing retry mechanism (same one
`ContactSyncJob` already relies on) without any new error class, queue,
or backoff config.

### Behaviour

| Scenario | Before | After |
|---|---|---|
| Editor not yet synced (school create race) | Mirror row goes `FAILED`
permanently | Job retries with polynomial backoff up to 10 times; lands
once Editor is in Salesforce |
| Contact not yet synced (transient upstream lag) | Mirror row goes
`FAILED` permanently | Same; self-heals when Contact arrives |
| Contact never created upstream (genuinely broken) | Silent `FAILED`
row in mirror | Job exhausts retries with a clear error message; visible
in GoodJob/Sentry |
| Student role | Returns early (unchanged) | Returns early (unchanged) |
| `SALESFORCE_ENABLED=false` | Discarded (unchanged) | Discarded
(unchanged) |

### What this does not fix

- 1,098 rejected schools without affiliations — working as designed.
- 3 schools whose users (1 live, 2 discarded/invalidated) have no
`Contact` in Salesforce — needs investigation upstream of editor-api.
- Existing `FAILED` mirror rows already in production — those need a
one-off cleanup (delete `sfid: nil` rows then re-enqueue `RoleSyncJob`);
already done for the 10 recoverable cases above.

## Tests

`spec/jobs/salesforce/role_sync_job_spec.rb`:

- Existing happy-path / save-failure / student-role / feature-flag-off
contexts retained, with parent rows now provisioned via `let!` so the
new guards pass.
- New contexts:
- `Salesforce::School` exists with `sfid: nil` → raises
`SalesforceRecordNotFound` (Editor not yet synced)
- No `Salesforce::School` row at all → raises `SalesforceRecordNotFound`
- `Salesforce::Contact` exists with `sfid: nil` → raises
`SalesforceRecordNotFound` (Contact not yet synced)
- No `Salesforce::Contact` row at all → raises
`SalesforceRecordNotFound`
  - In all failure cases, no row is written to the affiliation mirror.

## Test plan

- [x] CI rspec passes
- [x] Rubocop passes (verified locally)
- [ ] After deploy, monitor Heroku Connect error log for `Foreign key
external ID … not found` errors on `contact_editor_affiliation__c` —
should drop to ~0 for new schools
- [ ] Spot-check a freshly created school: confirm
`Contact_Editor_Affiliation__c` rows appear in Salesforce within retry
window

Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Author: 3 people

app/jobs/salesforce/role_sync_job.rb

@@ -18,6 +18,15 @@ def perform(role_id:)
 
       return if role.student?
 
+      # The Contact_Editor_Affiliation__c row uses Salesforce external-ID lookups to resolve
+      # its parent Editor__c (school) and Contact (user). If either parent has not yet been
+      # pushed to Salesforce, Heroku Connect rejects the INSERT permanently with a
+      # "Foreign key external ID ... not found" error and the row is stuck FAILED in the
+      # mirror. Raising SalesforceRecordNotFound here defers the affiliation write via the
+      # SalesforceSyncJob retry_on, giving the parent records time to land in Salesforce.
+      ensure_parent_synced!(Salesforce::School, :editoruuid__c, role.school_id, 'Editor__c')
+      ensure_parent_synced!(Salesforce::Contact, :pi_accounts_unique_id__c, role.user_id, 'Contact')
+
       sf_role = Salesforce::Role.find_or_initialize_by(affiliation_id__c: role_id)
       sf_role.attributes = sf_role_attributes(role:)
 
@@ -32,6 +41,13 @@ def perform(role_id:)
 
     private
 
+    def ensure_parent_synced!(model, external_id_field, external_id, label)
+      return if model.where(external_id_field => external_id).where.not(sfid: nil).exists?
+
+      raise SalesforceRecordNotFound,
+            "#{label} not yet synced for #{external_id_field}: #{external_id}"
+    end
+
     def sf_role_attributes(role:)
       mapped_attributes(role:).to_h do |sf_field, value|
         value = truncate_value(sf_field:, value:) if value.is_a?(String)

spec/jobs/salesforce/role_sync_job_spec.rb

@@ -6,6 +6,12 @@
   subject(:perform_job) { described_class.perform_now(role_id: role.id) }
 
   let(:role) { create(:role) }
+  let!(:sf_school) do
+    create(:salesforce_school, editoruuid__c: role.school_id, sfid: SecureRandom.alphanumeric(18))
+  end
+  let!(:sf_contact) do
+    create(:salesforce_contact, pi_accounts_unique_id__c: role.user_id, sfid: SecureRandom.alphanumeric(18))
+  end
 
   around do |example|
     ClimateControl.modify(SALESFORCE_ENABLED: 'true') { example.run }
@@ -53,6 +59,58 @@
     end
   end
 
+  context 'when the parent Editor__c is not yet synced to Salesforce' do
+    before { sf_school.update!(sfid: nil) }
+
+    it 'retries the job to defer the affiliation write' do
+      expect { perform_job }.to have_enqueued_job(described_class).with(role_id: role.id)
+    end
+
+    it 'does not write the affiliation to the mirror' do
+      perform_job
+      expect(Salesforce::Role.find_by(affiliation_id__c: role.id)).to be_nil
+    end
+  end
+
+  context 'when there is no Salesforce::School row for the role school' do
+    before { sf_school.destroy }
+
+    it 'retries the job' do
+      expect { perform_job }.to have_enqueued_job(described_class).with(role_id: role.id)
+    end
+
+    it 'does not write the affiliation to the mirror' do
+      perform_job
+      expect(Salesforce::Role.find_by(affiliation_id__c: role.id)).to be_nil
+    end
+  end
+
+  context 'when the parent Contact is not yet synced to Salesforce' do
+    before { sf_contact.update!(sfid: nil) }
+
+    it 'retries the job to defer the affiliation write' do
+      expect { perform_job }.to have_enqueued_job(described_class).with(role_id: role.id)
+    end
+
+    it 'does not write the affiliation to the mirror' do
+      perform_job
+      expect(Salesforce::Role.find_by(affiliation_id__c: role.id)).to be_nil
+    end
+  end
+
+  context 'when there is no Salesforce::Contact row for the role user' do
+    before { sf_contact.destroy }
+
+    it 'retries the job' do
+      expect { perform_job }.to have_enqueued_job(described_class).with(role_id: role.id)
+    end
+
+    it 'does not write the affiliation to the mirror' do
+      perform_job
+      expect(Salesforce::Role.find_by(affiliation_id__c: role.id)).to be_nil
+    end
+  end
+
   context 'when SALESFORCE_ENABLED is false' do
     around do |example|
       ClimateControl.modify(SALESFORCE_ENABLED: 'false') do