Skip to content

fix: restore earthdistance function search paths#5205

Draft
nebhale wants to merge 1 commit into
teslamate-org:mainfrom
nebhale:nebhale/earth-functions-search-path
Draft

fix: restore earthdistance function search paths#5205
nebhale wants to merge 1 commit into
teslamate-org:mainfrom
nebhale:nebhale/earth-functions-search-path

Conversation

@nebhale
Copy link
Copy Markdown

@nebhale nebhale commented Mar 13, 2026

Summary

Reapply the public search path for the ll_to_earth and earth_box functions after the earthdistance extension upgrade.

Why

TeslaMate previously added search_path = public for these functions in the PostgreSQL 17 compatibility work because earthdistance functions without an explicit search path can fail to resolve the earth type correctly.

Later, the earthdistance v1.2 upgrade migration updated the extension but did not restore that custom function metadata. That appears to drop the expected search path from the extension-managed functions.

This follow-up migration restores the public search path for both functions.

Context

Validation

  • mix format priv/repo/migrations/20260312120000_set_search_path_for_earthdistance_functions.exs
  • mix compile

@netlify
Copy link
Copy Markdown

netlify Bot commented Mar 13, 2026

Deploy Preview for teslamate ready!

Name Link
🔨 Latest commit 2647ff3
🔍 Latest deploy log https://app.netlify.com/projects/teslamate/deploys/69b37a95d489c3000877cbb1
😎 Deploy Preview https://deploy-preview-5205--teslamate.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 13, 2026

CLA assistant check
All committers have signed the CLA.

@swiffer
Copy link
Copy Markdown
Collaborator

swiffer commented Mar 14, 2026

this isn't needed anymore, is it ?

@JakobLichterfeld JakobLichterfeld modified the milestone: v3.0.1 Mar 14, 2026
@JakobLichterfeld JakobLichterfeld added the note:discussion Details or approval are up for discussion label Mar 14, 2026
@JakobLichterfeld JakobLichterfeld marked this pull request as draft March 14, 2026 11:42
@nebhale
Copy link
Copy Markdown
Author

nebhale commented Mar 20, 2026

@swiffer I back my teslamate instance with a Google Cloud SQL PG instance. I've been looking to upgrade form 17 to 18 and it fails some pre-upgrade checks (I have two schemas on the same database):

image

My guess is that you're right, PG itself doesn't require this but at least Google's managed instances do.

@swiffer
Copy link
Copy Markdown
Collaborator

swiffer commented Mar 20, 2026

Ok, seems to be problematic if using inplace upgrade / pg_restore without previously creating the extensions agian like we do in the docs.

# Stop the teslamate container to avoid write conflicts
docker compose stop teslamate

# Drop existing data and reinitialize (Don't forget to replace first teslamate if using different TM_DB_USER)
docker compose exec -T database psql -U teslamate teslamate << .
DROP SCHEMA public CASCADE;
DROP SCHEMA private CASCADE;
CREATE SCHEMA public;
CREATE EXTENSION cube WITH SCHEMA public;
CREATE EXTENSION earthdistance WITH SCHEMA public;
.

# Restore
docker compose exec -T database psql -U teslamate -d teslamate < teslamate.bck

# Restart the teslamate container
docker compose start teslamate

You have been able to upgrade correctly after setting the search_path?

@nebhale
Copy link
Copy Markdown
Author

nebhale commented Mar 21, 2026

Yes, I was able to run those two commands manually and upgrade the database. Restarting Teslamate against the modified and upgraded database does not appear to have any problems.

@swiffer
Copy link
Copy Markdown
Collaborator

swiffer commented Mar 26, 2026

Good to hear - if we set this via migration - are we able to drop creating the schema and extensions in our own restore?

@nebhale
Copy link
Copy Markdown
Author

nebhale commented Mar 26, 2026

My hypothesis is yes, but I've never upgraded this way, so I defer to others on an authoritative answer.

@swiffer
Copy link
Copy Markdown
Collaborator

swiffer commented Mar 26, 2026

I will give it a try over the weekend ;)

@brianmay
Copy link
Copy Markdown
Collaborator

Disclaimer: I don't really understand these functions, or why they are required. Or why setting the search path is important. So I might have got details wrong.

I can say that the migration will only run if it has not run before. So if you restore an old backup that hasn't had this migration, it will run the migration. But if you restore a recent backup that already has had this migration applied, it will not run this migration again. Because it is already recorded in the schema_migrations table.

Normally this wouldn't matter, because the backup would also have all the values from all the applied migrations.

If I look at my backup, after manually applying the changes in this migration, the backup does not contain the new search_path values, all I see in my backup is this line:

SELECT pg_catalog.set_config('search_path', '', false);

Which I think might be insufficient. ?

@github-actions
Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It could be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the Stale label Apr 28, 2026
@nebhale
Copy link
Copy Markdown
Author

nebhale commented Apr 29, 2026

I'd like to keep this alive and eventually get it merged if possible. The change does appear to be necessary for some installations and the change has been validated.

@nebhale nebhale marked this pull request as ready for review April 29, 2026 15:26
@brianmay
Copy link
Copy Markdown
Collaborator

As above, it makes me nervous that this value does not appear to be saved in backups. Do we need to update the restore documentation to include a step to manually fix this?

@github-actions github-actions Bot removed the Stale label Apr 30, 2026
@JakobLichterfeld JakobLichterfeld added the enhancement New feature or request label May 1, 2026
@JakobLichterfeld JakobLichterfeld added this to the v3.1.0 milestone May 1, 2026
@nebhale
Copy link
Copy Markdown
Author

nebhale commented May 1, 2026

Do we need to update the restore documentation to include a step to manually fix this?

Copy that. I'll update the PR.

@nebhale nebhale force-pushed the nebhale/earth-functions-search-path branch from 2647ff3 to 7b6c2af Compare May 2, 2026 15:28
@netlify
Copy link
Copy Markdown

netlify Bot commented May 2, 2026

Deploy Preview for teslamate ready!

Name Link
🔨 Latest commit 2de5913
🔍 Latest deploy log https://app.netlify.com/projects/teslamate/deploys/69f618ae0c4c300008db1b3d
😎 Deploy Preview https://deploy-preview-5205--teslamate.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@nebhale
Copy link
Copy Markdown
Author

nebhale commented May 2, 2026

Documentation updated and ready for re-review.

Reapply the public search_path for ll_to_earth and earth_box after the earthdistance extension upgrade.

TeslaMate previously set these functions to use the public schema to avoid lookup errors around the earth type. The later earthdistance update migration upgrades the extension but does not restore that custom function metadata, which can leave the functions without the expected search path.

This adds a follow-up migration that sets the search path for both functions back to public and updates restore documentation to include these changes.

Signed-off-by: Ben Hale <nebhale@nebhale.com>
@nebhale nebhale force-pushed the nebhale/earth-functions-search-path branch from 7b6c2af to 2de5913 Compare May 2, 2026 15:30
@JakobLichterfeld
Copy link
Copy Markdown
Member

Do we need to update the restore documentation to include a step to manually fix this?

No, the issue is only present when someone does not follow the restore docu:

Ok, seems to be problematic if using inplace upgrade / pg_restore without previously creating the extensions agian like we do in the docs.

So imo this PR is not needed

If I look at my backup, after manually applying the changes in this migration, the backup does not contain the new search_path values, all I see in my backup is this line:

SELECT pg_catalog.set_config('search_path', '', false);

Which I think might be insufficient. ?

And as you observed does not provide any meaningful value.

@JakobLichterfeld JakobLichterfeld marked this pull request as draft May 2, 2026 16:40
@brianmay
Copy link
Copy Markdown
Collaborator

brianmay commented May 2, 2026

I do think we need more data to justify the need for this. i.e. what steps are required to reproduce the search_path being unset.

For the record, AI is telling me the solution is not to rely on the search path at all, and prefix everything with "public.", e.g. public.earth_box(public.ll_to_earth(...)).

@JakobLichterfeld JakobLichterfeld removed this from the v3.1.0 milestone May 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request note:discussion Details or approval are up for discussion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants