Skip to content

TrackBatch#58

Closed
stereosteve wants to merge 2 commits intomainfrom
sp-track-batch
Closed

TrackBatch#58
stereosteve wants to merge 2 commits intomainfrom
sp-track-batch

Conversation

@stereosteve
Copy link
Copy Markdown
Contributor

This is a POC for moving some stuff out of GetTracks sql and into go with some parallel loader stuff.

This is similar to how things work in python or elemental or graphql with dataloader.

Expand full_tracks.go to see the meat.

The hope is that:

  • running multiple simpler queries in parallel could be faster than a single large query. This is because postgres query execution is typically single threaded.
  • We could in the future use a different datastore (i.e. ES) for personalization stuff... if we can't figure out how to make it fast enough in postgres land.
  • If we do explore graphql or expose some optionality in rest land (i.e. "skip personalization") this code would be relevant.

TODO:

  • resolve FolloweeReposts + FolloweeReposts2 data types

Extra credit:

  • bulk query for access
  • might be nice to just use pointers everywhere (for better or worse)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant