TimescaleDB local to cloud sync, wrong DBC warning#64
Merged
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| import logging | ||
| import time | ||
| from datetime import datetime, timezone | ||
| from typing import Optional |
|
|
||
| import logging | ||
| import time | ||
| from datetime import datetime, timezone |
| cur.execute(""" | ||
| SELECT add_compression_policy(%s, INTERVAL '2 days', if_not_exists => TRUE) | ||
| """, (self.cloud_table,)) | ||
| except Exception: |
| finally: | ||
| try: | ||
| local_conn.close() | ||
| except Exception: |
| pass | ||
| try: | ||
| cloud_conn.close() | ||
| except Exception: |
Expose local table management: add GET /api/local-tables and POST /api/select-local-table endpoints to list available local tables and switch the active local source (selection is blocked while a sync is running and it invalidates unsynced cache/state). Implement SyncEngine.list_local_tables to query the local DB information_schema for tables matching ^wfr[0-9].
Update the web UI to include a local table dropdown, loadLocalTables/selectLocalTable JS handlers, polling integration, and minor label tweak ("New" -> "New cloud").
Normalize the default local table name from wfr26test_base to wfr26base across config, .env.macbook and docker-compose defaults (also normalizing casing for TIMESCALE_TABLE), so the composed services and environment use the new base table name.
Introduce a new cloud-sync microservice to copy rows from local TimescaleDB to a cloud TimescaleDB. Adds a FastAPI-based API and static web UI (static/index.html), a SyncEngine (sync.py) that uses psycopg2 to upsert rows and manage dynamic signal columns, a simple config module, requirements, and a Dockerfile exposing port 8092 with a healthcheck. Integrates the service into docker-compose.macbook-base.yml (service definition, env vars, depends_on), adds deploy/pg_hba.conf and mounts it into the TimescaleDB container with a command to load the custom hba, and updates deploy/.env.macbook with example CLOUD_POSTGRES_DSN and LOCAL_TABLE overrides. Cloud DSN is optional; sync is gated on CLOUD_POSTGRES_DSN being configured.
Fix cloud-sync cursor; update deploy and status
Handle named-server cursor behavior in cloud-sync: fetch the first batch before reading cursor.description so column names and signal columns are populated correctly, and iterate fetchmany() in a proper loop to avoid missing batches. Update .env.macbook to point DBC_HOST_PATH to the secret DBC location. Clarify docker-compose.macbook-base usage and environment file path in comments, and add status and Cloud Sync access URLs. Enhance status UI to show the DBC file name (shortened), color-code it (red for example.dbc), and retain the existing warning banner behavior for the example DBC.
Use DBC_DISPLAY_NAME and don't clear frozenRef
Preserve frozenRef when clearing traces and add a configurable DBC display name.
Optional prompt-guide for CI
haoruizhou
force-pushed
the
timescale-sync
branch
from
2ae2bf0 to
3defdb0
Compare
Introduce a GitHub Actions workflow to package required Docker images into an offline tarball and publish it as an artifact (and as a release asset on publish). The workflow (/.github/workflows/offline-bundle.yml) pulls/builds the cloud-sync image, saves all images to universal-telemetry-software/deploy/offline/wfr-docker-images.tar, and uploads it; it can be triggered via workflow_dispatch, release (published), or pushes to timescale-sync/main/dev. Also add universal-telemetry-software/deploy/offline/.gitignore and a README with instructions to build, load, and run the MacBook offline stack.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.