Skip to content

rbio migration cleanup#3591

Open
davidsmejia wants to merge 5 commits into
davidsmejia/3572-08-ci-workflowfrom
davidsmejia/3572-09-cleanup
Open

rbio migration cleanup#3591
davidsmejia wants to merge 5 commits into
davidsmejia/3572-08-ci-workflowfrom
davidsmejia/3572-09-cleanup

Conversation

@davidsmejia
Copy link
Copy Markdown
Contributor

@davidsmejia davidsmejia commented May 26, 2026

Issue Number

#3572

Purpose/Implementation Notes

Every entry in scripts/ now has a replacement landed earlier in this stack (rbio commands, install/*.sh helpers, or relocated to infrastructure/). This PR ports the last script (calculate_ram_hours.py) to a Django management command, trims the last code-side dependency on scripts/common.sh, then deletes the directory and refreshes the READMEs to match.

Commits

This PR is split into four commits so reviewers can step through them independently:

  1. foreman: add calculate_ram_hours management command Ports scripts/calculate_ram_hours.py to a Django management command in foreman. Replaces pyrefinebio HTTP queries with direct ORM access via Sample.get_downloader_jobs() / get_processor_jobs(). Invocation: rbio compose:manage foreman calculate_ram_hours <accession> locally, or docker exec dr_foreman python3 manage.py calculate_ram_hours <accession> on the prod foreman EC2.

  2. infra/format_batch_with_env.sh: drop dead common.sh source + unused exports The script sourced ../scripts/common.sh solely to call get_docker_db_ip_address, which on the deploy box (no drdb container) returned an empty string. The exported DB_HOST_IP and EXTRA_HOSTS values are never substituted into any *.tpl.json template (verified by grep). Dead exports kept alive by a dead source. Removes the last code-side dependency on scripts/common.sh.

  3. remove scripts/ and obsolete shell wrappers; rbio --help is the entry point The big delete: 18 files. Every deletion has a landed replacement (or, in the case of foreman/test_survey.sh, is confirmed dead via grep of @tag("manual") across foreman tests).

  4. docs: refresh READMEs to point at rbio commands Walks README.md and foreman/README.md and replaces every reference to a deleted script with its rbio equivalent. Touches the Quick Start, Installation (manual + automatic), Postgres helpers, Testing, Surveyor commands, Downloader/Processor Job examples, Development Helpers, Docker Images, and Terraform sections.

Deletes in commit 3

scripts/ (12 files):

Deleted Replacement
calculate_ram_hours.py foreman/.../management/commands/calculate_ram_hours.py (added in commit 1)
common.sh callers inlined what they used; helpers either dead or replaced by --network refinebio_default
create_virtualenv.sh install/setup_venv.sh + rbio install:venv
install_all.sh install/install_deps.sh + rbio install:deps
kill_all_jobs.{py,sh} rbio ops:kill-jobs
prepare_image.sh rbio build <target>
run_all_tests.sh rbio test:all
run_full_pipeline.py rbio dev:pipeline
run_manage.sh rbio compose:manage <service> <args>
run_shell.sh rbio compose:manage <service> shell
update_docker_images.sh rbio build --push <group>

Other deletes (6 files):

Deleted Replacement
.github/scripts/run_terraform.sh inlined into remote_deploy.sh; final SSH calls rbio deploy:up
foreman/run_management_command.sh rbio compose:manage foreman <args>
workers/run_command.sh rbio compose:manage <worker> <args>
foreman/test_survey.sh dead (no @tag("manual") tests exist); the --exclude-tag=manual default in test:foreman is a tracked follow-up
workers/run_job.sh rbio dev:job <image> <subcommand> [args]
workers/run_janitor.sh rbio dev:janitor

What is NOT affected

The metasra README and the prod foreman EC2 user-data template reference a different run_management_command.sh that lives on the prod foreman instance (generated by user-data on that box). Those references are correct and intentionally unchanged.

Methods

N/A

Types of changes

  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Functional tests

N/A

Checklist

  • Lint and unit tests pass locally with my changes
  • I have added necessary documentation (if appropriate)

Screenshots

N/A

@davidsmejia davidsmejia changed the title move rbio migration cleanup May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant