rbio migration cleanup#3591
Open
davidsmejia wants to merge 5 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue Number
#3572
Purpose/Implementation Notes
Every entry in
scripts/now has a replacement landed earlier in this stack (rbio commands,install/*.shhelpers, or relocated toinfrastructure/). This PR ports the last script (calculate_ram_hours.py) to a Django management command, trims the last code-side dependency onscripts/common.sh, then deletes the directory and refreshes the READMEs to match.Commits
This PR is split into four commits so reviewers can step through them independently:
foreman: add calculate_ram_hours management commandPortsscripts/calculate_ram_hours.pyto a Django management command in foreman. Replaces pyrefinebio HTTP queries with direct ORM access viaSample.get_downloader_jobs()/get_processor_jobs(). Invocation:rbio compose:manage foreman calculate_ram_hours <accession>locally, ordocker exec dr_foreman python3 manage.py calculate_ram_hours <accession>on the prod foreman EC2.infra/format_batch_with_env.sh: drop dead common.sh source + unused exportsThe script sourced../scripts/common.shsolely to callget_docker_db_ip_address, which on the deploy box (nodrdbcontainer) returned an empty string. The exportedDB_HOST_IPandEXTRA_HOSTSvalues are never substituted into any*.tpl.jsontemplate (verified by grep). Dead exports kept alive by a dead source. Removes the last code-side dependency onscripts/common.sh.remove scripts/ and obsolete shell wrappers; rbio --help is the entry pointThe big delete: 18 files. Every deletion has a landed replacement (or, in the case offoreman/test_survey.sh, is confirmed dead via grep of@tag("manual")across foreman tests).docs: refresh READMEs to point at rbio commandsWalksREADME.mdandforeman/README.mdand replaces every reference to a deleted script with its rbio equivalent. Touches the Quick Start, Installation (manual + automatic), Postgres helpers, Testing, Surveyor commands, Downloader/Processor Job examples, Development Helpers, Docker Images, and Terraform sections.Deletes in commit 3
scripts/(12 files):calculate_ram_hours.pyforeman/.../management/commands/calculate_ram_hours.py(added in commit 1)common.sh--network refinebio_defaultcreate_virtualenv.shinstall/setup_venv.sh+rbio install:venvinstall_all.shinstall/install_deps.sh+rbio install:depskill_all_jobs.{py,sh}rbio ops:kill-jobsprepare_image.shrbio build <target>run_all_tests.shrbio test:allrun_full_pipeline.pyrbio dev:pipelinerun_manage.shrbio compose:manage <service> <args>run_shell.shrbio compose:manage <service> shellupdate_docker_images.shrbio build --push <group>Other deletes (6 files):
.github/scripts/run_terraform.shremote_deploy.sh; final SSH callsrbio deploy:upforeman/run_management_command.shrbio compose:manage foreman <args>workers/run_command.shrbio compose:manage <worker> <args>foreman/test_survey.sh@tag("manual")tests exist); the--exclude-tag=manualdefault intest:foremanis a tracked follow-upworkers/run_job.shrbio dev:job <image> <subcommand> [args]workers/run_janitor.shrbio dev:janitorWhat is NOT affected
The
metasraREADME and the prod foreman EC2 user-data template reference a differentrun_management_command.shthat lives on the prod foreman instance (generated by user-data on that box). Those references are correct and intentionally unchanged.Methods
N/A
Types of changes
Functional tests
N/A
Checklist
Screenshots
N/A