Remove intermediate datasets from working environment on program calls

[parmsam-pfizer]

Based on recent feedback from reviewers.

[robertdevine]

@parmsam-pfizer , @bms63 , @RConsortium/pilot5 - remove or rm will deliver the environment display request. The S system also provides capability to hide objects yet not appropriate to Hye Soo's request at the general meeting. Care should be taken with use of rm or remove - for example, removal of intermediate objects will not affect program operations yet removal of primitives may require re-running steps in the ADRG ('path' is an appropriate example). [The intermediate objects removed from the environment tab using remove or rm are easily re-created with each re-run of the production program.]

[bms63]

I don't remember Pilot 3 removing objects in a programmatic way to make things easy to view.

Could we suggest a line of code in the ADRG to only keep the final datasets in their environment using rm() ? I don't think we should do anything with this code in the programs as it might have untoward impacts.

@RConsortium/pilot5 ideas?

[robertdevine]

@RConsortium/pilot5 - I also recall from pilot 3 days that the intermediate datasets are present in the submission. We still have some sandboxes from the ADRG testing phase during p3 (pilot 3) - the intermediate datasets are in the environment as shown below. Not a major item to remove the intermediate datasets from the environment if need arises per the regulatory request. As noted the intermediate sets are re-created with a simple re-run of the production programs if further analysis is required.

[bms63]

did they want the final datasets for the TLFs as well - those are not as pretty to look at but we can do that.

Maybe we should have a cleaning script that could be run to remove everything in a non-invasive way. rm() always scares as it might hit the .Rprofile setup and then that might confuse them if they don't restart the session :(

[robertdevine]

Thanks @bms63, @parmsam-pfizer, @RConsortium/pilot5 - as noted at today's meeting seems appropriate to update p5 ADRG Appendix Sec. 9.5.5 to include a one-line rm(list(x,y,z, etc.)) command to pare down the environment per the regulatory request. Then if reviewers wish to re-run the production programs they can get a fresh display of the full environment allowing deeper analysis or even re-use the ADRG command to pare down the environment window without intermediate data sets. If this sounds reasonable, I'll update the ADRG and issue a PR.

[bms63]

keep_names <- c("data1", "data2")
dataframe_objs <- ls()[sapply(mget(ls(), .GlobalEnv), is.data.frame)]
rm(list = setdiff(dataframe_objs, keep_names))