create fitted model object with subset of variables

[jgabry]

Following up on this forum post from @ihrke, we should add a way to create fitted model objects using only a subset of the variables in the CSV files.

My first thought is we could add a variables argument to as_cmdstan_fit(). However, we would then need to either eliminate the variables from the CSV files or add some functionality to prevent them from being read in by subsequent calls to draws() or other methods that result in reading in variables.

@rok-cesnovar (or anyone else) any altnernative ideas or suggestions for how to handle not reading the unwanted variables in later when draws() is called after as_cmdstan_fit()?

[jwschroeder3]

Hi @jgabry, can this also be implemented for the initial fit object and resulting csv, rather than for cases where the user would like to pare back the variables to a sub-set after the csv file has already been written? In my case, I'm working with a model with millions of parameters. My current understanding is that I cannot limit which parameters are written to the csv at fit-time, but rather, I can only choose which to read later on. This results in a csv file containing all the variables, which is hundreds-of-Gigabytes. Parsing that csv file later to pare back the variables takes a very long time. I would much rather only write the variables I'll need at the outset. Can this be done? I'm happy to contribute if necessary!

(Edit): Just to be clear, what I would like in cmdstanr is something like rstan's pars and include arguments in cmdstanr's variational method.

[andrjohns]

Unfortunately that behaviour would have to be implemented in cmdstan as an option. It's been discussed before but does not look likely to have an implementation on the horizon: stan-dev/cmdstan#553

[jwschroeder3]

Thanks for your response, and thanks for providing that link. I read through the discussion, and I think it all makes sense.

[jgabry]

This is now implemented in #1121 via as_cmdstan_fit() to read a subset of variables from the CSV and create a fitted model object, but we still require a change in cmdstan itself (stan-dev/cmdstan#553) to avoid writing specific variables to CSV.