Pathfinder doc / behavior issues

[avehtari]

I titled this issue as generic Pathfinder doc / behavior issues, not to make separate for each small issue

$pathfinder() doc says

   draws: (positive integer) Number of draws to return after performing
          pareto smooted importance sampling (PSIS). This must be
          smaller than ‘single_path_draws * num_paths’.

Current behavior is that this is by default 1000 which can be bigger than ‘single_path_draws * num_paths’ and also it can be set to value which is bigger than ‘single_path_draws * num_paths’, without any warning.

[jgabry]

@avehtari Are you proposing just that we add a warning or change the behavior?

@WardBrian @SteveBronder Is this also the case with the cmdstanpy implementation or did we end up doing something different here accidentally?

[WardBrian]

We had discussions about automatically adjusting the num_single_draws to prevent this but Steve argued successfully that this would be too magic. Cmdstan got a warning for it in stan-dev/cmdstan#1221

[jgabry]

Oh ok great, so this warning will get handled in CmdStan itself.

[avehtari]

I'm fine that the document is updated to reflect that it doesn't need to be smaller

[jgabry]

Ok instead of "This must be smaller than ‘single_path_draws * num_paths’." should we just say "This should be smaller than ‘single_path_draws * num_paths’ or CmdStan will throw a warning." Or something else? (Currently it won't throw a warning but sounds like next version will)

[jgabry]

@avehtari are you ok with changing the doc to "This should be smaller than ‘single_path_draws * num_paths’ or CmdStan will throw a warning."? (although the warning won't come until the next version of CmdStan)

[avehtari]

Yes

[jgabry]

Ok I updated the doc to

This should be smaller than single_path_draws * num_paths (future versions of CmdStan will throw a warning).

I will leave this issue open and wait to close it until the CmdStan with the warning is released (at that time I'll update the doc again).

[avehtari]

Adding here another pathfinder issue
save_single_paths option does not seem to be working (installed the latest cmdstanr from github). With save_single_paths=TRUE (or =1) there are no files for individual paths and the only csv shows

#     save_single_paths = 0 (Default)

[avehtari]

Has this been fixed?

[jgabry]

Closing because I think everything in this issue is complete. #1192 finally fixed save_single_paths = TRUE, which I guess never got its own issue. There are a lot of pathfinder related things I've been fixing recently (e.g. #1192, #1205, #1206, #1207, etc.). I didn't do the initial implementation, so I wasn't aware of some of these things and other things I did know about but hadn't had time to fix yet