docs: pedagogical audit of varPro wrappers (gg_partial_varpro, gg_varpro, gg_udependent) (#95)

* docs(gg_partial_varpro): teach what varPro partialpro is doing

* docs(gg_varpro): teach what varpro variable priority is doing

* docs(gg_udependent): teach what cross-variable dependency is doing

* chore: open v2.7.3.9009 + NEWS for varPro pedagogical doc audit

* docs: enable roxygen2 markdown package-wide

Add Roxygen: list(markdown = TRUE) to DESCRIPTION so devtools::document()
auto-converts backticks / [fn()] / [pkg::fn()] in source roxygen to
\code{} / \link{} / \link[pkg]{} in the generated Rd. Existing Rd-style
markup keeps working; both styles now coexist. Saves the manual
conversion work the Copilot review on PR #94 flagged.

Two source-roxygen edits needed to keep R CMD check clean under markdown:
- R/help.R: randomForest[SRC] -> randomForestSRC (markdown read [SRC]
  as an unfinished link reference, producing a missing-link warning).
- R/gg_rfsrc.R::bootstrap_survival: 95\% -> 95% (markdown over-escaped
  the backslash, producing a malformed Rd with shifted section order).

Regenerates all 31 Rd files. No functional or rendered-content change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: address PR #95 Copilot review

- plot.gg_udependent: clarify that truly isolated nodes are dropped by
  gg_udependent() before plotting; reword 'Isolated' as 'low-degree'.
- gg_partial_varpro: fix varpro::partialpro -> varPro::partialpro (six
  instances) so \link{} renders correctly.
- plot.gg_varpro: clarify the cutoff line lives in z-units on the
  default axis and in raw-importance units when type='raw'; the
  numeric is the same, the scale is not.
- gg_varpro reference: complete the dangling 'arXiv 2409.' with the
  full arXiv:2409.09003 identifier and an https://arxiv.org link.

* docs: regenerate gg_isopro Rd under markdown mode post-rebase

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: ehrlinger

DESCRIPTION

@@ -1,7 +1,7 @@
 Package: ggRandomForests
 Type: Package
 Title: Visually Exploring Random Forests
-Version: 2.7.3.9008
+Version: 2.7.3.9009
 Date: 2026-05-21
 Authors@R: person("John", "Ehrlinger",
   role = c("aut", "cre"),
@@ -50,3 +50,4 @@ Suggests:
     callr
 VignetteBuilder: quarto
 Config/roxygen2/version: 8.0.0
+Roxygen: list(markdown = TRUE)

NEWS.md

@@ -1,8 +1,25 @@
 Package: ggRandomForests
-Version: 2.7.3.9008
+Version: 2.7.3.9009
 
 ggRandomForests v2.8.0 (development) — continued
 =================================================
+* Documentation: pedagogical pass over the varPro wrappers
+  (`gg_partial_varpro`, `gg_varpro`, `gg_udependent` and their `plot.*`
+  methods). Each help page now has explicit "What X is doing", "What's
+  in the output", and "What you use this for" sections so a reader new
+  to varPro can learn the underlying method (release rules, beta-entropy
+  dependency, parametric / nonparametric / causal partial estimators)
+  from the help page alone, not just the wrapper mechanics. No API or
+  behavioural change.
+* Documentation: enable roxygen2 markdown package-wide via
+  `Roxygen: list(markdown = TRUE)` in `DESCRIPTION`. New roxygen blocks
+  can use backticks and `[fn()]` link syntax; existing `\code{}` /
+  `\link{}` markup keeps working. Two source-roxygen edits to keep
+  R CMD check clean: `randomForest[SRC]` in `R/help.R` (markdown read
+  it as an unfinished link) becomes plain `randomForestSRC`; the `95\%`
+  escape in `R/gg_rfsrc.R::bootstrap_survival` becomes a literal `95%`.
+  No API or rendered-doc behavioural change beyond the conventions
+  switch.
 * New `gg_isopro()` and `plot.gg_isopro()`: tidy wrapper and ranked-elbow +
   density visualisation for `varPro::isopro` isolation-forest anomaly
   scores. `plot.gg_isopro()` takes `panel = c("both", "elbow", "density")`

R/gg_partial_varpro.R

@@ -1,19 +1,78 @@
 ##=============================================================================
 #' Partial dependence data from a varPro model
 #'
-#' \code{varpro::partialpro} returns one list, with continuous and
+#' \code{varPro::partialpro} returns one list, with continuous and
 #' categorical predictors mixed together. This function splits that list into
 #' two tidy data frames, one for each kind, and resolves the y-axis label the
 #' plot method will use.
 #'
-#' @param part_dta Partial plot data from \code{varpro::partialpro}.  Each
+#' @section What partialpro is doing:
+#' A partial dependence curve answers the question, "if I hold a single
+#' variable at a grid of values and average out everything else, how does
+#' the model's prediction move?" That is the same question \code{rfsrc}
+#' partial dependence answers. What \code{varPro::partialpro} adds is two
+#' wrinkles that are worth understanding before you read the curves.
+#'
+#' First, \code{partialpro} filters the partial grid through an isolation
+#' forest (Unlimited Virtual Twins, or UVT) so that unlikely combinations
+#' of the focal variable with the rest of the data are downweighted. The
+#' \code{rfsrc} version, by contrast, averages over the full marginal grid
+#' regardless of plausibility. So when a covariate is highly correlated
+#' with others, the two methods can disagree, and \code{partialpro}'s
+#' curve is the one restricted to the data manifold.
+#'
+#' Second, \code{partialpro} fits a local polynomial model to the
+#' predicted values rather than just plotting their mean. That gives
+#' three parallel curves per variable, stored as \code{yhat.par},
+#' \code{yhat.nonpar}, and \code{yhat.causal}, which the plot method
+#' overlays so you can see whether a smooth parametric story and the
+#' raw forest predictions are telling you the same thing.
+#'
+#' Interpretation of the y-axis depends on the outcome (per
+#' \code{varPro::partialpro}): response scale for regression, log-odds of
+#' the target class for classification, and either ensemble mortality
+#' (default) or RMST (if the original \code{varpro} call set
+#' \code{rmst}) for survival.
+#'
+#' @section What's in the output:
+#' We split \code{partialpro}'s mixed list into two tidy data frames so
+#' the plot method does not have to. A variable with more than
+#' \code{cat_limit} distinct grid points goes into \code{$continuous},
+#' one row per grid point with the column means of \code{yhat.par},
+#' \code{yhat.nonpar}, and \code{yhat.causal} stored as
+#' \code{parametric}, \code{nonparametric}, and \code{causal}. A
+#' variable at or below \code{cat_limit} goes into \code{$categorical},
+#' one row per observation per category level, carrying the same three
+#' columns unaveraged so the plot method can draw boxplots. Path C
+#' (\code{scale \%in\% c("surv","chf")}) takes a different route: we
+#' hand the underlying \code{rfsrc} forest to \code{gg_partial_rfsrc} so
+#' you get a survival-probability or cumulative-hazard curve on the
+#' usual rfsrc scale instead.
+#'
+#' @section What you use this for:
+#' \itemize{
+#'   \item read the marginal shape of a relationship the varpro model
+#'     found important — monotone, threshold, U-shape, flat;
+#'   \item compare the three partialpro estimators on the same variable
+#'     and flag the ones where parametric and nonparametric disagree —
+#'     those are the candidates for closer inspection;
+#'   \item report a survival partial dependence on the probability or
+#'     cumulative-hazard scale (\code{scale = "surv"} or \code{"chf"})
+#'     rather than the unbounded mortality scale.
+#' }
+#' A varpro partial dependence curve is a description of the model, not
+#' a causal effect. The \code{causal} column is varpro's local
+#' estimator, not a structural causal claim about the data-generating
+#' process.
+#'
+#' @param part_dta Partial plot data from \code{varPro::partialpro}.  Each
 #'   element must contain \code{xvirtual}, \code{xorg}, \code{yhat.par},
 #'   \code{yhat.nonpar}, and \code{yhat.causal}.  Supply at least one of
 #'   \code{part_dta} or \code{object}.
 #' @param object A fitted \code{varpro} object, the forest the partial data
 #'   came from.  When supplied it provides the provenance metadata, and when
 #'   \code{part_dta} is \code{NULL} it is passed to
-#'   \code{varpro::partialpro(object)} for you.  Required when
+#'   \code{varPro::partialpro(object)} for you.  Required when
 #'   \code{scale \%in\% c("surv","chf")}.
 #' @param scale Character; sets the y-axis label and, for survival forests,
 #'   the output type.  One of \code{"auto"} (default), \code{"mortality"},

R/gg_rfsrc.R

@@ -383,7 +383,7 @@ gg_rfsrc.rfsrc <- function(object, # nolint: cyclocomp_linter
 #' @param bs_samples Integer; number of bootstrap resamples.
 #' @param level_set Numeric vector of length 2 giving the lower and upper
 #'   quantile probabilities for the confidence band (e.g. \code{c(0.025, 0.975)}
-#'   for a 95\% CI).
+#'   for a 95% CI).
 #'
 #' @return A \code{data.frame} with one row per unique event time and columns
 #'   \code{value} (time), \code{lower}, \code{upper}, \code{median}, and

R/gg_udependent.R

@@ -7,6 +7,67 @@
 #' \code{\link[varPro]{sdependent}}, and returns them as a tidy list that
 #' \code{plot.gg_udependent} can draw as a network.
 #'
+#' @section What cross-variable dependency is doing:
+#' UVarPro (Zhou, Lu and Ishwaran, 2026) extends the varpro framework to
+#' the unsupervised setting: grow a forest without a response, then use
+#' the same region-release contrasts varpro uses for supervised
+#' importance to ask, "which variables explain the structure in the
+#' data?" The lasso-driven variant frames each region-release contrast
+#' as a classification task — does an observation belong to the region
+#' or to its release? — and fits a lasso logistic regression with the
+#' other variables as predictors. The coefficient on variable \eqn{j}
+#' in the model for variable \eqn{i}'s region-release contrast is the
+#' entry \eqn{I[i, j]} of the matrix \code{varPro::get.beta.entropy()}
+#' returns.
+#'
+#' Read that entry as "how much does knowing \eqn{j} help separate
+#' \eqn{i}'s region from its release". A large \eqn{I[i, j]} says
+#' \eqn{j} carries information about the structure varpro picked up in
+#' \eqn{i}. \code{varPro::sdependent} thresholds that matrix at a
+#' user-chosen cut and returns the set of "signal" variables — the
+#' nodes with high enough out-degree to be worth keeping. We pass the
+#' threshold through to \code{sdependent} and use the same matrix to
+#' weight the edges of the resulting graph.
+#'
+#' The graph is directed by default because \eqn{I[i, j]} and
+#' \eqn{I[j, i]} are separate lasso coefficients and need not agree;
+#' setting \code{directed = FALSE} collapses each pair by taking the
+#' larger of the two, which is appropriate when you only want to see
+#' that two variables are dependent, not which way the dependency
+#' reads.
+#'
+#' @section What's in the output:
+#' \code{$edges} has one row per surviving edge with the raw weight
+#' \code{I[i, j]} (or, for undirected graphs, the max of the two
+#' directions). \code{$nodes} has one row per surviving variable with
+#' its degree (out-degree for directed, total degree for undirected)
+#' and a \code{selected} flag for membership in the \code{sdependent}
+#' signal set. \code{$graph} is the same information packaged as an
+#' \code{igraph} object, with \code{weight}, \code{degree}, and
+#' \code{selected} attached so \code{plot.gg_udependent} can render it
+#' without recomputing anything.
+#'
+#' @section What you use this for:
+#' \itemize{
+#'   \item screen a wide unsupervised dataset for the small set of
+#'     variables UVarPro thinks are carrying the signal — the nodes
+#'     with high degree, or those flagged \code{selected = TRUE};
+#'   \item spot clusters of mutually dependent variables (hubs and the
+#'     spokes around them) that may be measuring the same underlying
+#'     construct;
+#'   \item compare two datasets, or two preprocessing pipelines, by
+#'     looking at how their dependency graphs change.
+#' }
+#' An edge in this graph is a statistical dependency in the unsupervised
+#' decomposition of the data — it is not a causal arrow. A high
+#' \eqn{I[i, j]} says \eqn{j} predicts \eqn{i}'s region membership,
+#' not that \eqn{j} causes \eqn{i}.
+#'
+#' @references
+#' Zhou, L., Lu, M. and Ishwaran, H. (2026). Variable priority for
+#' unsupervised variable selection. \emph{Pattern Recognition},
+#' 172:112727.
+#'
 #' @param object A fitted \code{uvarpro} object (required).
 #' @param threshold Numeric; the positive dependency threshold passed on to
 #'   \code{sdependent()}.  An edge \eqn{i \to j} is drawn when

R/gg_varpro.R

@@ -8,6 +8,70 @@
 #' For a classification forest you can also keep the class-conditional
 #' importances.
 #'
+#' @section What varpro is doing:
+#' Permutation importance asks "what happens to OOB accuracy when I scramble
+#' this variable?" That works, but it leans on artificial data — the
+#' permuted column — and the answer can be unstable when variables are
+#' correlated. The varpro framework (Lu and Ishwaran, 2024) replaces
+#' permutation with \emph{release rules}. The forest is grown with guided
+#' splitting; from a subset of trees varpro samples a collection of
+#' decision-rule branches; for each variable it then compares the
+#' response inside the rule's region to the response after the rule's
+#' constraint on that variable is "released". The size of that change,
+#' aggregated over many rules and trees, is the variable's importance.
+#' No synthetic covariates, no permutation — the contrast is between two
+#' real subsets of the data.
+#'
+#' Because varpro builds importance from rules sampled over trees, every
+#' tree contributes its own importance value for each variable. Those are
+#' the per-tree scores we summarise here. With \code{local.std = TRUE}
+#' (the default) the per-tree values are standardised by their column
+#' standard deviation so the column mean equals the aggregate z-score
+#' returned by \code{varPro::importance()}; that z-score is the canonical
+#' "is this variable in or out?" statistic, and \code{cutoff = 0.79} is
+#' varpro's default selection threshold.
+#'
+#' For a classification forest, varpro also returns a class-conditional
+#' z table: the same importance computed restricting attention to rules
+#' relevant to each class. \code{conditional = TRUE} keeps that table so
+#' the plot method can show which variables matter for which class
+#' rather than only in aggregate.
+#'
+#' @section What's in the output:
+#' \code{$imp} is the one-row-per-variable summary: aggregate z from
+#' \code{varPro::importance()}, plus a \code{selected} flag for
+#' \code{z > cutoff}. \code{$stats} holds the box quantiles
+#' (5/15/50/85/95 percentiles, plus the raw mean) computed from the
+#' per-tree matrix; these are what the boxplot draws. \code{$imp.tree}
+#' is the per-tree matrix itself, kept only when \code{faithful = TRUE}
+#' so the plot method can scatter individual tree values over the box.
+#' \code{$conditional} is the tidy class x variable z table, present
+#' only when \code{conditional = TRUE} and the family is
+#' classification.
+#'
+#' @section What you use this for:
+#' \itemize{
+#'   \item rank candidate variables by importance and pick a working set
+#'     above varpro's z cutoff;
+#'   \item see, via the boxplot's spread and the per-tree points
+#'     (\code{faithful = TRUE}), how stable each variable's importance
+#'     is across trees — a high median with a wide box is a different
+#'     story from a high median with a tight box;
+#'   \item for a classification forest, ask which variables drive which
+#'     class (\code{conditional = TRUE}) rather than just which
+#'     variables drive the model overall.
+#' }
+#' The z-score is a standardised ranking statistic, not a p-value or a
+#' probability. Two variables with the same z are "similarly important
+#' by this method", not "equally likely to be true signal". For a
+#' data-driven cutoff rather than the 0.79 default, see
+#' \code{varPro::cv.varpro}.
+#'
+#' @references
+#' Lu, M. and Ishwaran, H. (2024). Model-independent variable selection
+#' via the rule-based variable priority framework. \emph{arXiv preprint}
+#' \href{https://arxiv.org/abs/2409.09003}{arXiv:2409.09003}.
+#'
 #' @param object A fitted \code{varpro} object (required).
 #' @param local.std Logical; default \code{TRUE}.  When \code{TRUE} the
 #'   per-tree importances are put on the z-scale before the box statistics

R/help.R

@@ -58,8 +58,8 @@
 #'
 #' The \code{ggRandomForests} package contains the following data functions:
 #' \itemize{
-#' \item \code{\link{gg_rfsrc}}: randomForest[SRC] predictions.
-#' \item \code{\link{gg_error}}: randomForest[SRC] convergence rate based on
+#' \item \code{\link{gg_rfsrc}}: randomForestSRC predictions.
+#' \item \code{\link{gg_error}}: randomForestSRC convergence rate based on
 #' the OOB error rate.
 #' \item \code{\link{gg_roc}}: ROC curves for randomForest classification
 #' models.

R/plot.gg_partial_varpro.R

@@ -8,6 +8,33 @@
 #' \code{scale \%in\% c("surv","chf")} was passed to the extractor) are
 #' handed off to \code{\link{plot.gg_partial_rfsrc}} for drawing.
 #'
+#' @section Reading the partial dependence:
+#' For a continuous variable the x-axis is the variable's grid of values
+#' and the y-axis is the partial prediction; each of the three effect
+#' types (\code{parametric}, \code{nonparametric}, \code{causal}) is
+#' drawn as its own line. The shape of the line is the story: a clear
+#' slope says the model uses the variable, a flat line says it
+#' essentially does not, and a U-shape or a threshold says the effect
+#' is nonlinear in a way a single coefficient would miss. For a
+#' categorical variable the picture is a boxplot per level; here the
+#' eye is looking at level-to-level shifts in the centre of each box.
+#'
+#' Where the three effect types track each other, the parametric story
+#' is a fair summary of what the forest is doing. Where they fan
+#' apart — typically the parametric curve smoother than the
+#' nonparametric, or the causal curve flatter than either — the
+#' variable is one to inspect more carefully before reading a single
+#' effect off the plot.
+#'
+#' @section What this tells you:
+#' Use these curves to describe how the model uses each variable, not
+#' to claim how the world works. They are a window into the fitted
+#' relationship; they do not by themselves establish that intervening
+#' on the variable would move the outcome. For survival path-C
+#' (\code{scale = "surv"} or \code{"chf"}), the y-axis is on the
+#' probability or cumulative-hazard scale, which is usually the scale
+#' you want to report to a clinical audience.
+#'
 #' @param x A \code{\link{gg_partial_varpro}} object.
 #' @param type Character vector; one or more of \code{"parametric"},
 #'   \code{"nonparametric"}, \code{"causal"}.  Defaults to all three.

R/plot.gg_udependent.R

@@ -6,6 +6,34 @@
 #' set, and the width and opacity of an edge tell you how strong the
 #' dependency between its two variables is.
 #'
+#' @section Reading the network:
+#' Each node is a variable; each edge is a cross-variable dependency
+#' that cleared the threshold passed to \code{gg_udependent}. The
+#' Fruchterman-Reingold layout (the default) places mutually connected
+#' variables near each other, so the picture tends to show hubs and
+#' the clusters around them rather than a tidy ring. The eye usually
+#' goes first to the largest blue node — a variable that is both in
+#' the signal set and connects to many others is a hub of the
+#' dependency structure. Edges with wider, more opaque strokes are
+#' stronger dependencies; thin, faint edges sit near the threshold and
+#' are the ones that would disappear first if you raised it.
+#'
+#' Grey, low-degree nodes are the ones UVarPro thinks are not
+#' contributing much to the structure. (Truly isolated nodes are
+#' dropped by `gg_udependent()` before the graph is drawn — what you
+#' see is the connected component.) A cluster of mutually
+#' connected variables is worth checking for redundancy — they may be
+#' several views of the same underlying quantity.
+#'
+#' @section What this tells you:
+#' Use the figure to pick a working set of variables: the hubs and
+#' their immediate neighbours are the candidates UVarPro flags as
+#' carrying structure. If a cluster of high-degree variables looks
+#' like it might be measuring the same thing, that is a cue to look at
+#' their pairwise correlations or fit them as a block rather than
+#' individually. The threshold and layout are recorded in the caption
+#' so a different choice is easy to spot in a later figure.
+#'
 #' @param x A \code{gg_udependent} object from \code{\link{gg_udependent}}.
 #' @param layout Character; the igraph/ggraph layout algorithm.  Common
 #'   choices are \code{"fr"} (Fruchterman-Reingold, the default),