From 5e46e7d88951490aa76147cc21c71d96bde84a9b Mon Sep 17 00:00:00 2001 From: Chris Toney Date: Mon, 25 May 2026 19:32:27 -0600 Subject: [PATCH] add a section on using Pixi and other revisions Co-authored-by: Matthew Feickert --- vignettes/articles/r-and-gdal-in-conda.Rmd | 77 +++++++++++++++------- 1 file changed, 53 insertions(+), 24 deletions(-) diff --git a/vignettes/articles/r-and-gdal-in-conda.Rmd b/vignettes/articles/r-and-gdal-in-conda.Rmd index 3e1ef2921..ec9dc46c3 100644 --- a/vignettes/articles/r-and-gdal-in-conda.Rmd +++ b/vignettes/articles/r-and-gdal-in-conda.Rmd @@ -1,5 +1,5 @@ --- -title: "Using R and GDAL in conda" +title: "Using R and GDAL from conda-forge" output: html_document --- @@ -10,18 +10,18 @@ knitr::opts_chunk$set( ) ``` -DRAFT: last updated 2026-05-17 +DRAFT: last updated 2026-05-25 -Conda is an open-source, cross-platform, language-agnostic package manager and environment management system. It was originally developed to solve package management challenges faced by Python data scientists, and today is a popular package manager for Python and R [[Wikipedia](https://en.wikipedia.org/wiki/Conda_(package_manager))]. +Conda is an open-source, cross-platform, language-agnostic package manager and environment management system. It was originally developed to solve package management challenges faced by Python data scientists [[Wikipedia](https://en.wikipedia.org/wiki/Conda_(package_manager))]. Today, conda is a general purpose packaging format that covers a wide selection of programming languages, including C, C++, Fortran, Rust, Go, Python, and R. Conda uses the concept of channels as the base for hosting and managing packages. Conda is often associated with the commercial company Anaconda which provides the [Anaconda.org](https://anaconda.org/) channel of packages. Alternatively, the conda-forge project builds and distributes packages via [conda-forge.org](https://conda-forge.org/), specializing in hard-to-build or unique packages that often arise in a scientific computing context. Conda-forge is community-driven and community-curated. Both conda and conda-forge are [NumFocus-affiliated](https://numfocus.org/project/conda-forge) projects. Conda offers features potentially of interest to R users who work with geospatial data: -* install different versions of binary software packages and required dependencies on multiple supported platforms -* assemble sets of packages in isolation from the rest of the compute platform using environments +* install different versions of binary packages with their required dependencies on multiple platforms +* assemble sets of packages in isolation using environments * manage and switch between environments that use various packages for different projects -* access to current R or previous release version, a large number of CRAN packages, and current GDAL with dependencies +* access R and a large number of CRAN packages, along with current GDAL and its dependencies * benefit from lightweight [modular GDAL packaging](https://quansight.com/post/introducing-lightweight-versions-of-gdal-and-pdal/) based on deferred plugin loading * available plugins include format drivers that sometimes are unavailable in other package managers, such as the Arrow and Parquet vector drivers @@ -29,7 +29,9 @@ Conda-forge provides pre-compiled binary packages for **gdalraster** that track [![Conda Platforms](https://img.shields.io/conda/pn/conda-forge/r-gdalraster.svg)](https://anaconda.org/conda-forge/r-gdalraster) -## Installation +Several tools are available for managing packages in the [conda ecosystem](https://pixi.prefix.dev/latest/conda_ecosystem/). The next few sections describe `conda`, the original package manager written in Python. [Pixi](https://pixi.prefix.dev/latest/) might also be considered as a fast, modern, and reproducible package management tool. The section "Pixi for conda package management" further below describes a Pixi-equivalent setup. + +## Using conda from Miniforge Install Miniforge: https://conda-forge.org/download/ @@ -82,28 +84,30 @@ Note that conda-forge is the default channel in Miniforge, so it could be omitte R package names in conda-forge always have the `r-` prefix. When using R in `conda`, packages should be installed with the `conda` package manager rather than from source or binaries using `install.packages()` (i.e., should not mix packages from CRAN or other repositories such as R-universe). A web portal for exploring packages available in conda-forge can be found at: https://conda-forge.org/packages/. -## GDAL installation in conda +## Installing GDAL from conda-forge GDAL is available in conda-forge as a modular set of packages that take advantage of GDAL's support of deferred plugins for geospatial format drivers. This provides an efficient and streamlined approach to handling dependencies. The plugin system for `conda` was developed by [Quansight](https://quansight.com/) and [Hobu](https://hobu.co/), with more details at: https://quansight.com/post/introducing-lightweight-versions-of-gdal-and-pdal/ -The list of available packages is reproduced here. +The following packages are available from conda-forge. -GDAL conda-forge packages: +GDAL library packages: -* `libgdal-core`: core C++ library -* `libgdal`: core C++ library and all plugins except arrow/parquet -* `gdal`: python library without the plugins +* `libgdal-core`: core C++ library with a number of built-in drivers +* `gdal`: Python bindings and Python utilities +* `libgdal` meta package: core C++ library and all plugins except arrow/parquet -GDAL plugin conda-forge packages: +Driver plugins: * `libgdal-arrow-parquet`: `vector.arrow` and `vector.parquet` drivers as a plugin +* `libgdal-avif`: `raster.avif` driver as a plugin * `libgdal-fits`: `raster.fits` driver as a plugin * `libgdal-grib`: `raster.grib` driver as a plugin * `libgdal-hdf4`: `raster.hdf4` driver as a plugin * `libgdal-hdf5`: `raster.hdf5` driver as a plugin -* `libgdal-jp2openjpeg`:`raster.jp2openjpeg` driver as a plugin +* `libgdal-heif`: `raster.heif` driver as a plugin +* `libgdal-jp2openjpeg`: `raster.jp2openjpeg` driver as a plugin * `libgdal-kea`: `raster.kea` driver as a plugin * `libgdal-netcdf`: `raster.netcdf` driver as a plugin * `libgdal-pdf`: `raster.pdf` driver as a plugin @@ -112,13 +116,37 @@ GDAL plugin conda-forge packages: * `libgdal-tiledb`: `raster.tiledb` driver as a plugin * `libgdal-xls`: `vector.xls` driver as a plugin -The package `libgdal-core` is a dependency of `r-gdalraster` so installed automatically, but the additional driver plugins are optional. We'll install additional drivers before starting an R session in order to work with Parquet vector files. The following command installs `libgdal-arrow-parquet` in the active environment (assuming conda-forge is the default channel): +The package `libgdal-core` is a dependency of `r-gdalraster` so it is installed automatically, but the additional driver plugins are optional. We'll install additional drivers before starting an R session in order to work with Parquet vector files. The following command installs `libgdal-arrow-parquet` in the active environment (assuming conda-forge is the default channel): ```bash conda install libgdal-arrow-parquet ``` -## Working with gdalraster in conda +## Pixi for conda package management + +Pixi is an alternative to using `conda` as described above. It is a modern, Rust-based package manager with lockfiles, tasks, and multi-environment support. Pixi introduces a workspace-centric approach rather than focusing solely on environments. + +Installation instructions can be found in the Pixi [Getting Started](https://pixi.prefix.dev/latest/installation/) guide. Once installed you can create an environment in a pixi workspace, basically equivalent to the setup described above for conda: + +```bash +pixi init r_example +cd r_example +pixi add r-essentials r-gdalraster libgdal-arrow-parquet +pixel shell +``` + +The last command starts an interactive shell within the environment (use `exit` to leave the shell). You get a digest-level lock file automatically, so the environment is fully reproducible. + +Pixi also supports using tools and libraries globally, similar to conda's base environment without having to use an activate command: + +```bash +pixi global install gdal libgdal-core +pixi global install libgdal-arrow-parquet # if needed +``` + +As noted above for conda environments, R packages should be installed with `pixi add` in this case rather than from source or binaries using `install.packages()` (i.e., should not mix packages from CRAN or other repositories such as R-universe). + +## Example using gdalraster from conda-forge We can now start an R session in the active environment by typing R + \. @@ -130,7 +158,7 @@ The code below uses a sample GeoPackage file included with **gdalraster** contai library(gdalraster) #> GDAL 3.13.0 (released 2026-05-04), GEOS 3.14.1, PROJ 9.8.1 -# check that the Parquet driver is available +## verify the Parquet driver is available gdal_formats("Parquet") |> str() #> 'data.frame': 1 obs. of 17 variables: #> $ short_name : chr "Parquet" @@ -151,7 +179,7 @@ gdal_formats("Parquet") |> str() #> $ read_field_domains : logi FALSE #> $ creation_fld_dom_types : chr "" -# source GPKG with MTBS fire perimeters +## gdalraster example GeoPackage file with MTBS fire perimeters src <- system.file("extdata/ynp_fires_1984_2022.gpkg", package = "gdalraster") (src_lyr <- new(GDALVector, src, "mtbs_perims")) #> C++ object of class @@ -165,7 +193,7 @@ src <- system.file("extdata/ynp_fires_1984_2022.gpkg", package = "gdalraster") src_lyr$getFeatureCount() #> [1] 61 -# get info on GDAL algorithm usage +## get usage info for the algorithm gdal_usage("vector reproject") #> #> Usage: vector reproject [OPTIONS] @@ -243,13 +271,13 @@ gdal_usage("vector reproject") #> #> For more details: -# inverse project to WGS84 +## inverse project to WGS84 dst <- "/home/ctoney/data/ynp_fires_1984_2022.parquet" args <- list(input = src_lyr, output = dst, output_crs = "EPSG:4326") -# run the algorithm and close datasets when finished +## run the algorithm and close datasets when finished gdal_run("vector reproject", args, close = TRUE) #> ✔ Done (21ms) #> @@ -266,6 +294,7 @@ gdal_run("vector reproject", args, close = TRUE) #> • CRS: WGS 84 (EPSG:4326) #> • Geometry: MULTIPOLYGON +## verify the output dataset dst_lyr$info() #> INFO: Open of `/home/ctoney/data/ynp_fires_1984_2022.parquet' #> using driver `Parquet' successful. @@ -315,7 +344,7 @@ dst_lyr$info() #> ig_date: Date #> ig_year: Integer (0.0) -# check the linework +## check the linework features <- dst_lyr$fetch(-1) nrow(features) #> [1] 61 @@ -342,5 +371,5 @@ dst_lyr$close() * [R support in conda-forge](https://conda-forge.org/blog/2025/11/22/r-support/) * [r-gdalraster feedstock](https://github.com/conda-forge/r-gdalraster-feedstock) +* [GDAL Binaries: Cross-Platform Package Managers](https://gdal.org/en/stable/download.html#cross-platform-package-managers) * [Installing pip packages](https://www.anaconda.com/docs/getting-started/working-with-conda/packages/pip-install) -* [conda-forge status dashboard](https://conda-forge.org/status/)