7  Code that was not written for lazy arrays

The examples so far could all be rescued today by calling altarr_extract() in place of [. This chapter is about why that is not an answer.

The promise of a classless ALTREP array is that it is an array. Nobody has to write methods for it and nobody has to know it is lazy. Thirty years of R code takes arrays and indexes them with [. Every one of those functions works on a lazy array on stock R, correctly, and reads its data one element at a time. A special extract function helps only the code that calls it.

With the method in R, the planning happens underneath [, so it happens for all of them.

# 07-ordinary.R -- R code that was not written for lazy arrays. ASCII only.
#
# The same simulated store as 02-rectangles.R, with names on the time
# dimension. Every read below is plain base R.
here <- function() {
  f <- sub("^--file=", "", grep("^--file=", commandArgs(FALSE), value = TRUE))
  if (length(f)) dirname(f[1]) else "scripts"
}
source(file.path(here(), "_bench.R"))

x <- synthetic(c(1000, 800, 60), c(100, 100, 10),
               dimnames = list(NULL, NULL, paste0("t", 1:60)))

# negative, logical and character subscripts: R resolves them all
keep <- rep(c(TRUE, FALSE), 400)
bench("x[-(1:900), keep, \"t30\"]", x, x[-(1:900), keep, "t30"])

# a function written for ordinary arrays
time_mean <- function(a, i, j) apply(a[i, j, , drop = FALSE], c(1, 2), mean)
bench("apply() over x[101:150, 201:260, , drop = FALSE]", x,
      time_mean(x, 101:150, 201:260))

# a loop over time steps
bench("for each of 20 steps: sum(x[201:400, 301:500, k])", x,
      vapply(1:20, function(k) sum(x[201:400, 301:500, k]), numeric(1)))

# R's own wrapper object, made by changing attributes on a shared array
y <- x
dimnames(y) <- list(lon = NULL, lat = NULL, time = paste0("t", 1:60))
bench("after dimnames<- on a copy: y[1:250, 1:250, 1:15]", x, y[1:250, 1:250, 1:15])
finish()
2026-11-22 04:46 UTC on a5af98a4c685; R-devel patched r90638, stock r90638
Read Values Chunks Fetch calls, stock Fetch calls, patched Seconds, stock Seconds, patched Faster by Same values
x[-(1:900), keep, “t30”] 40,000 8 8 1 0.42 0.06 6.6x yes
apply() over x[101:150, 201:260, , drop = FALSE] 3,000 6 6 1 0.34 0.08 4.4x yes
for each of 20 steps: sum(x[201:400, 301:500, k]) 20 8 8 2 0.47 0.13 3.7x yes
after dimnames<- on a copy: y[1:250, 1:250, 1:15] 937,500 18 18 1 1.00 0.09 10.8x yes

Read by read:

Read Element calls, stock Element calls, patched
x[-(1:900), keep, “t30”] 40,000 0
apply() over x[101:150, 201:260, , drop = FALSE] 180,000 0
for each of 20 steps: sum(x[201:400, 301:500, k]) 800,000 0
after dimnames<- on a copy: y[1:250, 1:250, 1:15] 937,500 0

7.1 Where this reaches

The same thing holds further up the stack, in packages that sit on lazy arrays:

  • ndr (labelled n-dimensional arrays, in the manner of xarray) selects with do.call("[", ...). On stock R it has to detect a lazy array and call altarr_extract(). With the method it does not need to know.
  • GDAL7 presents GDAL multidimensional arrays and raster bands with as_altarr(). Given the whole request, a band indexed as b[rows, cols] can be one GDAL read of the window, with one AdviseRead ahead of it.
  • Any reader could do the same. The method is not specific to altarr: arrow, a NetCDF or HDF5 package, a database-backed matrix, a memory-mapped file class could each implement it.

7.2 Still open

  • Wrapper classes in packages. R’s own wrapper forwards the method. A package that defines its own ALTREP view over another object (rray 0.2.0 has one) cannot forward it, because the dispatch function is internal to R. Whether to export it is a question for R core. Not yet tested here.
  • Subassignment. x[i, j, k] <- value on a lazy array materializes it. A write-side method would be the companion to this one; it is not part of this proposal.
  • is.na() has no ALTREP hook, so mean(x, na.rm = TRUE) still walks elements. Subsetting first, then computing on the ordinary array that comes back, is the pattern that works, and it is the one this patch makes fast.