Renderers

A family draws frames either with its upstream library's code or with one of the shared renderers in stable_envs.render. Each class table on a family page names the renderer it uses on CPU and on CUDA. Each section below takes one shared renderer and gives the families that use it and what it promises about backends, exactness and cost. The Scene2D IR gives the tensor layout of a 2D scene, and The compiled variation covers StyleTensor, the form of a variation that renderers read.

Module Draws Used by Devices
scene, raster2d discs, convex polygons and capsules SimplePointMaze; the navigation dot renderers reuse its tiling Torch on CPU, Triton on CUDA
gfxdraw SDL_gfx primitives classic control Torch or Numba on CPU, Triton on CUDA
raycast3d MuJoCo primitive scenes OGBench mazes, fixed-camera DM Control tasks Torch and Numba on CPU, Torch and Triton on CUDA
cpu3d MuJoCo primitive scenes with a floor and a tracking camera DM Control body-tracking tasks on CPU Torch and Numba, CPU only
nvdiff textured, lit triangle meshes PyFlytRocketLanding CUDA only
mujoco_raster MuJoCo scenes, as MuJoCo's OpenGL renderer draws them OGBench Cube and Scene (backend="numba", the CPU default); Fetch render_semantic Numba, CPU only
mjwarp MuJoCo models DM Control body-tracking tasks on CUDA, Fetch on CUDA, OGBench Cube and Scene at supersample 2 or more CUDA only

The other MuJoCo paths use MuJoCo's OpenGL renderer. The benchmarks measure the throughput of the ray caster and MJWarp.

2D scenes and rasterizer

stable_envs.render.scene defines Scene2D, a batch of discs, convex polygons and capsules in draw order, and SceneBuilder, which appends them. A family that draws in 2D returns a Scene2D from Env.scene, and stable_envs.render.raster2d draws it into (B, H, W, 3) uint8 frames.

import torch
from stable_envs.render.raster2d import render
from stable_envs.render.scene import SceneBuilder

b = SceneBuilder(batch=2, device="cpu")
b.add_rect(center=(0.5, 0.5), size=(0.6, 0.2), color=(40, 90, 200))      # shared by both rows
b.add_disc(center=torch.tensor([[0.3, 0.5], [0.7, 0.5]]), radius=0.1,    # one center per row
           color=(220, 30, 30))
scene = b.build(background=(255, 255, 255), world=(0.0, 0.0, 1.0, 1.0))
print(scene.batch, scene.n_prims)  # 2 2
print(scene.kind.tolist())         # [[2, 1], [2, 1]]: a polygon, then a disc

frames = render(scene, out_size=64, supersample=2)
print(frames.shape, frames.dtype)                            # torch.Size([2, 64, 64, 3]) torch.uint8
print(frames[0, 32, 19].tolist(), frames[0, 2, 2].tolist())  # [220, 30, 30] [255, 255, 255]

Every SceneBuilder argument takes one value shared by the batch or one value per batch row. Coordinates are in the family's own world units, and world=(x0, y0, x1, y1) maps them onto the frame. Colors are RGB in 0 to 255. enabled turns a primitive off per row, and the scene keeps its shape when the number of visible objects changes. build raises ValueError when nothing was added. See The Scene2D IR for the tensor layout and the static-prefix cache that mark_static requests.

Primitives are composited in the order they were added. A static background is baked only when its geometry and appearance are the same for the whole batch. Output offsets are int64, which keeps the address calculation of large batches from overflowing. Outlines are unions of per-edge capsules with round corners, as in pygame's draw_fat_segment.

backend="auto" uses Triton when it's installed and the scene is on CUDA, and Torch otherwise. backend="triton" with a CPU scene raises ValueError. The Triton kernel computes each pixel's color in registers and writes only the result, while the Torch backend composites on a (B, 3, R, R) float canvas, one pass per primitive. Tests compare their pixels exactly. Both take out, a contiguous (B, H, W, 3) uint8 buffer that Env.render(out=...) passes through, and a training loop can reuse one buffer for every call. supersample=ss averages ss × ss subpixels per output pixel, which changes only edge pixels.

Building a Scene2D from state issues a few dozen small Torch operations from Python, and its cost changes little with batch size. The kernel is one launch that scales with output pixels. Below about a thousand environments, building the scene takes most of the time.

Batched 2-D scene description and the builder that fills it, consumed by raster2d.

Classes:

  • Scene2D –

    Batched 2-D primitives in draw order.

  • SceneBuilder –

    Append primitives in draw order and build a batched Scene2D.

Scene2D dataclass

Scene2D(kind: Tensor, geom: Tensor, fill: Tensor, line: Tensor, line_w: Tensor, background: Tensor, world: tuple[float, float, float, float] = (0.0, 0.0, 1.0, 1.0), flip_y: bool = False, static_prefix: int = 0)

Batched 2-D primitives in draw order.

world=(x0, y0, x1, y1) maps native coordinates onto the image and flip_y handles upward-positive axes. KIND_NONE disables a primitive per row.

Methods:

  • to –

    Move arrays and optionally convert floating dtype, preserving kind dtype.

  • validate –

    Validate shapes, int32 kinds, and positive world extents; return self.

Attributes:

  • n_prims (int) –

    Return primitive slots per environment, including disabled slots.

  • vertices (int) –

    Return polygon capacity in (x, y) pairs.

n_prims property

n_prims: int

Return primitive slots per environment, including disabled slots.

vertices property

vertices: int

Return polygon capacity in (x, y) pairs.

to

to(device: Any = None, dtype: Any = None) -> Scene2D

Move arrays and optionally convert floating dtype, preserving kind dtype.

validate

validate() -> Scene2D

Validate shapes, int32 kinds, and positive world extents; return self.

SceneBuilder

SceneBuilder(batch: int, device: Any = 'cpu', vertices: int = 4, dtype: dtype = float32, style: Any = None)

Append primitives in draw order and build a batched Scene2D.

A width-sized value is shared, (B, width) ((B, 1) for scalars) is per environment, and plural add methods also take (1, n, width) per-primitive values.

Methods:

  • add_disc –

    Add a filled disc with an optional concentric inner disc and independent color.

  • add_poly –

    Add a convex CCW polygon from (V, 2) or (B, V, 2) vertices.

  • add_segment –

    Add a capsule from endpoints and a half-width in world units.

  • add_rect –

    Add a rectangle with full width/height and optional rotation in radians.

  • add_segments –

    Add n capsules per row with endpoints (B, n, 2) and shared or per-primitive widths.

  • add_polys –

    Add convex CCW polygons (B, n, V, 2), rejecting excess vertices.

  • add_rects –

    Add rectangles with centers and full sizes (B, n, 2), optionally rotated per rectangle.

  • mark_static –

    Offer the primitives appended so far as a state-independent prefix for caching.

  • build –

    Build a validated nonempty scene.

add_disc

add_disc(center: Batched, radius: Batched, color: Batched, inner_radius: Batched = 0.0, inner_color: Batched | None = None, enabled: Batched | None = None) -> None

Add a filled disc with an optional concentric inner disc and independent color.

Parameters:

  • center (Batched) –

    Center (x, y) in world units.

  • radius (Batched) –

    Radius in world units.

  • color (Batched) –

    RGB in 0 to 255.

  • inner_radius (Batched, default: 0.0 ) –

    Radius of the inner disc (none if 0).

  • inner_color (Batched | None, default: None ) –

    RGB of the inner disc.

  • enabled (Batched | None, default: None ) –

    Whether each row draws the disc (all rows if None).

add_poly

add_poly(verts: Batched, color: Batched, outline_color: Batched | None = None, outline_w: Batched = 0.0, enabled: Batched | None = None) -> None

Add a convex CCW polygon from (V, 2) or (B, V, 2) vertices.

Repeat the last vertex when padding; exceeding vertex capacity raises ValueError.

Parameters:

  • verts (Batched) –

    Vertices in world units, counterclockwise.

  • color (Batched) –

    Fill RGB in 0 to 255.

  • outline_color (Batched | None, default: None ) –

    Outline RGB (no outline if None).

  • outline_w (Batched, default: 0.0 ) –

    Outline width in world units.

  • enabled (Batched | None, default: None ) –

    Whether each row draws the polygon (all rows if None).

add_segment

add_segment(a: Batched, b: Batched, half_w: Batched, color: Batched, enabled: Batched | None = None) -> None

Add a capsule from endpoints and a half-width in world units.

Parameters:

  • a (Batched) –

    First endpoint (x, y).

  • b (Batched) –

    Second endpoint (x, y).

  • half_w (Batched) –

    Half the capsule width.

  • color (Batched) –

    RGB in 0 to 255.

  • enabled (Batched | None, default: None ) –

    Whether each row draws the capsule (all rows if None).

add_rect

add_rect(center: Batched, size: Batched, color: Batched, angle: Batched | None = None, outline_color: Batched | None = None, outline_w: Batched = 0.0, enabled: Batched | None = None) -> None

Add a rectangle with full width/height and optional rotation in radians.

Parameters:

  • center (Batched) –

    Center (x, y) in world units.

  • size (Batched) –

    Full (width, height).

  • color (Batched) –

    Fill RGB in 0 to 255.

  • angle (Batched | None, default: None ) –

    Rotation in radians (none if None).

  • outline_color (Batched | None, default: None ) –

    Outline RGB (no outline if None).

  • outline_w (Batched, default: 0.0 ) –

    Outline width in world units.

  • enabled (Batched | None, default: None ) –

    Whether each row draws the rectangle (all rows if None).

add_segments

add_segments(a: Tensor, b: Tensor, half_w: Batched, color: Batched, enabled: Batched | None = None) -> None

Add n capsules per row with endpoints (B, n, 2) and shared or per-primitive widths.

add_polys

add_polys(verts: Tensor, color: Batched, outline_color: Batched | None = None, outline_w: Batched = 0.0, enabled: Batched | None = None) -> None

Add convex CCW polygons (B, n, V, 2), rejecting excess vertices.

add_rects

add_rects(center: Tensor, size: Tensor, color: Batched, angle: Batched | None = None, outline_color: Batched | None = None, outline_w: Batched = 0.0, enabled: Batched | None = None) -> None

Add rectangles with centers and full sizes (B, n, 2), optionally rotated per rectangle.

mark_static

mark_static() -> SceneBuilder

Offer the primitives appended so far as a state-independent prefix for caching.

The rasterizer bakes the prefix only if its values are uniform across the batch.

build

build(background: Batched, world: Sequence[float], flip_y: bool = False) -> Scene2D

Build a validated nonempty scene.

Parameters:

  • background (Batched) –

    Background RGB in 0 to 255.

  • world (Sequence[float]) –

    (x0, y0, x1, y1), the world rectangle the frame shows.

  • flip_y (bool, default: False ) –

    Set when the world's +y points up, since the image's +y points down.

Rasterize Scene2D discs, convex polygons, and capsules with Torch or a fused Triton kernel.

Functions:

  • render –

    Render uint8 RGB frames with Torch or Triton on the scene's device.

  • render_torch –

    Rasterize a Scene2D into (B, H, W, 3) uint8 frames on the scene's device.

  • render_triton –

    Rasterize a CUDA Scene2D into (B, H, W, 3) uint8 frames.

  • row_groups –

    Partition (B, 4) pixel boxes into row groups with smaller union windows.

  • tile_shape –

    Choose a (height, width) tile within the instances and max_width budgets.

render

render(scene: Scene2D, out_size: int | tuple[int, int], supersample: int = 1, backend: str = 'auto', out: Tensor | None = None) -> Tensor

Render uint8 RGB frames with Torch or Triton on the scene's device.

Parameters:

  • scene (Scene2D) –

    The scene to draw.

  • out_size (int | tuple[int, int]) –

    Output size, an int or (height, width).

  • supersample (int, default: 1 ) –

    Subpixels per output pixel along each axis.

  • backend (str, default: 'auto' ) –

    "auto" (Triton for CUDA scenes when it's installed), "triton" or "torch".

  • out (Tensor | None, default: None ) –

    A contiguous (B, H, W, 3) uint8 buffer to fill.

Returns:

  • Tensor –

    (B, H, W, 3) uint8 frames.

render_torch

render_torch(scene: Scene2D, out_size: int | tuple[int, int], supersample: int = 1) -> Tensor

Rasterize a Scene2D into (B, H, W, 3) uint8 frames on the scene's device.

render_triton

render_triton(scene: Scene2D, out_size: int | tuple[int, int], supersample: int = 1, out: Tensor | None = None, block: int | None = None, cull: bool | None = None) -> Tensor

Rasterize a CUDA Scene2D into (B, H, W, 3) uint8 frames.

block defaults to the raster2d_block setting and cull to a choice by primitive count. out must be contiguous uint8 storage with exactly B*H*W*3 elements.

row_groups

row_groups(box, r0: int, r1: int, c0: int, c1: int) -> list[tuple]

Partition (B, 4) pixel boxes into row groups with smaller union windows.

Returns (row_indices, r0, r1, c0, c1) tuples; row_indices=None means the whole batch.

tile_shape

tile_shape(H: int, W: int, instances: int, max_width: int) -> tuple[int, int]

Choose a (height, width) tile within the instances and max_width budgets.

The Scene2D IR

Scene2D is the interface between a 2D environment and the rasterizer. The environment turns (state, variation) into a Scene2D, and one kernel turns that into pixels. The kernel sees only the scene, never an environment handle, simulator or Python callback.

from stable_envs.render import raster2d
from stable_envs.render.scene import SceneBuilder

agent = torch.tensor([[256.0, 256.0], [100.0, 400.0]])  # one position per batch row

sb = SceneBuilder(batch=2, device="cpu", vertices=4)
sb.add_segment((5, 506), (5, 5), 2.5, (0, 0, 0))  # a wall, shared by the batch
sb.mark_static()                                  # request a bake of everything above
sb.add_disc(agent, 15.0, (65, 105, 225))          # (B, 2) centers, one per row
scene = sb.build(background=(255, 255, 255), world=(0, 0, 512, 512))

frames = raster2d.render_torch(scene, 96)
print(frames.shape, frames.dtype)  # torch.Size([2, 96, 96, 3]) torch.uint8

The IR has three primitive kinds, discs, convex polygons and capsules, composited back to front with flat colors and packed into fixed-width tensors:

Field Shape Meaning
kind (B, N) int32 0 disabled, 1 disc, 2 convex polygon, 3 capsule
geom (B, N, 2V) f32 disc: [cx, cy, r, inner_r, ...]. Polygon: [x0, y0, ... xV-1, yV-1] CCW. Capsule: [x0, y0, x1, y1, half_w, ...]
fill (B, N, 3) f32 interior color, 0–255
line (B, N, 3) f32 outline color, also the disc's inner color
line_w (B, N) f32 outline half-width in world units, where 0 disables it
background (B, 3) f32 per-environment background color
world 4 floats (x0, y0, x1, y1), the extent of the primitives
flip_y bool set when the environment's +y points up
static_prefix int number of leading primitives the environment asks to bake
  • kind == 0 disables a slot. A batch is one rectangular tensor: a scene whose primitive count depends on the state disables slots instead of changing shape, which allows CUDA graph capture and compile-time unrolling.
  • Draw order is index order, back to front, with no depth buffer.
  • Padding repeats the last vertex of a polygon. That makes a zero-length edge, which both backends drop from the signed distance (edges with L2 < 1e-12), and the outline stays correct.

Each environment keeps its own units, such as PushT's 512-unit board or CartPole's meters, and the rasterizer maps world onto the output image. Set flip_y instead of flipping coordinates in environment code. add_disc takes an optional inner_radius and inner_color, add_poly takes (B, V, 2) world vertices with an optional outline_color and outline_w, and add_rect wraps add_poly with an optional rotation. build() runs Scene2D.validate() once, which checks each tensor's shape, the int32 dtype of kind, and that world isn't degenerate.

Static prefix

SceneBuilder.mark_static() asks the rasterizer to draw the primitives added so far once into a cached base image that later frames start from. The whole batch shares the base image. It bakes only if the prefix is identical across batch rows, which one kernel checks against row 0, and otherwise draws every primitive per frame. The output is the same either way.

The cache key holds the exact bytes of the static primitives and the background color, the supersampled resolution, world, flip_y, the vertex count, the prefix length and the device. A variation that recolors a static primitive or the background therefore invalidates the bake. The rasterizer skips the bake when K/N × B × H × W (K static primitives out of N, at the supersampled resolution) is below raster2d._BAKE_MIN_WORK = 8e6 evaluations, and then no comparison or transfer happens. An environment that passes its StyleTensor to the builder (SceneBuilder(B, ..., style=sty)) also skips the comparison when the style differs per batch row.

Baking saves more work as the batch and the resolution grow. Below the work threshold the rasterizer does not bake, because the bake would cost more than it saves. The public PushT classes do not use this cache: they draw with their own Numba (CPU) and Triton (CUDA) rasterizers.

gfxdraw

stable_envs.render.gfxdraw is a batched Torch version of the SDL_gfx primitives behind pygame.gfxdraw, and classic control frames match Gymnasium's pygame drawing. On CUDA the Triton kernels in stable_envs.render.gfxdraw.fused draw, and on CPU a Numba twin in stable_envs.render.cpu_raster.classic draws when Numba is installed (classic extra), with the Torch functions as the fallback. With backend="auto", some CPU batches with the default appearance use Gymnasium's own renderer instead, and every choice gives the same frames.

It's separate from raster2d: SDL_gfx works in integers and takes Sint16 vertices. The module computes geometry in float64 and truncates every coordinate before drawing. Scanline fills, circle spans, clipping and packed-channel blending keep SDL's integer operation order, and the CUDA path repeats it. The conformance comparison found 0 differing pixels out of 250.9 M.

Batched SDL_gfx-style integer rasterization with Torch.

Compute geometry in float64 before truncating it, because float32 rounding can move a whole pixel row.

Classes:

  • Gfx –

    Cache drawing constants for one canvas size and device.

  • StyleCanvas –

    Canvas with per-environment color, antialiasing and visibility gates.

Functions:

  • polygon –

    Draw a convex polygon with an antialiased fringe and integer scanline fill.

  • circle –

    Draw ordered antialiased fringe passes, then the filled circle.

  • aaline –

    Clip and draw an antialiased edge with a solid initial pixel and no final endpoint.

  • fill_polygon_mask –

    Return a (B, H, W) scanline mask for convex integer polygons; see polygon_spans.

  • fill_circle_mask –

    Return a (B, H, W) circle mask for integer (B,) centers and a cached span table.

  • blend –

    Apply SDL_gfx packed-channel alpha blending with unsigned 32-bit wrapping.

  • blit_over –

    Blend a per-pixel-alpha surface onto opaque RGB using SDL_BlitSurface arithmetic.

  • rotate –

    Apply pygame's Vector2.rotate_rad arithmetic before the canvas's final vertical flip.

  • to_u8 –

    Truncate style colors to uint8 using the reference's integer conversion rule.

Gfx

Gfx(H: int, W: int, device)

Cache drawing constants for one canvas size and device.

Warm it up before CUDA graph capture, because creating its tensors during capture is invalid.

Methods:

  • disc –

    Return cached (S, 2) int64 filled-disc offsets for radius r.

  • verts –

    Cache float64 vertices so coordinate truncation retains reference precision.

  • color_u8 –

    Return a cached (3,) uint8 color for opaque fills.

disc

disc(r: int) -> Tensor

Return cached (S, 2) int64 filled-disc offsets for radius r.

verts

verts(vals) -> Tensor

Cache float64 vertices so coordinate truncation retains reference precision.

color_u8

color_u8(c) -> Tensor

Return a cached (3,) uint8 color for opaque fills.

StyleCanvas

StyleCanvas(gfx: Gfx, B: int, bg)

Bases: Canvas

Canvas with per-environment color, antialiasing and visibility gates.

use selects (B, 3) colors and gated restricts the primitives that follow.

Methods:

  • use –

    Set (B, 3) colors for subsequent primitives and return this canvas.

  • gated –

    Set a (B,) boolean visibility gate for subsequent primitives and return this canvas.

  • paint –

    Fill a window-sized mask using per-environment colors and the current visibility gate.

  • paint_spans –

    Fill the inclusive (B, rows) column spans of the window's rows with per-environment colors.

use

use(color) -> StyleCanvas

Set (B, 3) colors for subsequent primitives and return this canvas.

gated

gated(gate) -> StyleCanvas

Set a (B,) boolean visibility gate for subsequent primitives and return this canvas.

paint

paint(mask, color=None, window=None) -> None

Fill a window-sized mask using per-environment colors and the current visibility gate.

paint_spans

paint_spans(xa, xb, window=None, color=None) -> None

Fill the inclusive (B, rows) column spans of the window's rows with per-environment colors.

Only the span pixels are written, so a thin polygon costs its area rather than its window.

polygon

polygon(cv: Canvas, verts, color, color_t, bidx, kmax, aa: bool = True, fill_first: bool = False, window=None) -> None

Draw a convex polygon with an antialiased fringe and integer scanline fill.

verts should be float64 and window must contain the polygon footprint. fill_first=True fills before the fringe, which changes edge blending.

circle

circle(cv: Canvas, cx, cy, r, color, color_t, bidx, hw_t, stencil, aa: bool = True, window=None, disc=None, instances=None) -> None

Draw ordered antialiased fringe passes, then the filled circle.

disc is a filled-disc stencil (hw_t and window are used otherwise). With instances, the centers belong to those environments only.

aaline

aaline(cv: Canvas, x1, y1, x2, y2, color_t, kmax, bidx) -> None

Clip and draw an antialiased edge with a solid initial pixel and no final endpoint.

fill_polygon_mask

fill_polygon_mask(vx: Tensor, vy: Tensor, ys: Tensor, xs: Tensor, W: int) -> Tensor

Return a (B, H, W) scanline mask for convex integer polygons; see polygon_spans.

xs selects columns.

fill_circle_mask

fill_circle_mask(cx: Tensor, cy: Tensor, r: int, ys: Tensor, xs: Tensor, hw_t: Tensor) -> Tensor

Return a (B, H, W) circle mask for integer (B,) centers and a cached span table.

blend

blend(src: Tensor, dst: Tensor, w: Tensor, sh: Tensor) -> Tensor

Apply SDL_gfx packed-channel alpha blending with unsigned 32-bit wrapping.

Weight 256 is an opaque write; weight 255 still blends.

blit_over

blit_over(dst: Tensor, src_rgb: Tensor, alpha: Tensor) -> Tensor

Blend a per-pixel-alpha surface onto opaque RGB using SDL_BlitSurface arithmetic.

This uses alpha directly; SDL_gfx's packed-channel pixel blend uses another rule.

rotate

rotate(v: Tensor, a: Tensor) -> Tensor

Apply pygame's Vector2.rotate_rad arithmetic before the canvas's final vertical flip.

to_u8

to_u8(color: Tensor) -> Tensor

Truncate style colors to uint8 using the reference's integer conversion rule.

Ray casting and CPU3D

stable_envs.render.raycast3d intersects rays with spheres, capsules, cylinders, ellipsoids, boxes and finite planes. stable_envs.render.cpu3d uses the same intersections to draw a whole primitive scene on CPU when a moving camera rules out a static background, adding sky and floor textures, directional shadows and specular light. Its constructor rejects unsupported geometry and shadow-floor setups.

Fixed-camera rendering bakes a static RGB and depth plate, casts the moving primitives inside conservative screen bounds, and shades only the covered samples. Three MuJoCo conventions decide silhouette and highlight pixels: the mono camera averages its two stereo eyes, the depth buffer measures perpendicular distance while the compositor compares radial ray parameters, and specular light is added before texture modulation.

backend="auto" picks Triton for packed float32 inputs on CUDA and Torch otherwise. An explicit "triton" needs CUDA float32 inputs on one device with packed trailing axes and raises ValueError otherwise. The Torch cast takes bounds=False to cast every pixel, for diagnostics. In tests, override the backend in a scoped fixture rather than with a process-wide switch. On CPU, DM Control uses Numba kernels when Numba imports: cpu_raster.rays3d repeats the float32 operation order of the Torch intersectors and tests each tile of rays only against primitives its rays can reach, cpu_raster.suite3d composites the fixed-camera plate, and cpu_raster.scene3d draws a CPU3D scene. They use the IEEE square root, while Torch's CPU square root can round through MKL. Rare pixels differ from the Torch backend as a result.

Tests require exact agreement between the optimized compositor and its full-window reference, between the Torch and Triton casts on float32 inputs, and between bounded and unbounded Torch casts. Agreement with the upstream library's OpenGL frames is measured per class.

Batched analytic rendering of MuJoCo primitive geometry.

Torch and Triton share intersection and shading conventions. Moving primitives are composited over a fixed RGB/depth plate; compact windows avoid casting the static background again. Float operation order is part of the pixel contract.

Classes:

  • Camera –

    Pinhole camera with the near-plane frustum reported by mjvGLCamera.

  • Lights –

    Directional lights with unit directions toward each light and scalar intensity weights.

Functions:

  • intersect –

    Return ray parameters and world-space normals, with t=INF on misses.

  • cast_nearest –

    Return (ray parameter, world normal, primitive index); see cast_nearest_torch.

  • cast_nearest_torch –

    Return nearest ray parameters (B, N), normals (B, N, 3) and indices (B, N).

  • any_blocking –

    Return a (B, N) mask of hits before lim, excluding each ray's optional skip index.

  • any_blocking_torch –

    Return a (B, N) mask of hits with 0 < t < lim, excluding each ray's skip index.

  • cached_rays –

    Return cached camera rays, keyed by camera, resolution, device, and dtype.

  • shade –

    Return RGB (..., 3) in [0, 1]; white specular is added before texture modulation.

  • static_depth –

    Return nearest static-surface ray parameters (H*W,), in radial units rather than MuJoCo depth.

  • composite –

    Draw moving primitives over a static plate and return (B, H, W, 3) uint8 frames.

Camera dataclass

Camera(pos: tuple, forward: tuple, up: tuple, near: float, top: float, bottom: float, center: float = 0.0)

Pinhole camera with the near-plane frustum reported by mjvGLCamera.

Keeping the frustum supports off-center and stereo cameras.

Methods:

  • from_mjv –

    Build a mono camera from one eye or the average of the two stereo eyes.

  • rays –

    Return origin (3,) and unit directions (H*W, 3) through pixel centers, top row first.

  • project –

    Project world points (..., 3) to normalized device coordinates (..., 2).

from_mjv staticmethod

from_mjv(glcam_a, glcam_b=None) -> Camera

Build a mono camera from one eye or the average of the two stereo eyes.

rays

rays(H: int, W: int, device, dtype=float32) -> tuple[Tensor, Tensor]

Return origin (3,) and unit directions (H*W, 3) through pixel centers, top row first.

project

project(pts: Tensor, aspect: float = 1.0) -> Tensor

Project world points (..., 3) to normalized device coordinates (..., 2).

aspect is W/H. The result is used to choose conservative screen windows.

Lights dataclass

Lights(dirs: ndarray, ambient: ndarray, diffuse: ndarray, specular: ndarray, exponent: float)

Directional lights with unit directions toward each light and scalar intensity weights.

intersect

intersect(kind: int, size: Tensor, pos: Tensor, R: Tensor, o: Tensor, d: Tensor) -> tuple[Tensor, Tensor]

Return ray parameters and world-space normals, with t=INF on misses.

Poses (..., 3) and rotations (..., 3, 3) broadcast against rays (..., 3). Unsupported MuJoCo geometry kinds raise ValueError.

cast_nearest

cast_nearest(kinds: Kinds, sizes: Tensor, pos: Tensor, R: Tensor, o: Tensor, d: Tensor, backend: str = 'auto') -> Cast

Return (ray parameter, world normal, primitive index); see cast_nearest_torch.

auto uses Triton for packed float32 inputs on CUDA, otherwise Torch. An unknown backend or unsupported explicit triton request raises ValueError.

cast_nearest_torch

cast_nearest_torch(kinds: Kinds, sizes: Tensor, pos: Tensor, R: Tensor, o: Tensor, d: Tensor, *, bounds: bool = True) -> Cast

Return nearest ray parameters (B, N), normals (B, N, 3) and indices (B, N).

kinds holds K host codes; sizes and positions are (B, K, 3), rotations (B, K, 3, 3) and rays (B, N, 3). Misses have t=INF.

any_blocking

any_blocking(kinds: Kinds, sizes: Tensor, pos: Tensor, R: Tensor, o: Tensor, d: Tensor, lim: Tensor, skip: Tensor | None = None, backend: str = 'auto') -> Tensor

Return a (B, N) mask of hits before lim, excluding each ray's optional skip index.

Backend selection and errors follow cast_nearest.

any_blocking_torch

any_blocking_torch(kinds: Kinds, sizes: Tensor, pos: Tensor, R: Tensor, o: Tensor, d: Tensor, lim: Tensor, skip: Tensor | None = None, *, bounds: bool = True) -> Tensor

Return a (B, N) mask of hits with 0 < t < lim, excluding each ray's skip index.

Inputs follow cast_nearest_torch; lim and skip are (B, N).

cached_rays

cached_rays(cam: Camera, H: int, W: int, device, dtype=float32) -> tuple[Tensor, Tensor]

Return cached camera rays, keyed by camera, resolution, device, and dtype.

shade

shade(n: Tensor, v: Tensor, ldir: Tensor, lamb: Tensor, ldif: Tensor, lspec: Tensor, exponent: float, mat: Tensor, tex: Tensor, spec: Tensor) -> Tensor

Return RGB (..., 3) in [0, 1]; white specular is added before texture modulation.

static_depth

static_depth(cam: Camera, prims: Primitives, xpos, xmat, H: int, W: int, device) -> Tensor

Return nearest static-surface ray parameters (H*W,), in radial units rather than MuJoCo depth.

composite

composite(plate: Tensor, plate_t: Tensor, cam: Camera, prims: Primitives, xpos: Tensor, xmat: Tensor, lights: Lights, H: int, W: int, supersample: int = 1, out: Tensor | None = None, tile: int | None = None, mat_rgb: Tensor | None = None, size_scale: Tensor | None = None, shader=None, cast: Sequence[int] | None = None) -> Tensor

Draw moving primitives over a static plate and return (B, H, W, 3) uint8 frames.

plate is (H, W, 3) or (B, H, W, 3) uint8 and plate_t its (H*W,) radial depth. Poses are (B, P, 3) positions and (B, P, 3, 3) rotations; mat_rgb (B, P, 3) and size_scale (B,) override appearance per environment. shader(rows, hit, normal, view, prim, mat) replaces lights and returns colors in [0, 255]. cast limits the cast to these primitive indices, and the others must not cover any sample these can cover.

nvdiffrast

stable_envs.render.nvdiff rasterizes textured, lit triangle meshes on CUDA with nvdiffrast and returns uint8 frames. The PyFlyt rocket reuses its scene and shading code with its own rasterizer, which follows Bullet's triangle coverage rules. DM Control, Fetch and CPU3D borrow its material and texture helpers, effective_rgba and texture_array, which run without nvdiffrast. nvdiffrast is installed from source.

Projection uses a fixed arithmetic order, and every batch size takes the same matrix-multiply rounding path. Texture origin, row direction and byte rounding are explicit. The backend copies MuJoCo view arrays before later simulator calls can change them. Mipmapping and optional anisotropic filtering control texture footprints, with one tap by default, and antialiasing and supersampling control edge coverage. Each setting has its own cost and needs its own source-image check. The output isn't bit-exact with upstream OpenGL or TinyRenderer images. Exact tests check only that frames stay the same across batch sizes. On x86 CPU, nvdiffrast has a one-ULP batch dependence in a sparse stage, and no registered class uses that path on CPU.

Batched nvdiffrast rendering of MuJoCo geometry with MuJoCo-style lighting, textures, and skybox.

Classes:

  • Scene –

    Static triangle geometry, appearance and textures on one device.

  • BatchRenderer –

    Render geom transforms to (B, H, W, 3) uint8 frames with nvdiffrast on CUDA.

  • Camera –

    Pinhole camera looking along local -z, with matrix columns right, up and backward.

  • Lights –

    Resolved mjvScene lights, including the camera headlight; directions point away from the lights.

  • TinyShading –

    Bullet TinyRenderer lighting parameters for a z-up world; dir points toward the light.

Functions:

  • effective_rgba –

    Resolve geom RGBA, using its material only when geom RGBA matches the compiler default.

  • texture_array –

    Read texture tex of a compiled model as an (H, W, 3) uint8 array.

  • close_cached_renderer –

    Close the renderer held by a family cache entry before eviction.

Scene

Scene(m, groups=DEFAULT_GROUPS, device='cuda', nslice: int = NUMSLICES, nstack: int = NUMSTACKS, skip_alpha_zero: bool = True, tex_policy: str = 'pot')

Bases: Closeable

Static triangle geometry, appearance and textures on one device.

A renderer given this Scene borrows it by default, so close the Scene only after every borrower is done.

Methods:

  • from_triangles –

    Build a Scene from triangles in each geom's local frame.

  • clip –

    Return clip vertices (B, V, 4), world normals (B, V, 3), and world positions.

from_triangles classmethod

from_triangles(vert, norm, face, vert_geom, rgba, *, uv=None, specular=None, shininess=None, emission=None, textures=None, tex_of_geom=None, device='cuda', tex_policy: str = 'pot') -> Scene

Build a Scene from triangles in each geom's local frame.

Inputs are vert/norm (V, 3), face (F, 3), vert_geom (V,) and rgba (G, 4). Every face lies in one geom, tex_of_geom is -1 for untextured geoms, and render transforms use the same geom order.

clip

clip(xpos, xmat, view, proj)

Return clip vertices (B, V, 4), world normals (B, V, 3), and world positions.

BatchRenderer

BatchRenderer(m=None, res: int = 224, groups=DEFAULT_GROUPS, device: str = 'cuda', textures: bool = True, mipmap: bool = True, skybox: bool = True, quant: str = 'round', nslice: int = NUMSLICES, nstack: int = NUMSTACKS, supersample: int = 1, tex_policy: str = 'pot', aniso: int = 1, *, scene: Scene | None = None, shading: str = 'mujoco', tiny: TinyShading | None = None, tex_filter: str = 'linear', sample_offset=(0.0, 0.0))

Bases: Closeable

Render geom transforms to (B, H, W, 3) uint8 frames with nvdiffrast on CUDA.

Close it or use it as a context manager. A scene= argument is borrowed unless owns_scene=True.

shading="tiny" uses TinyShading parameters and ignores scene lights. sample_offset shifts pixel coverage; tex_filter selects linear or nearest sampling. Texture mipmapping and anisotropy are configured separately.

Methods:

  • set_skybox –

    Prepare the model's skybox texture when present.

  • resolve_from_mujoco –

    Resolve lights and camera through mjv_updateScene, including runtime defaults and the headlight.

  • render –

    Render device-resident xpos (B, G, 3) and xmat (B, G, 3, 3) to uint8 RGB.

set_skybox

set_skybox(m) -> None

Prepare the model's skybox texture when present.

resolve_from_mujoco

resolve_from_mujoco(m, d, camera=None, option=None)

Resolve lights and camera through mjv_updateScene, including runtime defaults and the headlight.

render

render(xpos, xmat, cam: Camera, out=None, mat_rgb=None)

Render device-resident xpos (B, G, 3) and xmat (B, G, 3, 3) to uint8 RGB.

G counts drawn geoms only, so select full-model arrays with scene.mjgeom first. mat_rgb (B, G, 3) overrides material colors for this call.

Camera dataclass

Camera(pos: ndarray, mat: ndarray, fovy: float, znear: float, zfar: float)

Pinhole camera looking along local -z, with matrix columns right, up and backward.

Position and orientation may carry a leading batch dimension.

Methods:

  • from_glcam –

    Build a mono camera by averaging the two resolved eyes, canceling their IPD offsets.

  • matrices –

    Return OpenGL view (..., 4, 4) and projection (4, 4) matrices.

from_glcam staticmethod

from_glcam(m, scn) -> Camera

Build a mono camera by averaging the two resolved eyes, canceling their IPD offsets.

matrices

matrices(aspect: float = 1.0)

Return OpenGL view (..., 4, 4) and projection (4, 4) matrices.

Lights dataclass

Lights(pos: ndarray, dir: ndarray, directional: ndarray, ambient: ndarray, diffuse: ndarray, specular: ndarray, attenuation: ndarray, cutoff: ndarray, exponent: ndarray, headlight: ndarray)

Resolved mjvScene lights, including the camera headlight; directions point away from the lights.

TinyShading dataclass

TinyShading(dir: tuple = (-50.0, 30.0, 100.0), ambient: float = 0.6, diffuse: float = 0.35, specular: float = 0.05, exponent: float = 2.0, color: tuple = (1.0, 1.0, 1.0))

Bullet TinyRenderer lighting parameters for a z-up world; dir points toward the light.

effective_rgba

effective_rgba(m, g: int) -> ndarray

Resolve geom RGBA, using its material only when geom RGBA matches the compiler default.

texture_array

texture_array(m, tex: int) -> ndarray

Read texture tex of a compiled model as an (H, W, 3) uint8 array.

close_cached_renderer

close_cached_renderer(c) -> None

Close the renderer held by a family cache entry before eviction.

MJWarp

stable_envs.render.mjwarp renders a batch of MuJoCo configurations on CUDA with MuJoCo Warp, each family with its own model preparation, static background and material handling. Model preparation selects the visible geom groups, moves invisible geometry out of them, sets shadow casting and records texture mappings MuJoCo Warp can't sample. The backend converts MuJoCo Warp's packed BGRA output to RGB, and tracking cameras need center-of-mass data.

  • GL-compatible rounding must be active when the backend captures the render graph. A captured graph keeps the pack function it was captured with.
  • Building a render context patches MuJoCo Warp's texture-upload and byte-packing helpers for the duration of the call. These patches aren't thread-safe: build contexts only while no other thread uses those helpers.
  • Importing the module sets WARP_CACHE_PATH to ~/.cache/warp when it's unset.
  • The light specialization applies only to the MuJoCo Warp version and model it supports, and other configurations use stock MuJoCo Warp.
  • Exact tests compare the backend with itself across changes, and texture filtering, coverage and shading leave a per-class MSE against the upstream library's OpenGL frames.

See OGBench for how Cube and Scene choose between the MuJoCo rasterizer, OpenGL and MJWarp.

MJWarp model preparation, texture uploads and batched CUDA rendering.

Importing it sets WARP_CACHE_PATH to ~/.cache/warp unless you've set it.

Classes:

  • BatchRenderer –

    Render (B, nq) joint positions to (B, H, W, 3) uint8 RGB on CUDA.

  • Prepared –

    Model edits and saved arrays needed to undo them.

Functions:

  • prepare_model –

    Prepare a model for MJWarp in place and return restoration data.

  • create_render_context –

    Create an MJWarp context with one upload of the model's texture blob.

  • release_render_context –

    Drop texture tensor keepalives attached to an MJWarp render context; repeated calls are safe.

  • release_cached_ctx –

    Drop texture keepalives from a cached family context before eviction.

  • gl_rounding –

    Temporarily round RGB channels before MJWarp packs them to uint8.

  • needs_com_pos –

    Return whether any camera or light tracks or targets a subtree center of mass.

BatchRenderer

BatchRenderer(m, batch: int, res: int = 224, camera: int = 0, groups=DEFAULT_GROUPS, textures: bool = True, shadows: bool = True, skybox: bool = True, prepare: bool = True, capture: bool = True)

Bases: Closeable

Render (B, nq) joint positions to (B, H, W, 3) uint8 RGB on CUDA.

Close it or use it as a context manager. With prepare=True the caller's MuJoCo model stays modified; restore_model with prepared undoes that.

Methods:

  • render –

    Render device-resident (B, nq) positions and return uint8 RGB frames.

render

render(qpos)

Render device-resident (B, nq) positions and return uint8 RGB frames.

Prepared dataclass

Prepared(hidden_geoms: list[int], hidden_sites: list[int], groups: tuple[int, ...], restore: dict, neutralised_spots: list[int] = list())

Model edits and saved arrays needed to undo them.

Methods:

  • summary –

    Describe the hidden elements and adjusted spot lights in one line.

summary

summary() -> str

Describe the hidden elements and adjusted spot lights in one line.

prepare_model

prepare_model(m, groups=DEFAULT_GROUPS, alpha_eps: float = 0.0, spot_semantics: bool = True) -> Prepared

Prepare a model for MJWarp in place and return restoration data.

Geoms and sites with alpha at most alpha_eps move to HIDDEN_GROUP, which the caller must not render.

create_render_context

create_render_context(m, tex_lod: int = 1, **kw)

Create an MJWarp context with one upload of the model's texture blob.

tex_lod > 1 area-downsamples textures before upload, which is a visual approximation. Don't create contexts from several threads at once, because this patches a process-global MJWarp helper.

release_render_context

release_render_context(rc) -> None

Drop texture tensor keepalives attached to an MJWarp render context; repeated calls are safe.

The caller must also drop the context to release its device arrays.

release_cached_ctx

release_cached_ctx(c) -> None

Drop texture keepalives from a cached family context before eviction.

gl_rounding

gl_rounding()

Temporarily round RGB channels before MJWarp packs them to uint8.

Enclose graph capture as well as warmup. This patches process-global MJWarp internals, so don't render concurrently.

needs_com_pos

needs_com_pos(m) -> bool

Return whether any camera or light tracks or targets a subtree center of mass.

MuJoCo rasterizer

stable_envs.render.mujoco_raster draws MuJoCo scenes on CPU with a Numba rasterizer that follows MuJoCo's fixed-function OpenGL renderer (render_gl3.c). It reads one scene model and one set of per-frame inputs. It is the default CPU renderer of the OGBench Cube and Scene tasks (backend="numba"), and Fetch uses it for render_semantic only. backend="opengl" stays the exact reference. On CUDA, the Cube and Scene backend="numba" raises ValueError.

Scene model

stable_envs.render.mujoco_scene.extract reads one mjv_updateScene of a model. It gives the drawn geoms with the triangles of their display lists, the materials and texture mappings, the lights, the camera, the textures with their mipmaps, the skybox and the shadow-map settings. mujoco_scene.pack turns the scene into flat arrays: corners, triangles, materials with a table of specular powers, textures, light rows, the camera row and the shadow matrices.

The extraction records the MuJoCo minor version. mujoco_scene.VERIFIED lists the scene features that were compared with the OpenGL renderer of each version. A scene that needs another feature raises NotImplementedError. Fetch uses MuJoCo 3.11. Cube and Scene use MuJoCo 3.17.

Per-frame inputs

A family gives these arrays for each frame:

Input Shape Content
gpos (B, G, 3) Geom positions
gmat (B, G, 9) Geom rotations, row major
rgba (B, G, 4) Geom colors; an alpha below 0.995 makes the geom translucent in that frame
cams (B, 12) Camera position, right, up and forward
lights (B, L, 23) Light rows; mujoco_scene.headlight_rows points the headlight along each camera
sky (B, 3) A flat sky color on the 0-255 scale, or -1 for the skybox

Rendering contract

  • Lighting is per vertex, in float64, with a local viewer, the headlight and the model lights.
  • Each frame has 4 samples per pixel at the sample positions of Mesa, shaded at the pixel center.
  • A shadow light adds its light where the shadow map has no nearer caster. The static geoms make one map per scene; the moving geoms make a map per frame.
  • Translucent geoms blend after the opaque geoms and the skybox, sorted by distance to the camera less their bounding radius, once front to back and once back to front.
  • Triangles are clipped at the near plane.
  • A draw of the same inputs gives the same frame each time, alone or in any batch.

Plates

Frames with the same camera, light rows, sky color and colors of the static geoms share a plate. The plate holds the static opaque geoms, their shadows and the skybox. Each frame starts from its plate and draws the other geoms.

Segmentation and depth

mujoco_raster.semantic gives the geom of each pixel center and its depth along the optical axis, as the segmentation and depth rendering of MuJoCo give them. A pixel shows the nearest opaque geom, or a translucent geom in front of it. Fetch and Cube/Scene give render_semantic(rows): the segmentation as (objid, objtype) pairs in the model of the reference environment, and the depth in model units.

The compiled variation

stable_envs._core.style.StyleTensor is the compiled form of variation: one flat (B, variation_dim) float32 tensor plus a layout that maps columns to leaves such as cart.color. Application code uses variation_spec, default_variation, sample_variation and render(..., variation=) instead.

import stable_envs as snv
from stable_envs._core.style import StyleTensor

with snv.CartPole(num_envs=2, device="cpu") as env:
    style = StyleTensor.from_space(env.variation_space, env.num_envs, device=env.device)
    print(style.values.shape, style.values.dtype)  # torch.Size([2, 22]) torch.float32
    print(style.get("cart.color").shape)           # torch.Size([2, 3])
    print(style.for_batch(4).values.shape)         # torch.Size([4, 22])

StyleTensor.for_batch is the only operation that may change the batch size of a compiled row. A shared style broadcasts its one row to the new batch size. A style whose rows differ raises ValueError, since nothing says which rows the smaller batch should keep. Slice the style together with the states instead.

Compiled appearance rows: a (B, style_dim) float32 tensor addressed by variation path.

StyleTensor.for_batch is the only operation that changes the batch size of a compiled row.

Classes:

  • StyleTensor –

    Appearance values of shape (B, style_dim), indexed by variation path.

Functions:

  • rows_are_equal –

    Test equality across batch rows, reducing all comparisons before one host read.

StyleTensor

StyleTensor(values: Tensor, layout: list[tuple[str, int, int]], shared: bool | None = None)

Appearance values of shape (B, style_dim), indexed by variation path.

layout lists (path, offset, width) for each leaf. shared records whether all rows are known to be equal; None defers the check until :attr:shared is read.

Methods:

  • get –

    Return the leaf view for path, or default when the layout has no such leaf.

  • to –

    Return a copy on device with dtype, keeping the layout and known sharedness.

  • for_batch –

    Return this style at batch rows.

  • from_space –

    Build float32 appearance rows from space defaults or supplied values.

Attributes:

  • shared (bool) –

    Whether all rows match; the first query may synchronize with the device.

shared property

shared: bool

Whether all rows match; the first query may synchronize with the device.

The result is cached. Do not mutate values after this property is read.

get

get(path: str, default=None)

Return the leaf view for path, or default when the layout has no such leaf.

to

to(device=None, dtype=None) -> StyleTensor

Return a copy on device with dtype, keeping the layout and known sharedness.

for_batch

for_batch(batch: int, owner: str = 'render') -> StyleTensor

Return this style at batch rows.

A matching batch returns self. A shared style broadcasts its row; a style with differing rows raises ValueError, since no row selection is implied.

from_space classmethod

from_space(space, batch: int, device='cuda', overrides: dict | None = None, values=None) -> StyleTensor

Build float32 appearance rows from space defaults or supplied values.

Values must contain one shared row or exactly batch rows. Overrides map paths to either one leaf value or one value per batch row; they apply last.

rows_are_equal

rows_are_equal(*tensors: Tensor) -> bool

Test equality across batch rows, reducing all comparisons before one host read.