Roadster¶
Roadster is five planning tasks for a car with nine cold-gas jets, simulated in MuJoCo. The car drives on its wheels and flies on its jets, and each class fixes one task of the simulator in stable_envs.envs.roadster.source. The state holds everything that determines the future, the variation sets wind and rain, and MuJoCo OpenGL draws the frames from a world-model camera.
# needs the roadster extra
import stable_envs as snv
with snv.RoadsterTrafficJam(num_envs=4, device="cpu") as env:
options = {"variation_values": {"weather.wind": [8.0], "weather.rain": [0.5]}}
state = env.reset(seed=0, options=options)
print(env.render(state).shape) # torch.Size([4, 96, 96, 3])
out = env.step(env.action_spec.sample(4, env.rng))
print(out["success"].shape) # torch.Size([4])
Install¶
The classes need the roadster extra (MuJoCo 3.17 and Pillow): install stable-envs[roadster]. MuJoCo Warp physics on CUDA needs the roadster-gpu extra, which adds MuJoCo Warp 3.17 and Warp.
Task¶
| Class | Task | Horizon (steps) |
|---|---|---|
RoadsterLoop |
drive the half-pipe loop, leave it inverted and roll upright | 60 |
RoadsterMission |
the full course: loop, three rings, pad landing | 450 |
RoadsterTrafficJam |
hop a traffic jam and land on the road beyond | 150 |
RoadsterRooftop |
fly from a tower roof around a taller tower to a helipad | 200 |
RoadsterCarHop |
jump sideways over a parked car with a barrel roll | 120 |
The action has four values in [-1, 1]. The environment applies each action at 10 Hz through a 50 Hz autopilot:
| Index | Control |
|---|---|
| 0 | forward acceleration in the heading frame (wheel torque on the ground, jets in the air) |
| 1 | sideways acceleration (the car banks into it) |
| 2 | vertical acceleration relative to hover; on the ground a positive value lights the lift jets |
| 3 | yaw rate |
The task's cost is the distance to the current subgoal plus the path through the remaining ones. The reward is the decrease of that cost, plus 10 on success and minus 10 on failure. The step output carries success, failed, terminated, truncated and the current cost. A batch row terminates on success or failure and truncates at the task's horizon, and the elapsed steps are part of the state.
stable_envs.envs.roadster.source.GoalExpert is a scripted expert that reads the simulator, for demos and data collection.
Simulation¶
simulation_backend selects the physics:
| Value | CPU device | CUDA device |
|---|---|---|
"auto" (default) |
CPU batch | CPU batch for small batches, MuJoCo Warp for large ones |
"native" |
CPU batch | CPU batch |
"mjwarp" |
error | MuJoCo Warp |
With "auto" on CUDA, a batch of large_backend_min_envs or more environments steps on MuJoCo Warp. When the setting is not set, the switch point comes from the tuning cache that stable-envs tune writes, else from the value the class ships, so without tuning the same batch size selects the same backend on every machine (Physics backends).
Both backends step every environment in one batch. The source's autopilot, drag and wind forces, fuel, ring and route bookkeeping, crash checks, cost, reward and success run on arrays of environments. MuJoCo steps the 10 substeps of each 50 Hz step one at a time, because the drag and wind force of a substep depends on the velocity after the previous one. The physics model is the source's world with MuJoCo contact sensors for the contacts that the task reads (tires on anything, on the pad, on the ground and on the goal roof, the hull on anything, the car on each obstacle). The visual geoms outside the car are not in it. Sensors and these geoms do not change the dynamics.
The CPU batch runs the task logic in NumPy and MuJoCo in one native mujoco.batch in float64. It steps bit for bit as the source environment: the same mj_step calls with the same applied forces, and the same float64 operations in the same order. NumPy computes a stack of small matrix products and least-squares solves with the same BLAS and LAPACK routines as one. A device="cuda" environment with the CPU batch returns its state on the GPU.
The MuJoCo Warp backend runs the task logic in float64 Torch on the GPU, without host reads in a step, and MuJoCo in float32, so a rollout follows the source approximately. Each 50 Hz step is three CUDA graph launches: the Torch phase before the substeps, one MuJoCo Warp graph of the 10 substeps with a Warp kernel for the drag and wind force before each substep, and the Torch phase after them. The autopilot's bounded least-squares jet allocation uses a precomputed pseudo-inverse for each set of free jets. With wind, the gusts of an action are drawn on the host from the generator of each environment, as the source does. An environment that ends within an action has then drawn the gusts of the whole action. Replays of the source's scripted expert under wind and rain (tests/gpu/test_roadster_mjwarp.py, 8 environments, 300 steps) keep the outcome of every environment and end within 3 steps of the source. The car stays within 0.01 m of the source's path in TrafficJam and CarHop, 0.3 m in Rooftop and 3.4 m in Loop until the episode ends. In Mission the paths separate after the loop, where the source itself moves 13 m by step 300 when its start velocity changes by 1e-9 m/s.
Resets draw the start, goal, traffic and parked cars of each environment with the source's random draws on the host, and write all environments to the physics batch in one call.
State¶
The state has one tensor for each dtype: roadster_state.float64, roadster_state.int64 and roadster_state.bool. Each field below is a slice of one of them. Read a field by name with snv.get_field(state, "sim.qpos"). The state is made of:
- MuJoCo's integration state, split into
sim.time,sim.qpos,sim.qvel,sim.warmstart,sim.ctrl,sim.qfrc_applied,sim.xfrc_applied,sim.mocap_posandsim.mocap_quat. The two mocap fields place the traffic and the parked cars. sim.qpos_pre, the pose that MuJoCo's derived quantities refer to after a step.- The autopilot commands and the task bookkeeping: status, target and route index, fuel, counters, flags and the previous cost.
- The goal, the route waypoints, the traffic jam, and the weather with its gust RNG.
- The exhaust history that the smoke is drawn from (
fx.countandfx.history, the last 75 steps of pose, velocity, jet commands and wind). lifecycle.elapsed_steps, and the physics leaves the episode was reset with (physics.weather.windandphysics.weather.rain).
set_state validates the whole batch before it changes any environment, and it refuses render rows, partial dictionaries and states of another task.
Large batches draw frames in render worker processes, with the same frames as one process. The settings of the roadster-render path set the batch size that starts workers and the number of workers. A step that needs a frame but no state draws it from the render-row fields of the live environments, and the frame equals what render(state) draws.
Rendering¶
render(state) draws the wm camera, which follows the car with a fixed world orientation, at 96 px by default, with MuJoCo's OpenGL renderer (MUJOCO_GL=egl or osmesa). It rasterizes on the GPU when the EGL driver is a GPU's (NVIDIA's, for example) and on the CPU otherwise (Mesa). render also accepts compact render rows (N, state_dim) from sample_states, on each device.
Frames show the car, its jet flames, exhaust smoke and blob shadow, the scenery and the rain, exactly as MuJoCo draws the live simulation at the same state. The smoke is regenerated from the exhaust history rather than simulated.
rendering.cinematic=1 draws the same state from the same wm camera on a dark-space scene, with 8× MSAA, real shadows, fog to black and post-processing (bloom, filmic grade, vignette). It leaves out grain and motion blur, and a state always gives the same pixels. It looks best at a larger size such as 512. Set it per environment with set_variation({"rendering.cinematic": [1]}), or per call with render(state, variation={"rendering.cinematic": [1]}). The installed variation records which look drew a frame, and the default, 0, is the raw world-model frame.
Variation¶
| Leaf | Kind | Values | Effect |
|---|---|---|---|
weather.wind |
physics | 0 to 20 m/s | mean wind speed, with gusts of about 30% of it |
weather.rain |
physics | 0 to 1 | drops, haze, and tire grip down to 0.55× in the heaviest rain |
rendering.cinematic |
visual | 0 or 1 | draws the dark-space look instead of the world-model frame |
The variation has two physics leaves and one visual leaf. The simulator sets the weather at reset, and set_variation and the reset options for the two physics leaves apply from the next reset. The defaults (0, 0) give the simulator's physics without weather, and the weather of a restored state shows in get_variation() for that episode.
Classes¶
RoadsterLoop¶
| Property | Value |
|---|---|
| Reference | roadster_jets/RoadsterPlan-v0:loop |
| Physics | native MuJoCo on CPU; MuJoCo Warp on CUDA |
| Requires CUDA | no |
| Renderer on CPU | opengl (MuJoCo) |
| Renderer on CUDA | opengl (MuJoCo) on CPU, frames copied to CUDA |
| Default frame | 96 × 96 × 3, uint8 |
| Action | float32 (4,), [-1, 1] |
| Clips | rendered on CPU at 160 px, then resized |
State (1 top-level keys)
| Key | Shape | dtype |
|---|---|---|
roadster_state |
3 leaves, 2377 values | bool, float64, int64 |
Variation space (3 leaves)
| Path | Type | Bounds or choices | Default | Sampled at reset |
|---|---|---|---|---|
rendering.cinematic |
Discrete | 2 choices from 0 | 0 | |
weather.rain |
Box (1,) | [0, 1] | 0 | |
weather.wind |
Box (1,) | [0, 20] | 0 |
RoadsterMission¶
| Property | Value |
|---|---|
| Reference | roadster_jets/RoadsterPlan-v0:mission |
| Physics | native MuJoCo on CPU; MuJoCo Warp on CUDA |
| Requires CUDA | no |
| Renderer on CPU | opengl (MuJoCo) |
| Renderer on CUDA | opengl (MuJoCo) on CPU, frames copied to CUDA |
| Default frame | 96 × 96 × 3, uint8 |
| Action | float32 (4,), [-1, 1] |
| Clips | rendered on CPU at 160 px, then resized |
State (1 top-level keys)
| Key | Shape | dtype |
|---|---|---|
roadster_state |
3 leaves, 2377 values | bool, float64, int64 |
Variation space (3 leaves)
| Path | Type | Bounds or choices | Default | Sampled at reset |
|---|---|---|---|---|
rendering.cinematic |
Discrete | 2 choices from 0 | 0 | |
weather.rain |
Box (1,) | [0, 1] | 0 | |
weather.wind |
Box (1,) | [0, 20] | 0 |
RoadsterTrafficJam¶
| Property | Value |
|---|---|
| Reference | roadster_jets/RoadsterPlan-v0:traffic |
| Physics | native MuJoCo on CPU; MuJoCo Warp on CUDA |
| Requires CUDA | no |
| Renderer on CPU | opengl (MuJoCo) |
| Renderer on CUDA | opengl (MuJoCo) on CPU, frames copied to CUDA |
| Default frame | 96 × 96 × 3, uint8 |
| Action | float32 (4,), [-1, 1] |
| Clips | rendered on CPU at 160 px, then resized |
State (1 top-level keys)
| Key | Shape | dtype |
|---|---|---|
roadster_state |
3 leaves, 2377 values | bool, float64, int64 |
Variation space (3 leaves)
| Path | Type | Bounds or choices | Default | Sampled at reset |
|---|---|---|---|---|
rendering.cinematic |
Discrete | 2 choices from 0 | 0 | |
weather.rain |
Box (1,) | [0, 1] | 0 | |
weather.wind |
Box (1,) | [0, 20] | 0 |
RoadsterRooftop¶
| Property | Value |
|---|---|
| Reference | roadster_jets/RoadsterPlan-v0:rooftop |
| Physics | native MuJoCo on CPU; MuJoCo Warp on CUDA |
| Requires CUDA | no |
| Renderer on CPU | opengl (MuJoCo) |
| Renderer on CUDA | opengl (MuJoCo) on CPU, frames copied to CUDA |
| Default frame | 96 × 96 × 3, uint8 |
| Action | float32 (4,), [-1, 1] |
| Clips | rendered on CPU at 160 px, then resized |
State (1 top-level keys)
| Key | Shape | dtype |
|---|---|---|
roadster_state |
3 leaves, 2377 values | bool, float64, int64 |
Variation space (3 leaves)
| Path | Type | Bounds or choices | Default | Sampled at reset |
|---|---|---|---|---|
rendering.cinematic |
Discrete | 2 choices from 0 | 0 | |
weather.rain |
Box (1,) | [0, 1] | 0 | |
weather.wind |
Box (1,) | [0, 20] | 0 |
RoadsterCarHop¶
| Property | Value |
|---|---|
| Reference | roadster_jets/RoadsterPlan-v0:carhop |
| Physics | native MuJoCo on CPU; MuJoCo Warp on CUDA |
| Requires CUDA | no |
| Renderer on CPU | opengl (MuJoCo) |
| Renderer on CUDA | opengl (MuJoCo) on CPU, frames copied to CUDA |
| Default frame | 96 × 96 × 3, uint8 |
| Action | float32 (4,), [-1, 1] |
| Clips | rendered on CPU at 160 px, then resized |
State (1 top-level keys)
| Key | Shape | dtype |
|---|---|---|
roadster_state |
3 leaves, 2377 values | bool, float64, int64 |
Variation space (3 leaves)
| Path | Type | Bounds or choices | Default | Sampled at reset |
|---|---|---|---|---|
rendering.cinematic |
Discrete | 2 choices from 0 | 0 | |
weather.rain |
Box (1,) | [0, 1] | 0 | |
weather.wind |
Box (1,) | [0, 20] | 0 |
Limitations¶
The drawn pose is the one MuJoCo's derived quantities refer to, which is one substep behind sim.qpos after a step.