Noisy dSprites

Overview

The Noisy dSprites dataset is a synthetic benchmark designed for disentangled and unsupervised representation learning. It is a variant of the original dSprites dataset, where random noise is added to the background of each image, while the object itself remains unchanged.

Compared to the original grayscale dSprites, this variant introduces background noise as a nuisance factor, enabling evaluation of robustness to noisy observations and analysis of how disentanglement methods perform when irrelevant background variation is present.

The dataset contains all possible combinations of six latent factors of variation inherited from dSprites, with each combination appearing exactly once.

Total images: 737,280
Image resolution: 64×64x3 (binary object with noisy background)

Latent Factors of Variation

The dataset is generated from six independent latent factors, consistent with the original dSprites specification. In this noisy variant, no additional latent factor is introduced; background noise is applied independently of the ground-truth factors.

Factor	Discrete Values	Continuous Values
`color`	{0}	1.0 (fixed, white)
`shape`	{0, 1, 2}	{1.0, 2.0, 3.0} (square, ellipse, heart)
`scale`	{0, …, 5}	Linearly spaced in [0.5, 1.0]
`orientation`	{0, …, 39}	Uniform in [0, 2π] radians
`posX`	{0, …, 31}	Normalized position in [0, 1]
`posY`	{0, …, 31}	Normalized position in [0, 1]

Each image corresponds to a unique combination of these six latent factors. The background noise does not affect the factor indexing or ordering.

Data Structure

When accessing an example using ds[i], you will receive a dictionary with the following keys:

Key	Type	Description
`image`	`PIL.Image.Image`	64×64×3 image with noisy background
`label`	`List[int]`	Discrete latent indices: `[color, shape, scale, orientation, posX, posY]`
`label_values`	`List[float]`	Continuous latent values corresponding to `label`
`color` … `posY`	`int`	Individual discrete latent factors
`colorValue` … `posYValue`	`float`	Individual continuous latent values

Note: In this Noisy variant, the object remains white and identical to the original dSprites. Only the background pixels are replaced with random noise, applied independently per image.

Usage Example

Basic Usage

from stable_datasets.images.dsprites_noisy import DSpritesNoisy

# First run will download + prepare cache, then return the split as a HF Dataset
ds = DSpritesNoisy(split="train")

# If you omit the split (split=None), you get a DatasetDict with all available splits
ds_all = DSpritesNoisy(split=None)

sample = ds[0]
print(sample.keys())

image = sample["image"]
factors = sample["label"]
factor_values = sample["label_values"]

# Optional: make it PyTorch-friendly
ds_torch = ds.with_format("torch")

Why No Train/Test Split?

The Noisy dSprites dataset does not define an official train/test split. It is intended for representation learning research, where models are trained to capture underlying factors of variation rather than to generalize across semantic classes.

Because the dataset is a complete Cartesian product of all factor combinations, common evaluation protocols rely on:

Factor-wise generalization
Metric-based disentanglement scores
Robustness to background noise
Controlled interventions on latent variables

References

Dataset repository: https://github.com/google-research/disentanglement_lib/
License: Apache License 2.0
Paper: Locatello et al., Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations, ICML 2019

Citation

@inproceedings{locatello2019challenging,
  title={Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations},
  author={Locatello, Francesco and Bauer, Stefan and Lucic, Mario and
          Raetsch, Gunnar and Gelly, Sylvain and
          Sch{\"o}lkopf, Bernhard and Bachem, Olivier},
  booktitle={International Conference on Machine Learning},
  year={2019}
}