SIGGRAPH Asia 2025

Bokeh Diffusion

Defocus Blur Control in Text-to-Image Diffusion Models

Armando Fortes Tianyi Wei Shangchen Zhou Xingang Pan

MMLab@NTU Nanyang Technological University
Paper arXiv Code Dataset BibTeX
Drag across the image to rack the bokeh.
Same scene, only the blur changes.
TL;DR

We condition text-to-image diffusion models on a defocus blur parameter, so the amount of bokeh can be set directly and changed without altering the scene. The method works with both Stable Diffusion and FLUX.

01Abstract

Recent advances in large-scale text-to-image models have revolutionized creative fields by generating visually captivating outputs from textual prompts; however, while traditional photography offers precise control over camera settings to shape visual aesthetics—such as depth-of-field via aperture—current diffusion models typically rely on prompt engineering to mimic such effects. This approach often results in crude approximations and inadvertently alters the scene content.

In this work, we propose Bokeh Diffusion, a scene-consistent bokeh control framework that explicitly conditions a diffusion model on a physical defocus blur parameter. To overcome the scarcity of paired real-world images captured under different camera settings, we introduce a hybrid training pipeline that aligns in-the-wild images with synthetic blur augmentations, providing diverse scenes and subjects as well as supervision to learn the separation of image content from lens blur. Central to our framework is our grounded self-attention mechanism, trained on image pairs with different bokeh levels of the same scene, which enables blur strength to be adjusted in both directions while preserving the underlying scene.

Extensive experiments demonstrate that our approach enables flexible, lens-like blur control, supports downstream applications such as real image editing via inversion, and generalizes effectively across both Stable Diffusion and FLUX architectures.

02Method

Two ingredients: a bokeh-conditioned denoiser that keeps the scene anchored while the blur changes, and a hybrid dataset that teaches the model to separate content from lens blur.

Architecture

···

Denoising network εθ

  • Text cross-attention frozen
  • Bokeh cross-attention trained, deeper layers
  • Grounded self-attention trained
Bokeh condition

Text & Bokeh Cross-Attention

Text features K V hi Q Bokeh feature Kb Vb Attention Attention × λ + h′i

The blur level gets its own keys and values, next to the frozen text attention.

Grounded Self-Attention

pivot hpiv Qp Kp target hi Ki Vi C × S R × h′i Ⓒ concat · Ⓡ repeat · Ⓢ softmax

The pivot fixes the layout; the target supplies the blur.

Training data

a

In-the-wild data collection

Real photos with authentic lens blur

  1. 1Collect
    In-the-wild photograph of cows NIKON D3200
    30 mm · 1/500 s

    Photos with EXIF lens metadata

  2. 2Annotate
    A countryside scene with cows in a lush green field…
    Estimated depth map Foreground mask

    Caption · Depth · Mask

  3. 3Filter
    • Aperture
    • Focal length
    • Fg / Bg area
    • Fg / Bg depth

    Reliable lens info, clear subject

  4. 4Balance

    Long-tail → uniform blur

b

Synthetic bokeh augmentation

One scene at several blur levels

  1. 1All-in-focus
    All-in-focus photo

    Image, depth and mask

  2. 2Defocus map
    Defocus map

    Blur radius K ~ 𝒰

  3. 3Render
    Rendered at small blur radius Rendered at medium blur radius Rendered at large blur radius
    K

    Same scene, different K

03Direct Bokeh Control

Sample an image directly at the bokeh level you want. The number on each image is the bokeh condition, on a scale from 0 (all-in-focus) to 30.

04Scene-Consistent Bokeh

Grounded self-attention keeps the scene fixed while the blur changes continuously. One dial drives every scene below. Click any scene to inspect it up close.

sharp strong bokeh

05Comparison with Non-generative Methods

Post-processing bokeh renderers depend on external depth estimation. Bokeh Diffusion instead leverages the generative prior of text-to-image models and our in-the-wild dataset to produce more natural bokeh in challenging areas, such as thin structures and translucent materials.

All-in-focus input
Input
Predicted depth map
Predicted depth (used by the renderer)
Bokeh Diffusion result
Depth estimation plus bokeh rendering result
Thin structures Depth + rendering Bokeh Diffusion

Drag across the image to compare.

Thin structures: depth-based rendering blurs the whiskers together with the background. Bokeh Diffusion keeps them sharp.

06Citation

@inproceedings{fortes2025bokeh,
  title     = {Bokeh Diffusion: Defocus Blur Control in Text-to-Image Diffusion Models},
  author    = {Fortes, Armando and Wei, Tianyi and Zhou, Shangchen and Pan, Xingang},
  booktitle = {Proceedings of the SIGGRAPH Asia 2025 Conference Papers},
  series    = {SA Conference Papers '25},
  publisher = {ACM},
  articleno = {160},
  pages     = {1--11},
  year      = {2025},
  doi       = {10.1145/3757377.3763906}
}