Diffusion Models

Research shows that diffusion image generators can train somewhat faster if they learn to reconstruct embeddings from a pretrained encoder that’s built for vision tasks like classification, segmentation, and retrieval — not image generation.

Image generators often mangle text. An open-weights model outperforms open and proprietary competitors in text rendering.

Current methods that produce 3D scenes from text or images are slow and produce inconsistent results. Researchers introduced a technique that generates detailed, coherent 3D scenes seconds.

Chain-of-thought reasoning can help autonomous vehicles decide what to do next.

A new image generator reasons over prompts to produce outstanding pictures.

The mixture-of-experts approach that has boosted the performance of large language models may do the same for video generation.

Diffusion transformers learn faster when they can look at embeddings generated by a pretrained model like DINOv2.

Diffusion models usually take many noise-removal steps to produce an image, which takes time at inference. There are ways to reduce the number of steps, but the resulting systems are less effective. Researchers devised a streamlined approach that doesn’t sacrifice output quality.

Materials that have specific properties are essential to progress in critical technologies like solar cells and batteries. A machine learning model designs new materials to order.

Typical large language models are autoregressive, predicting the next token, one at a time, from left to right. A new model hones all text tokens at once.

Last year, we saw an explosion of models that generate either video or audio outputs in high quality. In the coming year, I look forward to models that produce video clips complete with audio soundtracks including speech, music, and sound effects.

The gap is narrowing between closed and open models for video generation.

A new model improves on recent progress in generating interactive virtual worlds from still images.

Amazon introduced a range of models that confront competitors head-on.

Researchers devised a way to cut the cost of training video generators. They used it to build a competitive open source text-to-video model and promised to release the training code.