Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models

Zitong Huang*, Kaidong Zhang*, Yukang Ding, Chao Gao, Rui Ding, Ying Chen, Wangmeng Zuo (*equal contribution)

[Paper]

LocalDPO is an efficient post-training framework that aligns text-to-video models with human preferences by using an automated pipeline to create localized spatio-temporal preference pairs from real videos, enabling fine-grained region-aware optimization without the need for external critic models or multi-sample ranking.

LaCon: Late-Constraint Diffusion for Steerable Guided Image Synthesis

Chang Liu, Rui Li, Kaidong Zhang, Xin Luo, Dong Liu

[Paper] [Code]

LaCon is a flexible and efficient “late-constraint” paradigm that achieves steerable image synthesis by aligning external conditions with the internal features of pre-trained diffusion models to guide the sampling process without requiring extra modules or heavy parameter optimization.

StableV2V: Stablizing Shape Consistency in Video-to-Video Editing

Chang Liu, Rui Li, Kaidong Zhang, Yunwei Lan, Dong Liu

[Paper] [Code] [Project Page]

StableV2V presents a novel paradigm to perform video editing in a shape-consistent manner, especially handling the editing scenarios when user prompts cause significant shape changes to the edited contents.

Drim-NeRF: Diffusion-Based Restoration for Improving Neural Radiance Fields

Ganlin Yang, Kaidong Zhang, Jingjing Fu, Dong Liu

[Paper]

Drim-NeRF is a diffusion-based, backbone-agnostic restoration method that treats NeRF artifacts as a specific degradation model and utilizes optical flow warping along with feature-wrapping in the VAE decoder to produce high-fidelity, multi-view consistent enhancements for complex scenes.

Toward interactive image inpainting via robust sketch refinement

Chang Liu, Shunxin Xu, Jialun Peng, Kaidong Zhang, Dong Liu

[Paper] [Code]

This paper explores the guidance of sketch in image inpainting. It seperates the completion of sketch and image and designs two specialized networks for excellent image inpainting performance.

Exploiting Optical Flow Guidance for Transformer-Based Video Inpainting

Kaidong Zhang, Jialun Peng, Jingjing Fu, Dong Liu

[Paper] [Video]

This paper investigates how to use the power of optical flows to guide the video inpainting transformer systematically. It reaches SOTA video inpainting performance against all of the previous counterparts.

Flow-Guided Transformer for Video Inpainting

Kaidong Zhang, Jingjing Fu, Dong Liu

[Paper] [Code] [Video]

FGT is the first video inpainting framework that utilizes the optical flows to influence attention calculation in transfomer for video inpainting with higher fidelity.

Inertia-guided flow completion and style fusion for video inpainting

Kaidong Zhang, Jingjing Fu, Dong Liu

[Paper] [Code] [Video]

ISVI is a novel flow-guided video inpainting system that integrates inertia into flow completion and adopts additional style correction to produce completed videos with high fidelity.

A Dataset for Deep Learning-based Bone Structure Analyses in Total Hip Arthroplasty

Kaidong Zhang, Ziyang Gan, Dong Liu, Xifu Shang

[Paper] [Code]

The researchers propose an efficient bone structure annotation pipeline that combines non-learning-based segmentation (using graph-cut and gradient-based algorithms) with active-learning-based refinement to create a large-scale, high-quality THA dataset with minimal manual labeling.

Customized segment anything model for medical image segmentation

Kaidong Zhang, Dong Liu

[Paper] [Code]

SAMed is the seminal work that utilizes Segment Anything Model (SAM) in multi-organ segmentation. It achieves remarkable performance and demonstrates the feasibility of SAM in medical usage.