detectors/NPR/README.md

Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection

Original Paper: Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection.

Authors: Chuangchuang Tan, Huan Liu, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, Yunchao Wei.

Abstract

Recently, the proliferation of highly realistic synthetic images, facilitated through a variety of GANs and Diffu- sions, has significantly heightened the susceptibility to mis- use. While the primary focus of deepfake detection has tra- ditionally centered on the design of detection algorithms, an investigative inquiry into the generator architectures has remained conspicuously absent in recent years. This paper contributes to this lacuna by rethinking the architectures of CNN-based generator, thereby establishing a generalized representation of synthetic artifacts. Our findings illumi- nate that the up-sampling operator can, beyond frequency- based artifacts, produce generalized forgery artifacts. In particular, the local interdependence among image pixels caused by upsampling operators is significantly demon- strated in synthetic images generated by GAN or diffusion. Building upon this observation, we introduce the concept of Neighboring Pixel Relationships(NPR) as a means to cap- ture and characterize the generalized structural artifacts stemming from up-sampling operations. A comprehensive analysis is conducted on an open-world dataset, comprising samples generated by 28 distinct generative models. This analysis culminates in the establishment of a novel state-of- the-art performance, showcasing a remarkable 11.6% im- provement over existing methods

Please Cite

@inproceedings{tan2024rethinking,
title={Rethinking the up-sampling operations in cnn-based generative network for generalizable deepfake detection},
author={Tan, Chuangchuang and Zhao, Yao and Wei, Shikui and Gu, Guanghua and Liu, Ping and Wei, Yunchao},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={28130--28139},
year={2024}
}