Dunhuang Mural Restoration using Deep Learning

Abstract

As time goes by, the art pieces inside Dunhuang Grottoes have suffered from tremendous damage such as mural deterioration, and they are usually difficult to be repaired. Although we can achieve digital preservation by modeling the caves and preserving mural as textures in virtual environment, we still cannot have a glimpse of how the grottoes look like without damage. In this work, we propose a systematic restoration framework, which is based on Generative Adversarial Network (GAN) technique, for these high-resolution but deteriorated mural textures. The main idea is to make the machine learn the transformation between deteriorated mural textures and restored mural textures. However, the resolution of training texture images (i.e. 8192x8192) is too high to be applied with GAN technology directly due to GPU RAM limitation. Instead, our method restores a set of high-resolution yet color-inconsistent textures patch-by-patch and a set of low-resolution but color-consistent full textures, and then combines them to get the final high-resolution and color-consistent result.