Image inpainting, a crucial task within the broader field of image restoration, aims to reconstruct missing or corrupted regions in images, ensuring they blend seamlessly with their surroundings. In this work, we propose Diff-PMArt to focus on a specific and complex type of inpainting involving portrait-map-art (PMA) images, where artistic portraits are integrated within cartographic backgrounds. Our goal is to restore the map background after removing the portrait, ensuring that the reconstructed region harmonizes with the surrounding structure and visual patterns on the map. Our results demonstrate the ability of Diff-PMArt to produce high-quality, realistic inpainting for PMA images, significantly improving over previous methods in handling unknown regions. In contrast to existing solutions, our approach can inpaint unknown regions without annotations, addressing a more intractable problem where no prior knowledge of the mask or labels is available. The proposed framework is carefully designed with two modules: a Visual Object Extraction (VOE) module, which identifies and extracts the portrait region as a mask, and a Diffusion-based Image Reconstruction (DIR) module, which uses a diffusion process to progressively inpaint the masked region. Our experimental results and evaluations demonstrate that Diff-PMArt has ability to reconstruct visually realistic and semantically coherent background of PMA images.

Keywords -- PMA, map art, Diff-PMArt, diffusion, inpainting

Comparisons

Results on AI-Created PAM images