Exploration of Image Inpainting approaches and challenges: A Survey
Main Article Content
Abstract
The process of restoring missing areas of an image is referred to as "image inpainting." It is a significant challenge in the field of computer vision and a crucial feature that is utilized in a wide variety of image and graphics programs. Although image inpainting, also known as the art of repairing old and worn images, has been around for a couple of years, it has recently gained even more popularity due to recent developments in image processing techniques. Image inpainting can be thought of as the art of restoring old and worn images. Automatic image inpainting has become an important and challenging area of research in image processing due to the advancement of tools for image processing and the flexibility of digital image editing. Automatic image inpainting has found important applications in computer vision and is also becoming an important image processing application. Due to its high significance and effectiveness in a variety of image processing applications such as, for example, object removal, image restoration, manipulation, re-targeting, compositing, and image-based rendering, researchers have studied the image inpainting problem intensively over several decades. The process of eliminating or filling in a missing area in an image is referred to as "image inpainting," It is defined as in-depth knowledge of the image details in terms of its structure and texture. It is considered to be one of the most challenging subjects in image processing. This article presents a survey of most image inpainting techniques and summarizes them, along with comparisons that include the benefits and drawbacks of each method. These comparisons and summaries can assist researchers in evaluating their own proposed techniques against existing ones.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
IJCERT Policy:
The published work presented in this paper is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. This means that the content of this paper can be shared, copied, and redistributed in any medium or format, as long as the original author is properly attributed. Additionally, any derivative works based on this paper must also be licensed under the same terms. This licensing agreement allows for broad dissemination and use of the work while maintaining the author's rights and recognition.
By submitting this paper to IJCERT, the author(s) agree to these licensing terms and confirm that the work is original and does not infringe on any third-party copyright or intellectual property rights.
References
C. Guillemot and O. Le Meur, “Image inpainting: Overview and recent advances,” IEEE Signal Process. Mag., vol. 31, no. 1, pp. 127144, Jan.2014
Bertalmio, M., Sapiro, G., Caselles, V. and Ballester, C. (2000) Image Inpainting. Proceedings of ACM SIGGRAPH, New Orleans, July 2000, 417-424.
A. Criminisi, P. Prez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” IEEE Trans. Image Process., vol.13, no. 9, pp. 12001212, Sep. 2004.
A. Efros and T. K. Leung, “Texture synthesis by non-parametric sampling,” in Proc. 7th IEEE ICCV, vol. 2. Sep. 1999, pp. 10331038.
Criminisi A, Perez P, Toyama K. Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Processing. 2004; 13:1200–1212.
Criminisi A, Perez P, Toyama K. Object removal by exemplar-based image inpainting. Proc IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2003; p. 721–728.
Aujol J.F, Ladjal S, Masnou S. Exemplar-Based Inpainting from a Variational Point of View. SIAM Journal on Mathematical Analysis. 2010; 42:1246–1285.
M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proceedings of the 27th annual conference on Computer graphics and interactive techniques (SIGGRAPH '00), pp. 417–424, July 2000.
A. A. Efros and T. K. Leung, “Texture synthesis by non-parametric sampling,” in Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV'99), pp. 1033–1038, Corfu, Greece, September 1999.
[24] M. Bertalmio, L. Vese, G. Sapiro, and S. Osher, “Simultaneous structure and texture image inpainting,” IEEE Transactions on Image Processing, vol. 12, no. 8, pp. 882–889, 2003.
A. Rare?, M. J. T. Reinders, and J. Biemond, “Edge-based image restoration,” IEEE Transactions on Image Processing, vol. 14, no. 10, pp. 1454–1468, 2005.
O. Elharrouss, N. Almaadeed, S. Al-Maadeed, et al., Image inpainting: a review, Neural Process. Lett. 51 (2020) 2007–2028, https://doi.org/10.1007/s11063-019- 10163-0.
Z.P. Qiang, L.B. He, X. Chen, D. Xu, Survey on deep learning image inpainting methods, J. Image Graph. 24 (03) (2019) 0447–0463.
C. Barnes, E. Shechtman, A. Finkelstein, and B. G. Dan, ‘‘PatchMatch: A randomized correspondence algorithm for structural image editing,’’ ACM Trans. Graph., vol. 28, no. 3, pp. 1–11, 2009.
M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, ‘‘Image inpainting,’’ Siggraph, vol. 4, no. 9, pp. 417–424, 2005.
A. Criminisi, P. Perez, and K. Toyama, ‘‘Object removal by exemplarbased inpainting,’’ in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Nov. 2003, pp. 1–8
A. Efros and T. Leung, ‘‘Texture synthesis by non-parametric sampling,’’ in Proc. 7th IEEE Int. Conf. Comput. Vis., vol. 2, Sep. 1999, pp. 1033–1038.
F. Farahnakian, P. Liljeberg, and J. Plosila, ‘‘LiRCUP: Linear regression based CPU usage prediction algorithm for live migration of virtual machines in data centers,’’ in Proc. 39th Eur. Conf. Softw. Eng. Adv. Appl., Sep. 2013, pp. 357–364.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, X. Bing, and Y. Bengio, ‘‘Generative adversarial networks,’’ in Proc. Adv. Neural Inf. Process. Syst., vol. 3, 2014, pp. 2672–2680.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, ‘‘ImageNet classification with deep convolutional neural networks,’’ in Proc. Int. Conf. Neural Inf. Process. Syst., 2012, pp. 1097–1105.
Z. Qiang, L. He, and X. Dan, ‘‘Exemplar-based pixel by pixel inpainting based on patch shift,’’ in Proc. CCF Chin. Conf. Comput. Vis., 2017, pp. 370–382
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A. Efros, “Context Encoders: Feature Learning by Inpainting,” Proc. International Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio, “Generative Adversarial Nets,” in Advances in Neural Information Processing Systems (NeurIPS), 2014.
Chao Yang, Xin Lu, Zhe Lin, Eli Shechtman, Oliver Wang, and Hao Li, “High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis,” Proc. International Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa, “Globally and Locally Consistent Image Completion,” ACM Trans. on Graphics, Vol. 36, ?4, Article 107, Publication date: July 2017.
Ugur Demir, and Gozde Unal, “Patch-Based Image Inpainting with Generative Adversarial Networks,” https://arxiv.org/pdf/1803.07422.pdf.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep Residual Learning for Image Recognition,” Proc. Computer Vision and Pattern Recognition (CVPR), 27–30 Jun. 2016.
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” Proc. Computer Vision and Pattern Recognition (CVPR), 21–26 Jul. 2017.
Zhaoyi Yan, Xiaoming Li, Mu Li, Wangmeng Zuo, and Shiguang Shan, “Shift-Net: Image Inpainting via Deep Feature Rearrangement,” Proc. European Conference on Computer Vision (ECCV), 2018.
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S. Huang, “Generative Image Inpainting with Contextual Attention,” Proc. Computer Vision and Pattern Recognition (CVPR), 2018.
Yi Wang, Xin Tao, Xiaojuan Qi, Xiaoyong Shen, and Jiaya Jia, “Image Inpainting via Generative Multi-column Convolutional Neural Networks,” Proc. Neural Information Processing Systems, 2018.
Guilin Liu, Fitsum A. Reda, Kevin J. Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro, “Image Inpainting for Irregular Holes Using Partial Convolution,” Proc. European Conference on Computer Vision (ECCV), 2018.
Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Z. Qureshi, Mehran Ebrahimi, “EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning,” Proc. International Conference on Computer Vision (ICCV), 2019.
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas Huang, “Free-Form Image Inpainting with Gated Convolution,” Proc. International Conference on Computer Vision (ICCV), 2019.