Research and Implementation of Inpainting Algorithms for Old Photos

: In view of the scratches and red spots caused by the impact of time, environment and shooting equipment on the old photos, this paper uses MATLAB as image processing simulation software to carry out the research and implementation of image inpainting algorithm. By comparing and analyzing the repair effect and operational efficiency of the three basic models based on partial differential equations: BSCB model, TV model, and CDD model, the method of introducing weights into the TV model is selected to perform simple repair on color photos and black and white photos with scratches and red dots, and the repair results of old photos are presented through the MATLAB GUI interface. The experimental results show that the introduction of weight values in the TV model not only has a significant effect on solving the problem of unidirectional extension of the equal illuminance line in the TV model, but also has a good effect on repairing scratches and red dots in small-scale color and black and white photos.


Introduction
Due to the influence of time, environment and other factors, the degradation of old photos is quite complicated.How to remove all kinds of degradation factors in old photos has become one of the hotspots of image restoration technology research.Based on the BSCB model originally proposed by M.Bertalmio, Sapiro, Caselles et al. [1] , the image was repaired through continuous iteration and diffusion along the isopilance line to the area to be repaired.TV (Total Variation) model [2] , proposed by Chan et al.In 2000, which seeks functional functions, sets up anisotropy equations, and minimizes energy to achieve the purpose of restoring old photos.In the following year, they introduced curvature to solve the existing incoherence problem of TV model to achieve the restoration of old photos.Thus, the CDD model (Curvature Driven Dif-fusion) [3] was proposed.Suganuma et al. proposed an efficient hierarchical neural network structure in 2019 [4] , which performs multiple operations in parallel, weights them by attention mechanism, selects appropriate operations according to input, and better deals with multiple degenerated combinations.In 2020, Wan et al. proposed a triple domain transformation network specifically for old photo restoration [5] , which completes the mapping of old photo potential domains to target domains by reducing the potential domain spacing between real old photos and synthesized old photos.n2021, Chen Yuanyuan proposed to repair damaged old photos by Generative adversarial network [6] .This method avoids the problem of color disharmony and incoherence of the repair results by adding context attention modules, and enhances network stability by adding perceptual loss, style loss, and reconstruction loss to the Loss function.In 2022, Liu Jixin et al. proposed the restoration of old photos by integrating reference priori and generative priori [7] , which makes full use of the semantic information of reference apriority and the portrait apriority encapsulated by the Generative model, and has achieved advanced repair performance.By comparing and analyzing the repair effect and operation efficiency of three basic models, BSCB, TV and CDD, this paper introduces the method of weight into the TV model to repair the scratches and red dots of color photos and black and white photos, and has achieved certain results.

Image Repair Model Selection
In order to select an appropriate Inpainting model, three basic models based on partial differential equations are studied respectively.
(1) BSCB model The damaged image (b) corresponding to the original image (a) in Figure 1 is repaired by BSCB model, and the repair effect is (c).By comparing (a) and (b) in Figure .1, it can be seen that the BSCB model has a general repair effect on scratches, and does not process the edge information of the damaged area obviously, and the operation is complicated and the operation efficiency is low.(2) TV model The TV model is used to repair the two types of images with obvious edge information of the damaged area in Figure 2 and fuzzy edge information of the damaged area in Figure 3.By comparing the original image and the repaired image in Figure 2 and Figure 3, it is found that the TV model has a good repair effect for the area with obvious damaged edge information, while the area with fuzzy damaged edge information is prone to false edge phenomenon.However, the repair algorithm is simple and the repair efficiency is high.From the above analysis, it can be seen that BSCB model can complete normal damaged image restoration, but the restoration effect is average.The TV model has insufficient information processing ability for damaged edges and high repair efficiency.CDD model can complete the repair well, but the repair efficiency is low.In this paper, we choose to optimize the TV model and introduce weights to solve the false edge problem and improve the operation efficiency.

Algorithm Research
Set the damaged image as shown in Figure 5, where the area to be repaired is R and the area with complete information is L. Assuming that the pixel value of the repaired area is m, the weight (diffusion function) formula ( 1) is introduced to process the edge information of the damaged area based on the TV model.
With the weight g(|∇m|), it can be realized that in the process of restoration, those who give a gentle area with a small gradient value will give a smaller value, and those who give a sharp area with a large gradient will give a larger value.The final repair result is shown in formula (2).
For numerical discrete processing, the interval sampling is shown in Figure 6, where the sampling step h is set as 1, the center O point is the m coordinate of the target pixel, its eight neighborhood is N, S, W, E, NW, NE, SE, SW, and its half-pixel point is n, s, w, e.
Where m O 0 is the value of the undamaged image.Let λ L (O)=0, the result of one iteration of the target pixel O to be repaired can b expressed as formula (5).The result of n iterations is formula (6), which is the final repair result.

Algorithm Flow
In the experiment, we designed to repair scratches and red spots in the identified color and black and white photos, and the specific flow chart is shown in Figure 7.As can be seen from Figure 7, the algorithm execution process is as follows: the photos to be repaired are read, corresponding mask images are obtained according to the damage conditions, and the repair iterations are set as 10, 40, 70 and 100.The modulus and diffusion values of reference points are obtained according to formula (3) for the areas to be repaired in the mask images, and the pixel update is performed by substituting formula (5).When the set number of iterations is reached, the repair is complete.

Algorithm Implementation
This paper uses MATLAB programming and its GUI interface to complete the mask acquisition, repair and result display of scratches and red dots in color and black and white photos.
(1) Color photo scratch repair The mask image of color photo Figure 8 is shown in Figure 9, and the iterative repair results of scratches are shown in Figure 10.The relationship between the number of iterations and their repair time is shown in Table 1.   10 that as the number of iterations increases, the scratch repair effect becomes more obvious.When the number of iterations is 70, the scratch repair of color photos can be basically realized.
(2) Black and white photo scratches repair The iterative repair results for scratches in black and white photo Figure 11 are shown in Figure 13, and the mask image is shown in Figure 12.The relationship between the number of iterations and their repair time is shown in Table 2.   13, it can be seen that for areas with small scratch ranges, the repair effect is obvious.If the scratch range is large, it is difficult to achieve a significant repair effect.Compared with Table 1 and Table 2, it can be seen that the repair efficiency of scratches in color pictures is high, while the repair efficiency of scratches in black and white photos is low for large-scale repair.
(3) Color photos red dot repair The iterative repair results for red dots in color photo Figure 14 are shown in Figure 16, and the mask image is shown in Figure 15.The relationship between the number of iterations and their repair time is shown in Table 3.   16 that for red dots repair of color pictures, the repair effect is obvious when the number of iterations reaches 100, but the data in Table 3 shows that the repair time is relatively long.
(4) Black and white photos red dot repair The iterative repair results for red dots in black and white photo Figure 17 are shown in Figure 19, and the mask image is shown in Figure 18.The relationship between the number of iterations and their repair time is shown in Table 4.   19 that the repair effect of red dot repair for black and white photos is obvious, but the problem of repairing edge blur still exists for the repaired photos.Comparing the time data in Table 3 and Table 4, the red dot repair efficiency of black and white photos is higher than that of color photos.

Conclusions
In this paper, weights are introduced into the TV model to repair the red spots and scratches in black and white photos and color photos.The repair results show that the improved method has obvious effect on the repair of scratches in a small range of black and white photos and color photos, and the repair efficiency is high.When the number of iterations reaches 100, the repair time is stable within 1 minute.For the repair of red dots, the repair effect is obvious, but the repair result is prone to edge blur, and the overall repair efficiency is relatively low compared with scratches.When there are too many red dots to be repaired and the number of iterations is set to 100, the repair time is basically maintained within 3 minutes under the operation of different devices.

Figure 1 :
Figure 1: Repair results of BSCB model

Figure 2 :Figure 3 :Figure 4 :
Figure 2: Image repair of damaged areas with obvious edge information

Figure 6 : 2 ℎ ( 3 )
Figure 6: Target pixel O and its neighborhood points Let v 1 and v 2 represent the horizontal and vertical components of∇m |∇m| in the area to be repaired, then the gradient value of the half-pixel point is formula (3).

Table 1 :
Relationship between scratch repair time and iterations of color photos

Table 2 :
Relationship between scratch repair time and iterations of black and white photos

Table 3 :
Relationship between red dot repair time and iterations of color photos

Table 4 :
Relationship between red dot repair time and iterations of black and white photos