1. The Fundamental Problem of Inpainting

Mathematically, let an image frame be denoted as a continuous domain Ω ⊂ ℝ2. Inside Ω, let δΩ be an occluded region (the target watermark, logo, or scratch mask). The objective of an inpainting algorithm is to synthesize values for every pixel p ∈ δΩ using the known information from the surrounding boundary ∂δΩ.

The reconstructed boundary must satisfy two essential visual criteria:

  1. Geometric continuation: Edges and structural lines arriving at ∂δΩ must continue smoothly into δΩ along their isophote (constant intensity) directions.
  2. Photometric consistency: Color and gradient transitions must match the surrounding neighborhood without unnatural discontinuities or blur halos.

2. Alexandru Telea’s Fast Marching Method (FMM)

Published in 2004, Alexandru Telea’s algorithm remains one of the fastest and most reliable spatial inpainting methods used in computer vision pipelines today (implemented as cv2.INPAINT_TELEA in OpenCV).

Telea's algorithm models inpainting as the propagation of a boundary interface marching inward toward the center of the missing region:

Iq(p) = I(q) + ∇I(q) ⋅ (p - q)
I(p) = ∑q ∈ Bε(p) w(p, q) ⋅ Iq(p) / ∑q ∈ Bε(p) w(p, q)

Here, the weighting function w(p, q) combines three critical factors:

  • Direction component: Prioritizes pixels located along the boundary normal vector, maintaining continuous contour flow.
  • Geometric distance: Decreases exponentially as distance ||p - q|| increases, avoiding contamination from distant irrelevant features.
  • Level set distance: Ensures pixels closer to the original boundary curve are computed before pixels deeper in the unknown territory.

3. Bertalmio-Sapiro Navier-Stokes Inpainting

Marcelo Bertalmio, Andrea Bertozzi, and Guillermo Sapiro introduced an alternative approach inspired by 2D fluid dynamics (the Navier-Stokes equations, implemented as cv2.INPAINT_NS).

In this model, the image intensity I(x, y) represents the stream function of a two-dimensional fluid, while the image Laplacian ΔI represents fluid vorticity. By propagating vorticity along isophote lines into the damaged zone, edges are smoothly propagated without synthetic smearing.

4. The Video Challenge: Spatial vs. Temporal Consistency

While static image inpainting only needs to solve for spatial smoothness, video inpainting introduces a critical third dimension: time (t).

Method Computational Speed Temporal Flickering Risk Best Use Case
FFmpeg Bilinear Delogo Ultra-Fast (>120 fps) Extremely Low Stationary corner bugs, broadcaster logos, camera date stamps.
Telea Fast Marching Fast (40–60 fps) Low on static cameras Irregular shape scratches, thin watermark text lines.
Navier-Stokes Fluid Moderate (20–35 fps) Low on smooth backgrounds Gradient backgrounds, soft skies, architectural lines.
Deep Flow-Guided Transformers Slow (1–5 fps) Requires heavy optical flow Dynamic objects crossing across complex moving backgrounds.

5. Production OpenCV Python Implementation

Below is a production-ready snippet demonstrating OpenCV frame-by-frame inpainting for a designated coordinate mask:

import cv2 import numpy as np def inpaint_frame(frame: np.ndarray, x: int, y: int, w: int, h: int, radius: int = 3) -> np.ndarray: # 1. Generate single-channel 8-bit binary mask mask = np.zeros(frame.shape[:2], dtype=np.uint8) mask[y:y+h, x:x+w] = 255 # 2. Execute Alexandru Telea Fast Marching Inpainting cleaned = cv2.inpaint(frame, mask, inpaintRadius=radius, flags=cv2.INPAINT_TELEA) return cleaned

Conclusion

For practical video restoration—especially with high-resolution 1080p and 4K footage—bilinear spatial interpolation (as deployed in our Online Video Studio) and Telea Fast Marching provide the optimum balance of mathematical elegance, frame-to-frame stability, and lightning-fast hardware throughput.