Segmentation-based method for motion-compensated frame interpolation

Claim 3

The method claimed in claim 1, further comprising the steps of: e) identifying a second foreground for the two successive frames; f) treating the first foreground as the background for the second foreground; and g) applying steps b) through d) to the second foreground and the first background.

Claim 4

The method claimed in claim 1, wherein step a) of identifying a foreground and a background is performed by a user employing a graphical user interface.

Claim 5

The method claimed in claim 1, wherein step b) of estimating a velocity vector field for the foreground, and a velocity vector field for the background includes the steps of: i) computing a velocity vector field for a coarsest resolution level; ii) project the velocity vector field from the previously computed resolution level onto a next finer resolution level; iii) computing a velocity vector field at the finer resolution level; iv) repeating steps b) and c) until a final finest resolution level is reached; and v) interpolating the velocity vector field at the finest resolution level to the pixel resolution level.

Claim 6

The method claimed in claim 1, wherein step d) of obtaining the intermediate image by using the foreground velocity vector field to perform motion compensated interpolation for the foreground in the intermediate image and using the background velocity vector field to perform motion compensated interpolation for the background in the intermediate image, further comprises the steps of: i) for each pixel in the intermediate image, if it is a foreground pixel, using the foreground velocity vector at that pixel to find the corresponding points in the two frames, and computing the time weighted average of the two points; and ii) if it is a background pixel, finding the corresponding points in the two frames using the background velocity vector and if they are both background points, computing the time weighted average of the two points; if only one of the two corresponding points is a background point, using the value of the background point; and if they are both foreground point determining the value of the pixel from the values of neighboring background pixels in the intermediate image.