US6192145B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for three-dimensional scene processing using parallax geometry of pairs of points
Assignee: Sarnoff Corp
•
Filing: 1997-02-11
•
Granted: 2001-02-20
•
Expires: 2017-02-11
Abstract
Abstract A method and apparatus for processing a geometric relationship between the image motion of pairs of points over multiple image frames representing a three-dimensional scene. This relationship is based on the parallax motion of points with respect to an arbitrary planar surface, and does not involve epipolar geometry. A constraint is derived over two frames for any pair of points, relating their projective structure (with respect to the plane) based only on their image coordinates and their parallax displacements. Similarly, a 3D-rigidity constraint between pairs of points over multiple frames is derived. Also disclosed are applications of these parallax-based constraints to solving three important problems in 3D scene analysis: (i) the recovery of 3D scene structure, (ii) the detection of moving objects in the presence of camera induced motion, and (iii) the synthesis of new camera views based on a given set of views. Moreover, this approach can handle difficult situations for 3D scene analysis, e.g., where there is only a small set of parallax vectors, and in the presence of independently moving objects.
Inventors
- 👤 Padmanabhan Anandan
- 👤 Michal Irani
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-24 21:17
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/20; G06T7/246; G01; G01B; G01B11/00; G01B11/02; G01B11/04; G06T15/00; G06T15/10; G06T7/50; G06T7/55; G06T7/579; G06T7/593; G06V; G06V10/00; G06V10/10; G06V10/12; G06V10/14; G06V10/147; H; H04; H04N; H04N13/00; H04N13/20; H04N13/204; H04N13/207; H04N13/221; G06T2207/00; G06T2207/10; G06T2207/10004; G06T2207/10012; G06V2201/00; G06V2201/12; H04N13/239; H04N2013/0074; H04N2013/0081
Claims Analysis
Total Claims: 25
Claim 1
Independent Claim
161 words
Method for image processing to detect a moving object comprising the steps of: a) receiving a plurality of two dimensional images representative of a scene, each image being taken at a respectively different time; b) computing a planar homography aligning a surface in the scene among the plurality of images; c) computing parallax vectors with respect to respective corresponding points in each of two images of the plurality of images; d) using the parallax vectors and additional information in the plurality of images, computing a parallax rigidity constraint for the plurality of images, the parallax rigidity constraint being independent of any epipole; e) applying the parallax rigidity constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax rigidity constraint; and f) using the generated information to perform moving object detection related processing on the received plurality of images.
Claim 2
Dependent Claim
161 words
Method for image processing to detect a moving object comprising the steps of: a) receiving a plurality of two dimensional images representative of a scene, each image being taken at a respectively different time; b) computing a planar homography aligning a surface in the scene among the plurality of images; c) computing parallax vectors with respect to respective corresponding points in each of two images of the plurality of images; d) using the parallax vectors and additional information in the plurality of images, computing a parallax rigidity constraint for the plurality of images, the parallax rigidity constraint being independent of any epipole; e) applying the parallax rigidity constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax rigidity constraint; and f) using the generated information to perform moving object detection related processing on the received plurality of images.
Claim 3
Dependent Claim
61 words
The method of claim 1 , wherein the parallax vectors are computed with respect to a pair of corresponding points in the two images and the additional information used to compute the parallax rigidity constraint is a pair of points in a third image of the plurality of images which corresponds to the pair of corresponding points in the two images.
Claim 4
Dependent Claim
46 words
The method of claim 1 , wherein the parallax vectors are computed with respect to a pair of corresponding points in the two images and the additional information used to compute the parallax rigidity constraint is a third corresponding point in each of the two images.
Claim 5
Dependent Claim
161 words
Method for image processing to recover scene structure comprising the steps of: a) receiving a plurality of two dimensional images representative of a scene, wherein each image is taken from an arbitrary camera position; b) computing a planar homography aligning a surface in the scene among the plurality of images; c) computing parallax vectors with respect to respective corresponding points in each of two images of the plurality of images; d) using the parallax vectors and additional information in the plurality of images, computing a parallax rigidity constraint for the plurality of images, the parallax rigidity constraint being independent of any epipole; e) applying the parallax rigidity constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax structure constraint; and f) using the generated information to perform recovery of scene structure related to the received plurality of images.
Claim 6
Dependent Claim
161 words
Method for image processing to recover scene structure comprising the steps of: a) receiving a plurality of two dimensional images representative of a scene, wherein each image is taken from an arbitrary camera position; b) computing a planar homography aligning a surface in the scene among the plurality of images; c) computing parallax vectors with respect to respective corresponding points in each of two images of the plurality of images; d) using the parallax vectors and additional information in the plurality of images, computing a parallax rigidity constraint for the plurality of images, the parallax rigidity constraint being independent of any epipole; e) applying the parallax rigidity constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax structure constraint; and f) using the generated information to perform recovery of scene structure related to the received plurality of images.
Claim 7
Dependent Claim
161 words
Method for image processing to generate new views comprising the steps of: a) receiving a plurality of two dimensional images representative of a scene, each image taken from an arbitrary camera position; b) computing a planar homography aligning a surface in the scene among the plurality of images; c) computing parallax vectors with respect to respective corresponding points in each of two images of the plurality of images; d) using the parallax vectors and additional information in the plurality of images, computing a parallax rigidity constraint for the plurality of images, the parallax rigidity constraint being independent of any epipole; e) applying the parallax rigidity constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax rigidity constraint; and f) using the generated information to perform new view generation related to and based on the received plurality of images.
Claim 8
Dependent Claim
161 words
Method for image processing to generate new views comprising the steps of: a) receiving a plurality of two dimensional images representative of a scene, each image taken from an arbitrary camera position; b) computing a planar homography aligning a surface in the scene among the plurality of images; c) computing parallax vectors with respect to respective corresponding points in each of two images of the plurality of images; d) using the parallax vectors and additional information in the plurality of images, computing a parallax rigidity constraint for the plurality of images, the parallax rigidity constraint being independent of any epipole; e) applying the parallax rigidity constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax rigidity constraint; and f) using the generated information to perform new view generation related to and based on the received plurality of images.
Claim 9
Dependent Claim
61 words
The method of claim 5 , wherein the parallax vectors are computed with respect to a pair of corresponding points in the two images and the additional information used to compute the parallax rigidity constraint is a pair of points in a third image of the plurality of images which corresponds to the pair of corresponding points in the two images.
Claim 10
Dependent Claim
46 words
The method of claim 5 , wherein the parallax vectors are computed with respect to a pair of corresponding points in the two images and the additional information used to compute the parallax rigidity constraint is a third corresponding point in each of the two images.
Claim 11
Dependent Claim
179 words
Apparatus for image processing comprising: a source of two dimensional images representative of a scene; a computer processor for processing the two dimensional images including: means for receiving a plurality of two dimensional images representative of a scene; means for computing a planar homography aligning a surface in the scene among the plurality of images; means for computing parallax vectors with respect to the planar homography of a plurality of points among the plurality of images; means for computing a parallax-related constraint directly from the positions of the plurality of points and the parallax vectors, the parallax-related constraint being independent of any epipole; means for applying the parallax-related constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax-related constraint; means for using the generated information for an image processing task related to the received plurality of images and generating an output related thereto; and an output device for presenting the output of the image processing task.
Claim 12
Dependent Claim
179 words
Apparatus for image processing comprising: a source of two dimensional images representative of a scene; a computer processor for processing the two dimensional images including: means for receiving a plurality of two dimensional images representative of a scene; means for computing a planar homography aligning a surface in the scene among the plurality of images; means for computing parallax vectors with respect to the planar homography of a plurality of points among the plurality of images; means for computing a parallax-related constraint directly from the positions of the plurality of points and the parallax vectors, the parallax-related constraint being independent of any epipole; means for applying the parallax-related constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax-related constraint; means for using the generated information for an image processing task related to the received plurality of images and generating an output related thereto; and an output device for presenting the output of the image processing task.
Claim 13
Dependent Claim
15 words
The apparatus of claim 8 , wherein the source of images includes a video camera.
Claim 14
Dependent Claim
15 words
The apparatus of claim 8 , wherein the source of images includes an infra-red camera.
Claim 15
Dependent Claim
15 words
The apparatus of claim 8 , wherein the source of images includes a storage device.
Claim 16
Dependent Claim
15 words
The apparatus of claim 11 , wherein the storage device includes a video tape recorder.
Claim 17
Dependent Claim
14 words
The apparatus of claim 11 , wherein the storage device includes a disk drive.
Claim 18
Dependent Claim
163 words
A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps of: a) receiving, in digital form, a plurality of two dimensional images representative of a scene; b) computing a planar homography aligning a surface in the scene among the plurality of images; c) computing parallax vectors with respect to the planar homography for a plurality of points among the plurality of images; d) computing a parallax-related constraint directly from the positions of the plurality of points and the parallax vectors, the parallax-related constraint being independent of any epipole; e) applying the parallax-related constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax-related constraint; and f) using the generated information for an image processing task related to the received plurality of images.
Claim 19
Dependent Claim
163 words
A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps of: a) receiving, in digital form, a plurality of two dimensional images representative of a scene; b) computing a planar homography aligning a surface in the scene among the plurality of images; c) computing parallax vectors with respect to the planar homography for a plurality of points among the plurality of images; d) computing a parallax-related constraint directly from the positions of the plurality of points and the parallax vectors, the parallax-related constraint being independent of any epipole; e) applying the parallax-related constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax-related constraint; and f) using the generated information for an image processing task related to the received plurality of images.
Claim 20
Dependent Claim
17 words
The computer readable medium of claim 14 , wherein the parallax-related constraint is a parallax structure constraint.
Claim 21
Dependent Claim
17 words
The computer readable medium of claim 14 , wherein the parallax-related constraint is a parallax rigidity constraint.
Claim 22
Dependent Claim
61 words
The medium of claim 16 , wherein the parallax vectors are computed with respect to a pair of corresponding points in the two images and the additional information used to compute the parallax rigidity constraint is a pair of points in a third image of the plurality of images which corresponds to the pair of corresponding points in the two images.
Claim 23
Dependent Claim
46 words
The medium of claim 16 , wherein the parallax vectors are computed with respect to a pair of corresponding points in the two images and the additional information used to compute the parallax rigidity constraint is a third corresponding point in each of the two images.
Claim 24
Dependent Claim
156 words
Method for image processing comprising the steps of: a) receiving a plurality of two dimensional images representative of a scene, the images taken from arbitrary camera positions; b) computing a parallax-related constraint for a plurality of points within the plurality of images including the steps of: i) warping the plurality of images to compensate for effects of scaling and rotation around the camera positions; ii) computing parallax vectors for the plurality of points; and iii) computing the parallax-related constraint directly from the positions of the plurality of points and the parallax vectors, the parallax-related constraint being independent of any epipole; c) applying the parallax-related constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax-related constraint; and d) using the generated information for an image processing task related to the received plurality of images.
Claim 25
Dependent Claim
156 words
Method for image processing comprising the steps of: a) receiving a plurality of two dimensional images representative of a scene, the images taken from arbitrary camera positions; b) computing a parallax-related constraint for a plurality of points within the plurality of images including the steps of: i) warping the plurality of images to compensate for effects of scaling and rotation around the camera positions; ii) computing parallax vectors for the plurality of points; and iii) computing the parallax-related constraint directly from the positions of the plurality of points and the parallax vectors, the parallax-related constraint being independent of any epipole; c) applying the parallax-related constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax-related constraint; and d) using the generated information for an image processing task related to the received plurality of images.
References Cited (Backward Citations)
US3715482A
Cited
US4573191A
Cited
US4654872A
Cited
US4669048A
Cited
US4982438A
Cited
US5448687A
Cited
US4985856A
Cited
US4908573A
Cited
US5076687A
Cited
US5490133A
Cited
US5249035A
Cited
US5528194A
Cited
US5185671A
Cited
US5577130A
Cited
US5390024A
Cited
US5680474A
Cited
US5495576A
Cited
US5623528A
Cited
US5764871A
Cited
US5586231A
Cited
US5473364A
Cited
US5703961A
Cited
US5644651A
Cited
US5821943A
Cited
US5852672A
Cited
US5696551A
Cited
US5694531A
Cited
US5706416A
Cited
US5828793A
Cited
US5986668A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2004-02-20 | 2004-08-21 | 2005-02-20 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2008-02-20 | 2008-08-21 | 2009-02-20 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2012-02-20 | 2012-08-21 | 2013-02-20 | $7700.00 | Window Closed / Lapsed |