Patent US5581276A successfully refreshed from Google Patents.
US5581276A
Expired / Lapsed
US
⏱️ 0.0 years remaining
3D human interface apparatus using motion recognition based on dynamic image processing
Assignee: Toshiba Corp
•
Filing: 1993-09-08
•
Granted: 1996-12-03
•
Expires: 2013-09-08
Abstract
Abstract A 3D human interface apparatus using a motion recognition based on a dynamic image processing in which the motion of an operator operated object as an imaging target can be recognized accurately and stably. The apparatus includes: an image input unit for entering a plurality of time series images of an object operated by the operator into a motion representing a command; a feature point extraction unit for extracting at least four feature points including at least three reference feature points and one fiducial feature point on the object, from each of the images; a motion recognition unit for recognizing the motion of the object by calculating motion parameters, according to an affine transformation determined from changes of positions of the reference feature points on the images, and a virtual parallax for the fiducial feature point expressing a difference between an actual position change on the images and a virtual position change according to the affine transformation; and a command input unit for inputting the command indicated by the motion of the object recognized by the motion recognition unit.
Inventors
- 👤 Roberto Cipolla
- 👤 Yasukazu Okamoto
- 👤 Yoshinori Kuno
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-31 02:11
CPC Classifications
G; G06; G06F; G06F3/00; G06F3/01; G06F3/017; G06T; G06T7/00; G06T7/20; G06T7/246; G06V; G06V40/00; G06V40/20; G06V40/28
Claims Analysis
Total Claims: 18
Claim 1
Independent Claim
155 words
A 3D human interface apparatus for facilitating 3D pointing and controlling command inputs by an operator, comprising: image input means for entering a plurality of time series images of an object operated by the operator into a motion representing a command; feature point extraction means for extracting at least four feature points including at least three reference feature points and one fiducial feature point on the object, from each of said images; motion recognition means for recognizing the motion of the object by calculating motion parameters, according to an arline transformation determined from changes of positions of the reference feature points on said images, and a virtual parallax for the fiducial feature point expressing a difference between an actual position change on said images and a virtual position change according to the affine transformation; and command input means for inputting the command indicated by the motion of the object recognized by the motion recognition means.
Claim 2
Dependent Claim
155 words
A 3D human interface apparatus for facilitating 3D pointing and controlling command inputs by an operator, comprising: image input means for entering a plurality of time series images of an object operated by the operator into a motion representing a command; feature point extraction means for extracting at least four feature points including at least three reference feature points and one fiducial feature point on the object, from each of said images; motion recognition means for recognizing the motion of the object by calculating motion parameters, according to an arline transformation determined from changes of positions of the reference feature points on said images, and a virtual parallax for the fiducial feature point expressing a difference between an actual position change on said images and a virtual position change according to the affine transformation; and command input means for inputting the command indicated by the motion of the object recognized by the motion recognition means.
Claim 3
Dependent Claim
39 words
The apparatus of claim 1, wherein the motion recognition means calculates the motion parameters for a relative motion of an observer taking said images with respect to the object, according to the affine transformation which represents the relative motion.
Claim 4
Dependent Claim
33 words
The apparatus of claim 2, wherein the motion recognition means calculates the motion parameters by eliminating factors not due to the motion of the object from invariant features derived from the affine transformation.
Claim 5
Dependent Claim
38 words
The apparatus of claim 3, wherein the motion recognition means eliminates the factors not due to the motion of the object which are obtained by using a translational motion direction on said images determined from the virtual parallax.
Claim 6
Dependent Claim
89 words
The apparatus of claim 1, wherein the feature point extraction means also extracts a multiplicity of feature points distributed over an entire imaging field of said images, and which further comprises: object detection means for detecting the object in said images by calculating translational motion directions on said images for said multiplicity of feature points, plotting a distribution plot of the translational motion directions over the entire imaging field, and separating an area having identical and distinctive translational motion directions distributed therein in the distribution plot as the object.
Claim 7
Dependent Claim
63 words
The apparatus of claim 5, wherein the object detection means calculates the translational motion directions according to virtual parallaxes for said multiplicity of feature points, each virtual parallax expressing a difference between an actual position change on said images and a virtual position change according to an affine transformation determined from changes of positions of said multiplicity of feature points on said images.
Claim 8
Dependent Claim
76 words
The apparatus of claim 1, wherein the feature point extraction means also extracts a multiplicity of feature points distributed over the object in said images, and which further comprises: structure extraction means for extracting a structure of the object by determining relative positions of said multiplicity of feature points in a depth direction on said images according to a ratio of the virtual parallaxes for each two closely located ones of said multiplicity of feature points.
Claim 9
Dependent Claim
34 words
The apparatus of claim 7, wherein the structure extraction means also determines a texture data indicating a gradient of the object surface according to the determined relative positions of said multiplicity of feature points.
Claim 10
Dependent Claim
44 words
The apparatus of claim 1, wherein the command input means determines the command indicated by the motion by generating a gesture pattern expressing the motion and making a matching of the generated gesture pattern with one of predetermined gesture patterns corresponding to predetermined commands.
Claim 11
Dependent Claim
31 words
The apparatus of claim 1, further comprising: display means for displaying a 3D model which moves in accordance with the motion indicated by the command inputted by the command input means.
Claim 12
Dependent Claim
19 words
The apparatus of claim 10, wherein the display means also displays said images in superposition to the 3D model.
Claim 13
Dependent Claim
159 words
A 3D human interface apparatus for facilitating 3D pointing and controlling command inputs by an operator, comprising: image input means for entering a plurality of time series images of an object operated by the operator into a motion representing a command; feature point extraction means for extracting a multiplicity of feature points distributed over an entire imaging field of said images; object detection means for detecting the object in said images by calculating translational motion directions on said images for said multiplicity of feature points, plotting a distribution plot of the translational motion directions over the entire imaging field, and separating an area having identical and distinctive translational motion directions distributed therein in the distribution plot as the object; motion recognition means for recognizing the motion of said area separated as the object by image processing said images; and command input means for inputting the command indicated by the motion of the object recognized by the motion recognition means.
Claim 14
Dependent Claim
159 words
A 3D human interface apparatus for facilitating 3D pointing and controlling command inputs by an operator, comprising: image input means for entering a plurality of time series images of an object operated by the operator into a motion representing a command; feature point extraction means for extracting a multiplicity of feature points distributed over an entire imaging field of said images; object detection means for detecting the object in said images by calculating translational motion directions on said images for said multiplicity of feature points, plotting a distribution plot of the translational motion directions over the entire imaging field, and separating an area having identical and distinctive translational motion directions distributed therein in the distribution plot as the object; motion recognition means for recognizing the motion of said area separated as the object by image processing said images; and command input means for inputting the command indicated by the motion of the object recognized by the motion recognition means.
Claim 15
Dependent Claim
59 words
The apparatus of claim 12, wherein the object detection means calculates the translational motion directions according to virtual parallaxes of the feature points, each virtual parallax expressing a difference between an actual position change on said images and a virtual position change according to an arline transformation determined from changes of positions of the feature points on said images.
Claim 16
Dependent Claim
212 words
A 3D human interface apparatus for facilitating 3D pointing and controlling command inputs by an operator, comprising: image input means for entering a plurality of images of an object operated by the operator into a motion representing a command; feature point extraction means for extracting a multiplicity of feature points distributed over the object in said images; structure extraction means for extracting a structure of the object by determining relative positions of said multiplicity of feature points in a depth direction on said images according to a ratio of virtual parallaxes for each two closely located ones of said multiplicity of feature points, each virtual parallax expressing a difference between an actual position change on said images and a virtual position change according to an arline transformation determined from changes of positions of the feature points on said images; motion recognition means for recognizing the motion of the object by image processing said images; command input means for inputting the command indicated by the motion of the object recognized by the motion recognition means; and display means for displaying a 3D model having the structure of the object as extracted by the structure extraction means which moves in accordance with the motion indicated by the command inputted by the command input means.
Claim 17
Dependent Claim
212 words
A 3D human interface apparatus for facilitating 3D pointing and controlling command inputs by an operator, comprising: image input means for entering a plurality of images of an object operated by the operator into a motion representing a command; feature point extraction means for extracting a multiplicity of feature points distributed over the object in said images; structure extraction means for extracting a structure of the object by determining relative positions of said multiplicity of feature points in a depth direction on said images according to a ratio of virtual parallaxes for each two closely located ones of said multiplicity of feature points, each virtual parallax expressing a difference between an actual position change on said images and a virtual position change according to an arline transformation determined from changes of positions of the feature points on said images; motion recognition means for recognizing the motion of the object by image processing said images; command input means for inputting the command indicated by the motion of the object recognized by the motion recognition means; and display means for displaying a 3D model having the structure of the object as extracted by the structure extraction means which moves in accordance with the motion indicated by the command inputted by the command input means.
Claim 18
Dependent Claim
34 words
The apparatus of claim 14, wherein the structure extraction means also determines a texture data indicating a gradient of the object surface according to the determined relative positions of said multiplicity of feature points.
References Cited (Backward Citations)
US4685143A
Cited
US4843568A
Cited
US4977603A
Cited
US5252951A
Cited
JPH036782A
Cited
JPH036780A
Cited
US5263097A
Cited
JP3006780U
Cited
JP3006782U
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1999-12-03 | 2000-06-03 | 2000-12-03 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2003-12-03 | 2004-06-03 | 2004-12-03 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2007-12-03 | 2008-06-03 | 2008-12-03 | $7700.00 | Window Closed / Lapsed |