US7194134B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Hierarchical, probabilistic, localized, semantic image classifier
Assignee: Microsoft Technology Licensing LLC
•
Filing: 2001-01-02
•
Granted: 2007-03-20
•
Expires: 2021-01-02
Abstract
Abstract Described herein is a technology for semantically classifying areas of an image (and/or the images themselves) as one of a number of multiple discriminating categories. More particularly, the technology employs one or more hierarchical, probabilistic techniques for performing such classification. Such technology is particularly useful in fields of image classification and image retrieval. The architecture of such technology employs multiple hierarchical layers. The architecture is based on modeling class likelihoods at each of such layers separately and then combining these to form an overall estimate of the posterior, conditioned on the data. The task of combining the estimated class likelihoods at each layer is made more efficient by assuming statistical independence between layers. This abstract itself is not intended to limit the scope of this patent. The scope of the present invention is pointed out in the appending claims.
Inventors
- 👤 David Benedict Bradshaw
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-25 00:37
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/50; G06F16/58; G06F16/583; G06F16/5838; G06V; G06V30/00; G06V30/10; G06V30/24; G06V30/248
Claims Analysis
Total Claims: 22
Claim 1
Independent Claim
110 words
A method of semantically classifying an image, the method comprising: forming a group of hierarchical layered blocks from the image, each block within the group being only partially coextensive with the other blocks of the group; determining a posterior estimate of class membership of the group of hierarchical layered blocks, the estimate being based upon class likelihoods of the hierarchical layered blocks in the group, such likelihood being conditioned on data extracted from hierarchical layered blocks in the group; semantically classifying a portion of such image based upon the posterior estimate of class membership conditioned on the data extracted from the group of hierarchical layered blocks local to such portion.
Claim 2
Independent Claim
110 words
A method of semantically classifying an image, the method comprising: forming a group of hierarchical layered blocks from the image, each block within the group being only partially coextensive with the other blocks of the group; determining a posterior estimate of class membership of the group of hierarchical layered blocks, the estimate being based upon class likelihoods of the hierarchical layered blocks in the group, such likelihood being conditioned on data extracted from hierarchical layered blocks in the group; semantically classifying a portion of such image based upon the posterior estimate of class membership conditioned on the data extracted from the group of hierarchical layered blocks local to such portion.
Claim 3
Dependent Claim
47 words
A method as recited in claim 1 , wherein the determining comprises: determining an estimated class likelihood of each block in the group of hierarchical layered blocks; combining the estimated class likelihoods of hierarchical layered blocks in the group into a posterior estimate of semantic class membership.
Claim 4
Dependent Claim
18 words
A method as recited in claim 1 , wherein the determining comprises extracting low level features from blocks.
Claim 5
Dependent Claim
16 words
A method as recited in claim 3 , wherein the low-level features comprise color and textures.
Claim 6
Dependent Claim
19 words
A method as recited in claim 4 , wherein the extracting extracts color low-level features by an Ohta decomposition.
Claim 7
Dependent Claim
21 words
A method as recited in claim 4 , wherein the extracting extracts texture low level features by a complex wavelet transform.
Claim 8
Dependent Claim
17 words
A method as recited in claim 1 , wherein the class likelihoods are estimated in one-dimensional space.
Claim 9
Dependent Claim
20 words
A method as recited in claim 1 , wherein a classification that results from the semantically classifying step is binary.
Claim 10
Dependent Claim
46 words
A method as recited in claim 1 , wherein a classification that results from the semantically classifying step is selected from a group consisting of these binary sets: sky or not-sky; grass or not-grass; natural or man-made; inside or outside; hair or not-hair; face or not-face.
Claim 11
Dependent Claim
39 words
An image retrieval method comprising: a semantic image classification method as recited in claim 1 ; searching for images matching a given query in an image library containing images having portions thereof classified using the semantic image classification method.
Claim 12
Dependent Claim
22 words
A computer-readable storage medium having computer-executable instructions that, when executed by a computer, performs the method as recited in claim 1 .
Claim 13
Dependent Claim
117 words
A semantic image classification system, comprising: a block analyzer configured to extract low-level features of blocks of an image and estimate class likelihoods for each block, a class being a discriminating semantic classification and a block being a portion of the image; a combiner configured to generate a posterior estimate of class membership based on combining estimated class likelihoods of hierarchical sets of blocks, a hierarchical set of blocks being a hierarchical organized and associated blocks that are only partially coextensive with one another; an image classifier configured to determine and classify one of multiple discriminating semantic classifications to localized portions of the image based upon the posterior estimate of class membership of blocks comprising such portions.
Claim 14
Dependent Claim
117 words
A semantic image classification system, comprising: a block analyzer configured to extract low-level features of blocks of an image and estimate class likelihoods for each block, a class being a discriminating semantic classification and a block being a portion of the image; a combiner configured to generate a posterior estimate of class membership based on combining estimated class likelihoods of hierarchical sets of blocks, a hierarchical set of blocks being a hierarchical organized and associated blocks that are only partially coextensive with one another; an image classifier configured to determine and classify one of multiple discriminating semantic classifications to localized portions of the image based upon the posterior estimate of class membership of blocks comprising such portions.
Claim 15
Dependent Claim
57 words
A system as recited in claim 12 further comprising a hierarchy definer configured to subdivide the image into multiple hierarchical sets of blocks, each successively lower layer of a hierarchical set of blocks comprising one or more blocks which are smaller than and at least partially coextensive with one or more blocks in a layer immediately above.
Claim 16
Dependent Claim
16 words
A system as recited in claim 12 , wherein the low-level features comprise color and textures.
Claim 17
Dependent Claim
20 words
A system as recited in claim 14 , wherein the block analyzer extracts color low-level features by an Ohta decomposition.
Claim 18
Dependent Claim
21 words
A system as recited in claim 14 , wherein the block analyzer extracts texture low-level features by a complex wavelet transform.
Claim 19
Dependent Claim
18 words
A system as recited in claim 12 , wherein the class likelihoods are estimated in one dimensional space.
Claim 20
Dependent Claim
41 words
A system as recited in claim 12 , wherein the discriminating semantic classifications are selected from a group consisting of these binary sets: sky or not-sky; grass or not-grass; natural or man-made; inside or outside; hair or not-hair; face or not-face.
Claim 21
Dependent Claim
45 words
An image retrieval system comprising: a semantic image classification system as recited in claim 12 ; a semantic image querier configured to search for images matching a given query in an image library containing images having portions thereof classified using the semantic image classification system.
Claim 22
Dependent Claim
32 words
A computer-readable storage medium having stored thereon a data structure, comprising an image library containing images having portions thereof classified using a semantic image classification system as recited in claim 12 .
References Cited (Backward Citations)
US5872864A
Cited
US5822452A
Cited
US5995651A
Cited
US6317517B1
Cited
US6594383B1
Cited
US6678421B1
Cited
US20020122596A1
Cited
US7020330B2
Cited
US6826300B2
Cited
US6801672B1
Cited
US7035467B2
Cited
US7039239B2
Cited
US7043474B2
Cited
US20050105776A1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2010-03-19 | 2010-09-18 | 2011-03-20 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2014-03-19 | 2014-09-18 | 2015-03-20 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2018-03-19 | 2018-09-18 | 2019-03-20 | $7700.00 | Window Closed / Lapsed |