US20040221237A1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Methods and apparatuses for interactive similarity searching, retrieval and browsing of video
Assignee: Fujifilm Business Innovation Corp
•
Filing: 2004-06-03
•
Granted: —
•
Expires: 2024-06-03
Abstract
Abstract Methods for interactive selecting video queries consisting of training images from a video for a video similarity search and for displaying the results of the similarity search are disclosed. The user selects a time interval in the video as a query definition of training images for training an image class statistical model. Time intervals can be as short as one frame or consist of disjoint segments or shots. A statistical model of the image class defined by the training images is calculated on-the-fly from feature vectors extracted from transforms of the training images. For each frame in the video, a feature vector is extracted from the transform of the frame, and a similarity measure is calculated using the feature vector and the image class statistical model. The similarity measure is derived from the likelihood of a Gaussian model producing the frame. The similarity is then presented graphically, which allows the time structure of the video to be visualized and browsed. Similarity can be rapidly calculated for other video files as well, which enables content-based retrieval by example. A content-aware video browser featuring interactive similarity measurement is presented. A method for selecting training segments involves mouse click-and-drag operations over a time bar representing the duration of the video; similarity results are displayed as shades in the time bar. Another method involves selecting periodic frames of the video as endpoints for the training segment.
Inventors
- 👤 Jonathan Foote
- 👤 Andreas Girgensohn
- 👤 Lynn Wilcox
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-01 16:14
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/70; G06F16/78; G06F16/783; G06F16/7847; G06F16/7864; G06V; G06V20/00; G06V20/40; G06V20/48; G06F16/73; G06F16/732; G06F16/7328; G06F16/738
Claims Analysis
Total Claims: 16
Claim 1
Independent Claim
49 words
1 . A method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; and training a statistical model using the reduced feature vectors.
Claim 2
Dependent Claim
49 words
1 . A method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; and training a statistical model using the reduced feature vectors.
Claim 3
Dependent Claim
39 words
2 . A method as in claim 1 , further comprising the steps of: for each frame of the video, obtaining a reduced feature vector; and computing a similarity score using the reduced feature vector and the statistical model.
Claim 4
Dependent Claim
27 words
3 . A method as in claim 2 , further comprising the step of: segmenting the video into similar and non-similar segments based upon the similarity scores.
Claim 5
Dependent Claim
56 words
4 . A method as in claim 2 , wherein the steps of obtaining reduced feature vectors corresponding to frames of the training video segment and, for each frame of the video, obtaining a reduced feature vector are performed by retrieval of the reduced feature vectors from a precomputed feature vector database corresponding to the video.
Claim 6
Dependent Claim
44 words
5 . A method as in claim 2 , wherein the steps of obtaining reduced feature vectors corresponding to frames of the training video segment and, for each frame of the video, obtaining a reduced feature vector are performed transforming frames of the video.
Claim 7
Dependent Claim
39 words
6 . A method as in claim 1 , wherein each reduced feature vector corresponding to a frame of the training video segment includes features representing chromatic components of the frame and features representing luminance components of the frame.
Claim 8
Dependent Claim
25 words
7 . A method as in claim 6 , wherein each reduced feature vector includes fewer features representing chromatic components than features representing luminance components.
Claim 9
Dependent Claim
46 words
8 . A method as in claim 1 , wherein each reduced feature vector corresponding to a frame of the training video segment includes features representing red components of the frame, features representing green components of the frame, and features representing blue components of the frame.
Claim 10
Dependent Claim
39 words
9 . A method as in claim 3 , wherein the step of segmenting the video into similar and non-similar segments based upon the similarity scores is performed by comparing the similarity scores to an interactively defined similarity threshold.
Claim 11
Dependent Claim
88 words
10 . A computer system, comprising: a processor; a user interface; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; and training a statistical model using the reduced feature vectors.
Claim 12
Dependent Claim
88 words
10 . A computer system, comprising: a processor; a user interface; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; and training a statistical model using the reduced feature vectors.
Claim 13
Dependent Claim
137 words
11 . A computer system, comprising: a display; a user interface; a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of presenting a video within a video browser, comprising the steps of: providing a display window for viewing the video; displaying a time bar within the video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each frame in the video to the training video segment using shades of the time bar at positions corresponding to each frame to indicate the similarity measure.
Claim 14
Dependent Claim
137 words
11 . A computer system, comprising: a display; a user interface; a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of presenting a video within a video browser, comprising the steps of: providing a display window for viewing the video; displaying a time bar within the video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each frame in the video to the training video segment using shades of the time bar at positions corresponding to each frame to indicate the similarity measure.
Claim 15
Dependent Claim
113 words
12 . A computer system, comprising: a display; a user interface; a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of presenting a video within a web-based interface, comprising the steps of: displaying periodic frames of the video separated by a predetermined time interval; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each displayed periodic frame in the video to the training video segment using shades surrounding each displayed periodic frame to indicate the similarity measure.
Claim 16
Dependent Claim
113 words
12 . A computer system, comprising: a display; a user interface; a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of presenting a video within a web-based interface, comprising the steps of: displaying periodic frames of the video separated by a predetermined time interval; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each displayed periodic frame in the video to the training video segment using shades surrounding each displayed periodic frame to indicate the similarity measure.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
Maintenance schedule is calculated for granted US utility patents.