US6002998A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Fast, efficient hardware mechanism for natural language determination
Assignee: International Business Machines Corp
•
Filing: 1996-09-30
•
Granted: 1999-12-14
•
Expires: 2016-09-30
Abstract
Abstract A language in which a document is written is identified by comparing the words of a document to the most frequently used words in a plurality of candidate languages. The words are stored in a plurality of sets of word tables, each set of word tables for storing a selected set of most frequently used words in a respective candidate language according to letter pairs in the words. In the preferred embodiment, each of the word tables is an N×N bit table, where each bit represents a given letter pair at a particular place in one of the most frequently used words in a respective candidate language. A set of table access registers, is used for accessing a respective set of word tables to compare words from the document to words stored in the word tables; each table access register accesses word tables for a respective candidate language. One or more word counting registers count a number of matches for a respective candidate language. A comparator selects a candidate language which corresponds to the word counting register having the highest count as the language in which the document is written.
Inventors
- 👤 Michael John Martino
- 👤 Robert Charles Paulsen, Jr.
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-31 03:52
CPC Classifications
G; G06; G06F; G06F40/00; G06F40/20; G06F40/263
Claims Analysis
Total Claims: 20
Claim 1
Independent Claim
169 words
A system for identifying a language in which a document is written, comprising: a plurality of sets of word tables, each set of word tables for storing a selected set of most frequently used words in a respective candidate language according to letter pairs in the words in wherein each of the word tables is an N×N bit table, wherein each bit represents a given letter pair at a particular place in one of the most frequently used words in a respective candidate language; a set of table access registers, each table access register for a respective candidate language, for accessing a respective set of word tables to compare words from the document to words stored in the word tables; a set of word counting registers, each word counting register for counting a number of matches for a respective candidate language; and a comparator for selecting a candidate language which corresponds to the word counting register having the highest count as the language in which the document is written.
Claim 2
Independent Claim
169 words
A system for identifying a language in which a document is written, comprising: a plurality of sets of word tables, each set of word tables for storing a selected set of most frequently used words in a respective candidate language according to letter pairs in the words in wherein each of the word tables is an N×N bit table, wherein each bit represents a given letter pair at a particular place in one of the most frequently used words in a respective candidate language; a set of table access registers, each table access register for a respective candidate language, for accessing a respective set of word tables to compare words from the document to words stored in the word tables; a set of word counting registers, each word counting register for counting a number of matches for a respective candidate language; and a comparator for selecting a candidate language which corresponds to the word counting register having the highest count as the language in which the document is written.
Claim 3
Dependent Claim
43 words
The system as recited in claim 1 wherein the sets of tables comprise a plurality of tableaus of tables, each tableau for storing respective length words and further comprising a table access register for accessing word tables in each respective tableau of tables.
Claim 4
Dependent Claim
39 words
The system as recited in claim 2 further comprising: length determining means for determining lengths of words in the document; truncating means for truncating words in the document to a maximum length of words stored in the word tables.
Claim 5
Dependent Claim
34 words
The system as recited in claim 2 further comprising length determining means for determining lengths of words in the document wherein only words having lengths equal to those stored in the tableaus are compared.
Claim 6
Dependent Claim
35 words
The system as recited in claim 4 wherein the bits stored in the word tables for the letter pairs of the word from the document are ANDed together to determine whether there is a match.
Claim 7
Independent Claim
115 words
A system for identifying a language in which a document is written, comprising: a set of word tables for storing a selected set of most frequently used words in a candidate language according to letter pairs in the words, wherein each of the word tables is an N×N bit table, wherein each bit represents a given letter pair at a particular place in one of the most frequently used words in a respective candidate language; a table access register for accessing a respective set of word tables to compare words from the document to words stored in the word tables; and a word counting register for counting a number of matches for a candidate language.
Claim 8
Independent Claim
115 words
A system for identifying a language in which a document is written, comprising: a set of word tables for storing a selected set of most frequently used words in a candidate language according to letter pairs in the words, wherein each of the word tables is an N×N bit table, wherein each bit represents a given letter pair at a particular place in one of the most frequently used words in a respective candidate language; a table access register for accessing a respective set of word tables to compare words from the document to words stored in the word tables; and a word counting register for counting a number of matches for a candidate language.
Claim 9
Dependent Claim
51 words
The system as recited in claim 6 wherein a plurality of passes through the document are made, each pass loading the selected set of words in the set of word tables for a respective candidate language, the language of the document determined by the pass having the greatest number of matches.
Claim 10
Dependent Claim
43 words
The system as recited in claim 6 wherein words of different lengths are stored in the set of word tables and a blank character is used for words of a length less than a maximum length stored in the set of word tables.
Claim 11
Dependent Claim
34 words
The system as recited in claim 6 wherein at least one set of most frequently used words for a candidate language is selected to reduce strong aliasing with words of a different candidate language.
Claim 12
Dependent Claim
17 words
The system as recited in claim 6 wherein the word tables are stored in content addressable memory.
Claim 13
Dependent Claim
64 words
The system as recited in claim 6 wherein at least one of the candidate languages is an ideographic language which uses a single word table loaded with a double byte character set (DBCS) representation of the ideographs; a comparator for selecting a candidate language which corresponds to the word counting register having the highest count as the language in which the document is written.
Claim 14
Independent Claim
175 words
A method for identifying a language in which a document is written, comprising the steps of: storing a plurality of sets of word tables, each set of word tables for storing a selected set of most frequently used words in a respective candidate language according to letter pairs in the words, wherein each of the word tables is an N×N bit table, wherein each bit represents a given letter pair at a particular place in one of the most frequently used words in a respective candidate language; accessing the word tables with a set of table access registers, each table access register for accessing a respective set of word tables to compare words from the document to words stored in the word tables; counting a number of matches with a set of word counting registers, each word counting register for counting a number of matches for a respective candidate language; and selecting a candidate language which corresponds to the word counting register having the highest count as the language in which the document is written.
Claim 15
Independent Claim
175 words
A method for identifying a language in which a document is written, comprising the steps of: storing a plurality of sets of word tables, each set of word tables for storing a selected set of most frequently used words in a respective candidate language according to letter pairs in the words, wherein each of the word tables is an N×N bit table, wherein each bit represents a given letter pair at a particular place in one of the most frequently used words in a respective candidate language; accessing the word tables with a set of table access registers, each table access register for accessing a respective set of word tables to compare words from the document to words stored in the word tables; counting a number of matches with a set of word counting registers, each word counting register for counting a number of matches for a respective candidate language; and selecting a candidate language which corresponds to the word counting register having the highest count as the language in which the document is written.
Claim 16
Dependent Claim
44 words
The method as recited in claim 12 wherein the sets of tables comprise a plurality of tableaus of tables, each tableau for storing respective length words and a separate table access register is used for accessing word tables in each respective tableau of tables.
Claim 17
Dependent Claim
35 words
The method as recited in claim 13 further comprising the steps of: determining lengths of words in the document; truncating words in the document to a maximum length of words stored in the word tables.
Claim 18
Dependent Claim
31 words
The method as recited in claim 13 further comprising the steps of: determining lengths of words in the document; comparing words having lengths equal to those stored in the respective tableau.
Claim 19
Dependent Claim
37 words
The method as recited in claim 15 further comprising the step of ANDing the bits stored in the word tables for the letter pairs of the word from the document to determine whether there is a match.
Claim 20
Dependent Claim
44 words
The method as recited in claim 12 further comprising the steps of: determining whether the number of matches for the respective candidate languages has diverged by a predetermined amount; and responsive to the determination of the predetermined amount of divergence, halting the comparison process.
References Cited (Backward Citations)
US4610025A
Cited
US4829580A
Cited
WO1991002347A1
Cited
US5062143A
Cited
US5182708A
Cited
US5392419A
Cited
US5371807A
Cited
US5418951A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-12-13 | 2003-06-14 | 2003-12-14 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-12-13 | 2007-06-14 | 2007-12-14 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-12-13 | 2011-06-14 | 2011-12-14 | $7700.00 | Window Closed / Lapsed |