Translation of a neural network into a rule-based expert system

Claim 1

A method comprising the steps in the following order of: performing an initial backpropagation to train a neural network with plurality of layers having: an input layer of input units; and different layers of processing elements including an output layer having output processing elements and at least one hidden layer having hidden processing elements, each processing element having input weights; comparing hidden processing elements so as to find hidden processing elements common to at least one given layer of said plurality of layers and which have similar patterns of information; replacing the similar pattern hidden processing elements with a single hidden processing element having information from one or more of the similar pattern hidden processing elements until each hidden processing element in each given layer has a distinctive pattern of information; and performing a further backpropagation on the neural network.

Claim 2

A method comprising the steps in the following order of: performing an initial backpropagation to train a neural network with plurality of layers having: an input layer of input units; and different layers of processing elements including an output layer having output processing elements and at least one hidden layer having hidden processing elements, each processing element having input weights; comparing hidden processing elements so as to find hidden processing elements common to at least one given layer of said plurality of layers and which have similar patterns of information; replacing the similar pattern hidden processing elements with a single hidden processing element having information from one or more of the similar pattern hidden processing elements until each hidden processing element in each given layer has a distinctive pattern of information; and performing a further backpropagation on the neural network.

Claim 7

A method comprising the steps in the following order of: performing an initial backpropagation to train a neural network with plurality of layers having: an input layer of input units; and different layers of processing elements including an output layer having output processing elements and at least one hidden layer having hidden processing elements, each processing element having input weights; finding hidden processing elements common to at least one given layer of said plurality of layers and having similar patterns of information; replacing the similar pattern hidden processing elements with a single hidden processing element having information from one or more of the similar pattern hidden processing elements until each hidden processing element in each given layer has a distinctive pattern of information; and performing a further backpropagation on the neural network; and further comprising, following said further backpropagation: translating knowledge in the output layer and each hidden layer of the neural network into a layer set of rules, there being a layer set of rules corresponding to each layer of said output layer and each hidden layer, there being undefined hidden concepts embodied in the layer sets of rules; rewriting rules from the layer set of rules by reformulating rules from one layer in terms of rules of another layer to eliminate undefined hidden concepts and thereby generate a rewritten set of rules, the rewritten set of rules comprising rules in IF-THEN form; and storing said rewritten rule set in a memory.

Claim 8

A method comprising the steps in the following order of: performing an initial backpropagation to train a neural network with plurality of layers having: an input layer of input units; and different layers of processing elements including an output layer having output processing elements and at least one hidden layer having hidden processing elements, each processing element having input weights; finding hidden processing elements common to at least one given layer of said plurality of layers and having similar patterns of information; replacing the similar pattern hidden processing elements with a single hidden processing element having information from one or more of the similar pattern hidden processing elements until each hidden processing element in each given layer has a distinctive pattern of information; and performing a further backpropagation on the neural network; and further comprising, following said further backpropagation: translating knowledge in the output layer and each hidden layer of the neural network into a layer set of rules, there being a layer set of rules corresponding to each layer of said output layer and each hidden layer, there being undefined hidden concepts embodied in the layer sets of rules; rewriting rules from the layer set of rules by reformulating rules from one layer in terms of rules of another layer to eliminate undefined hidden concepts and thereby generate a rewritten set of rules, the rewritten set of rules comprising rules in IF-THEN form; and storing said rewritten rule set in a memory.

Claim 9

An apparatus comprising: a neural network having an input layer of input units and different layers of processing elements including an output layer having output processing elements and at least one hidden layer having hidden processing elements, each processing element having input weights; and means for training said neural network including: means for performing an initial backpropagation on the neural network; means for finding hidden processing elements common to at least one given hidden layer and which similar patterns of information; means for replacing the similar pattern hidden processing elements within the given hidden layer with a single hidden processing element having information from one or more of the similar pattern hidden processing elements such that each hidden processing element in a layer eventually has a distinctive pattern of information; and means for performing a further backpropagation on the neural network; and wherein said neural network is a hardware network with said input units, output processing elements, and hidden processing elements all being hardware components; and further comprising: a translating means for translating knowledge in a neural network having an input layer of input units, and different layers of processing elements including an output layer of output processing elements and at least one hidden layer of hidden processing elements, said translating means operable to translate knowledge in the outer layer and each hidden layer of the neural network into a layer set of rules, there being undefined hidden concepts embodied in the layer sets of rules; a rewriting means for rewriting rules from the layer set of rules by reformulating rules from one layer in terms of rules of another layer to eliminate undefined hidden concepts and thereby generate a rewritten set of rules in IF-THEN form, the rewritten set of rules having an equivalent performance to the neural network; a memory for receiving and storing the rewritten set of rules.

Claim 10

An apparatus comprising: a neural network having an input layer of input units and different layers of processing elements including an output layer having output processing elements and at least one hidden layer having hidden processing elements, each processing element having input weights; and means for training said neural network including: means for performing an initial backpropagation on the neural network; means for finding hidden processing elements common to at least one given hidden layer and which similar patterns of information; means for replacing the similar pattern hidden processing elements within the given hidden layer with a single hidden processing element having information from one or more of the similar pattern hidden processing elements such that each hidden processing element in a layer eventually has a distinctive pattern of information; and means for performing a further backpropagation on the neural network; and wherein said neural network is a hardware network with said input units, output processing elements, and hidden processing elements all being hardware components; and further comprising: a translating means for translating knowledge in a neural network having an input layer of input units, and different layers of processing elements including an output layer of output processing elements and at least one hidden layer of hidden processing elements, said translating means operable to translate knowledge in the outer layer and each hidden layer of the neural network into a layer set of rules, there being undefined hidden concepts embodied in the layer sets of rules; a rewriting means for rewriting rules from the layer set of rules by reformulating rules from one layer in terms of rules of another layer to eliminate undefined hidden concepts and thereby generate a rewritten set of rules in IF-THEN form, the rewritten set of rules having an equivalent performance to the neural network; a memory for receiving and storing the rewritten set of rules.