Multimedia data transmission system

Claim 2

A vector quantizer for encoding digital signals with high efficiency, characterized in that a codebook having the learning function is formed, input vector and deciding information of predetermined dynamic codebook are transmitted for matching of dynamic codebooks in the transmitting and receiving sides, and a means for executing, at a high speed, the vector quantization through parallel use of a fixed codebook and a plurality of random access dynamic codebooks which are connected to the intermediate hierarchy of said fixed codebook comprised of a plurality of quantizing representative vectors arranged in the tree-structure, sequentially storing said input vectors, as the new quantizing representative vectors into the predetermined dynamic codebook if the minimum distortion in the tree-searched vector quantization is larger than a certain value, and simultaneously erasing the old quantizing representative vectors stored in the past.

Claim 3

An interframe vector quantization encoding and decoding apparatus comprising: a frame memory for storing video signal corresponding to at least one frame; a subtractor for subtracting interframe forecast video signal series read from said frame memory to input video signal series and obtaining interframe difference signal series; a vector quantization encoding and decoding unit for performing scalar quantization or vector quantization of input vector based on wave form distortion calculated in process that mean value separation input vector obtained by blocking the interframe forecast video signal is subjected to vector encoding quantization by an inner product vector quantization encoder if the wave form distortion is larger than threshold value and updating code books in sequence and using it to perform subsequent encoding and decoding operation, or for updating the code books in sequence based on value that the minimum distortion obtained in quantization process of performing mean value separation normalization vector quantization is weighted by amplitude gain and performing the encoding and decoding operation and obtaining the encoding data and the decoding interframe difference signal series; an adder for adding the decoding difference signal series to the interframe forecast video signal series and outputting the decoding video signal series to said frame memory; a variable length encoding unit for converting the encoding data into variable length encoding and outputting variable length encoding data; a transmission buffer for storing the variable length encoding data temporarily and smoothing the amount of generating information and outputting it as transmission data at a constant speed and also estimating the amount of generating information in one frame unit; and an encoding control unit for generating encoding control parameter to perform adaptive control of operation of said vector quantization encoding and decoding unit based on the amount of generating information in one frame unit.

Claim 4

An interframe vector quantization encoding and decoding apparatus comprising: a frame memory for storing video signal corresponding to at least one frame; a subtractor for subtracting interframe forecast video signal series read from said frame memory to input video signal series and obtaining interframe difference signal series; a vector quantization encoding and decoding unit for performing scalar quantization or vector quantization of input vector based on wave form distortion calculated in process that mean value separation input vector obtained by blocking the interframe forecast video signal is subjected to vector encoding quantization by an inner product vector quantization encoder if the wave form distortion is larger than threshold value and updating code books in sequence and using it to perform subsequent encoding and decoding operation, or for updating the code books in sequence based on value that the minimum distortion obtained in quantization process of performing mean value separation normalization vector quantization is weighted by amplitude gain and performing the encoding and decoding operation and obtaining the encoding data and the decoding interframe difference signal series; an adder for adding the decoding difference signal series to the interframe forecast video signal series and outputting the decoding video signal series to said frame memory; a variable length encoding unit for converting the encoding data into variable length encoding and outputting variable length encoding data; a transmission buffer for storing the variable length encoding data temporarily and smoothing the amount of generating information and outputting it as transmission data at a constant speed and also estimating the amount of generating information in one frame unit; and an encoding control unit for generating encoding control parameter to perform adaptive control of operation of said vector quantization encoding and decoding unit based on the amount of generating information in one frame unit.

Claim 5

An interframe vector quantization encoding and decoding apparatus as set forth in claim 2, wherein input vector obtained in blocking of the input video signal series by the first stage vector quantizer is quantized by a plurality of output vectors which are cut out of prescribed address of said frame memory and outputted from code books, and if the minimum distortion obtained in the quantization process exceeds the quantization threshold value, first stage mean value estimated by a mean value calculating unit is interpolated and content of the code books is updated and the first stage mean value interporated is outputted as first stage vector quantization decoding series to said subtractor and said adder, and also a first stage vector quantization encoding and decoding unit for adding the first stage mean value to a first stage index for discrimination of the output vectors and outputting the first stage vector quantization encoding data to said variable length encoding unit is installed at output stage of said frame memory.

Claim 6

An interframe vector quantizer comprising: a band dividing unit for converting input video signal series into a plurality of frequency band separation input video signal series divided per space frequency band; a first frame memory for storing one frame of the frequency band separation input video signal series outputted from said band dividing unit, and for outputting one frame of prescribed frequency band input video signal series in time division; a second frame memory for supplying frame delay to prescribed frequency band decoding video signal series outputted by an adder hereinafter described and for generating prescribed frequency band interframe forecast signal; a subtractor for subtracting prescribed frequency band interframe forecast signal outputted by said second frame memory from prescribed frequency band input video signal series outputted by said first frame memory and for generating prescribed frequency band interframe difference signal; a dynamic vector quantization encoding unit for performing dynamic vector quantization encoding of prescribed frequency band interframe difference signal outputted by said subtractor in accuracy corresponding to level of space frequency and for generating prescribed frequency band encoding data; an encoding control unit for controlling write/read of said first frame memory and said second frame memory and for controlling vector quantization characteristics of said dynamic vector quantization encoding unit; a dynamic vector quantization decoding unit for converting prescribed frequency band encoding data generated in said dynamic vector quantization encoding unit into prescribed frequency band interframe decoding difference signal; an adder for adding prescribed frequency band interframe decoding difference signal from said dynamic vector quantization decoding unit to prescribed frequency band interframe forecast signal from said second frame memory and for generating prescribed frequency band decoding video signal series and inputting it to said second frame memory; a variable length encoding unit for performing variable length encoding of prescribed frequency band encoding data outputted by said dynamic vector quantization encoding unit; a buffer for temporarily storing data subjected to variable length encoding from said variable length encoding unit; and a line interface for transmitting the variable length encoded data outputted from said buffer as transmission signal.

Claim 7

An interframe vector quantizer comprising: a band dividing unit for converting input video signal series into a plurality of frequency band separation input video signal series divided per space frequency band; a first frame memory for storing one frame of the frequency band separation input video signal series outputted from said band dividing unit, and for outputting one frame of prescribed frequency band input video signal series in time division; a second frame memory for supplying frame delay to prescribed frequency band decoding video signal series outputted by an adder hereinafter described and for generating prescribed frequency band interframe forecast signal; a subtractor for subtracting prescribed frequency band interframe forecast signal outputted by said second frame memory from prescribed frequency band input video signal series outputted by said first frame memory and for generating prescribed frequency band interframe difference signal; a dynamic vector quantization encoding unit for performing dynamic vector quantization encoding of prescribed frequency band interframe difference signal outputted by said subtractor in accuracy corresponding to level of space frequency and for generating prescribed frequency band encoding data; an encoding control unit for controlling write/read of said first frame memory and said second frame memory and for controlling vector quantization characteristics of said dynamic vector quantization encoding unit; a dynamic vector quantization decoding unit for converting prescribed frequency band encoding data generated in said dynamic vector quantization encoding unit into prescribed frequency band interframe decoding difference signal; an adder for adding prescribed frequency band interframe decoding difference signal from said dynamic vector quantization decoding unit to prescribed frequency band interframe forecast signal from said second frame memory and for generating prescribed frequency band decoding video signal series and inputting it to said second frame memory; a variable length encoding unit for performing variable length encoding of prescribed frequency band encoding data outputted by said dynamic vector quantization encoding unit; a buffer for temporarily storing data subjected to variable length encoding from said variable length encoding unit; and a line interface for transmitting the variable length encoded data outputted from said buffer as transmission signal.

Claim 8

An interframe vector quantizer as set forth in claim 1, said dynamic vector quantization encoding unit comprising: a mean value separation unit for converting the prescribed frequency band interframe difference signal into vector and separating mean value and for performing scalar quantization encoding to the mean value and outputting the mean value separation input vector and the mean value encoding data; code books including a fixed code book for storing a plurality of normalization output vectors with mean "0" and magnitude "1" and a dynamic code book for storing normalization output by normalizing the mean value separation input vector as normalization output and having the storage content updated at any time; an inner product vector quantization encoding unit for calculating inner product of the mean value separation input vector supplied from said mean value separation unit with each normalization output vector stored in the fixed code book and the dynamic code book of said code books and detecting the normalization output vector to give the maximum inner product being "0" or more, and for outputting amplitude encoding data defined by value of the maximum inner product and index for discriminating the normalization output vector to give the maximum inner product if distortion estimated based on amount of the mean value separation input vector and amount of the maximum inner product is less than the distortion threshold value transmitted from said encoding control unit, and for inverting sign of the amplitude encoding data and outputting it and outputting the mean value separation input vector from said mean value separation unit as it is in place of the index if the distortion is larger than the distortion threshold value; a scalar quantization encoding unit for performing scalar quantization of the mean value separation input vector per sample supplied from said inner product vector quantization encoding unit and generating the mean value separation vector encoding data; a selector for selecting the index and the mean value separation vector quantization data by the amplitude encoding data; a block counter for counting the prescribed frequency band interframe difference signal in block unit and generating frequency band discrimination signal; a block discrimination unit for performing decision of significant block/insignificant block using the block discrimination threshold value transmitted from said encoding control unit and outputting the block discrimination signal; and a normalization unit for normalizing the mean value separation input vector from said inner product vector quantization encoding unit and generating the normalization output vector and outputting it to the dynamic code book in said code books. wherein as the prescribed frequency band encoding data, for said significant block, the block discrimination signal, the mean value encoding data, the amplitude encoding data and one of the index selected by said selector or the mean value separatior vector encoding data are outputted, and for said insignificant block, only the block discrimination signal is outputted.

Claim 9

A multimedia data transmission system comprising: a means for configurating transmission frames of 80/160/320/480/560-bit lengths having the repetition frequency of 100 Hz uniformly collecting 80 basic time slots, specified without relation to the transmission rate, of 1/2/4/6/7-bit lengths having the repetition frequency of 8 kHz corresponding to the transmission rate of 8/16/32/48/56 Kbps, a means for configurating a multiframe by uniformly collecting 16 transmission frames without relation to the transmission rate, a means for multiplexing a multimedia data, in every said basic time slot, transmission frame or multiframe, such as encoded video data or encoded voice data having the code transmission rate which is, for example, an integer time of 8 Kbps or matching said transmission frame or multiframe, and a communication control means for notifying the receiving side by arranging a synchronous code for identifying partition of transmission frame and multiframe and a bit assigning information for indicating bit assigning of multimedia data to a predetermined bit in the basic time slot.

Claim 10

A multimedia data transmission system comprising: a means for configurating transmission frames of 80/160/320/480/560-bit lengths having the repetition frequency of 100 Hz uniformly collecting 80 basic time slots, specified without relation to the transmission rate, of 1/2/4/6/7-bit lengths having the repetition frequency of 8 kHz corresponding to the transmission rate of 8/16/32/48/56 Kbps, a means for configurating a multiframe by uniformly collecting 16 transmission frames without relation to the transmission rate, a means for multiplexing a multimedia data, in every said basic time slot, transmission frame or multiframe, such as encoded video data or encoded voice data having the code transmission rate which is, for example, an integer time of 8 Kbps or matching said transmission frame or multiframe, and a communication control means for notifying the receiving side by arranging a synchronous code for identifying partition of transmission frame and multiframe and a bit assigning information for indicating bit assigning of multimedia data to a predetermined bit in the basic time slot.

Claim 11

A multimedia data transmission system according to claim 6, wherein the number of basic time slots in the transmission frame and the number of transmission frames in the multiframe are set to constant values independent of the transmission rates of 8/16/32/48/56 Kbps and the additional information including synchronous code and bit assigning information is set to a common value corresponding all transmission rates with the same control means.

Claim 12

A multimedia data transmission system according to claim 6, wherein a bit rate for interface is fixed to 64 Kbps in the transmission path having the transmission rate of 8/16/32/48/56 Kbps and the same transmission path interface is used, not depending on the transmission rate of 32/48/56 Kbps, with a means for converting the transmission rate to the interface bit rate through matching of transmission rate by adding dummy bit of 7/6/4/2/1-bit lengths to 8/16,/32/48/56 Kbps for each basic time slots and a means for extracting only the valid bits by eliminating dummy bits with a timing signal of 8 kHz synchronized with the partition of basic time slot obtained from periodicity of dummy bit in the receiving side or the basic time slot supplied from the interface.

Claim 13

In an interframe encoding apparatus for encoding, with high performance, a series of input video signals introducing conditional replenishment in unit of block, an encoding control system characterized in that a means is provided for controlling the threshold value to be applied to the entire part of frame so that, in case a predetermined value which is smaller than the block discriminating threshold value already used once is to be set as a new block discriminating threshold value in such a process as controlling a block discriminating threshold value for executing said conditional replenishment in unit of frame, a new block threshold value is set to a value which is reduced step by step, while sequentially widening the region within the frame to which said threshold value is applied, and such thres­hold value is finally set to said predetermined value.

Claim 14

In an interframe encoding apparatus for encoding, with high performance, a series of input video signals introducing conditional replenishment in unit of block, an encoding control system characterized in that a means is provided for controlling the threshold value to be applied to the entire part of frame so that, in case a predetermined value which is smaller than the block discriminating threshold value already used once is to be set as a new block discriminating threshold value in such a process as controlling a block discriminating threshold value for executing said conditional replenishment in unit of frame, a new block threshold value is set to a value which is reduced step by step, while sequentially widening the region within the frame to which said threshold value is applied, and such thres­hold value is finally set to said predetermined value.

Claim 15

An encoding control system comprising a means for executing frame encoding to the region in the frame to which a new block discrimination threshold value is applied and is enlarged up to the region sequentially designated in the predetermined sequence synchronized by the sending and receiving, setting said region in unit of frame so that the frame as a whole is subjected to the frame encoding after the constant framing time and transmitting the sequence information indicating passage of such frame time for each frame and periodically executing the frame/interframe mixed encoding mode for the interframe encoding to the regions other than those designated above, wherein said frame/interframe mixed encoding mode and the interframe encoding mode using threshold value control are switched applicably.

Claim 16

An encoding control system comprising a means for executing frame encoding to the region in the frame to which a new block discrimination threshold value is applied and is enlarged up to the region sequentially designated in the predetermined sequence synchronized by the sending and receiving, setting said region in unit of frame so that the frame as a whole is subjected to the frame encoding after the constant framing time and transmitting the sequence information indicating passage of such frame time for each frame and periodically executing the frame/interframe mixed encoding mode for the interframe encoding to the regions other than those designated above, wherein said frame/interframe mixed encoding mode and the interframe encoding mode using threshold value control are switched applicably.

Claim 17

An image encoding transmission apparatus with motion compensating processing comprising: a preprocessing means for reading a dynamic image signal for each frame, producing a pixel signal through A/D conversion, forming blocks each of which is formed by the predetermined number of adjacent pixels on the image and generating image vector signal consisting of pixel signal groups for each block, a motion compensating means for generating a plurality of reference blocks with reference to the current block unit from a reproduced signal of the previous frame and detecting a motion vector of the image vector signal by searching the reference block which is most approximated to the image vector signal, a vector quantizing encoding means for compressing and encoding such image vector signal on the basis of the searched reference block and motion vector by utilizing correlation between frames, a send buffer for sending, frame by frame, the encoded image vector signal to the transmission line, and a vector quantizing decoding means for decoding the image vector signal encoded in the vector quantizing encoding means and generating reproduced signal by adding the vector signal of reference block searched to the obtained decoded vector signal, characterized by further comprising; an encoding accuracy control means for changing the encoding accuracy of the vector quantizing encoding means at the predetermined period in accordance with the amount of send encoding information of the encoded image vector signal to be temporarily stored in the send buffer, an adaptive spatial filter for executing the smoothing process of the reproduced signal to use the pixel values including, at the predetermined ratio, the pixel values approximated to each pixel value of the reproduced signal, and a smoothing characteristic control means for controlling ON/OFF conditions of smoothing process of the adaptive spatial filter on the basis of the motion vector, enhancing a degree of smoothing operation of adaptive spatial filter in case the encoding accuracy is low and lowering the degree of smoothing operation in case the encoding accuracy is high.

Claim 18

An image encoding transmission apparatus with motion compensating processing comprising: a preprocessing means for reading a dynamic image signal for each frame, producing a pixel signal through A/D conversion, forming blocks each of which is formed by the predetermined number of adjacent pixels on the image and generating image vector signal consisting of pixel signal groups for each block, a motion compensating means for generating a plurality of reference blocks with reference to the current block unit from a reproduced signal of the previous frame and detecting a motion vector of the image vector signal by searching the reference block which is most approximated to the image vector signal, a vector quantizing encoding means for compressing and encoding such image vector signal on the basis of the searched reference block and motion vector by utilizing correlation between frames, a send buffer for sending, frame by frame, the encoded image vector signal to the transmission line, and a vector quantizing decoding means for decoding the image vector signal encoded in the vector quantizing encoding means and generating reproduced signal by adding the vector signal of reference block searched to the obtained decoded vector signal, characterized by further comprising; an encoding accuracy control means for changing the encoding accuracy of the vector quantizing encoding means at the predetermined period in accordance with the amount of send encoding information of the encoded image vector signal to be temporarily stored in the send buffer, an adaptive spatial filter for executing the smoothing process of the reproduced signal to use the pixel values including, at the predetermined ratio, the pixel values approximated to each pixel value of the reproduced signal, and a smoothing characteristic control means for controlling ON/OFF conditions of smoothing process of the adaptive spatial filter on the basis of the motion vector, enhancing a degree of smoothing operation of adaptive spatial filter in case the encoding accuracy is low and lowering the degree of smoothing operation in case the encoding accuracy is high.

Claim 19

A vector quantizing encoding transmission apparatus comprising: a preprocessing means for reading a video signal, forming blocks, each of which is comprised of a specified number of adjacent pixels of image and generating input vector to output it, a codebook for previously storing a plurality of the representative vector for quantization which is a pattern of input vector and given the predetermined index data, and a vector quantizing encoding means for selecting the representative vector for quantization which is most approximated to the input vector from the codebook and outputting the index data of representative vector for quantization as the video transmitting data, said apparatus further comprising; an encoding control means for measuring selection frequency of representative vector for quantization, executing code assigning control to assign the short code length index data sequentially to the vectors from the representative vector for quantization having a high selection frequency depending on the result of measurement, comparing the minimum distortion value of the representative vector for quantization selected at the time of vector quantization and the input vector with a threshold value for vector updata, and rewriting the input vector, when said minimum distortion value is larger than the threshold value, to the representative vector for quantization with low selection frequency to send update discrimination information, index data and update representative vector, and a decoding control means for rewriting the representative vector for quantization of codebook in the receiving side in accordance with the update discrimination information received, measuring selection frequency of the representative vector for quantization from the index data received and assigning a short index data sequentially from the representative vector for quantization with high selection frequency on the basis of the result of measurement, wherein the index data of representative vector for quantization and the representative vector for quantization in the codebook are updated in accordance with the transmitting image.

Claim 20

A vector quantizing encoding transmission apparatus comprising: a preprocessing means for reading a video signal, forming blocks, each of which is comprised of a specified number of adjacent pixels of image and generating input vector to output it, a codebook for previously storing a plurality of the representative vector for quantization which is a pattern of input vector and given the predetermined index data, and a vector quantizing encoding means for selecting the representative vector for quantization which is most approximated to the input vector from the codebook and outputting the index data of representative vector for quantization as the video transmitting data, said apparatus further comprising; an encoding control means for measuring selection frequency of representative vector for quantization, executing code assigning control to assign the short code length index data sequentially to the vectors from the representative vector for quantization having a high selection frequency depending on the result of measurement, comparing the minimum distortion value of the representative vector for quantization selected at the time of vector quantization and the input vector with a threshold value for vector updata, and rewriting the input vector, when said minimum distortion value is larger than the threshold value, to the representative vector for quantization with low selection frequency to send update discrimination information, index data and update representative vector, and a decoding control means for rewriting the representative vector for quantization of codebook in the receiving side in accordance with the update discrimination information received, measuring selection frequency of the representative vector for quantization from the index data received and assigning a short index data sequentially from the representative vector for quantization with high selection frequency on the basis of the result of measurement, wherein the index data of representative vector for quantization and the representative vector for quantization in the codebook are updated in accordance with the transmitting image.

Claim 21

Learning-type vector quantizer for effectively encoding digital signals, wherein it comprises an upstream fixed code book composed of a plurality of representative quantization vectors, a downstream random access variable code book and encoder-decoders for vector-quantization operation different in block size connected respectively to each code book, so that, if no encode operation is carried out in the upstream vector quantization encoder, effective vector quantization operation is carried out in the downstream vector quantization encoder using a predetermined threshold value on the input image divided into pieces by reducing the production of information.

Claim 22

Learning-type vector quantizer for effectively encoding digital signals, wherein it comprises an upstream fixed code book composed of a plurality of representative quantization vectors, a downstream random access variable code book and encoder-decoders for vector-quantization operation different in block size connected respectively to each code book, so that, if no encode operation is carried out in the upstream vector quantization encoder, effective vector quantization operation is carried out in the downstream vector quantization encoder using a predetermined threshold value on the input image divided into pieces by reducing the production of information.

Claim 23

Learning-type vector quantizer as claimed in claim 1, wherein, in order to search rapidly and precisely a representative quantization vector in the variable code book used in a vector quantizer, characteristics of the representative quantization vectors are classified in several groups, a region is prepared in the code book for each characteristic; and the representative quantization vectors having the same characteristics are arranged according to the order of frequency, so that registration and renewal of the representative quantization vectors in the code book is carried out effectively.

Claim 24

Learning-type vector quantizer as claimed in claim 13, wherein the vector quantization operation is carried out effectively by connecting the variable code book to the upstream vector quantization encoder and the fixed code book to the downstream vector quantization encoder, while reducing the production of information.

Claim 25

Learning-type vector quantizer, wherein at the time of vector quantization of input vectors by using a plurality of representative quantization vectors stored in a random access code book, the indexes of the representative quantization vectors selected from the code book are rearranged according to the order of the selection, and when the minimum distortion at the time of the vector quantization for an input vector is larger than a predetermined value, the input vector is taken as the representative quantization vector and the latter replaces the oldest selected representative quantization vector derived from the rearrangement of the indexes to form a code book having a learning function, and in order to make code books at the transmission block and the reception block are the same, the input vector is transmitted to the reception block from the transmission block to carry out vector quantization using the newly formed code book.

Claim 26

Learning-type vector quantizer, wherein at the time of vector quantization of input vectors by using a plurality of representative quantization vectors stored in a random access code book, the indexes of the representative quantization vectors selected from the code book are rearranged according to the order of the selection, and when the minimum distortion at the time of the vector quantization for an input vector is larger than a predetermined value, the input vector is taken as the representative quantization vector and the latter replaces the oldest selected representative quantization vector derived from the rearrangement of the indexes to form a code book having a learning function, and in order to make code books at the transmission block and the reception block are the same, the input vector is transmitted to the reception block from the transmission block to carry out vector quantization using the newly formed code book.

Claim 27

Learning-type vector quantizer as claimed in claim 15, wherein based on the selection frequency of a representative quantization vector obtained by the rearrangement of the indexes, a code allocation operation is carried out in which a shorter code length is allocated to a representative quantization vector having a higher probability to be selected.