Adaptive audio delay control for multimedia conferencing

Claim 2

A multimedia conferencing system comprising: a plurality of multimedia terminals, each of said multimedia terminals comprising: an audio channel for transmitting audio information to at least one other multimedia terminal of said plurality of multimedia terminals; a monitor for determining if the audio channel is active; and a controller for controlling an amount of delay of the audio being transmitted on the audio channel based on the determination by the monitor; and a bridge for receiving the audio information from said audio channel and transmitting the audio information received thereat to said at least one other multimedia terminal, wherein the bridge uses a number of floor changes occurring within a predetermined amount of time to determine the number of multimedia terminals having users that have been actively speaking in the multimedia conference.

Claim 3

The multimedia conferencing system according to claim 1, wherein the monitor monitors the audio being transmitted on the audio channel to determine if silence or active speech is occurring.

Claim 4

The multimedia conferencing system according to claim 2, wherein when the monitor determines that active speech has been occurring for longer than a predetermined amount of time, the audio delay is increased with respect to a video delay in a corresponding video channel.

Claim 5

The multimedia conferencing system according to claim 2, wherein when the monitor determines that silence has been occurring for longer than a predetermined period of time, the audio delay is decreased with respect to a video delay in a corresponding video channel.

Claim 6

The multimedia conferencing system according to claim 3, wherein the audio delay is gradually and steadily increased until at least perceptual lip sync is achieved with a corresponding image being transmitted in said video channel and having said video delay.

Claim 7

The multimedia conferencing system according to claim 3, wherein the audio delay is gradually increased in steps until at least perceptual lip sync is achieved with a corresponding image being transmitted in said video channel and having said video delay.

Claim 8

The multimedia conferencing system according to claim 1, wherein the monitor comprises a voice energy monitor for monitoring voice energy of audio on the audio channel.

Claim 9

The multimedia conference terminal according to claim 2, wherein said controller comprises: a silence counter for counting duration of silence responsive to an output signal of said monitor; a speech counter for counting duration of speech responsive to an output of said monitor; and an audio delay controller coupled to said silence and speech counters for controlling said amount of delay based on duration of silence and speech as determined by said counters.

Claim 10

A multimedia conferencing terminal comprising: at least one output audio channel for transmitting to at least one other terminal audio information via a bridge; at least one input audio channel for receiving from the at least one other terminal audio information via said bridge; at least one monitor for determining if the audio channels have been active; and a controller for controlling delay of the audio being transmitted on the at least one output audio channel based on the determination by the monitor, wherein the bridge uses a number of floor changes occurring within a predetermined amount of time to determine the number of multimedia terminals having users that have been actively speaking in the multimedia conference.

Claim 11

A multimedia conferencing terminal comprising: at least one output audio channel for transmitting to at least one other terminal audio information via a bridge; at least one input audio channel for receiving from the at least one other terminal audio information via said bridge; at least one monitor for determining if the audio channels have been active; and a controller for controlling delay of the audio being transmitted on the at least one output audio channel based on the determination by the monitor, wherein the bridge uses a number of floor changes occurring within a predetermined amount of time to determine the number of multimedia terminals having users that have been actively speaking in the multimedia conference.

Claim 12

The multimedia conferencing terminal according to claim 9, wherein said controller is also operable to control audio delay on the input audio channel based on the determination by the monitor.

Claim 13

The multimedia conferencing terminal according to claim 9, wherein when the monitor determines that only one of said audio channels has been active for longer than a predetermined period of time, the audio delay is increased until perceptual lip sync is reached.

Claim 14

The multimedia conferencing terminal according to claim 11, wherein when the monitor determines that silence has been occurring on a particular channel for longer than a predefined period of time, the audio delay is decreased to be lower than a corresponding video delay.

Claim 15

The multimedia conferencing terminal according to claim 9, wherein when the monitor determines that active speech has been occurring on both channels, the audio delay on at least the output channel is set lower than corresponding video delay.

Claim 16

A multimedia conferencing system comprising: a plurality of multimedia terminals each transmitting and receiving video data and audio data, the transmitted video data being coded at transmitting ones of said terminals and subsequently decoded at receiving ones of said terminals wherein a video delay is introduced; and a bridge for receiving the audio and video data from the transmitting multimedia terminals, and transmitting the video and audio data received thereat to the receiving multimedia terminals, said bridge using a number of floor changes occurring with a predetermined amount of time to determine the number of multimedia terminals having users that have been actively speaking in the multimedia conference, wherein when users of more than one of the multimedia terminals have been actively speaking in the multimedia conference, audio delay is decreased below that of video delay of corresponding video, and when a user of only one of the multimedia terminals is actively speaking in the multimedia conference for a predetermined amount of time, the audio delay is increased to substantially match the video delay.

Claim 17

A multimedia conferencing system comprising: a plurality of multimedia terminals each transmitting and receiving video data and audio data, the transmitted video data being coded at transmitting ones of said terminals and subsequently decoded at receiving ones of said terminals wherein a video delay is introduced; and a bridge for receiving the audio and video data from the transmitting multimedia terminals, and transmitting the video and audio data received thereat to the receiving multimedia terminals, said bridge using a number of floor changes occurring with a predetermined amount of time to determine the number of multimedia terminals having users that have been actively speaking in the multimedia conference, wherein when users of more than one of the multimedia terminals have been actively speaking in the multimedia conference, audio delay is decreased below that of video delay of corresponding video, and when a user of only one of the multimedia terminals is actively speaking in the multimedia conference for a predetermined amount of time, the audio delay is increased to substantially match the video delay.

Claim 18

The multimedia conferencing system according to claim 13, wherein when the bridge determines that the number of floor changes is greater than a predetermined number, the audio delay is decreased below that of the video delay.

Claim 19

The multimedia conferencing system according to claim 13, wherein when the bridge determines that the number of floor changes is less than a predetermined number, the audio delay is increased to substantially match the video delay.

Claim 20

The multimedia conferencing system according to claim 14, wherein the bridge comprises an audio portion and a video portion, said audio portion including: a multi-channel audio delay unit having a plurality of delay paths each coupled to an audio output channel of one of said multimedia terminals; a multi-channel audio monitor coupled to audio output channels of said multimedia terminals and operative to provide a plurality of output signals each indicative of audio activity occurring on a corresponding one of said audio output channels; an adaptive audio delay controller connected to receive each of said output signals from said monitor and to provide control signals, based on said output signals, to said audio delay unit to control delay in said delay paths; and an audio mixer coupled to said delay unit for mixing audio from said delay paths and providing a plurality of mixed audio outputs for transmission back to said receiving multimedia terminals.

Claim 21

The multimedia conferencing system according to claim 17, wherein said adaptive audio delay controller is operable to control delay in a particular delay path based on audio activity in both the corresponding audio output channel and on audio activity in other audio output channels of the other multimedia terminals.

Claim 22

The multimedia conferencing system according to claim 17, wherein said adaptive audio delay controller is operable to control delay in a particular delay path based on audio activity in both the corresponding audio output channel and on audio activity in other audio output channels of the other multimedia terminals, by taking into account the audio delays in said other audio output channels.

Claim 23

The multimedia conferencing system according to claim 14, further comprising means for implementing asymmetrical audio delay wherein audio delay of a monologue speaker who has been actively speaking for at least said predetermined amount of time substantially matches video delay of the monologue speaker, while audio delay of other speakers are minimally delayed, to thereby enable interruptions or questions of the other speakers to be minimally delayed.

Claim 26

The method of performing multimedia conferencing according to claim 21, wherein the step of determining if speech is actively occurring on an audio channel is performed by monitoring voice energy on the audio channel.

Claim 27

The method of performing multimedia conferencing according to claim 21, wherein the step of determining if speech is actively occurring on an audio channel is performed by monitoring outgoing audio to determined if speech is actively occurring.

Claim 28

The method of performing multimedia conferencing according to claim 21, wherein the step of determining if speech is actively occurring on an audio channel is performed by monitoring incoming audio on the audio channel to determine if speech is actively occurring.

Claim 29

The method according to claim 21, wherein said step of increasing an audio delay to at least substantially a delay of corresponding video if speech has been actively occurring for more than a predetermined amount of time implements audio delay for a monologue speaker, and said method further comprises the step of minimally delaying audio of other speakers to thereby enable interruptions of the monologue speaker or questions by the other speakers to be minimally delayed.

Claim 32

The method according to claim 26, wherein if only said particular audio channel has been active for more than a predetermined period of time, and said particular audio channel becomes silent, then said minimized delay difference of said particular channel is maintained unless one or more of the other channels becomes active.