Unified mixing, speaker selection, and jitter buffer management for multi-speaker packet audio systems

DETAILED DESCRIPTION A receiver manages multiple speakers in a packet network.
A packet gateway receives audio packets from the multiple speakers over the packet network.
Memory in the receiver stores the audio packets and information about the multiple speakers in the telephone call.
A processor selects which speaker audio packets and speaker information to retain in memory.
The processor determines which of the selected-audio packets to store in memory-and mix together to produce an audio output signal by determining from the stored speaker information which of the multiple speakers are actively talking.
The speaker information is kept-in an indexed data array that identifies the speakers, a talking status for the speakers; and a pointer to buffers in memory that retain the audio packets for the speakers.
Speaker entries in the data array also include a Least Recently Used (LRU) time indicating a local time the last audio packet was received for the speaker.
The processor uses the LRU time to determine which speakers are actively talking and which speakers have stopped talking.
A Talkspurt Packet Count indicates a single connected segment of audio.
The processor uses the Talkspurt Packet Count to distinguish audio packets coming from a speaker who is actively talking from audio packets containing background noise.
The receiver identifies the status of the speaker entries in the data array as actively talking (A), not actively talking but valid (V) or not in use (F).
Depending on available processing resources, the speaker status, LRU time, and Talkspurt Packet Count, speaker entries are stored, discarded or changed in the data array and audio packets from speakers are either stored or discarded in memory.
The invention solves the multi-speaker problem by using an adaptive speaker management scheme to intelligently select which speaker states and audio to retain and process.
The receiver-based system is especially effective for demanding applications such as distance learning, where a professor is speaking most of the time, but an occasional question from a few of many listening students could arrive at any time