Class AudioPeerConnection

A WebRTC peer connection with one Opus audio stream each way, as 1:1 voice calls use: SDP offer and answer, ICE with trickled candidates, DTLS-SRTP, and RTP; and, when the peer offers it and ReceiveVideo is set, VP8 video received.

Inheritance

object

AudioPeerConnection

Implements

IMediaConnection

IAsyncDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Uniform.Matrix.WebRtc

Assembly: Uniform.Matrix.WebRtc.dll

Syntax

public sealed class AudioPeerConnection : IMediaConnection, IAsyncDisposable

Remarks

Received video comes as whole frames in VideoReceived; the connection sends RTCP receiver reports every second, each with a REMB of the loss-based estimate, NACKs for gaps, and picture loss indications while a keyframe is needed. The caller calls CreateOffer(bool) and later AcceptAnswer(string); the callee calls AcceptOffer(string). Both trickle the candidates of CandidateGathered to the peer, give the peer's to AddRemoteCandidate(string), and call ConnectAsync(CancellationToken). The offerer controls ICE, and the answerer is the DTLS client, as browsers do. Datagrams are told apart by their first byte as RFC 7983 says.

Constructors

AudioPeerConnection(PeerConnectionOptions?)

Create a connection; nothing happens until an offer is made or accepted.

Declaration

public AudioPeerConnection(PeerConnectionOptions? options = null)

Parameters

Type	Name	Description
PeerConnectionOptions	options

Fields

FrameSamples

Samples per 20 ms frame at Opus's 48 kHz RTP clock.

Declaration

public const int FrameSamples = 960

Field Value

Type	Description
int

Properties

BundleIndex

The index of the section of BundleMid, for the sdpMLineIndex of trickled candidates.

Declaration

public int BundleIndex { get; }

Property Value

Type	Description
int

BundleMid

The media ID of the first section this side accepted, which carries the bundled transport, e.g. for the sdpMid of trickled candidates; null before any.

Declaration

public string? BundleMid { get; }

Property Value

Type	Description
string

Ice

The ICE agent, once an offer is made or accepted; e.g. to see the selected candidates.

Declaration

public IceAgent? Ice { get; }

Property Value

Type	Description
IceAgent

IsRestarting

Whether an ICE restart is under way, between its offer and its connecting.

Declaration

public bool IsRestarting { get; }

Property Value

Type	Description
bool

ReceivesVideo

Whether the answer accepted the peer's video, so frames may come in VideoReceived.

Declaration

public bool ReceivesVideo { get; }

Property Value

Type	Description
bool

SendsOrientation

Whether sent frames carry their rotation, as both sides took the video orientation extension; an SFU such as LiveKit does not.

Declaration

public bool SendsOrientation { get; }

Property Value

Type	Description
bool

SendsVideo

Whether the answer sends video to the peer, with SendVideoAsync(byte[], CancellationToken, int).

Declaration

public bool SendsVideo { get; }

Property Value

Type	Description
bool

State

The state.

Declaration

public PeerConnectionState State { get; }

Property Value

Type	Description
PeerConnectionState

VideoPacketsNacked

Video packets of the peer asked again, with NACKs, after a gap.

Declaration

public long VideoPacketsNacked { get; }

Property Value

Type	Description
long

VideoPacketsResent

Video packets sent again because the peer asked, with a NACK.

Declaration

public long VideoPacketsResent { get; }

Property Value

Type	Description
long

Methods

AcceptAnswer(string)

Accept the peer's answer to this side's offer.

Declaration

public void AcceptAnswer(string answer)

Parameters

Type	Name	Description
string	answer

Exceptions

Type	Condition
InvalidOperationException	When no offer was made, or the answer has no Opus audio.

AcceptOffer(string)

Accept the peer's offer, and start gathering; return the answer, in which this side is the DTLS client. An offer without audio, as an SFU makes before anyone publishes, connects the transport for media that a later offer adds.

Declaration

public string AcceptOffer(string offer)

Parameters

Type	Name	Description
string	offer

Returns

Type	Description
string

Exceptions

Type	Condition
InvalidOperationException	When the offer has no transport over DTLS-SRTP.

AcceptRestartAnswer(string)

Take the peer's answer to this side's restart offer.

Declaration

public void AcceptRestartAnswer(string answer)

Parameters

Type	Name	Description
string	answer

Exceptions

Type	Condition
InvalidOperationException	When the answer has no Opus audio.

AcceptRestartOffer(string)

Answer the peer's new offer: an ICE restart when its credentials changed, with a new agent this side is controlled in, else the same connection, with the media the offer adds, as an SFU offers each track published; this side keeps its DTLS role either way.

Declaration

public string AcceptRestartOffer(string offer)

Parameters

Type	Name	Description
string	offer

Returns

Type	Description
string

Exceptions

Type	Condition
InvalidOperationException	Before the first offer and answer, or when the offer has no transport.

AddRemoteCandidate(string)

Add a candidate the peer trickled, as its candidate: attribute, to the restarting agent while ICE restarts; unusable ones are ignored.

Declaration

public void AddRemoteCandidate(string candidate)

Parameters

Type	Name	Description
string	candidate

CancelRestart()

Abandon an ICE restart under way.

Declaration

public void CancelRestart()

ConnectAsync(CancellationToken)

Connect ICE, then DTLS; return whether media can flow.

Declaration

public Task<bool> ConnectAsync(CancellationToken cancellationToken = default)

Parameters

Type	Name	Description
CancellationToken	cancellationToken

Returns

Type	Description
Task<bool>

CreateOffer(bool)

Return an offer of audio, and with video of video both ways, and start gathering; this side controls ICE. The answer decides which ways the video goes.

Declaration

public string CreateOffer(bool video = false)

Parameters

Type	Name	Description
bool	video

Returns

Type	Description
string

CreateRestartOffer()

Start an ICE restart, e.g. after the network changed: a new agent with new credentials gathers on the networks there are now, while DTLS and SRTP go on; return the offer for the peer. This side controls the new agent.

Declaration

public string CreateRestartOffer()

Returns

Type	Description
string

Exceptions

Type	Condition
InvalidOperationException	Before the first offer and answer.

DisposeAsync()

Close DTLS and ICE.

Declaration

public ValueTask DisposeAsync()

Returns

Type	Description
ValueTask

ReconnectAsync(CancellationToken)

Connect the restarting agent, and move the media to it; return whether it connected. The old agent goes on until then.

Declaration

public Task<bool> ReconnectAsync(CancellationToken cancellationToken = default)

Parameters

Type	Name	Description
CancellationToken	cancellationToken

Returns

Type	Description
Task<bool>

RequestBitrateAsync(long)

Cap the bitrate asked of the peer's video at bitsPerSecond, e.g. as this device cannot decode more, and ask at once with a REMB; every report then asks for the loss-based estimate within the cap. Nothing without received video.

Declaration

public Task RequestBitrateAsync(long bitsPerSecond)

Parameters

Type	Name	Description
long	bitsPerSecond

Returns

Type	Description
Task

RequestKeyframeAsync()

Ask the peer for a keyframe of its video with an RTCP picture loss indication, e.g. after a frame failed to decode; nothing without received video.

Declaration

public Task RequestKeyframeAsync()

Returns

Type	Description
Task

SendAudioAsync(byte[], int, CancellationToken)

Send a frame of Opus, samples long at 48 kHz; dropped until connected.

Declaration

public ValueTask SendAudioAsync(byte[] opus, int samples = 960, CancellationToken cancellationToken = default)

Parameters

Type	Name	Description
byte[]	opus	The encoded frame.
int	samples	Samples it covers, which advance the RTP timestamp.
CancellationToken	cancellationToken	Cancels the send.

Returns

Type	Description
ValueTask

SendVideoAsync(byte[], CancellationToken, int)

Send a VP8 frame of this side's video, captured now, in RTP packets of at most 1100 bytes, so they fit a path's MTU with SRTP and TURN; dropped until connected or when the peer receives no video.

Declaration

public ValueTask SendVideoAsync(byte[] frame, CancellationToken cancellationToken = default, int rotation = 0)

Parameters

Type	Name	Description
byte[]	frame	The encoded frame, as the encoder wrote it.
CancellationToken	cancellationToken	Cancels the send.
int	rotation	Clockwise rotation in degrees that shows the picture upright, 0, 90, 180 or 270, sent as its video orientation when the peer takes it.

Returns

Type	Description
ValueTask

SkipAudio(int)

Skip samples without sending, e.g. while muted, so the timestamps stay in time; the next packet is marked.

Declaration

public void SkipAudio(int samples = 960)

Parameters

Type	Name	Description
int	samples

Events

AudioReceived

Raised for each audio packet received, decrypted, on the socket's read loop; its payload is Opus.

Declaration

public event Action<RtpPacket>? AudioReceived

Event Type

Type	Description
Action<RtpPacket>

BitrateRequested

Raised when the peer asks to limit the video sent to a bitrate, in bits per second, with a REMB, e.g. as it cannot decode more; on the socket's read loop.

Declaration

public event Action<long>? BitrateRequested

Event Type

Type	Description
Action<long>

CandidateGathered

Raised for each local candidate, to trickle to the peer.

Declaration

public event Action<IceCandidate>? CandidateGathered

Event Type

Type	Description
Action<IceCandidate>

GatheringCompleted

Raised when all candidates are gathered, e.g. to tell the peer that no more come.

Declaration

public event Action? GatheringCompleted

Event Type

Type	Description
Action

KeyframeRequested

Raised when the peer asks for a keyframe of the video sent, e.g. after it lost packets or joined; the encoder should make the next frame one. On the socket's read loop.

Declaration

public event Action? KeyframeRequested

Event Type

Type	Description
Action

StateChanged

Raised when State changes.

Declaration

public event Action<PeerConnectionState>? StateChanged

Event Type

Type	Description
Action<PeerConnectionState>

VideoReceived

Raised for each whole VP8 frame received, in order, on the socket's read loop; after lost packets, from the next keyframe.

Declaration

public event Action<Vp8EncodedFrame>? VideoReceived

Event Type

Type	Description
Action<Vp8EncodedFrame>

Implements

IMediaConnection

IAsyncDisposable