Don't restart DTLS after receiving new tls-id while passive

[alexbakker]

When the remote end of a DTLS media session changes their tls-id in their answer in response to a re-INVITE, they are the one to initiate a new DTLS handshake. Previously, rtpengine would always perform a DTLS restart after seeing a new tls-id, even if rtpengine's DTLS role is currently passive. This could abort an in-progress handshake, leaving the remote end retransmitting DTLS packets without ever getting a response. This patch addresses that.

[rfuchs]

Is that the correct behaviour though? Shouldn't we expect a new DTLS connection when the tls-id value changed in an answer?

The RFC doesn't seem to describe that situation at all.

[alexbakker]

That's my understanding based on RFC 8842 section 5.4:

When an offerer receives an answer that establishes a new DTLS association based on criteria defined in Section 3.1, if the offerer becomes DTLS client (based on the value of the SDP "setup" attribute value [RFC4145]), the offerer MUST establish a DTLS association. If the offerer becomes DTLS server, it MUST wait for the answerer to establish the DTLS association.

And then section 3.1 states:

The intent to establish a new DTLS association is explicitly signaled using SDP, by changing the value of the SDP "tls-id" attribute defined in this document;

So, if we're passive and the remote and changes its tls-id, rtpengine must wait and not initiate a DTLS restart.

[rfuchs]

So, if we're passive and the remote and changes its tls-id, rtpengine must wait and not initiate a DTLS restart.

dtls_restart merely resets the existing DTLS connection though, and doesn't necessarily initiate a new (active/outgoing) connection. So it would set up a new passive DTLS connection context and wait for a new incoming DTLS connection, which I think is correct?

[alexbakker]

The new handshake might already be in progress when the answer starts getting processed though. And like you said, a DTLS restart resets the existing DTLS connection context and thus wipes that state. That's the issue I encountered.

[rfuchs]

Ok, but what if the existing DTLS context is fully established?

So, say: initial offer/answer establishes DTLS. Re-invite offer comes in, same tls-id in outgoing offer, so DTLS context remains established. Re-invite answer comes in, has new tls-id. If we just ignore it, then the existing DTLS context doesn't get reset, and the new handshake would fail.

[alexbakker]

Hm, yes, you're right. Skipping DTLS restart in all cases is too broad. I suppose we need to check the state of the DTLS context to decide what to do.

[alexbakker]

I've been thinking about this some more and I'm struggling to come up with a comprehensive fix for this. One approach could be to never restart DTLS on answer in case we're passive and the tls-id changes, but instead restart on receive of a new ClientHello on a fully established DTLS context. I believe that would cover all cases. What do you think?

[rfuchs]

I'm not sure myself. AFAICT the underlying problem is being able to tell which DTLS connection/context is associated with which remote tls-id.

I'm wondering if the same thing could happen if we're active. We start establishing a new outgoing DTLS connection while the remote isn't expecting it (i.e. DTLS is already connected, we think it belongs to the old tls-id and do a reset, remote thinks it already belongs to the new tls-id).

Also I think the same problem could occur with a half-established DTLS connection? ClientHello was already processed but DTLS isn't complete yet, then a new tls-id comes in, together with a fresh ClientHello? I'm not sure if OpenSSL would accept that without a reset.

Either way, I think what you're suggesting makes sense and would be a good improvement.