NetworkTransform sync can fail when Interpolation is off due to network conditions

[zachstronaut]

Description

It is possible due to network conditions (jitter and packet loss non-zero) for NetworkTransform to receive in TransformStateUpdate() two incoming NetworkTransformState values in the same Unity frame.

If NetworkTransformState A contains an updated rotation but NetworkTransformState B only contains an updated position, then the m_LocalAuthoritativeNetworkState in ApplyAuthoritativeState() on the next OnUpdate() will only contain the data for state B, so the rotation change will never happen and the two clients will now be out of sync (different rotations).

I'm not exactly sure why Interpolation negates this problem, but I would suppose it has something to do with the fact OnNetworkStateChanged() is called on both A and B when they are received, so the data isn't lost to the interpolaters?

The problem will occur with real world network problems, but if you simulate something terrible like Delay=250, Jitter=40, Loss=20 it becomes a lot easier to reproduce.

Environment

OS: Windows 11
Unity Version: 6000.1.3f1
Netcode Version: 2.4.2
Netcode Topology: Client-Server

[zachstronaut]

You can also cause position to lose sync between clients this way.

Imagine NetworkTransformState A had a X position change but NetworkTransformState B had a Z position change, and both of those arrive on the same frame. ApplyAuthoritativeState() will not have access to the X coordinate data.

[zachstronaut]

The NetworkObject and NetworkTransform settings:

--

Here is a simple character controller that can be used to desync rotation. Hold D until walking due East and then hold A until walking due West. The rotation won't always sync on the non-host client by the time the position only data is being sent.

using UnityEngine;
using Unity.Netcode;

public class FakePlayer2 : NetworkBehaviour
{
    public float speed = 10f;
    public float turnSpeed = 500f;

    void Update()
    {
        if (IsOwner)
        {
            bool isMoving = false;
            Vector3 lookTarget = Vector3.zero;

            if (Input.GetKey(KeyCode.W))
            {
                isMoving = true;
                lookTarget += Vector3.forward;
            }
            if (Input.GetKey(KeyCode.S))
            {
                isMoving = true;
                lookTarget += -Vector3.forward;
            }
            if (Input.GetKey(KeyCode.A))
            {
                isMoving = true;
                lookTarget += -Vector3.right;
            }
            if (Input.GetKey(KeyCode.D))
            {
                isMoving = true;
                lookTarget += Vector3.right;
            }

            if (isMoving)
            {
                transform.rotation = Quaternion.RotateTowards(transform.rotation, Quaternion.LookRotation(lookTarget, Vector3.up), Time.deltaTime * turnSpeed);
                transform.position += transform.forward * Time.deltaTime * speed;
            }
        }
    }
}

--

And here is a simple character controller that can cause position to get out of sync. Simply hold any single direction (WASD) a bit and then switch to holding another different direction. Then, for example, the X changes won't always sync before the exclusively Z changes start arriving.

using UnityEngine;
using Unity.Netcode;

public class FakePlayer : NetworkBehaviour
{
    public float speed = 10f;

    void Update()
    {
        if (IsOwner)
        {
            if (Input.GetKey(KeyCode.W))
            {
                transform.position += transform.forward * Time.deltaTime * speed;
            }
            if (Input.GetKey(KeyCode.S))
            {
                transform.position -= transform.forward * Time.deltaTime * speed;
            }
            if (Input.GetKey(KeyCode.A))
            {
                transform.position -= transform.right * Time.deltaTime * speed;
            }
            if (Input.GetKey(KeyCode.D))
            {
                transform.position += transform.right * Time.deltaTime * speed;
            }
        }
    }
}

[zachstronaut]

Incidentally, is it possible that re-parenting and teleportation both potentially also suffer from bugs caused by network conditions -- multiple re-parenting or multiple teleporting messages arriving on the same Unity frame?

[zachstronaut]

Potential solution:

if an unconsumed (by OnUpdate) m_LocalAuthoritativeNetworkState exists when you receive new state, the previous and the new one could be combined.

For example, if the prev unconsumed m_LocalAuthoritativeNetworkState had rotational changes or an X position change, but the new state only has Z position change, you could add the new Z position change to the pending local authoritative state.

[NoelStephensUnity]

You can also cause position to lose sync between clients this way.

Imagine NetworkTransformState A had a X position change but NetworkTransformState B had a Z position change, and both of those arrive on the same frame. ApplyAuthoritativeState() will not have access to the X coordinate data.

Ok, I will look into this issue when interpolation is disabled...it shouldn't lose state... what should happen is that it "should" collapse state onto the current state...where it would look like:

• State update for tick 333 with position X update and state update for tick 334 with position Z update are received on the same frame due to poor network conditions.
• State update for tick 333 is applied to the local state.
• State update for tick 334 is applied to the local state.

So...that is what "should" be happening... but it looks like I might have introduced a bug (possibly when I did the interpolator update) for when interpolation is disabled.

I wrote a preliminary integration test to see if this issue was specific to the application of state updates and that side of things seems to be working:
(Added to the NetworkTransformGeneral.cs tests locally)

[Test]
public void TestMultipleStateSynchronization([Values] bool isLocal, [Values] bool timeTravelBetweenStateUpdates)
{
    m_NonAuthoritativeTransform.Interpolate = false;
    m_NonAuthoritativeTransform.InLocalSpace = isLocal;

    // Assure no new state updates are pushed.
    TimeTravel(0.5f, 60);

    // Get the non-authority's state
    var localState = m_NonAuthoritativeTransform.LocalAuthoritativeNetworkState;
    // Simulate a state update
    localState.UseInterpolation = false;
    localState.CurrentPosition = new Vector3(5.0f, 0.0f, 0.0f);
    localState.HasPositionX = true;
    localState.PositionX = 5.0f;
    localState.NetworkTick++;

    var lastStateTick = localState.NetworkTick;
    // Apply the simualted state update to the non-authority instance
    m_NonAuthoritativeTransform.LocalAuthoritativeNetworkState = localState;
    m_NonAuthoritativeTransform.ApplyAuthoritativeState();

    // Validate the state update was applied
    var xValue = isLocal ? m_NonAuthoritativeTransform.transform.localPosition.x : m_NonAuthoritativeTransform.transform.position.x;
    Assert.IsTrue(xValue == 5.0f, $"[Test1][IsLocal: {isLocal}] X axis ({xValue}) does not equal 5.0f!");
    // Simulate both having time between state updates and having state updates delivered back to back on the same frame
    if (timeTravelBetweenStateUpdates)
    {
        TimeTravelAdvanceTick();
    }

    // Get the non-authority state
    localState = m_NonAuthoritativeTransform.LocalAuthoritativeNetworkState;

    //Assure we have not received any state updates from the authority that could skew the test
    Assert.IsTrue(localState.NetworkTick == lastStateTick, $"Previous Non-authority state tick was {lastStateTick} but is now {localState.NetworkTick}. Authority pushed a state update.");

    // Simualate a 2nd state update on a different position axis
    localState.HasPositionX = false;
    localState.HasPositionZ = true;
    localState.PositionZ = -5.0f;
    localState.NetworkTick++;
    m_NonAuthoritativeTransform.LocalAuthoritativeNetworkState = localState;
    m_NonAuthoritativeTransform.ApplyAuthoritativeState();
    var zValue = isLocal ? m_NonAuthoritativeTransform.transform.localPosition.z : m_NonAuthoritativeTransform.transform.position.z;
    xValue = isLocal ? m_NonAuthoritativeTransform.transform.localPosition.x : m_NonAuthoritativeTransform.transform.position.x;

    // Verify the previous state update's position and current state update's position
    Assert.IsTrue(xValue == 5.0f, $"[Test2][IsLocal: {isLocal}] X axis ({xValue}) does not equal 5.0f!");
    Assert.IsTrue(zValue == -5.0f, $"[Test2][IsLocal: {isLocal}] Z axis ({zValue}) does not equal -5.0f!");
}

Is there anything else about the configuration I am missing? It looks like the NetworkTransform (from your screenshot above) is set to local space. Is the associated NetworkObject parented under something when this happens?

Trying to get a replication of the issue.

I'm not exactly sure why Interpolation negates this problem, but I would suppose it has something to do with the fact OnNetworkStateChanged() is called on both A and B when they are received, so the data isn't lost to the interpolaters?

If interpolation is enabled it will add the entry in the interpolation queue with the tick time stamp. Interpolation entries are then processed in their chronological order. Interpolators will hold the current interpolated state of the axis in question as well as the target values that it is interpolating towards. The exception to this if the state update is older than the current entry it will be dropped (theoretically this shouldn't happen when using reliable delivery, the default, but can happen if using unreliable deltas).

However, if you have something like this:

Tick 333: Z-Axis position updated in state
Tick 334: Quaternion updated in state

For the position, the entry will be added to the position interpolator and should be processed.
For the rotation, the entry will be added to the rotation interpolator and should be processed.

Since position, rotation, and scale have their own interpolators and since interpolators are computing the xyz axis (or xyzw for quaternions) values together it will assures that non-authority instances will reach their last known target state (and any other pending states in the queue).

[zachstronaut]

Is there anything else about the configuration I am missing? It looks like the NetworkTransform (from your screenshot above) is set to local space. Is the associated NetworkObject parented under something when this happens?

Yes, they are parented under another NetworkObject. These are player avatars that are spawned and end up parented to the game level.