In distributed systems, physical clocks on independent nodes deviate due to thermal variations and crystal imperfections. Clock synchronization minimizes skew (instantaneous difference) by correcting for drift (deviation rate).

Clock Offset and Delay Estimation (NTP)

The Network Time Protocol (NTP) estimates the offset between a client (C) and a server (S) using four timestamps: -T_1: Client sends request. -T_2: Server receives request. -T_3: Server sends response. -T_4: Client receives response.

The Math

Round-trip delay (\delta):

Clock Offset (\theta):

This calculation assumes network symmetry (\text{delay}_{C \to S} \approx \text{delay}_{S \to C}). Asymmetry is the primary source of error in NTP.## NTP Stratum Hierarchy

Trust is organized into strata:

• Stratum 0: Reference clocks (GPS, Atomic clocks).
• Stratum 1: Servers directly attached to Stratum 0.
• Stratum 2: Synced via network from Stratum 1.

Correction Mechanisms

1. Stepping (The Jump): For large offsets (>125\text{ms}), the clock is immediately set. This breaks monotonicity.
2. Slewing (The Drift): For small offsets, the kernel subtly speeds up or slows down the clock (typically at$500\text$) to converge without jumps.

Precision Time Protocol (PTP / IEEE 1588)

PTP is required for sub-microsecond precision (e.g., HFT, 5G base stations). It differs from NTP by using hardware timestamping at the MAC/PHY layer, bypassing OS kernel jitter.

Logical vs. Physical Time

Because physical clocks cannot be perfectly synced, distributed databases use logical time for ordering:

• LamportClocks: Tracks partial ordering based on causality.
• VectorClocks: Detects concurrent writes.
• Hybrid Logical Clocks (HLC): Combines the monotonicity of logical clocks with the approximate wall-clock time of physical clocks.

Operational Failure Modes

1. Leap Second Smearing: Abruptly adding a second can crash applications. Modern providers (Google, AWS) "smear" the extra second over 24 hours.
2. Virtualization Skew: Hypervisor context switching can cause massive, unpredictable clock jumps in VMs. Use the hypervisor's specific clock source (e.g., kvm-clock).
3. Partitioned NTP: If a cluster loses access to external Stratum 1 sources, nodes will drift relative to each other. Use a local Stratum 1 source (GPS receiver) for isolated environments.

Implementation Checklist

• Run chronyd instead of ntp for faster convergence.
• Monitor offset and jitter via chronyc sources -v.
• Disable ntpdate cronjobs; they cause non-monotonic jumps.
• Use TrueTime (Google Spanner approach) if you require external consistency across global regions.