Blog · August 4, 2026

Why latency can make a fast circuit look slow

One of the more persistent mistakes in throughput troubleshooting is to treat a disappointing single-stream TCP result as conclusive evidence of circuit impairment. That conclusion is sometimes warranted. Quite often, however, it is not. In many cases the transport is functioning normally and the observed limitation is being imposed by TCP flow-control mechanics rather than by the underlying circuit.

This distinction matters because it changes both the diagnosis and the remedy. If the path has nontrivial round-trip delay and the sender is not permitted to keep enough data in flight, measured throughput can remain well below line rate even when the optics, interfaces, routing path, and provider transport are all behaving properly. In those situations, the test result is telling you something useful, but not necessarily the thing many people assume it is telling them.

The correct starting point is not suspicion. It is arithmetic. Before assigning blame to the carrier or declaring a high-capacity path defective, it is necessary to account for round-trip time, effective TCP window size, stream count, host buffer behavior, and test methodology. Without that context, throughput testing turns into a guessing exercise dressed up as engineering.

The basic idea

TCP must maintain a sufficient quantity of unacknowledged data in flight if it is to utilize the available path bandwidth. If the sender is allowed to place only a modest amount of data on the wire before it must wait for acknowledgments, then round-trip delay becomes a direct limiter of achievable throughput. As latency rises, the amount of in-flight data required to keep the path full rises with it.

The result is the number of bytes that must remain in flight if the path is to be fully utilized. If the effective TCP window is materially smaller than that value, the path will not be full. The transport may be capable of far more throughput than the test is demonstrating, but the sender is simply not being allowed to sustain enough outstanding data to prove it.

A simple example

Take a 1 Gbps path with a 10 ms RTT. The bandwidth-delay product works out to roughly 1.25 MB of data in flight:

If the effective TCP window is materially smaller than that value, then a result well below line rate should not be considered surprising. In such a case, the circuit is not necessarily slow; the flow simply lacks sufficient in-flight capacity to fill it.

Why single-stream tests mislead people

A useful field example appears in a Cisco community discussion involving a 1 Gbps point-to-point fiber circuit with approximately 10 ms of round-trip latency. The initial single-stream result was only about 116 Mbps. When the tester increased the stream count to ten or raised the TCP window to 2 MB, throughput improved into roughly the 650 to 800 Mbps range. The lesson is straightforward: a poor default single-stream result does not, by itself, establish that the circuit is poor.

Meraki's iPerf guidance reaches much the same conclusion from an operational standpoint: on higher-capacity links, parallel streams and window tuning are often necessary when the objective is to exercise the path in a manner that better reflects its actual capacity. That is consistent with what experienced operators see in the field.

What the math looks like in practice

Here is the way I usually explain it when someone is trying to set expectations around a high-latency path:

On a 10 Gbps path with roughly 118 ms of round-trip delay, the amount of in-flight data required to fill the path is substantial when compared with a short-haul metro circuit. This is why transcontinental and international TCP tests so often create the appearance of transport underperformance when the more accurate explanation is that the endpoints are operating with windows too small for the path under examination.

Using one of the examples I keep handy:

This is the point that most often catches people off guard. A 10 Gbps handoff tends to create the assumption that any reasonably competent test should immediately produce multi-gigabit throughput. TCP does not honor that assumption. If the sender and receiver are not maintaining sufficient in-flight data, measured throughput can remain poor even while the underlying circuit is operating exactly as designed.

Checking what Linux is willing to do

On Linux, I like to look at the receive memory settings first:

Most modern Linux systems implement dynamic TCP window scaling, which is helpful, but it does not guarantee that every test will converge on ideal throughput. Host tuning, congestion control behavior, application design, CPU availability, and socket buffer limits remain material variables. The presence of window scaling is not the end of the discussion. It is merely a prerequisite for having the discussion intelligently.

UDP testing is useful

When the objective is to separate TCP behavior from path behavior, UDP testing can be useful. UDP removes the acknowledgment and windowing mechanics inherent to TCP, which makes it a practical tool for distinguishing between transport-layer limitations and the raw forwarding characteristics of the path itself.

UDP, of course, has its own caveats. If the offered rate exceeds what the path can sustain cleanly, the result becomes a measure of packet loss and jitter rather than a pure indicator of throughput. In one example, a UDP test targeting 1 Gbps produced roughly 465 Mbps with approximately 21 percent packet loss. That does not invalidate the method. It simply indicates that the path or the endpoints were not carrying that offered rate without impairment.

Useful public iPerf targets

Hurricane Electric's public iPerf targets are handy when you want different MTU options to test against:

I also still like a longer test interval when I want to watch behavior settle out:

The takeaway

The practical conclusion is uncomplicated: a circuit should not be judged solely by the result of a single default TCP stream across a path with meaningful RTT. Calculate the bandwidth-delay product first. Then evaluate window size, stream count, host tuning, offered load, and test direction. Only after those variables have been accounted for is it technically responsible to attribute the observed result to the transport.

None of this implies that every poor iPerf result is harmless. Policing, queueing, duplex mismatches, optical faults, undersized hardware, poor endpoint implementation, and unstable hosts remain common causes of degraded performance. The point is narrower and more important: if throughput improves materially when the TCP window is increased or when parallel streams are introduced, then TCP mechanics are almost certainly part of the explanation and should be analyzed before the transport is condemned.

In practice, this is where disciplined troubleshooting saves time. The most reliable way to waste hours in network operations is to demand line-rate performance from a test that was never structured to make line rate possible. The arithmetic is not glamorous, but it is definitive, and it prevents a remarkable amount of unnecessary speculation.