Anything that causes a single process to send a large number of line rate packets back to back onto the wire is just making burstiness in the Internet worse (and there are already phenomena that tend to coalesce packets into bursts as it is). So such bursts cross the Internet, where they enter their customer’s single, bloated, stupid broadband device.

Part of why we like fq_codel so much is that its natural behavior is to break up such bursts at any bottleneck, reducing the burst problem.

Needless to say, this is not good for latency, and may cause collateral damage of all sorts.

Note that the people are not watching latency (particularly from the customer’s end). That is as important/more important than bandwidth per CPU cycle: web sites are “stickier” the faster they are *to the user of that web site*. Getting packets out of your data center with the least CPU cycles may not achieve that goal, but sometimes the opposite.

We’ve been around the mulberry bush many times on offloading TCP itself since the 1980′s. But this means you get two maintenance headaches: both your OS, and the firmware in the NIC. Not a good idea.

It is also a classic example of “if a little is good, a lot must be better”. Any of these techniques may be helpful at scale 2 or 4 (they provide most of their benefit). But then people seem to think “it must be even better to turn the knob to 11″… It seldom is… But then the knob gets turned anyway.

The reason is because cwnd can grow more rapidly under FAST, and still avoid wrecking itself by exceeding line capacity, the way loss-based algorithms would if they grew too fast. So when a AIMD algorithm like Reno or BIC hits loss and cuts its cwnd in half, FAST can take over that newly available bandwidth faster than the loss-based one does. Vegas lost to Reno mostly because it didn’t grow to where it should be fast enough, not because the idea is inherently broken.

It’s true that FAST will also back off gradually as BIC grows, but BIC will still overreach and get its cwnd cut in half again before long, and it will usually happen sooner the 2nd time around when it’s sharing a bottleneck with a FAST flow (there is some oscillation, but see http://www.omikk.bme.hu/collections/phd/Villamosmernoki_es_Informatikai_Kar/2010/Sonkoly_Balazs/tezis_eng.pdf for an analysis of the fairness characteristics).

If I switch all my servers to delay based congestion control, the world is a worse place for me, until everyone else follows my lead. This is a Prisoner’s Dilemma; if we all co-operated by deploying delay based congestion control everywhere, it would be a net improvement. However, if you defect (by sticking to current congestion control methods), the best option for me is to defect too; if I cooperate and deploy delay based CC when you haven’t, your services are going to have an advantage over mine at every bottleneck link.

Further, this gives me an incentive to undo my deployment of delay based congestion control – I will get an advantage over you until you do the same. This makes delay based congestion control everywhere inherently unstable – you lose if someone else does packet loss based congestion control.

Your proposed solution involves “make and deploy a new protocol to communicate intent from the home router to the broadband head end”. Perhaps I’m misunderstanding, but I’m confused as to how that’s an easier problem to solve than “get most servers to use delay-based congestion control”.

And of course it doesn’t ruin your whole day to download a 5-packet text article, or most of the ajax junk you get when you leave random pages open. We’re mostly talking about being able to make phone calls while your computer decides to get software updates, and your kids are watching a movie. Right?

If that’s typically more or less accurate, then although it’s true that one big download from a server that hasn’t upgraded can be a problem, the more servers that are using delay-based congestion control, the fewer problems you should have. So I’m just saying you might want to include it in the list of solutions worth considering.

While delay based congestion control may be useful, I don’t understand how to get everyone using them, even if they work fine (which they may). All it takes is one TCP flow to a server that that does not use them to ruin your whole day.

Full disclosure: I work at Fastsoft, so I’m biased. (And I don’t mean to claim that our products will solve every transport-based performance problem. I’m still working on that.)

But I encourage you to read the papers and follow the math. If you have a way to have your downloads coming only from servers that use delay-based congestion control, that gives you another solution to the realtime performance problems caused by large buffers. (As far as I can tell, that’s not an idea in conflict with CoDel, btw.)