Rant warning: there is no single right answer for buffering, ever… (part 2)

It’s clear my previous post was ill formed.  Let me clarify a bit.

  1. I’m really, really, really, happy to see the work in Tomato USB w/reduced buffer bloat, as I am in the work going on to control buffering in DOCSIS (cable modems). Let me make this clear up front as somewhat of an apology to hechacker1.The enemy of the good is the perfect, and we can and should do what we can quickly to suffer less.
  2. Work to fix bufferbloat is going to be both a “do what we can right away” activity, as well as a long term fundamental redesign problem.  I am deeply unhappy as to quite the depth of that redesign problem.
I’d been thinking of the buffer management problem as a two part problem; the OS level queuing and buffering has been divorced from the device drivers, and how to better integrate the buffer management.  But hechacker1’s post made it clear it was yet more complex. Somehow, we need to get to a more intelligent unified view of queuing across all of these to handle the dynamic range found in today’s networks. Linux queue disciplines themselves may have independent buffering as well as in this example. The integration problem  is therefore more deep. I expect we’ll find similar issues elsewhere in other systems too.
Van Jacobson had warned me last fall of just how challenging the buffering problem was, and I had understood (part) of what he had told me; but it’s clear it has yet more dimensions than I had appreciated then.  That’s my deep unhappiness.

2 Responses to “Rant warning: there is no single right answer for buffering, ever… (part 2)”

  1. hechacker1 Says:

    I’m curious if my other comment will make it out of moderation.

    But, thanks for the follow up post. Your rant is clearly directed at the scope of the problem. I agree. It’s hard enough just trying to figure out where exactly all the dark buffers are in just one little router.

    I was thinking about this a bit, and perhaps a quick fix is to attempt to disable (or minimize) the buffers of the hardware itself. Right now the wireless driver of the WRT54-GL has 512TX buffers hard coded in.

    So even if you apply proper software QoS with lower limits, there’s always the network card that will happily accept up to 512 packets.

    I’m not quite sure how the chain of processing goes, but I assume it’s: Drivers -> Linux Kernel -> Software queue.

    It would seem each has there own set of limits:

    Drivers->Default to 512 buffers in most cases. Or 256 For Windows.

    Kernel-> txqueuelen parameter, often set to 1000. I don’t think Windows has an equivalent.

    Software queues->Can usually be set, but the widely popular SFQ uses 128 packets by default. pfifo is worse at 256 packets.

    QoS at the hardware would probably be the best in terms of efficiency and speed, but the cheap network cards don’t really do this.

    Fortunately, using software queuing is really fast, even for these little ARM routers. So perhaps the best we can do at this at the moment is to just minimize the network card’s buffers, and push all the queuing and QoS up to the software schedulers.

    It would seem we could use any of these locations (Drivers, txqueuelen, schedulers) to impose an overall limit. But only software schedulers let you do reordering and buffering.

    My build of Tomato’s firmware is basically focusing on tuning the built in software schedulers to be more sane about how much they buffer before dropping packets.

    In doing so, I’m also trying to disable (or minimize) all other buffering points in the line of processing packets. That would get my router closer to having one place where all the packets get managed.


    Eventually I want to look at the new schedulers proposed at bufferbloat.net and perhaps integrate them.

    • gettys Says:

      Sorry; just catching up. You caught me just before the weekend, and I wanted to do the part two post, since I had clearly not stated things the way I had intended to.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: