Apple Patents Portrait-Landscape Flipping: the patent system is broken…

I noticed with interest Slashdot’s article last week on Apple Patenting Portrait-Landscape flipping based on control of one or more accelerometers in Slashdot last week.  As I work at Bell Labs these days, I don’t read patents, so I’ll just go on the summary I read there.

Here’s some prior art from June 2001.  In that period, at Compaq/HP’s Cambridge Research Laboratory, we had ported Linux to the iPAQ handheld (with touch screen & expansion capability). Colleagues of mine, including Jamey Hicks, Andy Christian, Frank Bomba, Ben Curis had built an expansion pack for the iPAQ, called the BackPAQ (just like Apple has an I fetish, Compaq had a paq fetish and liked “i”s as well), with accelerometer, camera, and additional expansion capability including additional battery, for our (and other’s) research as part of “Project Mercury”; it was obvious that such devices would become standard in short order, but no device at the time had them integral. Quite a few BackPAQ’s were built and distributed to researchers around the world (small number of hundreds, if I remember correctly). We wrote some papers, distributed a bunch of BackPAQ’s to like minded researchers around the world, and demonstrated the code at the Usenix conference and elsewhere, and published all the code on handhelds.org (which seems down at the moment). The probability of Apple employees having seen this device and it rotating the screen is an absolute certainty; not only did we show the BackPAQ off at numerous conferences, but we built significant numbers used at universities.

It was blindingly obvious to us that hooking up the accelerometer to be able to rotate the screen would be “a good idea”.  Keith Packard and I wrote the xrandr X Window System extension specifically to support screen rotation, for the iPAQ handheld using his TinyX driver (the X extension then became a standard part of the X Window System releases in X.org).  I wrote (in an hour or two) the first version of the xaccel daemon that took the accelerometer data and controlled the screen rotation.  I first packaged it (in ipgk format, for the iPAQ Familiar Linux distribution) on June 11, 2001 to enable the code’s distribution. Ironically, I like what I remember of xaccel’s behaviour better than what I now see on the iPhone and the iPad I own.

Since I can’t go reading Apple’s patent itself, I’ll just note:

• This is a handheld device, with 802.11 wireless (later versions of the iPAQ became phones).
• It has a touch screen
• It has an accelerometer in the BackPaq
• It used the data from the accelerometer with simple heuristics to control the orientation (portrait or landscape) of the screen (in this case, running the X Window System

Now, maybe you’d like to quibble and claim the idea of putting an accelerometer in a hand-held device is non-obvious.  I think it was pretty obvious, myself, and doing that goes to the group working on Project Mercury. I don’t remember any patent being filed there. And having done so, it seemed obvious to hook it up to the screen. I know we did not file any patents. Are either of these ideas worth a patent? Personally, I think both ideas are pretty obvious, the first idea more original than the second.

But I’m sure the first handheld device with touch screen, with accelerometer, rotating the screen under control of that accelerometer was in my hand running my code below, sometime in the year 2000 or 2001 (I haven’t tried to excavate the exact date),  and that it was widely published on the Internet and used by hundreds of people.

Since handhelds.org seems down at the moment, I spent 5 minutes digging around for the code itself elsewhere.  It’s short enough I include it below (looks like the copyright notice got cut and pasted from the xrandr code); it was called xaccel.c, strangely.

Update 1:

Comments make it clear I fired before aiming carefully: the patent at question apparently is on multitouch gestural overrides to accelerometer screen flipping, apparently. If so, my apologies to Apple.

We have three problems here:

1. prior art, which may not apply to my example certainly we did not have a multi-touch screen to play with and did not explore that area.
2. Obviousness may be in the eye of the beholder, but certainly I’ve seen ideas which were non-obvious. The current broken patent system is encouraging filing of patents just for protection of every trivial idea, and to use as weapons against competitors, whether there is merit in them or not.
3. the treble damages problem, which is why I did not go read the patent in the first place, and stifles actual innovation (independent of whether you thing software patents are a good or bad idea, being unable to know what is going on elsewhere defeats part of the original bargain of why patents were granted in the first place.

And I still like my algorithm better than what I experience on the iPad, which often flips the screen when I don’t want it to flip and begs out for overriding.

Update 2

Jaharks of CMU in a comment below notes that the Itsy folks did gesture based screen rotation on the Itsy.  Quite a few Itsy‘s (the spiritual predecessor to the iPAQ, and to my knowledge the first handheld device to run Linux, and the inspiration/cause of our handhelds.org work) were built and distributed to universities, along with the source code.

/*
 * $XFree86: xc/lib/Xrandr/Xrandr.c,v 1.4 2001/06/07 15:33:43 keithp Exp $
 *
 * Copyright  2000 Compaq Computer Corporation, Inc.
 *
 * Permission to use, copy, modify, distribute, and sell this software and its
 * documentation for any purpose is hereby granted without fee, provided that
 * the above copyright notice appear in all copies and that both that
 * copyright notice and this permission notice appear in supporting
 * documentation, and that the name of Compaq not be used in advertising or
 * publicity pertaining to distribution of the software without specific,
 * written prior permission.  Compaq makes no representations about the
 * suitability of this software for any purpose.  It is provided "as is"
 * without express or implied warranty.
 *
 * COMPAQ DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL COMPAQ
 * BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION
 * OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
 * CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 *
 * Author:  Jim Gettys, Compaq Computer Corporation, Inc.
 */

#include <stdio.h>
#include <X11/Xlib.h>
#include <X11/Xproto.h>
#include <X11/extensions/Xrandr.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <math.h>
#include <stdlib.h>
#include <sys/poll.h>
#include <errno.h>

#define ACC_DEV "/dev/backpaq/accel"
#define ANGLE (.5 * 256)
#define RMAX (.75 * 256)
#define RMIN (-.75 * 256)

#define POLLRATE 100
#define NSAMPS 4

static int err = 0;

int
main (int argc, char **argv)
{
  Display *dpy;
  XRRScreenSize   *sizes;
  XRRScreenConfiguration *sc;
  int		    nsize;
  Window	    root;
  int		    n = 0, i;
  Status	    status;
  Rotation	    rotation, current_rotation, rotations;
  XRRScreenChangeNotifyEvent event;
  static Rotation old = 0;
  int accfd;
  int nsamps = 0;
  struct {
    short x_accel;
    short y_accel;
  } racc;

  struct accel {
    int x_accel;
    int y_accel;
  };
  struct accel acc[NSAMPS];
  struct accel ave;
  struct accel cal = { 0, 0 };

  int calibrated = 0;
  int product;
  int asq = ANGLE * ANGLE;

  pid_t pid;

  /*
   * this is coded to sleep for 10 seconds if either:
   *  the X server isn't running, or:
   *  the accelerometer isn't present, so it can run under any circumstance.
   */

  pid = fork();
  if (pid == -1) {
    fprintf(stderr, "Fork failed errno = %d\n", errno);
  }
  if (pid > 0) {
    printf("%d\n", pid);
    exit(0);
  }
  for (;;) {

    dpy = XOpenDisplay (NULL);
    if (dpy == NULL)    {
      sleep(10);
    }

    root = DefaultRootWindow (dpy);

    sc = XRRGetScreenInfo (dpy, root);

    accfd = open(ACC_DEV, O_RDONLY);

    if (accfd == -1) {
      sleep(10);
    }
    else {
      while (1) {
	n += 1;
	if (n >= NSAMPS) n = 0;

	poll (NULL, 0, POLLRATE);
	if (read (accfd, &racc, sizeof(racc)) == -1) {
	  goto leave;
	}

	nsamps += 1;
	/* compute running average, yes, ma, there are better ways */
	acc[n].x_accel = (int) racc.x_accel;
	acc[n].y_accel = (int) racc.y_accel;

	ave.x_accel = 0; ave.y_accel = 0;
	for (i = 0; i < NSAMPS; i++) {
	  ave.x_accel += acc[i].x_accel;
	  ave.y_accel += acc[i].y_accel;
	}

	ave.x_accel = ave.x_accel / NSAMPS - cal.x_accel;
	ave.y_accel = ave.y_accel / NSAMPS - cal.y_accel;

	/* lame attempt at self calibration */

	if (nsamps == NSAMPS) {
	  cal = ave;
	  if (argc > 1) printf("cal x = %d; cal y = %d\n", cal.x_accel, cal.y_accel );
	}
	product = ave.x_accel * ave.x_accel +
	  ave.y_accel * ave.y_accel;

	if (argc > 2) printf("ave x = %d, ave y = %d\n", ave.x_accel, ave.y_accel);

	if (product > asq ) {
	  rotation = 0;

	  /*
	   * if(ave.x_accel > RMAX) rotation = 8;
	   * if(ave.x_accel < RMIN) rotation = 2;
	   * if(ave.y_accel > RMAX) rotation = 4;
	   * if(ave.y_accel < RMIN) rotation = 1;
	   */

	  if(ave.x_accel > RMAX) rotation = 1;
	  if(ave.x_accel < RMIN) rotation = 4;
	  if(ave.y_accel > RMAX) rotation = 8;
	  if(ave.y_accel < RMIN) rotation = 2;
	  /* don't try to rotate unless needed */
	  if (rotation && (rotation != old)) {
	    status = XRRSetScreenConfig (dpy, sc, DefaultRootWindow (dpy),
					 (SizeID) 0, 0, rotation, CurrentTime);
	    if (status == RRSetConfigSuccess) old = rotation;
	  }
	}
      }
      leave: XRRFreeScreenInfo(sc);
    }
  }
}