To Open The Sky

Adding a Second CPU to the HP Kayak

The Black Art of Benchmarking

Benchmarking's goal is simple: To measure the performance of a given computer. This can be done by comparing it to some standard machine such as the manufacturer's base model, or perhaps to a supercomputer. (The latter comparison is done most often for scientific problems, and if done for your personal machine, is certain to induce envy. The envy is pointless because, even if you are someone like Larry Ellison, you are unlikely to be able to afford a supercomputer. The situation is much like a private pilot comparing his Cessna 172 to a Boeing 747. In purpose as well as in cost, those planes are in different categories.)

Ideally, you'd have access to the comparison machine and be able to run the same program on it as on yours. In practice, this is seldom possible even for businesses. Therefore, benchmarking is usually done with a standardized suite of programs designed to subject the computer to a realistic processing load. The benchmarking suite produces a set of numbers that can be compared to published results for various machines.

There are pitfalls in this. First, the programs might be biased in favor of one manufacturer's machines; but this is rare because most benchmarking suites come from independent firms. Of more importance are the many differences in possible computer configuration, and the difficulty of finding published results for computers that match yours in all but one factor.

In addition to the obvious ones of CPU and hard disk speed, some of the factors that affect performance are:

• Amount of cache memory, and how it is coupled to the CPU
• Amount of main memory
• Type of video card, and how much VRAM it carries
• I/O bandwidth, which determines how fast the CPU can exchange data with other components

Even assuming all the machine's components are perfectly matched, there is often a host of software settings that have to be optimized for best performance. Considering all this, many people do not bother with benchmarking; they simply buy a better computer if the one they have doesn't get the job done as fast as they like.

Here are comparisons of some flavors of Pentium-II Kayaks using 3D graphics benchmarks.

Here are some results using the floating-point routines of the SPEC2000 benchmark suite on 800 MHz Intel Pentium-III Coppermine machines. The results compare the two front-side bus (FSB) speeds of 100 and 133 MHz. The source (see table footer) goes beyond this to extrapolate performance to the 400MHz FSB used by Pentium-4 CPUs.

And here are some benchmarks for the current and past few generations of CPU chips from Intel Corporation. Note the omission from the published data of several important factors.