How do you define performance?

System performance is an interesting concept; everyone seems to define it differently. To some, it involves chewing through a complex spreadsheet. To others, it's how fast a 3D video sequence can be rendered, or how easily Web pages are served up.

Call me a rebel, but after years of living off of a BlackBerry, my thinking has evolved. As much as I focused on megahertz and gigahertz for much of my computing life, the most important criteria for me these days are how fast the thing turns on, and how long it stays on before I have to recharge it.

It goes against everything we grew up with, of course, as the PC industry has always been obsessed with the electronic era equivalent of brute force horsepower, rather than eco-friendly efficiency metrics. But when I'm sitting in a deserted airport terminal with an hour's worth of work to power through and only 45 minutes of remaining battery life, the last thing I'm thinking about is processor performance.

Machines (and uses) have changed

However we choose to define performance, we've always assumed that performance measurement benchmarks were run on machines that had already been powered on and booted up. We also assumed that these machines were plugged into a power outlet. These days, these assumptions are laughable, because outside of cubicle-dwellers sweating over dusty desktops for their corporate overlords, few of us leave our computers on 24/7/365. We take them with us to places like Starbucks, where finding a power outlet is about as challenging as locating the lost city of Atlantis, and we don't have the luxury of time to sit and wait while our machines boot up.

This shift in performance metric priority is being driven by two major changes in attitude and technology:

• Green matters. Conserving power didn't matter much when 30-pound desktops ruled the planet and no one paid attention to how much all that electricity cost. Now that we've mobilized, disconnected, and learned the value of power-performance curves, the cost and availability of power are suddenly much more important to us. It isn't feasible to leave a laptop or other mobile device perpetually powered on. Even if we didn't run the risk of frying our laptops inside their padded cases, their three-hour battery life would leave us high and dry before lunchtime.
• Software deployment has evolved. Corporate software distribution methods that required conventional desktop machines to be powered on at night, are yesterday's news. Today, commodity chipsets, firmware, and monitoring software allow machines to be remotely -- and automatically -- woken up, updated, and shut down. Web-based remote management solutions keep mobile machines in top form as well. In many cases, cloud-based software infrastructure is largely obviating the need to update client code in the first place. One reason that everyday users neither know nor care when Google Docs adds a new feature is because the software no longer makes a big deal of it. Windows tends to announce when it's been patched or updated, but Web apps typically don't.

Beyond eco-friendliness, better, smarter client hardware, network control, and an increasingly thin software stack, lies another simple truth: Today's mobile machines lead very different lives than more sedentary desktop hardware we all grew up with. We shut them down and hibernate them before shlepping them from one place to another, and when we get to our destination and open the lid, we expect things to happen quickly. The more dynamic nature of the modern workplace, coupled with the transition of mobile handsets from basic voice devices to full-featured computing devices, has radically altered our expectations of how these devices -- mobile or not, pocketable or not -- should work.

Thankfully, the industry is tentatively taking notice. The turtle-slow speed with which the average computer wakes itself up, freshens its desktop, and presents its happy face to its owner, is finally getting some attention from hardware vendors. HP's QuickWeb and Dell's Latitude ON, for example, boot into reduced-scale environments for quick access to simple apps like e-mail and contact management. Beyond launch performance, once these machines are on, the amount of time they actually remain on is also finally becoming a key concern for hardware makers.

I'd like to ask them what took so long, but that would be cliché. What matters is that this real-world performance metric is no longer being ignored. And as operating systems initially developed for handsets such as Google Android start showing up on netbooks and other lower-end machines, end users will increasingly expect these larger devices to offer the same kind of instant-on, battery-friendly experience provided by more traditional handhelds.

Not so fast

As heartening as it is to see vendors finally wake up to power-on and battery life performance, we're not quite up to speed yet in terms of standardizing and normalizing the messages they send to the broad base of consumer and enterprise buyers.

For example, Nokia claims its new Booklet 3G will run for 12 hours before it needs to be recharged. In the absence of standardized power-based performance metrics, however, it's difficult to take Nokia at its word. Is it 12 hours of watching a movie? Is wireless on or off? If so, is it Wi-Fi, 3G or both? Since vendors have no established baseline for these performance claims, they're free to say pretty much whatever they want.

As a result, they'll continue to use rather useless metrics (like how many cells their batteries have) to flog their machines. But computer vendors making a big deal out of six- or nine-cell batteries aren't much removed from automakers hawking the size of a given vehicle's gas tank. In both cases, we still don't know how far the thing will go. In the real world, we continue to lack industry-accepted parameters akin to the EPA's fuel economy standards that allow computer buyers to make apples-to-apples comparisons.

Change often comes to an industry when consumers demand it. And as we spend ever greater stretches of time using battery-powered devices far from home, the onus is on us to challenge vendors to get straight with realistic methods of measuring such real-world performance parameters like battery life and power-on availability. I'm sure I'm not the only one who looks forward to evaluating comparable machines based on performance figures that weren't first invented by the marketing department.

Carmi Levy is a Canadian-based independent technology analyst and journalist still trying to live down his past life leading help desks and managing projects for large financial services organizations. He comments extensively in a wide range of media, and works closely with clients to help them leverage technology and social media tools and processes to drive their business.