The need for always available email had me recently equipping up with a Motorola Q Windows Mobile 5-based smartphone. With Exchange Direct Push email capabilities (where the device opens an idled HTTP connection, being notified expediently when new messages are available with the minimum of data throughput), it has served the core purpose admirably, and is a wonderfully handy little device.

The addition of wireless, always-available communications has made it infinitely more useful than prior abandoned outings with PDAs.

Ultimately it's a Blackberry(TM) competitor. While I'm only 60km from RIM headquarters, for me this was a better device than the more commonly chosen option. In this case the technology infrastructure didn't require a third-party to unnecessarily act as an middleman of messages.

I don't only use it for email, though. Every now and then the device serves secondary duty as a web browsing tool: The landscape QVGA screen isn't exactly copious, but it's enough for basic browsing for some sites, catching up on tech news and happenings in situations where a traditional network isn't available, and I don't want to open up a laptop.

This blog looks great on it. The "content" column perfectly filling the screen by luck rather than intent. The homepage requires an excessive 229KB of transfers, but at least most of those bytes are filled with content text. Nonetheless, I think I'm going to change the settings to show fewer days of history on the main page.

Despite the grand pronouncements by the telcos about their high speed, next generation networks, the speed is often closer to dial-up, and where throughput is high the latency is often poor, making pages with dozens of elements a time consuming affair. Even with a speedy connection, many telcos have low throughput limits with exorbitant fees beyond that.

Loading a page like Joel's discussion page -- a very basic text discussion site -- remarkably pegs in at 280KB or so of transfers (grab a copy of the extraordinary Firebug add-in for Firefox and look at the Net tab. It is often eye opening), overwhelmingly for scripts that have no use on the discussion site.

Sure, caching helps for subsequent requests, but on small devices there's often little room set aside to cache 100s of KBs of irrelevant scripts. Worse, the linked versioning -- adding a date version number as a parameter -- used by Joel and crew has seen the cached scripts invalidated frequently.

It's too bad there wasn't a, err, "function-level linking" for JavaScript, automatically eliminating all of the unused script from pages that don't require it.

One of the better sites for mobile browsing is Google: Recognizing the limits of the device (presumably by noting the Mozilla/4.0 (compatible; MSIE 4.01; Windows CE; Smartphone; 176x220) user agent string -- not sure why it says 176x220 when the actual resolution of the device is 320x200), it renders a pared-down (even more!) version of the search engine, better still automatically proxying search results through an agent that filters pages down to more mobile friendly forms. Very nice.

It's a good thing that Google filters results, as many sites just render terribly in the small confines of QVGA on a Windows Mobile 5 device.

Obviously the mobile browsing market is a tiny, but growing, contingent of users, but it is something I'm going to pay more heed to. Too often we presume that everyone has a 7Mbps high speed pipe feeding an ultra high resolution display, when that isn't always the case. As smartphones continue to take off, and providers facilitate use by easing off on the restrictions and excessive charges, it's going to become a very important market.

The success of Vista is of obvious importance to any developer targeting any Vista-only or Vista-enhanced technology, so the latest news that Vista sales are strong and higher than expected is of obvious interest.

It appears that Vista has been a stunning success for Microsoft, surprizing even the optimists inside the Redmond empire. This stands is a stark contrast with reports of general consumer apathy about this release, and many dire predictions about Vista's adoption (predictions that are dubious, given that Vista is pretty much assured a reasonable level of success given its automatic sale to virtually anyone buying a new PC. Corporations naturally delay adoption of new operating systems, as they have with every prior edition, so only a delusional expected it to storm across business desktops).

You can see their Q3 return on the SEC site. Under the Client division, you can clearly see that revenue has rocketed up, from $3.151 billion in Q3 2006, to $5.272 billion in Q3 2007.

There are a couple of massive, almost-Enronesque caveats to these numbers that deserve some serious scrutiny before you start your Vista-only product launch.

Firstly, they deferred $1.2 billion of XP sales from Q1 and Q2 2007 -- sales that qualified for the upgrade coupon -- rolling it into this result.

Client revenue increased for the three months ended March 31, 2007, primarily reflecting licensing of Windows Vista, including recognition of approximately $1.2 billion of revenue previously deferred in fiscal year 2007 pending the January 2007 release to consumers.

Counting out that rather dubious bit of accounting trickery drops the gross revenue to a growth of just $0.8 billion over the year earlier.

Secondly, some of the gain is attributed to the price premium applied to Vista Premium (effectively imposing a price increase over prior XP licenses, as most PC makers automatically deliver the Premium edition.)

During the quarter, the OEM premium mix increased 18 percentage points over the prior year to 71% driven by the demand for Windows Vista Home Premium.

Not to mention that the computer market in general has grown over a year earlier.

Based on our preliminary estimates, total worldwide PC shipments from all sources grew 10% to 12% from the third quarter of the previous year and approximately 8% to 10% from the first nine months of the previous year driven by strong consumer demand in both emerging and mature markets.

Not quite as successful as some reports are claiming: After a year of market growth, subtracting the hard-to-rationalize rolling-forward trickery, and considering that the price for the operating system was effectively raised via the Premium edition, and suddenly the situation doesn't look quite as rosey.

Rough back-of-a-napkin calculations have OS unit sales remaining relatively flat after incorporating in market growth. If this exceeded inside-of-Microsoft estimates, then clearly they're either lying, or they were expecting an implosion. Applauding about Vista's lofty percentage of OS sales should be quelled by the reality that virtually every new client PC OS sold now is Vista -- if a corporation does want XP, their recourse now is to buy a Vista license that grants them the right to install XP, though it'll carefully get added as another vote of confidence by the Microsoft beancounters.

There are several other surprizes in their Q3 report. For instance that the entertainment division (Xbox, Zune) saw revenue drop over a year earlier, and that the Online Services Business -- this is where Microsoft put a lot of attention recently, with the huge push of the Live platform -- saw a marginal revenue increase, with a significant loss increase.

The only bright spot of the whole quarterly report, from my perspective, is the business systems division: Office 2007 has seen good adoption and has very healthily contributed to earnings.

Reports that this quarterly report validates Vista's success are unfounded. Further, it puts a huge question mark over Microsoft's web and hardware initiatives (the complete failure of the diversification to actually add to the bottom line, instead of just drowning in losses -- excused early on as toothing pains, but there doesn't seem to be a point when they'll actually make money -- should raise serious concern. Microsoft is still held aloft by Windows and Office).

Being able to quickly and easily build team projects on a newly imaged PC is a development process necessity: A new team member, with not a whit of project knowledge beyond where to find the simple build instructions, should be able to follow a sequence of clearly documented steps -- automated where possible -- painlessly generating a build.

No unnecessary mapped drives and hard-coded UNC locations. No undocumented but necessary third-party tools at hard-coded locations. No byzantine by-hand registrations and muddifications.

This holds true for open source projects as well. While a grizzled kernel hacker obviously doesn't need hand-holding, they didn't start as a grizzled kernel hacker. At some point they were new to the code, and the number of obstacles they faced in those early days were probably significant indicators of the likelihood that they would stick with it, overcoming administrative type nuisances and getting to the point where they were actually working on the code itself.

Some may see the barrier to entry that often exists as a useful filter, only letting only the best of the best through, but that contention seems dubious. More likely an onerous getting-started process simply demotivates a lot of great talents from even bothering. Being an expert C++ developer doesn't mean that one wants to spend a day messing around with cygwin packages and dependencies, setting up countless poorly or incorrectly documented environment variables and configurations.

On this theme, I recently took a look at the state of the bleeding edge of Firefox -- I think Firefox 3 is going to be one of the most important applications in years, and is going to completely redefine the entire industry -- and was very pleasantly surprized to find how stunningly gorgeous the build process now is for a Visual Studio-using-Windows developer.

-Download and install Mozilla Build.

-Run the appropriate start-msvc batch file (e.g. start-msvc8.bat for Visual Studio 2005 users). I updated mine to set the CL environment variable the compilation flags that I wanted, as opposed to passing them on the --enable-optimize parameter of the .mozconfig file).

-In the appropriate location -- / is fine, given that it's actually in the msys subdirectory and not really at the root, get the client.mk file via the following trivial command.

 cvs -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co mozilla/client.mk

-Navigate into that folder
 
 cd mozilla

-Do a CVS get of the appropriate project (originally I was getting the source outside of the make script using the excellent TortoiseCVS, however it turns out that you can't just wholesale grab the tree, and should stick to the integrated CVS functionality).

 make -f client.mk checkout MOZ_CO_PROJECT=browser

-Configure an appropriate /mozilla/.mozconfig file (note that Windows will block setting that filename directly. Do a mv move command in the MINGW32 shell after saving from notepad or wherever. You'll likely just copy the block on the linked page for the appropriate project, however if you're adventurous you might try out the configurator tool).

-Build it!

 make -f client.mk build MOZ_CO_PROJECT=browser

This is the slickest, most painless process for such a large scale application that I've ever seen. I can just re-checkout and build daily if I'd like to be on the razor edge, though sometimes that will mean a broken build.

Now I'm running an ultra-optimized, stack-protected custom build of Firefox 3.

I'm actually delving through the code with relative ease, testing my custom changes absolutely painlessly (in my case, curiousity brought me into the javascript engine, found in the js subdirectory. While the code is inherently advanced -- it is a remarkably complex product -- it is reasonably easy to follow around and get a feel for).

Brilliant. Absolutely brilliant. Now I just have to find a way to put some obnoxious exhaust pipes on this bad boy.

When People Danced to the Macarena

The exploding importance of the Internet in the mid-90s brought tremendous change to the technology market. It forced industry leaders and followers to hastily adapt to the new opportunities and challenges.

It was a do-or-die time, and you had to embrace and adapt, or get extinguished.

To everyone but Microsoft, it seemed.

Despite the hurricane-force winds of change around them, the industry leading behemoth looked to be stuck in a recursive loop. While upstarts were racing in every direction, envisioning and implementing new uses for this growingly accessible platform, Microsoft seemed to be busy navel gazing, more worried about how to maintain the status quo.

Despite the relative success of Windows 95 -- the long-overdue migration to mainstream 32-bit computing -- Microsoft's slow-moving heft seemed to make them incredibly vulnerable during this critical transition period, making them appear a lumbering giant that could be toppled by the smallest adversary.

The young upstart Netscape appeared a likely candidate to shoot the mortal stone: Sales of Netscape's server and browser products yielded a revenue growth curve exceeding that of any software company in history. They were actually running a profitable business, which was a remarkable feat for a technology company at a time.

Their Netscape Navigator browser had fortified a seemingly insurmountable position in the marketplace . The company image was hip, and Mozilla adorned swag was flying out of their online store.

In that era of seemingly boundless opportunity, inebriated with the seemingly limitless potential of the company that he co-founded, Marc Andreessen made the infamous comment that the Netscape Navigator browser, coupled with the Java platform, would reduce Windows to an "unimportant collection of slightly buggy device drivers."

By then Gates had penned his famous internal "Internet Memo", demanding that the company focus on the Internet. The cruise ship Microsoft was ever so slowly changing course.

While the overwhelming majority of Microsoft's renewed focus turned out to be largely useless "internet-enabled" bedazzling of existing products -- the oft-lauded "turn on a dime" fiction about Microsoft's Internet revolution is grossly overstated -- where it really counted, the browser, Microsoft executed very well.

Microsoft's browser offering quickly became good enough that that average user couldn't be bothered to download and configure a competitor's products on their new PC (Microsoft didn't have to provide a better product, or even as good for that matter: It just had to be good enough to dissuade an average user from seeking out alternatives. This is a bundling reality used in all industries).

Add to that the fact that Netscape's development cycles got longer and longer, their innovation dried up, and their product got buggier.

Eventually Internet Explorer was the winning product on merit alone.

Soon we had an internet full of "Made for Internet Explorer" buttons. Much of the non-academic web had been Microsoft-ized, and you couldn't play unless you went where Microsoft was going today.

The rest is, of course, history: Internet Explorer rocketed to success, almost entirely at the expense of Netscape.

Knowing how things turned out, with the all-knowing clarity of hindsight, Andreessen's claims of course look like foolish bravado. Even at the time it sounded like nonsense: Java applets had shown little promise, delivering terrible performance, atrocious interfaces, and an awkward, crippled interaction with their host environment. The browser wasn't much better, limited mostly to rendering personal pages full of blink tags and gaudy color schemes.

I recall reading that quote from Andreessen back then (I believe in a Dvorak article in PC Magazine), puhshaw-ing in disbelief. I couldn't believe his audacity, and as a junior Windows-targeting developer at the time, with perhaps a bit of a fear of change (nobody likes when their skills, even at a beginner stage, are being obsoleted), I cheered on a Microsoft response.

"Bill Gates is going to CRUSH this guy!" I thought.

And of course Microsoft easily won that battle.

But are they losing the war?

The Pillars of Our Reliance on Windows

Windows as an operating system certainly has a lot going for it: It is feature rich, demonstrates a lot of technical excellence, and can credibly measure up against any competitor.

Yet for many users over the past decade, there was no choice: Windows was obligatory. It was exactly this hegemony that Andreessen felt his platform was upsetting.

His prediction was just a decade or so early. And instead of Java being their tag-team partner, it's JavaScript/AJAX, Flash, and the innovation and power of modern console gaming.

"I dual-boot to play games"

I hit a local department store recently to look for some educational games for my pre-school aged daughter. This location never had an extensive PC software selection, but I was still surprised to find the entire section had been removed, save for a couple of relics sitting in a discount bin.

The entire area was taken over by game console and handheld software.

Thinking this was an anomaly; I drove across town and checked their competition, and then their competition's competition, only to find the same at each: No PC software at all was for sale.

No games. No typing tutors. No foreign language training. No photo management software. No pre-school aged games.

Baffled, I hit the local EB Games location. Over the years I'd purchased dozens of PC games there, so I was shocked to find no PC software at all (the exception being a couple of ratty late-90s era boxes in a wire-mesh bin).

Determined, I ventured to the local Future Shop (the Canadian equivalent to Best Buy, and in fact the chain was acquired by Best Buy a few years ago, causing much confusion as it came in concert with the actual Best Buy chain itself) to find a small PC software section. While it was much smaller than it once was -- where once there were rows dedicated to just productivity applications, now a miniscule little section caters to the entire gamut of software -- at least it was something.

However compelling, my personal anecdote doesn't really prove much, but it does correlate with industry metrics that have shown retail PC software sales to end users to be stagnating or in freefall. Businesses keep buying their Office and Windows licenses, of course, and niche groups keep satisfying their business need, but what once was a vibrant retail market for applications and games has virtually disappeared. Some of this has been supplanted by online purchases, including some new electronic delivery method (which is how I got Half-Life 2 -- an impulse purchase is well catered to by a simple online purchase with immediate satisfaction), but much of it has just disappeared.

Consumers just aren't consuming PC software anymore.

The reasons are obvious.

Deja Vu All Over Again - The Rise of Console Gaming

On the gaming front, the PC has seen incredible competition from gaming consoles. Not only have those competitors evolved into technical heavyweights, the simplification of the entire gaming genre has equalized the playing field: Where once a mouse and a keyboard were mandatory to play any decent game, most popular games now feature simple interfaces that are equally accommodated on any platform, and the complex simulator type games, once the consistent chart toppers, are largely unloved.

You don't need a mouse to interact with an onscreen flower menu. You don't need a keyboard to communicate via a headset and in-game Voice-over-IP.

Consoles aren't the only reason for PC gaming's decline -- general internet use has taken a lot of time that people would have spent gaming, some of that time being spent being entertained by the countless Flash-based, cross-platform games available now.

Doomsayers have being declaring the death of PC gaming for years, as generations of consoles have come and then gone and Windows gaming has remained, but never has it seemed as likely to actually happen. In response, Microsoft is attempting some Windows gaming branding; perhaps realizing that it was a linchpin of their occupation of the home; but their intervention is likely too late.

So what does any of this have to do with Windows and Netscape and buggy device drivers?

One of the primary reasons many users felt tied to the Windows platform was gaming: If you wanted to play any of the prominent games at the time, that collection of slightly buggy device drivers was very important, and the game-du-jour was usually very tightly coupled with the platform. Aside from a couple of exceptions, PC gaming overwhelmingly meant Windows gaming.

The Netscape browser certainly wasn't a replacement for this. Neither was the Java platform.

This situation led many prospective Windows migrants to declare that they would make the move to Linux or the Mac or FreeBSD or whatever, if only they could run their current gaming obsession on it. Dual-booting is a half-measure that seldom held, and the direct graphics card access meant that gaming couldn't be accommodated via virtualization, so more often than not they just stuck with Windows.

"But my applications only run on Windows!"

Across the industry hundreds of thousands of solutions have migrated to the web, and if anything the pace is accelerating. Despite Microsoft submarining the overly-capable Internet Explorer team -- a team that brought us many of the innovations that we now enjoy in competing browsers -- the genie was out of the bottle: Many had experienced the incredible platform freedom, wonderful deployment model, and rich interfaces provided by web applications.

The classic computer purchase justifications (as stated by a million pleading children trying to convince their parents that a new gaming rig will be productive for the household) -- balancing the checkbook, storing recipes, authoring and sending letters (now email), maintaining databases -- can all be very competently accomplished online, from any modern browsers available on dozens of platforms. In many ways the experience is superior online, given the accessibility of the data from anywhere at any time.

Not every task can be performed online or from a web browser, and for those needs a plethora of cross-platform, often open source options have appeared (ex. GIMP, Open Office). Yet it remains that for an average user, the overwhelming percentage of their computer time now will be spent in their add-in enabled web browser, perhaps accessorized by one of countless available, many-network supporting IM clients.

Which is, of course, where we circle back to Andreessen's prediction: The most popular, and arguably capable, cross-platform browser is the Firefox browser. It is the phoenix (and was originally named firebird) that rose out of the ashes of the collapse of Netscape, the source code open sourced and revitalized with a many year reworking. While its market share numbers remain relatively small, its influence has been absolutely extraordinary. Even for sites that see 100% Internet Explorer users, the freedom and diversity offered by Firefox often leads enlightened development teams to ensure that they facilitate it just as well.

The rules of the game have completely changed. While many were prematurely declaring the end of Microsoft's dominance for years (every year for the past 7 years or so has been declared "The Year of Linux" by some open source evangelist or other), it has been years since the field has been so open for actual competition.

It has been a long time since the choice of platform held so few caveats and limitations.

We are entering a glorious time when the operating system really is an unimportant collection of device drivers, no longer driving completely unreleated application choices.

The Usenet Legacy

A decade+ ago, most "online" comments were conceived and birthed in feature-rich, fat-client applications. These were tools that generally offered a rich gamut of functionality: spell-checking, automatic intelligent threading, offline composition, selective content blocks (such as plonking unliked trolls, censoring expletives), automatic notification of certain keywords or topics, alongside a wide breadth of additional capabilities.

You could read and participate in conversations on a massive array of topics, from law and order, to product support forums for a particular vendor's database product, to the seedier side of the alt hierarchy. All using the same client application that you were comfortable with, configured just the way you liked it.

After authoring your brilliant, convincing argument (or your question about what video card to buy or how to call a certain API function) and hitting send, the application would queue it up much like an outgoing email, and when the opportunity arose (when you dialed up to your local BBS), it would send it to your local server via a standard protocol, where it would be shared with a decentralized universe of servers.

Usually your brilliant literary gem would be immediately visible to the world -- limited only by the rate of propagation -- though a small number of newsgroups had post moderation that requiring each new addition to first be approved.

The Advantage of Standards

This standardized protocol, message format, and distribution mechanism allowed for rich client functionality without reinventing it for every single newsgroup. Imagine how absurd it would have been if you used a different set of tools, with a different set of functions, to interact with comp.sys.ibm.pc.hardware.video than you did with comp.sys.ibm.pc.hardware.sound?

Just as importantly, the standard message format and transport protocol allowed for very easy indexing and archiving -- easily searchable across time and space by whichever search vendor did the best job. This is how we got the incredible functionality of DejaNews (which was later purchased by Google and rebranded as Google Groups), which managed to reach its indexing fingers back a decade earlier than it was even imagined.

If you do software development, you've probably found newsgroups to be by far the most useful resource to search when looking for answers: While a normal web search will yield thousands of noise responses and pay sites begging for money to see the answer (that they usually ripped from a usenet newsgroup), a quick tab over to the groups will usually immediately find the archive of someone who faced a similar question or problem, and the helpful replies.

Of course Usenet is still around and very much alive, and some sites still use NNTP. Unfortunately the quantity of useful answers has been declining, or at least that's my impression, as more and more conversations are being siphoned off into poorly structured, often unindexed islands of information.

Why is every new web app creating yet another terrible reinvention of a container for discussions? Why are we functionally stepping back 20 years for every single new forum (see Digg, YouTube, Reddit, and others for examples of colosally broken discussion systems that people interact with despite their enormous failures, having no alternatives. There are a few, Slashdot for instance, that are moderately evolved, but it took half a decade to achieve a somewhat usable system, and even then the failings are numerous)?

Worse still, why are so many sites storing conversations and threads in isolated silos of data, stored and communicated in completely non-standardized ways. I can easily find and reference threads that I reminisce reading on a usenet newsgroups 14 years ago (usually for "I told you so!" purposes), yet it's often impossible to find a thread or comment on a modern web forum even if I remember seeing it a month ago.

This isn't an argument for a return to the days of yore, and I'm candy-coated the history and usability of Usenet, but it does seem like a lot of people are continually reinventing the wheel, ignoring the lessons of the past. 

It does seem like the value of each additional piece being added to the global solution set is being diminished or completely lost: Where once we had clearly defined domains of information, clearly deliminated and indexed by topic, with a clear threading organization and meta-data structure (author, date, what other comment entry it's a reply to, and so on) that could easily be interpreted by anyone who understood the NNTP spec, now we're at the point where search engines have to try to interpret a million variations of rendering engines, inevitably losing most context and metadata, and that's only if they happen to even crawl across the conversations in the first place.

Somehow we need to find a medium, taking from the past while incorporating modern technology. Perhaps a new embedded commenting structure should be an addition to Firefox 3.

The Ultimate Goal

• Standard message structure and accessibility for archiving and indexing. Deja News provided an incredible example of the value this brought to the table.
• Standardized authoring tools and structure - a threaded discussion forum has almost exactly the same needs as every other threaded discussion forum. Users spend so much time authoring comments that it is remarkable that we haven't long had a rich <comment></comment> HTML element as a supersized TEXTAREA, supporting all of the nuances and features shared by virtually every conversation site.

We've been using Microsoft's Team Foundation Server for version control and basic work item/requirements/bug tracking for about 9 months now. All in all it's a good tool, though really it still feels like a version 0.8 beta that got pushed out the door a little early.

The Good

• It's a world better than Visual SourceSafe
  
• Superb integration with Visual Studio
  
• Performance is great locally, but also over a low-speed pipe
  
• Security is well-defined and adequate
  
• Branching and merging works well
  
• Multiple files are checked in as a changeset, rather than individually
  
• The requirements/work items/bugs functionality is half-decent, albeit very unpolished and feature-poor

The Bad

• Some basic functionalities, such as rollback, are missing (which is why I say that it seems like a pre-release beta in ways). There is a powertool hack that lets you rollback a changeset, in which case it checks out all of the affected files, does a get of the pre-checked out files, and then checks in the old file(s), leaving the "rolled back" version in the history.
  
• The application tier is incredibly fragile. If an enterprising team member decides to do some clean-up directly in the TfsVersionControl data-tier database (getting around missing functionality in the tools -- for instance there is no way to permanently delete, aka destroy, in the tools, remarkably, so if someone checks in a 500MB file and you want to remove it, you're forced to do it directly in the database), you will discover that a single missing related record -- the database doesn't define or enforce foreign-keys, so it isn't going to block the DELETE command there -- will cause the application tier to die a hundred deaths, excepting out on null values and other inanities. This is made far worse by the fact that the application tier caches a lot of lookup data, so check-ins/outs will work for a while after these related records were moved, making a simple database rollback impossible. Instead you need to go through every database manually rationalizing all of the data, determining where the application tier is dying.
  
  The worst part is that many of the unfound lookups should be complete non-events, either not displaying the unrelated records, or putting in placeholders while indicating that an error occurred. The idea that it takes down the entire version control system is completely unreasonable, and it really made me question the implementation.
  
• Offline support is non-existent in the front-end tools, with it expecting a constant connection to the source control web services. There is a dubious powertool that manually works by removing the project from source control, and upon reconnection you tell it to reattach and then do a sync.

The Ugly

• The application tier won't install on a domain controller, or on a 64-bit system, both limitation being entirely unreasonable and hindering for small shops.

All in all it's pretty decent, but I think they called it done a little early.

Software development is a difficult task to meter.

It's not for lack of trying.

For decades consultants have been evangelizing methods which, they claim, would allow an unskilled, casual observer to easily measure and compare productivity in a contextually agnostic way.

Their ultimate goal: To allow a drop-in manager, with only a superficial knowledge of the activities, skills, and complexities of a task or project, to easily compute metrics by which to dole out the frequency and intensity of whippings and rewards.

[Aside: Before anyone incorrectly presumes any of this is critical of software development managers as a group or individually, realize that it is nothing of a sort: I start with a brief analysis of the goal of such simplistic measures -- most organizations would like positions, including management, to be lower-skill and easier (cheaper) to fill, and such a simplification of the role is definitely in their interest, just as many dream of the panacea of no-skill, factory-type software development -- and then actually question the fact that developers themselves are often guilty of quoting these metrics. 9 times out of 10, developers have only themselves to blame for a lot of the problems with the profession. This is not yet another boring us-versus-them war cry pandering piece, like those that top the meme charts frequently]

The same methods -- if they worked as promised -- could be used to chart project progress ("We're 7868.2 ConsultaMarks towards the 11273.9 estimated for the entire project!").

Instead of relying upon the from-the-trenches observations of Randal the development group manager -- a grizzled vet of software development who manages with a hands-on style by becoming intricately aware of the domain challenges and unique contributions of each team member -- Lynn, the parachuted in middle manager, wants some simple numbers that can be sorted like her mutual fund returns, giving her some available sacrificial lambs when the next diversion-from-massive-executive-fumbles headcount reduction comes due.

Many proposed solutions have come and gone, with the most persistent being the infamous SLOC (Source Lines of Code)/LOC measure.

Source Lines Of Code

SLOC, if you haven't been afflicted with it, is an easily computed count of the number of lines of code in a given project/component/object (although first you have to agree on the definition of a "line of code", and this is a point of debate among SLOC champions). It's often used to count the number of lines of tested, complete code added by a particular contributor (easily accomplished with many source code repositories), allowing for the easy creation of nice little charts like the one above.

SLOC does have some quasi-legitimate uses: Given a common programming language and domain complexity, SLOC magnitude differences have a moderate correlation with general project size, and at the method level it is a rough indicator of gross complexity (see the article FxCop & Cyclomatic Complexity for a discussion of a loosely related metric, which is the number of intermediate language instructions generated from a method).

Applied at the individual or group level, usually as a cheap substitute for good management and project awareness, SLOC measurements are likely to encourage very destructive behaviors: Copy/paste coding, limited reuse of existing code found elsewhere in the organization and the industry, little motivation to prune code where necessary, overly convoluted coding, motivation for employees to only take on trivial coding tasks, and so on.

The Lemon Slice Lemon Roast

Envision a system that ranked cooks by the number of lemons they use to provide a restaurant's service each night: You're going to end up with a lot of dishes featuring copious stacks of lemons, even if ultimately it compromises the quality and organizational health of the establishment. While in some situations you could conceivably roughly compare overall restaurant success by the number of lemons they go through in a period, the comparison only holds true if all else remains equal (e.g. if otherwise the restaurants are very comparable, such as two restaurants serving Thai food): A deli restaurant might use very few lemons despite a healthy customer turnover, where an equally successful Greek restaurant might go through hundreds.

Far more logical would be to measure the number of dishes served -- while still imperfect, it would be much more useful than the LemonMetric. There is no comparable measure, with a similar level of granularity, as "dishes served" in software development (don't even think of mentioning the highly ambiguous "function point" metric as a simile).

Preaching To The Absentee Choir

"Geez...we all know that there are significant problems with the SLOC metric!" many will inevitably retort. "This is old news. You're preaching to the choir!"

"...but having said that, I saw a recent article that claimed that the average developer does {X} lines of vetted code a year. Are they really that slow? Me and my team must generate at least 20{X} a month! I hear that some superstars are responsible for 200,000 SLOC a year. They must be awesome!"

Comments just like that are probably being typed into a TEXTAREA at this very moment.

Why do so many comments about productivity -- even in the comfort of secret No-PHB hideouts -- inevitably elicit gloating commentary about personal SLOC accomplishments? Why do we hear gushing superlatives about the "superstars who push out 100s of thousands of SLOC a year"?

Why do so many in this industry perpetuate this destructive myth?

~SLOC

Let me flip this metric on its head, and state that, if anything, for a certain domain of project, and a certain class of developer, a high rate of SLOC can actually indicate poor programming practices.

In the nascent days of software development, many teams had a compiler or an interpret and that was pretty much it. They were responsible for building the majority of functionality from scratch. The pace of SLOC creation was tremendous (especially given that much of that implementation was trivial, allowing them to code as fast as they typed. Little time needed to be spent problem solving or planning: It doesn't require a superstar to code yet another string copy function).

As time went on, organizations compiled volumes of reusable internal code for all of their domain specific problems.

From an individual developer perspective, no longer was it acceptable to simply "run and start coding". Now you had to spend some of your time learning, assessing, and implementing shared internal code in your projects.

And it wasn't just inhouse: The frameworks and libraries provided with our tools have been growing by leaps and bounds, immediately solving a huge range of traditional problems and tasks with well tested, robust, feature rich solutions.

In the industry as a whole, code sharing has become widespread, with excellent code being available for virtually all common (and even uncommon) tasks.

So many solutions are available in the industry and supplied within our libraries/frameworks, that even organizational code reuse can be indicative of a problem.

Yet somewhere out there someone is hand-writing an FTP client implementation. Somewhere developers are wasting a tremendous number of man-hours by poorly, and unintentionally, duplicating code that exists in the frameworks and libraries that they're already using, or which can be easily found in license compatible open source projects.

Not Invented Here

A part of the reason for this is laziness -- it's a real bother having to look through the documentation and amongst search engine results, and that's hardly as much fun as just coding. Another part of the reason is a classic perception flaw that virtually all developers suffer from: Endless optimism about the capabilities and quality of the code we produce -- which we always think we'll finish much quicker than we really will -- coupled with an unreasonable pessimism about the applicability or worth of code we could source from another group in the organization, or from an external source.

I'm often guilty of these failures of perception, as are the overwhelming majority of developers.

Conclusion

Rarely does a developer actually tread across new ground (and I'm certainly not just talking about business back-end "CRUD" developers -- even in signal processing, embedded development, game development, and other less common branches of software development, most of the "solution" is the integration of existing work in novel ways, adding an envelope and façade of customization).

For the rest of us, our job is partly to generate the generally small amount of niche-specific code, usually aiming to build it with the most concise -- aka minimal -- code necessary, with the bulk of our time being in the analysis and integration of the extraordinary volumes of available solutions.

Where niche, custom code is necessary, generally it will be for a non-trivial task, and the SLOC pace will be unavoidably glacial.

For the overwhelming majority of developers in the industry, the only value of SLOC measures is as a warning sign, not an indication of progress.