The Enemy of the Good

August 3, 2011

I’ve pursued programming as a hobby and occasional sideline income for nearly twenty years. Like a skipping stone, I go long distances between needing to do any development, only occasionally dipping into the water. This is not an efficient way to be a programmer, but I’m not a programmer, I’m a renaissance geek – for me programming is a means not an end.

Having to re-learn the current state of the art every so often for a specific project without staying immersed in the development of the technologies involved has given me perspective on a few issues that everyday professional programmers are often too close to see.

Programming is getting harder. 

Because necessity is always the mother of my coding inventions, I typically re-enter the programming world running and with an eye towards finding the fastest route from a blank screen to the (typically not very complicated) application in my head. In the last 10 years especially, I’ve noticed a trend; every time I set out with a newer toolset to build a simple project, it’s harder to do the same simple stuff than the previous time. Each time the tools update, the simple stuff gets harder. It takes more lines of code to do the same job, and the code is more obtuse. Why should this be?

Nearly seven years ago, I developed a data driven website for a friend using the then-new Ruby on Rails platform. Any of you who were involved with Rails at that time remember the original screencast, wherein a functional web application (a blog) was created from a blank screen, right before your eyes, with a handful of very simple code, and in very short order. This was a revolution! Sign me up! Very quickly, I had developed the app, and all was well. A few years later, time came to update the app, and by this time Rails had graduated a whole version number. Maybe it was time to re-work it for the current version, I thought. Since so much had changed, I decided to go to the RoR website and look at the current version of the famous “blog screencast” – It was a complete mess.

Suddenly, creating the same simple web app wasn’t simple anymore. Vast chunks of prepared code were being pasted in to make the demo as quick as the first demo had been typing all the code from scratch. What the hell happened?

The same thing that happened to VisualBasic between 6.0 and VisualBasic.NET -

Some professional programmer decided it needed to be able to be all things to all people and made it so “powerful” that it wasn’t usable anymore.

Programming languages and frameworks are hard to create, so they are written by quite advanced programmers. Advanced programmers are surrounded by advanced programming work in demanding environments, and that makes them want to develop all their tools for that environment. This brings me to my next observation…

Professionals have a distorted definition of Good.

If you ask a typical cross section of professional developers about the two framework/languages I just mentioned (VisualBasic and Ruby on Rails) chances are most of them will tell you they suck. They’ll also say this about Arduino, Fritzing, RealStudio, Processing, Hypercard (Yes, I’m old in computer years) and any other simple tool for doing simple rapid development. They’ve seen how these things are implemented under the hood, they’ll say, and it’s a complete mess! What a horror show they’ll describe; it’s inefficient, lacks power and has poor flexibility.

They’re not wrong exactly, they just don’t live in the same world occupied by the rest of us. The longer you practice programming the better you get at it. The better you get at it the more complex the projects you end up working on. By the time you are a proper professional, you are working to standards which are so different from what rapid development packages are ostensibly made for that there is virtually no resemblance.

What they are really saying is that those tools would suck for what they do all day. Invariably, what they do all day is heavy stuff. Heavy work requires heavy toolchains, and they judge all toolchains by those standards.

But judging rapid development systems by high-end development standards is like judging street cars by race track standards. Yes it lacks hundreds of horsepower and carbon ceramic brakes, but since I use it to buy groceries and haul myself to the office, so what?

Yes, rapid development tools have inefficient implementation, don’t scale well, and can’t do everything but since I’m not developing inertial guidance systems for moon rockets or hosting solutions for, so what?

These “failings” don’t actually matter in most real world applications, for one simple reason:

Most needful applications are simple.

Anybody who’s worked in the white collar world long enough has encountered a variation of the Dire Spreadsheet. You know the kind I mean, the quivering 19 sheet mass of heavy duty formulas and barely literate scripting which Somebody Who Knew Excel created to do (or nearly do) a job that really needed to be done by either a simple database or a simple piece of custom software.

This is the space in which most software for the small and medium business is written (or rather, should be written). The band of problems in between the capabilities of the spreadsheet or scripted-augmented PDF form but beneath proper heavy duty enterprise development is where the vast majority of the need for custom software exists in the business world. Moreover, for every Dire Spreadsheet or Really Bad Access Database that attempts to fill one of those voids, there are a dozen more companies who simply give repetitive data processing and storage work to human beings with pens and paper forms or perpetually updated word processing documents. Tens of billions of dollars in productivity are lost to this “Undeveloped Band” every year.

These applications-in-potentiâ are child’s play by serious programming standards. They’re so simple, in fact, that there’s no money in creating them even for single independent developers.  With a simple language and development tools, the businesses themselves could plug these holes with perfectly serviceable software. Those tools are thin on the ground, because programming tools are made by professional programmers with professional expectations to match. Institutions without full time developers can’t meet the technical barriers of entry to tools designed for full time programmers and they can’t afford to hire the folks who can. This brings us to the final point :

Most needful software never gets developed at all. 

The promise of the computer in the workplace is largely stillborn. Every day, in nearly every enterprise, people waste human resources on problems that computers are better at solving, all for lack of customized software solutions.

Somewhere, a younger man or woman than I should be looking to solve this problem. They should be looking to turn the Undeveloped Band into a fortune, and a changed world. They should be looking to put software development into the hands of non professionals in the same way that layout was put into the hands of non professionals by desktop publishing.

I hope I get to see it.

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>