Software Design's Dirty Little Secret

We spend a lot of time wringing our hands about how to do a better job gathering requirements. I think this bit reflects a very common scenario in how we convince clients that we're experts, and how things really end up going. We put forth some process or methodology that sounds impressive, like it's really going to squeeze every bit of uncertainty out of the project as early as possible and lead us down a clean, straight trail to the solution. Utter crap, yet we continue to push it because that is what clients demand we tell them. So the Design Observer lets us in on a version of the dirty little secret, one that applies in good part to software design as well:

When I do a design project, I begin by listening carefully to you as you talk about your problem and read whatever background material I can find that relates to the issues you face. If you’re lucky, I have also accidentally acquired some firsthand experience with your situation. Somewhere along the way an idea for the design pops into my head from out of the blue. I can’t really explain that part; it’s like magic. Sometimes it even happens before you have a chance to tell me that much about your problem! Now, if it’s a good idea, I try to figure out some strategic justification for the solution so I can explain it to you without relying on good taste you may or may not have. Along the way, I may add some other ideas, either because you made me agree to do so at the outset, or because I’m not sure of the first idea. At any rate, in the earlier phases hopefully I will have gained your trust so that by this point you’re inclined to take my advice. I don’t have any clue how you’d go about proving that my advice is any good except that other people — at least the ones I’ve told you about — have taken my advice in the past and prospered. In other words, could you just sort of, you know...trust me?

So, is that how it really goes? Well, most of the time, it sort of goes that way. It has to. If custom software development were not so nebulous and prone to reexamination and modification at every step, it wouldn't be custom software. In other words, you would instead solve your well-defined problem by going to CompUSA and buying a solution for $79.99. The next alternative would be to modify your problem so it could be solved by the $79.99 solution. Some people do just that, and successfully. Using Microsoft Excel is a good example. It's probably the most widely used database application on the planet, though it was never intended to be a database.

If you want truly custom software to help manage a truly unique problem (and lots of people have such a need), you should know what it takes. Creativity. A prescribed process can't get you 100% of the way there. You must be willing to accept at least a good portion of the process to be empirical. That means it's not clean, it's not predictable and it's not going to make you comfortable.

When a prospective consulting firm approaches you and claims they have a neat, linear, sequential process that they have proven over countless years, they are setting your expectations in such a way that you are likely to be frustrated. In other words, they are partially to completely full of crap.