Software Design's Dirty Little Secret

We spend a lot of time wringing our hands about how to do a better job gathering requirements. I think this bit reflects a very common scenario in how we convince clients that we're experts, and how things really end up going. We put forth some process or methodology that sounds impressive, like it's really going to squeeze every bit of uncertainty out of the project as early as possible and lead us down a clean, straight trail to the solution. Utter crap, yet we continue to push it because that is what clients demand we tell them. So the Design Observer lets us in on a version of the dirty little secret, one that applies in good part to software design as well:

When I do a design project, I begin by listening carefully to you as you talk about your problem and read whatever background material I can find that relates to the issues you face. If you’re lucky, I have also accidentally acquired some firsthand experience with your situation. Somewhere along the way an idea for the design pops into my head from out of the blue. I can’t really explain that part; it’s like magic. Sometimes it even happens before you have a chance to tell me that much about your problem! Now, if it’s a good idea, I try to figure out some strategic justification for the solution so I can explain it to you without relying on good taste you may or may not have. Along the way, I may add some other ideas, either because you made me agree to do so at the outset, or because I’m not sure of the first idea. At any rate, in the earlier phases hopefully I will have gained your trust so that by this point you’re inclined to take my advice. I don’t have any clue how you’d go about proving that my advice is any good except that other people — at least the ones I’ve told you about — have taken my advice in the past and prospered. In other words, could you just sort of, you know...trust me?

So, is that how it really goes? Well, most of the time, it sort of goes that way. It has to. If custom software development were not so nebulous and prone to reexamination and modification at every step, it wouldn't be custom software. In other words, you would instead solve your well-defined problem by going to CompUSA and buying a solution for $79.99. The next alternative would be to modify your problem so it could be solved by the $79.99 solution. Some people do just that, and successfully. Using Microsoft Excel is a good example. It's probably the most widely used database application on the planet, though it was never intended to be a database.

If you want truly custom software to help manage a truly unique problem (and lots of people have such a need), you should know what it takes. Creativity. A prescribed process can't get you 100% of the way there. You must be willing to accept at least a good portion of the process to be empirical. That means it's not clean, it's not predictable and it's not going to make you comfortable.

When a prospective consulting firm approaches you and claims they have a neat, linear, sequential process that they have proven over countless years, they are setting your expectations in such a way that you are likely to be frustrated. In other words, they are partially to completely full of crap.

Software Architecture - Why it Matters

I have worked with a few people over the years whom I would describe as the "just get it done and soon" type. These guys (these few have all been men) eschew putting much thought into the software they build, or thinking about kaizen or continuous improvement. Working with them is frustrating because they tend to create systems that are better torn down than modified. In other words, thanks to their myopia and/or hubris (to the sales manager: "sure, we can do that in two weeks!"), the software they build has massive design debt and huge costs to maintain. (I am not going to get into the other extreme right now -- the guys who suffer from analysis paralysis -- but that, too, can be a big yet much different problem...)

The well known benefits of good design are written about throughout the software development world. But the one benefit that may be most important is this: good software architecture is motivational.

Working for a company that supports thoughtful software design ignites pride. Having people at all levels of a company that understand what good architecture is makes this possible. For me personally, it is a rush knowing that everyone from developers on my team to the CEO understands that this philosophy contributes to:

• A positive perception of our product
• Motivation among the team, and
• Increased profitability.
  

Motivation and internal pride glue together the customer satisfaction and financial sides of the business. Good software architecture is necessary to have a successful software business. Customer satisfaction and long term financial health are keys to "success," in my opinion. That's not all that defines success, but for me those two things are required.

I have seen the architecture weenies that get so bogged down in design issues that they make slow and pained progress. However, that is usually when they are in an organization that is unbalanced and indulges them because it lacks good business people. It is more often, in my experience, that the siren song of "just get it done now, it's good enough" can influence people in almost any organization, and is therefore more dangerous.

ASP.NET ObjectDataSource and Business Objects

This past Friday's ASP.NET MVP Chat was very good, a lot of participation by well known experts. I learned a lot just sticking around reading both the experts' answers and the side banter.

Anyway, one issue that has been bugging me is whether to use declarative data binding in ASP.NET 2.0. So I asked Scott Guthrie if it was really a prime time thing or just something to "give good demo." He and Scott Mitchell both suggested it's the way to go, so I poked around and found the following:

ASP.NET 2.0 Data Tutorials
Peter Kellner's SQLDataSource Comments
Brendan Tompkins ObjectDataSource/GridView Experience
DataTableAdapter Article at CodeProject
Rocky Lhotka's ObjectDataSource Frustration

I am not liking what I see.

The gist of all this seems to be that the ASP.NET 2.0 data binding controls are fine for ADO.NET data objects, but when you have custom business objects, a lot of coding jujitsu may be required. In other words, not only are you not saving time because you have to code a fair bit to make things like sorting and filtering work, but you are taking a leap of faith that the magic stuff being done for you behind the scenes is going to just work. The posts on the ASP.NET forums indicate a lot of issues that may consume quite a bit of time to debug. Again, it seems like for ADO.NET data objects, it works mostly, but for custom business objects, the jury is out.

Peter Kellner's experience (above) is another twist. To use SqlDataSource when a query requires filtering based on the current user id (a pretty common requirement), he documents a somewhat arcane technique to get this to work declaratively. That's cool, but now we're creating more "maintenance debt," and I'm left wondering what else is lurking to trip me up if I go down this recommended path.

We really should be able to implement some well-defined interfaces in our business objects and just have this stuff work. That has to be the best approach, and I have to think this is very close to being possible right now. I am going to check this out a bit more before I give up and go back to "DataSource = " and "DataBind()" and other trusted but monotonous ways.