Visual Studio Find using RegEx

Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaargh!

Why couldn't Microsoft make the regex search use some existing regex dialect? Maybe they did (randomly!) pick one, but it's not the regex dialect they use in the .NET Framework.

I admit to always finding another way when faced with searching for something like the following lines (which happens to be a task for me today), but I'm keen to do it more easily:

CREATE PROCEDURE

CREATE    PROCEDURE


So using Microsoft's VS.NET syntax, it's this:

^create:Zs+procedure

That's good to know: :Zs is a SPACE, and :Zs+ is ONE OR MORE SPACES. And if I want to find these:

CREATE  TABLE

CREATE    FUNCTION

CREATE VIEW


It's this:

^create:Zs+:w

So :Zs+:w is ONE OR MORE SPACES FOLLOWED BY A WORD.

And I'm going to refer back to this post because I won't remember this two months from now.

Durable Information

Email is a wonderful way to communicate. I keep in touch with people I would have long ago neglected because of it. It's almost free. It's fast. It's super convenient. It's flexible, and allows me to send not just my words, but pictures, files, links and even viruses if I'm irresponsible. It's fairly private, and can be totally secure if I choose the right provider and tools. OK, spam is a nightmare, but more or less, email is a great thing for communicating.

Unfortunately, email doesn't cut it for storing information that is durable, a phrase that I'm sort of adopting or making up for the context of this post. By durable I mean the following:

• It needs to be found later by someone other than the email sender or recipient
• It constitutes a record of a decision or other important information that is of interest to a group outside of the email sender or recipient
  
• It is "owned" by someone other than the email sender or recipient.

So rather than elaborating on these cryptic bullet points, here's an example...

I work on a small software project (I am the sole developer) for a client of my employer. The client and I communicate frequently about the state of project tasks, why certain things aren't meeting his needs, etc. This goes on over the course of one "release" of the project, or about six weeks. He emails me, I respond, he responds, etc. Then I find a problem, I fire off a message, he ignores it, we revisit it a month later without remembering much about it... and on and on.

This is typical small company project management, which is to say, it's not being managed. The only way to get a complete picture of what was decided, how it was interpreted, what is left to do, how to login to the FTP server -- whatever -- is to sift through and piece together artifacts from the following:

• My inbox
• My sent items
• The client's inbox
• The client's sent items
• Our IM logs
• Some documents squirreled away on my company's network
• His [My Documents] folder
• etc.

The basic question when people default to blasting emails to think they are communicating effectively is this: Who OWNS THE INFORMATION? I would say the client and my company own the information, not me. So why would I think just leaving it in my inbox is appropriate?

Yes, my company owns my inbox, but that is not the point. The issue is how to reduce the friction created by my laziness. That my company owns the information that I am mismanaging means I am obligated to store these bits of decisions and communications in such a way that it can be found. Found by my managers, auditors, the next developer who takes on this project after I'm on another, the client, QA people so they can tweak a test plan et. al.

At the very least, emails should be saved off to a public folder. This is a pain, however. A good project management tool (such as basecamp) is much preferred for aggregating this type of communication and preserving it. Email should be used for notification or quick comments. Anything more long-lived than spur-of-the-moment has to be put somewhere else. Then it can be searched, prioritized, validated, transferred and generally managed. Durable information, not fleeing bits of conversation scattered across multiple inboxes.

Harder to Type, Easier to Maintain

One of my favorite software books, Steve McConnell's Code Complete, rails against hard-coded strings because they make an application difficult to maintain. At "typing time," it's the easiest and quickest way to do things. Thereafter, though, you've created a debt that you will pay down for the life of a project.

When we work with DataSets, DataTables and other means of packaging data, it is hard to avoid littering code with literal strings that represent column names. This is not helped by our tools -- how hard would it have been for Microsoft to expose an enum for column names from a DataTable object in a generated typed DataSet? (And why is "strongly-typed DataSet" preferred over "typed DataSet"?)

Several years ago, when I first started using .NET, I created an database column enum and constants generator. Point it to a database and let it generate enums or constants (in C# or VB.NET) for every table and view, or pick the ones you want. It's part of the suite of tools I use to create the data layer and business layer in my applications.

It's a simple thing. So instead of writing code like this:

30 dRow["LastName"] = "Smith";

31 dRow["FirstName"] = "Bob";

I can write it like this:

30 dRow[CustomerColumn.LastName] = "Smith";

31 dRow[CustomerColumn.FirstName] = "Bob";

A typical application is going to have references to columns in many places:

• Assigning values to columns
• Persisting data
• Filtering (using the DataTable's .Select method or a DataView's .RowFilter property)
• Sorting
• Ad-hoc querying
• Data binding and other tasks above in the designer for a form or web control

Aside from the visual designer code, giving up hard-coded strings for enums or constants is a very small change to make. If When your database schema changes, just regenerate the source file for the enums/constants and fix up your code. Most changes will now break the application at compile time, not run time. This makes it trivial to identify and fix these issues. A large application with thousands of references and many developers just cannot be maintained with hard-coded strings for column names. Catching these errors at run time requires 100% code coverage, and happens way too far downstream to NOT lose you money.

GridViews and other designer-based code may still be a chore, but you're not making things worse because that's how they are whether or not the rest of your source code has hard-coded values.

Does "LastName" look nicer than CustomerColumn.LastName? Probably. Is "LastName" easier to type? I'd say so. But it creates a mess, one that too many developers are keen to accept, unfortunately. Keep in mind, what I'm talking about here is for developers who haven't adopted object relational mapping or other means of encapsulating and abstracting object data. If you're still using DataSets and DataTables, you need to consider the friction your are accepting by hard-coding string column names, primary keys, default values, filters and so forth.

Agile FUD

Why do some people argue as though "agile" is some grand methodology, a prescribed process, comes with a big book and a two week training course and has a rigid set of mantras? It doesn't as best I can tell. I have adopted some agile practices in my daily routine, such as unit testing (though not TDD as of now), intensive refactoring, simple and frequent communication and continuous integration. There are a lot of agile practices that I do not have the expertise to try right now, or the right projects to try them on or the right team to try them with. I am basically fitting what I can into my current environment, reading all I can and reexamining every month or so to identify what else I can apply or do differently.

I think this is the whole point of agile. That is, there is no one prescribed way. Rather, there are principles or fundamentals, and the practices reflect these in a sort of toolbox approach. And above all, the personnel involved are intelligent and experienced enough to use these things to improve how they create software and in turn improve the software they create.

For example, sometimes we have projects that are remediation types of engagements. In other words, we are hired to fix something and it may last for a month. Can I do TDD? Should I set up a continuous integration server? It depends. I have enough experience to know which tools to apply to solve this problem.

So when I read something like this that portrays agile as some all-or-nothing "methodology," it makes me wonder first, if people like this are using some prescribed methodology already or instead have none whatsoever, and second, if they aren't just looking for a way to bash these ideas by portraying them as something they are not. That post at secretgeek makes it sound as though agile is loaded with ideas that contradict each other. I don't think this is the gist at all.

One tool from the toolbox that has had a huge impact on how I write code is designing for testability. This is one of the first things I consider when creating any code. It forces me to think about:

• How to structure the solution in Visual Studio .NET. I always create a ".Tests" project in addition to the main project (it used to be ".Test" until one of the versions of the Test Driven .NET add-in for VS.NET caused problems in projects named this way). Instead of creating some goofy form to drive my tests and therefore requiring me to manually test each time I make changes, all of my public methods can be launched through my unit tests. Right-click, Run Tests. Low stress. This is pretty standard stuff.
  
• How to design a class. When thinking about how to test it, you must immediately consider what the constructor does, how to initialize state, whether a static class is a good idea for the scenario, etc. Having to change these things later puts you into a much more defensive mentality, where you feel like you are constantly catching up and rethinking very basic design decisions. Again, pretty standard. Thinking about testing forces better design decisions.
  
• How to design its interface. When you know how your class can be used (i.e., because you thought about "exercising" it with unit tests at the outset), your interfaces are much cleaner and more consistent throughout a project.
  
• How and where to initialize and destroy resources. Basic stuff, but again, having to tweak things later can be annoying, and being inconsistent has subtle implications for robustness.
  
• Where to stash configuration information so that it is accessible in a very flexible way. Does the class need a SqlConnection? How do you provide it? Do you have different databases for development, testing, staging, performance testing and production? Instead of cluttering the class with code that accepts these, is it better to make it config-based? If you do, your ".Tests" project needs to be defined a certain way with its own config file, etc...
• How NUnit can automatically run my unit tests to help me control regression. Now that you've thought through these other things, everything is set up to do this. You can make changes and perform major refactoring at will and know what the health of your application is.

The principle here is that these things enable me to set up my project to accommodate change and ensure quality is built in from the start of the project. That's the fundamental requirement to doing these things.

Pretty standard stuff. I am amazed that it's still not universal.

ASP.NET 2.0 DataBinding Redux

In a previous post I mentioned my skepticism about using ObjectDataSource to bind business objects in ASP.NET 2.0. My desire to "implement some well-defined interfaces in our business objects and just have this stuff work" is touched on by the EntitySpaces guys, but it's more than just what I mentioned and it's touched a nerve.

I think the musings that Microsoft may have yanked what support there was for design-time data binding in 2.0 in order to develop their own ORM functionality in ADO.NET v.next is likely correct. Just like Sandcastle-NDoc, Team System-NUnit-NAnt and the other things I mentioned previously, it seems Microsoft is going to go their own way and quash a good portion of the open source efforts to build on their tools.

SQL 2005 Script Files and VSS

Using Microsoft Visual SourceSafe 6.x and Microsoft SQL Server Management Studio (for SQL Server 2005) is a bit of a pain for scripting database changes. One problem is if you don't save the script file as ANSI, VSS cannot use its diff feature to show you the results. It will simply say, "Binary files differ." You need to save it as ANSI, since that's all VSS 6.x supports for visual diffs.

To save your .sql files as ANSI:

• Instead of just clicking the Save button, select File | Save As...
  
  
  
  
• Click the little arrow on the Save button and choose Save with Encoding...
  

• Select the encoding and the line options, then click OK.
  
  

Then you can check the .sql file into VSS. Subsequent edits will not require you to go through all of these dialogs to properly save the file. Management Studio will use the file's current encoding and line break settings thereafter, so you can just (thankfully) click Save.

RFPs

Interesting take on RFPs at airbag. We have been occupied with some proposals recently, and I have to agree this is rarely a very fruitful way to land new business. So many of these RFPs are really for one or more of the following purposes:

• The company sending it out can satisfy some requirement that they found a consultant in a competitive manner
• They've already found a consultant and need to appear that they did it in a competitive manner
• They are looking for the lowest cost bidder and that is their top priority
• They are not clear about what they want and are soliciting ideas through the RFP process.

The last point is actually positive, in my opinion. If you work for a consulting firm who has the capability that they are seeking, at least generally, the RFP can open the door for a dialog. If you are sharp and experienced, you can put some good ideas on their table. Responding to an RFP without having that dialog is rarely productive.

Some of the comments to the airbag post really hit home:

"The problem is rarely defined and the client usually has a vendor and solution already in mind."

"They take a lot of time, you only have a marginal chance of winning the work. It really just drives your overhead up with a lot of non-billable work, making the clients that didn't send you a RFP essentially pay more."

"The traditional RFP process is damn near a no-win situation for us. We ran the numbers and we've got a horrible track record when it comes to winning work of an RFP. Given the time it takes to address them (much more than other ways of bringing work in) we've realized that RFPs aren't really something we should be spending a whole lot of time on."

"Submitting a "proposal" of any sorts before there's been any communication is akin to ordering a bride online. It may work for some people but it's unlikely they'll have a relationship that'll last. Too much room for misunderstandings and miscommunications."

Software Design's Dirty Little Secret

We spend a lot of time wringing our hands about how to do a better job gathering requirements. I think this bit reflects a very common scenario in how we convince clients that we're experts, and how things really end up going. We put forth some process or methodology that sounds impressive, like it's really going to squeeze every bit of uncertainty out of the project as early as possible and lead us down a clean, straight trail to the solution. Utter crap, yet we continue to push it because that is what clients demand we tell them. So the Design Observer lets us in on a version of the dirty little secret, one that applies in good part to software design as well:

When I do a design project, I begin by listening carefully to you as you talk about your problem and read whatever background material I can find that relates to the issues you face. If you’re lucky, I have also accidentally acquired some firsthand experience with your situation. Somewhere along the way an idea for the design pops into my head from out of the blue. I can’t really explain that part; it’s like magic. Sometimes it even happens before you have a chance to tell me that much about your problem! Now, if it’s a good idea, I try to figure out some strategic justification for the solution so I can explain it to you without relying on good taste you may or may not have. Along the way, I may add some other ideas, either because you made me agree to do so at the outset, or because I’m not sure of the first idea. At any rate, in the earlier phases hopefully I will have gained your trust so that by this point you’re inclined to take my advice. I don’t have any clue how you’d go about proving that my advice is any good except that other people — at least the ones I’ve told you about — have taken my advice in the past and prospered. In other words, could you just sort of, you know...trust me?

So, is that how it really goes? Well, most of the time, it sort of goes that way. It has to. If custom software development were not so nebulous and prone to reexamination and modification at every step, it wouldn't be custom software. In other words, you would instead solve your well-defined problem by going to CompUSA and buying a solution for $79.99. The next alternative would be to modify your problem so it could be solved by the $79.99 solution. Some people do just that, and successfully. Using Microsoft Excel is a good example. It's probably the most widely used database application on the planet, though it was never intended to be a database.

If you want truly custom software to help manage a truly unique problem (and lots of people have such a need), you should know what it takes. Creativity. A prescribed process can't get you 100% of the way there. You must be willing to accept at least a good portion of the process to be empirical. That means it's not clean, it's not predictable and it's not going to make you comfortable.

When a prospective consulting firm approaches you and claims they have a neat, linear, sequential process that they have proven over countless years, they are setting your expectations in such a way that you are likely to be frustrated. In other words, they are partially to completely full of crap.