June 2004 - Posts
Tech-Ed Europe 2004 is in its second day today, and so far it's been interesting. There've been some sessions about the Visual Studio 2005 Team System, and an interesting Birds-Of-A-Feather talk with James Newkirk of Team System and NUnit fame. That talk was an interesting chat in a small room, with nothing but unit-testing-interested people, and we all talked about how to motivate people to do unit testing, how to test database classes, etc. All in all, great fun, with Benjamin Mitchell reporting on his blog. I'm sure I'm going to do a piece on database testing soon, including source code.
XP
Microsoft is actually using Tech-Ed to evangelize XP -- no, not Windows XP or Office XP, but Extreme Programming. There are a couple of sessions on the subject here; so far, unfortunately, I haven't had the opportunity to join one yet (I'm spending a great deal of time at the U2U booths), but it's great to see that Microsoft is making an effort to promote agile methodologies. The more people who understand and practice these principles, the better. Thanks guys!
IBF
About a month ago, Microsoft launched Beta 1 of the Information Bridge Framework, which allows you to very easily turn Office applications into smart clients for your client/server applications. It's great to see it in action, integrating for example a personnel database with Word and Excel in just a few minutes. Surprisingly, only one (1) session here at Tech-Ed is about this fascinating new product, and it wasn't even mentioned in the Keynote... There's still some work for Microsoft there.
TechEd Europe 2004 is upon us, and I'm really looking forward to it. If you'd like to drop by for a chat, you can find me at one of the U2U booths. I'll be blogging about TechEd and about Smart Development, of course, all through the week.
Last Friday, Jelle Druyts organized the first Belgian Geek Dinner, which was a modest but definite success. We all had a great time, even my girlfriend despite all the geek-speak. A few pictures can be found at Patrick Verbruggen's blog (I'm the geeky character with the ring beard, which, incidentally, is gone now). Thanks for a great night everyone, and let's repeat that pretty soon! (I'm thinking of a Belgian Geek Poker Night, but not many people like to play poker with me because I also like to perform magic tricks with cards...)
Maybe I should ask this directly to Eric Gunnerson or other guys from Redmond, but here goes:
I would like the .Net language compilers to support given coding standards, and refuse to compile code that doesn't meet them.
Update: make that "I would like the MSBuild system to support given coding standards, and refuse to build solutions that don't meet them." Makes a lot more sense... Thanks to Scott Galloway for remarking that.
One example is a maximum Cyclomatic Complexity for methods. Cyclomatic Complexity, also called McCabe Complexity or McCabe's Metric, is a software metric that computes the complexity of a function. In short, it equals 1 + the number of branches (ifs, whiles, foreaches etc.). A CC of more than 20 is considered too complex; so if a method has a CC of more than 20, it should be refactored.
Wouldn't it be cool if MSBuild would refuse to compile methods that exceeded a given (configurable?) complexity? This would mean developers wouldn't even be allowed to check in code that is too complex. Of course, you should be able to turn this off, so that you'd still be able to work in "Quick & Dirty" mode for a while. But once it's decided that all code must meet given minimum standards, why not let the development environment enforce them?
Another example: the compilers could refuse to compile classes that contain only static (Shared) methods and (a) constructor(s). This type of code is a violation of good coding behavior, so the developer would either have to remove the constructor(s) or make (some) methods un-static, so to speak. (As Marc Sigrist remarked, the example is badly chosen because its situation may actually be what the author wants (if the constructor is private). Thanks Marc!)
What do you guys think?
As much as I love the MSDN Library, and as much as I am very much impressed with Microsoft's efforts to provide us architects and developers with as much support as they can -- much more than any other development tools vendor --, I have some issues with the MSDN Library and the SDKs that Microsoft provides.
Don't get me wrong, I don't hate Microsoft--hey, I used to work for the company and I still love it. So don't bash me for bashing Microsoft. At least not today.
I just think Microsoft, as a provider of the platform, the tools and much of the technical resources that go with them, has a responsibility to teach the architects and developers who work on their platform how to do it right. And, even with all the richness that's in MSDN, in some areas they fall short of that. Many code samples in the Library and the SDKs use, well, Worst Practices.
A couple of examples to illustrate:
- ASP.NET Pages that access the database directly. This is a very bad example, while at the same time the company (and especially its consulting division, Microsoft Consulting Services) is trying to evangelize N-Tier Design;
- VB6 code in VB.NET Samples. No, we shouldn't be using CInt() and Left$ anymore, thank you, they are not O-O but imperative functions. I understand that Microsoft provided these methods into the Microsoft.VisualBasic namespace so that VB6 developers wouldn't feel left out, but the reality of it is that developers who used to work in VB6 continue to use their old habits, even if .Net is a totally different environment. But how are VB.NET developers supposed to know they need to use Convert.ToInt32() and string.Substring() instead if their VB6 counterparts are found in so many examples? Especially if the documentation encourages old-style VB6?
- .Net Naming Conventions as described in the .Net Framework Design Guidelines and enforced by FxCop: these rules are violated by MSDN Library code examples almost everywhere. There are namespaces with underscores in their names, identifiers with 3 or more capitals in a row or--can you believe it?--Hungarian notation and "m" prefixes for member variables;
- Use of public fields rather than properties. This goes hand-in-hand with the violation of naming conventions and the fact that "property" values cannot be validated and are thus always assumed right;
- The Exception constructor documentation gives an example of how to implement custom exceptions--but they derive from System.Exception rather than the recommended ApplicationException or even the base exception found in the Exception Management Application Block.
I think the issue is, basically, that MSDN doesn't enforce coding conventions or even naming conventions in their examples. This is strange because these are the lowest-hanging fruit in the Best Practices Tree and MSDN, by its very nature, should lead by example (right...?)
So, a request for the MSDN Team and the team program managers responsible for the SDKs: please, please, pretty please, can you create a set of coding conventions and stick to them, as well as to the "standard" Fx naming conventions? Can you please make these public and make sure that all contributors to MSDN, worldwide, stick to them? Please use your influence not just to get more architects and developers onto the platform, but also to make them create better, more robust, more maintainable applications.
Many thanks in advance!!
Man, I can't tell you how good it feels to finally get that off my chest!
In response to my previous post Iterative Development (Part I), and the resulting comments, Peter Stuer made some very interesting remarks:
While each of the agile methodologies in theory stresses ceremony and artifact creation should be in line with the project type, the >culture< flowing out of the agility groups (maybe unintended) emphasizes minimalism in these issues more that "adequateness". For some types of projects and teams, "minimalism" simply won't cut it. E.g. in long running projects with a high team turnover, the “implicitness” of too much knowledge can become a real problem.
I agree. That is one of the reasons I think documentation ought to be regarded a prime deliverable, and no iteration can be considered complete if the documentation isn't up-to-date. This includes documentation of design and code, and of course a good user's manual and maybe a hands-on lab by the time the project is actually delivered to the customer. And, to use Peter's term, the documentation should be adequate for newcomers to the project as well as old hands...
Peter also writes:
Also the organizational structure of the setup in large projects can be an inhibitor. When you are dealing with many partners and subcontractors the contractual setup often confines the “fluidity” of the method. Agility needs trust, which might unfortunately not always be practical to achieve.
I agree with this too. Especially XP stresses the concept of Collective Ownership, which states that everyone's co-responsible for everything, and that everyone should be able to tackle any task -- and should be moved around in the team. MSF doesn't talk about that but the practice in Redmond is that everyone is owner of his or her particular piece of code, and that nobody else touches it.
I prefer the middle ground. I am very much convinced that development teams need both generalists and specialists. Some tasks require a high degree of specialization in order to do them right; one example is database design and optimization. While most developers are surely competent to do a basic database design, and to write stored procedures, a good DBA can do it better and knows how to squeeze more performance out of the system. So I think some roles should be assigned to specialists only, and the project lead or technical lead should be pragmatic about whom gets to play which role.
This leads to a different kind of fluidity and a different type of needed trust. Team members depend on each other, as always, but since everyone's responsibility is confined to what he or she knows best (in case of specialists), not too many people are actually moved around. Team members can actually take ownership of "their" piece of the pie. I think this leads to better end results, and certainly to more accountability.
The Zero-Defect Mindset
So what about Collective Ownership? XP argues that this principle makes sure there are no turf wars within the team. That sounds about right, but I think the same can be achieved by MSF's principle of the Zero-Defect Mindset, which states that every member of the team should strive for maximum quality, and everyone's accountable for their own work (see also what this other ex-Microsoftie, Joel Spolsky, has to say about this).
In practice, developers should make sure that code that contains known defects gets fixed as soon as possible. That is reflected in Smart Development's notion of alternating "new features" and "bug-fixing/optimization" iterations (more on that later), and the notion that no new features should be implemented when there are still known bugs in the system. It doesn't mean code has to be perfect, just that the measured quality of the output should be as high as possible.
Teams whose members all have the Zero-Defect Mindset don't need Collective Ownership.
Corporate Culture
This is a caveat, of course. Introducing an agile methodology, or even basic agile practices, into an organization whose culture isn't compatible with it, is very hard at best. Getting team members to adopt a Zero-Defect Mindset requires that they be open to reflecting about their work, challenging themselves and taking pride in the team and in their own output. Unfortunately, many companies and organizations have a culture where such a mindset or such behavior is not encouraged, and people are not held accountable for the quality of their work.
What are your experiences with corporate culture in such matters?
My colleague Patrick Tisseghem (a.k.a. Profke) has just published an article about building custom Alert Result Channels in SharePoint Portal Server 2003 on international MSDN. Congratulations Patrick, it's really cool stuff!
Last time, I wrote an intro to Smart Development (SD), summing up the 7 basic principles that Smart Development practices embody. SD is not really a new methodology, but rather an amalgam of the most useful principles and practices of XP and MSF, with a number of new practices and conventions thrown in.
Smart Development states, for instance, that Iterative Development makes many project problems go away, or at least makes them easier to control. Inter-project dependencies and feature creep (the problem of building in too many features, many of which nobody will use) are managed more easily using Release Planning. Quality control is done by adopting the practices of an automated Daily Build process, Measuring Quality, Bug Tracking and setting Coding Standards.
For today’s installment, we’ll focus on the first SD principle, Iterative Development.
Waterfall
Many development projects use the Waterfall model, which has been in use since the dawn of time and which just doesn’t want to die—just yet. The Waterfall model states that you should organize development projects in a single “chain”: specification first, then design, then implementation, then testing and finally deployment. The problem with this model is that, the later an error is discovered, the more expensive it becomes to fix.
For example, say that during the test phase, a design error is uncovered which makes it impossible to reach the required level of availability. This requires new design work, changes to implementation etc., and these changes may be significant. The error is expensive, because it not only requires a lot more work to correct errors, it’s also a strain on the schedule, to say the least. And, as many project schedules go, will the project manager delay the project? Most probably not. In many cases, he’ll decide to cut testing and documentation to make sure the deadline isn't missed.
But there are many, many more problems with the waterfall model. Requirements change, and when they do during development, more often than not the design is not adapted to accommodate new requirements—it’s up to the developers to come up with the necessary changes. This leads to undocumented and inconsistent design and unnecessary complexity. Testing is usually extremely limited, and as mentioned before, testing gets sacrificed for more development time. Teams can also change, and the waterfall model makes it harder to cope with changing project velocity. A one-shot development process becomes a beast that is hard to control, especially once it starts wobbling. And what happens if the budget is slashed half-way? What if, after a few months of work, it becomes apparent that the original schedule was much too optimistic and that the project will cost a lot more? It’s no coincidence that so many development projects get canceled because they run out of money.
Iterative
With the Iterative model, project development is divided up into a number of iterations, or cycles.
it•er•a•tion (noun)
(1) The action or a process of iterating or repeating; (a) a procedure in which repetition of a sequence of operations yields results successively closer to a desired result; (b) the repetition of a sequence of computer instructions a specified number of times or until a condition is met
(2) One execution of a sequence of operations or instructions in an iteration
Merriam-Webster Online Dictionary (http://www.m-w.com) |
Every iteration has its own phases of specs & design, implementation etc. The project grows from simple to complex, and in the beginning of each iteration, the new features to add are selected and the model is modified to accommodate the new features.
Very important: Every iteration ends in the delivery of a stable build, even starting with the very first iteration. This build is tested and test-deployed to make sure that the project is (still) deployable, which makes for no deployment surprises when version 1.0 is released. Project dynamics are easier to control and there are generally less unknowns during the project lifecycle.
Changes are limited per iteration. Only a few new features are added, but they are added completely: in design, implementation and documentation. The project’s deliverables are all up-to-date at the end of each iteration. One of the advantages of this approach is that when new people join the project, they can get their hands on the latest documents to get up to speed. Another advantage is that it is, at all times, easy to measure how the project is doing and how much work remains. There are many more advantages, which will become apparent in this and future blog entries.
The sum of iterations make up the final project. So in the meantime, the team has gone through the specification & design phases, development, test and (test) deployment many times. Changes to requirements and design are easier to accommodate.
Moreover, if the most urgent features are implemented first, the impact of budget slashes becomes far smaller. The team has produced deployable deliverables that produce business value, even if they may not contain all features that were foreseen originally.
Iteration Duration
How long does an iteration take? It depends on the project and even the iteration; there’s no clear-cut, one-size-fits-all answer.
I find that a duration of between 1 and 3 weeks works best: short enough to keep dynamic projects under control and to add small feature sets, and long enough to be able to do serious development work and plan ahead “just enough.” I think that the duration of an iteration is best determined by the project lead or technical lead, and that not all iterations should necessarily have the same duration. For example, when the deadline for releasing version 1.0 comes closer, one may choose 2-week iterations over the 3-week period adopted earlier, in order to accommodate important late-breaking changes better. If there are multiple projects being developed simultaneously, and there are inter-project dependencies, then this may also have an impact on iteration duration.
This is a topic we’ll cover more deeply when we talk about Release Planning.
In the next installment of Smart Development Principle #1: Iterative Development, we'll talk about iteration types, phases and versioning.
Jelle Druyts is organizing Belgium's first Geek Dinner, for Belgian "geeks", in Antwerp. We'll be there! Great initiative, Jelle! By the way -- I love your post on Reflector, definitely one of the best .Net utilities available today.
Here's an idea for Microsoft's Visual Studio.Net Editor team, or for the guys doing SharpDevelop or other IDEs...
What about having a code editor with code visibility modes? For example, Full Mode, Debug Mode and Release Mode.
When editing in Debug mode, stuff like Debug.Assert statements and #if DEBUG... #endif code would be visible; when editing in Release mode, they would not (but #if RELEASE...#endif etc. would). When editing in Full Mode, everything would be visible.
This way, developers can switch off display of Debug stuff when they really need to focus on algorithms and other implementation details, without being bothered by ugly-looking Debug code.
What do you think?
Microsoft Research is publishing some white papers about C-Omega, a research programming language (actually an extension to C#). It contains a "control flow extension for asynchronous wide-area concurrency (formerly known as Polyphonic C#)" and "a data type extension for XML and table manipulation (formerly known as Xen and as X#)".
It looks intriguing. Especially if you remember that Whidbey's Generics also started out as a Microsoft Research extension to C#...
Asserting yourself is a best practice. Especially when you write code. Or rather, you should assert that your code is correct.
What I'm talking about, of course, is not psychology or assertiveness, but correct and abundant use of Debug.Assert. This wonderful method, which dates back to the old C days, is executed in Debug versions of our code and verifies whether given assumptions are true. If not, the code exits and writes an error message to the Debug output, which we can read in Visual Studio.Net or which we can log to a text file, for example, using a TraceListener.
Misconceptions about Debug.Assert?
Misconception: Debug.Assert is for input parameter validation only
Hold on, hold on. Yes, I'm sure most of us know Debug.Assert and that many developers use it, maybe all the time, for input parameter validation on non-public methods. It's excellent for that, as is illustrated in this example:
Example 1
1: protected static double OptimizedLength(string text, int start, int end) { 2: Debug.Assert((text != null) && (text.Length > 0), "OptimizedLength: Input 'text' is empty.");
3: Debug.Assert(start > 0, "OptimizedLength: start is out of range.");
4: Debug.Assert(end >= start, "OptimizedLength: end is not after or equal to start.");
5:
6: //Rest of code here
7: }
But it can help us even better! Debug.Assert, in my humble opinion, is the most underused tool available in the .Net world, while it should, in fact, litter all code. Even in Microsoft's own example code, there's hardly any trace of it (pun unintended). It seems that us developers have some misconceptions about it, or fail to see the full range of its applicability.
Example 2
Consider this simple piece of code:
1: XmlNode pathNode = node.Attributes["HintPath"];
2: string xmlPath = pathNode.Value.Replace(".dll", ".xml"); 3: Console.WriteLine("XML Path: " + xmlPath); What's wrong here?
If anything goes wrong during execution of this code, it's hard to tell why