Architecture
Posts about Architecture and Design
I guess no one can ever say that the VS dev team isnt responsive.. after all the posts in various forums about the lack of support for class libraries on the AD surface, this has now been made available. Unfortunately its only available on Orcas Beta 1 (as a Power Toy) . Check out the post and screenshot at :
http://blogs.msdn.com/vsarch/. Apparently Orcas has some underlying infrastructure that allows it to support such out of band developments so this wont work on VS2005 which is a pity, but if we have waited so long, its only another few months to wait for the final orcas release with all these goodies.
Another really good thing is that Steve Cook and Gareth Jones et al have published a new book on
Domain Specific Development with VS. It should be available now on Amazon (the link is too long to paste here- check it out).
Martijn Hoogendoorn has re-posted the link to his utility to programmatically generate orchestrations (now that he is working for Microsoft the old link on his previous blog no longer works). Check it out
here. He has used the Microsoft.Biztalk.IMS.dll which contains interfaces such as IBtsVsDocument and the Biztalk Object Model dll to do most of the generation. This is quite different to Jon Flanders Pattern Wizards which have a wizard written over a bunch of VS2005 templates to generate what you need. In this codebase, Martijn has one class which generates a sample akin to the SDK CallOrchestration sample. Its pretty nifty and demonstrates how one can use the Biztalk TypeSystem to generate ports, message references etc. One thing which puzzled me is where he got the Biztalk.IMS.dll from!! Jon has that in his codebase too but i couldnt find that anywhere on my machine! Does anyone know where we can get that or where some documentation would be available?
[Update: Martijn informed me that the DLL is in the GAC and so it is!!]So when would you want to code gen orchestrations? After all, using the orchestration designer isnt exactly rocket science and now that there is a wealth of community wisdom in the realm of patterns (encapsulated in Jons tool), why do we need this? I look at this technique as facilitating one layer of abstraction over the designer itself. Take the "Solution Designer" that Eddie Churchill demo'ed in a channel9 video If you look at that, it shows how the Team Architect AD surface can be used to sketch your solution end to end and then generate the solution structure with stub orchestrations, schemas, maps etc. Thats a good application of DSL technology (although in the demo version it used windows forms, not the DSL toolkit which wasnt stable enough to be used at that time).
Anyway, back to the discussion. One thing about good architectures is that they should also be consistent (along with being flexible). So if you have a bunch of backend systems that expose similar interfaces to be integrated with (WS, queues, file ports etc) and if you have a choice of which interface to use, while you are well within your rights to use any way of connecting to them, IMO, a better way would be to establish a consistent pattern of connecting to them so that you can make sure your bases are all covered, your development team knows whats expected and you have a standard base for delivering estimates. So if you do this, then you can fairly well predict exactly what your "connector" looks like and then, if you have worked out the contract for your schemas etc, then you can just go ahead and code gen the entire lot. Use the VS automation model to generate the solution structure, add the various projects in programmatically and then inside each of them, use these utilities (like Martijns) to generate the rest (and in a future version of BizUnitExtensions, autogen your tests as well). This way we can maximize our "design contracts".
I do wish the Biztalk team would make the WebService Publishing Wizard available programmatically. (I guess the WSE publishing wizard is probably on its way out now that WCF is in production). This would help even more code generation. By the way, i heard that Eddie has left the BTS team and so the solution designer has been discarded. Anyone know the skinny on this? I would be terribly disappointed if this is the case because to me, that was the most compelling feature in the "future/vNext" Biztalk release (perhaps it was post R2).
By the way, I also came across another tool named BridgeWerx sometime ago that used a custom interface to sketch your biztalk solution and generate the codebase. I wonder if they have moved into the Microsoft DSL space now.
I've asked Martijn about the possibility of making this into a CodePlex project (although the source code is available with the download, putting it up as a project would make it more visible and get more collaboration). I'm sure that with more hands and more time (overall) there could be much more powerful things that could be done with the tool.
Of course, everyone is entitled to their own opinion and i am certainly not paid by Microsoft to be a "defender of the faith". Nevertheless, i do come across inane comments that i cannot just ignore. I ended up on the ASP.NET forum by accident today on a topic of Modular ASP.NET architectures when i was looking for some WCSF or DNN related things (i honestly cant remember which). Anyway, this chap made the following comment when asked about frameworks that help promote modularity.
WCSF is a bunch of source code produced by a couple MS developers to promote .Net 3.0. Believe it or not, .Net 3.0 is a failed product, not many acceptance, because it's not solving any real problems. I don't think you want to mess up with WWF in your project so far. Once you start to use WCSF, you got sucked into .Net 3.0. You spend 80% of your time to solve all weird technical problems, conventions, and finally you found it's not
.Net 3.0 problem, it's your own problem because you didn't do it in .net 3.0 way. Before you know that, you run out of time, you never have time to look at your real problem: The business problem you want to solve.Eh? what on earth was that about? So heres my reply in full having asked him not to slag off WCSF in such a way if he couldnt understand it or the concept of software factories
(1) WCSF delivers sorely needed guidance in building nLayered ASP.NET applications. Everyone has their own pet theories and approaches on how things should be done and as the saying goes "one mans meat is another mans poison". We are rarely, if ever ,going to agree with another developer on the "best" approach to anything because to each of us, our approach is the correct one and the other guy doesnt know what he is talking about. Its time MS made a stand and told us what they think is a good way of using their technology and those who wanna go along can do so. If you dont like it, dont use it.
Yes, it validates the concept of Software Factories, GAT, GAX etc along the way. It is not intended to promote .NET 3.0 but to also show us how WF can be used to do pageflow in a way thats much better than the old UIP block and ultimately even if it does show off the features of .NET 3.0
so what? lots of people want to know how to incorporate .NET 3 into their work now. Besides, nobody claimed that it is perfect or that it is the
only way to do stuff.
(2). NET 3.0 is NOT a failed product because (a) its not a product (sic!) and (b) its been released only just now. Its just more of the .NET framework and its too early to say if that it has succeeded or failed. You obviously havent done much distributed computing or you would appreciate what WCF does in unifying the communications stack and what it brings to the table for SOA and large scale integration. I use Biztalk almost 99% of the time now and i can fully well understand the huge problems that WCF solves and what WF does. I am not a front end programmer so i dont really have much to say about WPF but from an architects point of view a stack that can help unify the approach to the desktop and the web is definitely valuable. Try building solutions that go beyond a few web forms and incorporate orchestration and integration and then you can see the problems that .NET 3.0 is intending to solve
(3) Every technology/framework will prescribe that you do something in a particular way. Its just part of software development. If you want to use
DNN, you have to buy into their plug-in module concept. You want to use the Ent Lib? then you have to do stuff in a particular way. The point of
WCSF is that it should let you focus on the business problem not the plumbing and wasting time creating skeleton projects, editing config files
to put the same gunk in day after day etc. If its a complex approach, so be it. There are no silver bullets. (And besides, the WCSF team has been asking for feedback and even have a rating system where you get more points for criticising it than if you applaud it).
(4) Finally dont forget that the ASP.NET development team has also been involved in it. Yes, the same folks who wrote the technology that you are
using to solve your "real world problems". They obviously know what the real world is like. .
Now heres the thing. He probably tried using WCSF is an early CTP and couldnt figure it out and just to give him the benefit of the doubt, lets assume that the problems with it are real and that it is just clunky. Even if that were the case, thats still not an excuse to go to town about .NET 3.0 and call it a failed product! Theres no real correlation between a framework and a factory that implements some of the frameworks features. Factories are new. PAG is doing its best to provide good usable stuff but GAT and GAX are known to be difficult, hardcore technologies now and it isnt easy to understand them either. Im just getting started with automation and code gen the GAT way and its still tough going but that aint .NETs problem.
Anyway, im glad to get that off my chest. As always, feedback is welcome!!
Finally, something on practical application design without all the academic theory. Most of the articles i have got to read on the subject of writing maintainable code are just long winded theory stuff about interfaces and mocking and DIP etc without actually being concrete and understandable. But now, heres a brilliant explanation from Jeremy Miller on writing maintainable code replete with a real life example and some extra scenarios and thoughts thrown in.
Check it out at
http://codebetter.com/blogs/jeremy.miller/archive/2007/01/08/Orthogonal-Code.aspx
Now if only more people could explain refactoring and design patterns in that way, it would make that world much easier to get into.
I spent a rather interesting day at Microsoft , Reading at the Grey Matter Architect Forum. The focus of todays sessions was all about how other companies/partners are using VSTS and extending it. Heres some of my thoughts on the sessions today and on some of the things that sort of emerged along the way (at least, in my mind).
(1) MSF and VSTS: The session was by far too short. Alan Cameron Wills is a very engaging speaker and this talk came across very well. The first half was familiar territory since i have been looking at VSTS since TechEd 2004 and started using it recently. It was good to see that MSF is getting into the Factories space so that the process guidance can be tailored to fit with the factory approach. It would have been good to delve into more detail about what MS's plans are but we didnt have enough time. There were a couple of things that came up in the Q&A session at the end where Alan indicated that the CMMI template will have enhancements because MS is working with SEI to make things better. I gather that some extra stuff will also turn up in the Agile template. He did make a good point that we arent really expected to use the entire process template as is and that most companies will resort to customizing it in one form or another. As such, the out of the box templates are only a starting point, and they dont claim to be the final word on how you implement your process, so whatever enhancements MS provides would be cool but not something that should stop us from getting on with things now. Although the process templates are customizable, i gather (from other articles and books) that in the currentl scenario, this is not for the faint of heart. MSF is moving to a more 'composable' approach where instead of siloed "Agile" or "CMMI" templates, there will be a solid catalog of practices to choose from to make the process you want. There, of course, needs to be validation to check that we dont come up with some weird unworkable combination of practices.
I like the way VSTS chucks a ton of work items at you when you instantiate a team project. It gave me a lot of food for thought when starting a recent project. Its quite common to look down the list and think "Ooops!! forgot to sort that task out" so it helps keep you honest and on track. One thing I personally would like to see is "
Project Profiles". What i mean is this: if you take a large SOA project, when you instantiate it in CMMI and Agile (for instance, SCRUM) , what does the guidance look like? how does it differ amongst the two templates? what do the out of the box iterations look like? what do they say you should focus on first and what you should avoid? Similarly how about a B2B or a B2C project? what are the best practices there? when should we rely on the factories and when should we look elsewhere? This would be very helpful in figuring out which approach you want to take for your particular project. I guess that for very experienced architects the answers would be obvious, but this would be a solid fallback and a guide/checklist for those of us who arent exactly gurus in this.
(2) SCRUM and VSTS: Colin Bird from Conchango presented this. As quite a few people know, Conchango developed a SCRUM plugin for VSTS (collaborating with Ken Schwaber). Actually I didnt know it was free till today. Now i gotta go download it !! It was quite a dep talk on how to scale Agile with VSTS. Since I am not all that familar with Scrum (i mean i know what Sprints are etc but not very much more), i cant really comment on the content but it was quite interesting to hear what they have learned from large projects (some distributed and some co-located). These talks make me feel rather wistful sometimes. I wish I could be in a project where you can just have a quick/solid planning workshop and then go heads down in development while someone sorts out the impediments for you. If things arent fitting in the sprint, re-prioritise with the involvement of the client and move it to the next sprint. Wow!! I've heard this from many people including some of my friends who've worked with companies that are very heavily into Agile methodologies (names withheld). This has never happened to me. Its always been the case that my clients want the stuff done fixed price and they have commercial deadlines so they have to know when we are going live before we start!! Im sure im not alone in this!!
(3) Select Perspective: Rather all-encompassing toolset. I've heard a lot about it so it was good to see a demo. I did however disagree with the label "agile" applied to it by the speaker. I will agree that its adaptable and that we need to be more adaptable and flexible in the way we pick our processes and practices for the type of project at hand and it that sense the toolset with its collected "executable wisdom" helps us to be "agile", but this much of a modelling heavy environment surely cannot be a proper "agile" candidate can it? Isnt one of the main principles of agile to focus on working code rather than heavy documentation and models? If we spend so much time working out various architecture models then wheres the code? which stakeholder is going to pay for pictures? Now dont get me wrong. I'm all for setting out a blueprint before coding and detest the cowboys who plunge into code without a thought to the overall architecture.(okay, so if the whiteboard or the metaphor is enough and your stakeholders are happy with that, go for it!) My clients need to know how the system is broadly going to function and what the moving pieces look like at a high level and in my opinion any architect worth his or her salt who has worked in a particular domain for a while should be able to provide this info without having to first write a hundred unit tests and all the code for it. And im sure that the MDA camp would say "yeah, but spend time on the PIM and push the magic button and voila, its the PSM/code, so you havent wasted any time making that nice pretty UML model ". Something doesnt feel right to me. However, im not slagging off the toolset per-se. It looks quite solid and very formal risk averse environments would enjoy using things like that. I just dont think it should be labelled "agile".
(4) Essential Unified Process: Star Wars!! The Empire strikes back!! It looks like this is the return of the unified process albeit in a more "developer friendly" way. Take heart young Jedi, your process guardian is near!! IMO, the most outstanding thing (which unfortunately was only alluded to and not demonstrated) was the "intelligent agents", your persona/role buddies who can take on some of your workload in producing/reviewing artifacts etc, but otherwise nothing really stood out. Sure, its looks cool that you now have electronic storycards in VSTS and its nice to know that all the process vendors recognise that people are going to mix and match and are opening up their toolsets to allow this. IMO, this is a big plus and boost for Microsoft (but thats a post for another day). But for the folks who like the EssUP approach and who are investing in VSTS, this integration is sure to mean a great deal.
Now, how many feathers have i ruffled today?