"this is a well-known problem with code files that have desigable classes in them - the class that is to be designed has to be the first class in the file.  If you do the same thing in windows forms you get the following error: the class Form1 can be designed, but is not the first class in the file. Visual Studio requires that designers use the first class in the file. Move the class code so that it is the first class in the file and try loading the designer again."

It turns out I had changed a struct that was defined first in my file to a class. I moved that class to the end of the file and "mirabile dictu" everything worked.

So if this is a well known problem, why can't we get an error message just like in the Windows Forms case?

While it was clearly my mistake, Microsoft has a share of the blame here. Clearly this requirement makes it easier to build the workflow designer. It would have been just as easy to check if this class was not defined first, and issue an error message.

You can use the debugger in Visual Studio 2008 to step into some of the .NET Enterprise Libraries. This is a feature I asked for a number of years ago. Scott Guthrie just blogged about this. Here is his post and it includes the list of libraries to which this applies and some information about how to do it:

http://weblogs.asp.net/scottgu/archive/2008/01/16/net-framework-library-source-code-now-available.aspx

 Here is a short MSDN video about this:

http://www.microsoft.com/uk/msdn/screencasts/screencast/304/Debugging-Into-the-NET-Framework-Source-Code-with-Visual-Studio-2008.aspx

The Windows Workflow Foundation (WF) ships with a Policy Activity that allows you to execute a set of rules against your workflow. This activity contains a design time rules editor that allows you to create a set of rules. At run time, the Policy Activity runs these rules using the WF Rules engine.

Among other features, the rules engine allows you to prioritize rules and to set a chaining policy to govern rules evaluation.  The rules engine uses a set of Code DOM expressions to represent the rules. These rules can be run against any managed object, not just a workflow. Hence, the mechanisms of the rules engine have nothing to do with workflow. You can actually instantiate and use this rules engine without having to embed it inside of a workflow. You can use this rules engine to build rules-driven .NET applications.

I gave a talk at the last Las Vegas VSLive! that demonstrates how to do this. The first sample in the talk uses a workflow to demonstrate the power of the rules engine. The second and third samples use a very simple example to demonstrate how to use the engine outside of a workflow.

Two problems have to be solved.  You have to create a set of Code DOM expressions for the rules. You have to host the engine and supply it the rules and the object to run the rules against.

While the details are in the slides and the examples, here is the gist of the solution.

To use the rules engine at runtime, you pull the workflow rules out of some storage mechanism. The first sample uses a file. A WorkflowMarkupSerializer instance deserializes the stored rules to an instance of the RuleSet class.  A RuleValidation instance validates the rules against the type of the business object against which you will run the rules against. The Execute method on the RuleExecution class is used to invoke the rules engine and run the rules.

How do you create the rules? Ideally you would use some domain language, or domain based application, that would generate the rules as Code DOM expressions. If you were masochistic enough, you could create those expressions by hand.

As an alternative, the second sample hosts the Workflow rules editor dialog (RuleSetDialog class) to let you create the rules. Unfortunately, like the workflow designer, this is a programmer's tool, not a business analyst's tool. A WorkflowMarkupSerializer instance is used to serialize the rules to the appropriate storage.

I would be interested in hearing about how people use this engine to build rules driven applications.

My series of four digitial articles have been published by Addison-Wesley. You can get the links to purchase them and the associated source code from my web site.

I have tried to explain, in practical terms, what you need to know to actually build real world software using Windows Workflow. There is a tiny amount of theory to explain the underpinnings. The vast majority of the explanation uses code examples to illustrate all the key points. The last shortcut in the series has two extended examples that illustrate how to build custom activities.

If you were to believe some of the more vocal advocates of the agile approach to building software, you would think that the word planning is evil. If you change the comparison to agility and discipline, then the contrast is more interesting.

This is exactly what Barry Boehm and Richard Turner do in their book "Balancing Agility and Discipline". They argue that the agile approach to software development, and the discipline approach to software development have much to teach each other. Any given project must achieve the right mix of agility and discipline based on the nature of the project.

One of the more interesting graphs in their book makes this point clear:

You have to understand:

How mission critical your project is?

How many people are on the project?

How experienced are your people?

Do the people on your project handle uncertainty well, or do they prefer order?

How dynamic can the changes to requirements be?  Do you understand the domain model well?

As the graph makes clear, the more experienced developers you have that can handle vagueness and ambiguity, the more likely you can use more agile methods. On the other hand, the more mission critical, the more lives at stake, and the larger the project, the more project discipline you need.

This is not an all or nothing choice. As the book makes clear, most of your projects will need some combination of agility and discipline. Nonetheless, no method is a silver bullet.

Arthur Pyster (the Deputy Chief Information Officer for the FAA) writes in his foreword that even building air traffic control systems can incorporate some agile processes.

I had not read their book in 2005 when I made this entry to my blog: http://www.reliablesoftware.com/DasBlog/PermaLink,guid,847f84e5-3629-400b-807b-c6f8a1345454.aspx. Reading this book reinforced my belief that one must continually evaluate the risk associated with a project, and adopt the appropriate methods that reduce that risk.

Here are good instructions on how to install RC1 for the .NET Framework 3.0: http://blogs.msdn.com/pandrew/archive/2006/09/07/745701.aspx. People, including myself, have been having problems getting the Workflow Extensions for Visual Studio 2005 installed. I moved the installer file (Visual Studio 2005 Extensions for Windows Workflow Foundation RC5(EN).exe) to a different directory from the other installation files. The workflow extensions then installed just fine.

David Chappell (http://www.davidchappell.com/HTML_email/Opinari_No16_8_06.html) argues that SOA may not foster the service reuse that everyone has been hoping for. I think his analysis is correct, but I think with business services we at least have a reasonable hope of achieving reuse. Here we are least dealing with the way things actually happen in the world as opposed to programmer abstractions such as objects or components. That combined with the looser coupling of services gives me some hope.

The reason why frameworks like .NET are successful is they reflect years and years of experience with programming problems. Many examples of reuse (such as file systems and compilers) are so embedded in our experience that we no longer see them for what they are.

Reuse may fail here as well for all the reasons mentioned in David Chappell's analysis. At least now I feel we are on the right track.

The Reference Model for Service Oriented Architecture defines a vocabulary for building service-oriented systems. Put together by a technical committee operating under the auspices of the OASIS standards organization, it is the result of individuals and organizations representing vendors, users, governments, consulting organizations, and academic institutions.

The Reference Model (RM) sees SOA as a means for organizing and using distributed capabilities that may be under the control of different ownership domains. The RM is not an architecture. It does not attempt to make any architecture normative. It does not try to make any standard or set of standards normative.

It does provide a common set of semantics that can be used across different implementations. This does sound rather fancy. Nonetheless, just like Moliere's bourgeois gentlemen that found out he was speaking prose all his life, many industries have been using reference models all along. They just never had to define them explicitly.

An architect for a residential dwelling knows that if they use the term door or window, the builder will understand what is meant. There are widely varied implementations of doors and windows depending, for example, if you are building a space station or an igloo. Nonetheless, everyone knows what the terms mean. Many of these terms are codified in building codes, and by standards bodies, and have evolved over the years. The software architecture community moves too quickly for such evolution; this is where standards organizations can help.

Software architectures, for sure, can have views and viewpoints, but the terms in which they are discussed have to be understood.

The core concepts that the RM discusses are service, visibility, execution context,  service description, real world effect,  interaction, and contract and policy.

I will discuss these core concepts over the next few posts.

None of this work is going on in isolation, or is it intended to denigrate other work such as the WS* specifications, or organizations such as the ISO, IEEE, IETF, the Ontolog Forum or other groups. The reference model just supplies standard definitions so that it becomes easier for each group to communicate with the others.

I have updated the workflow examples on my site to the most recent Workflow version.

I would like to thank all those who helped me achieve an Microsoft MVP award for Visual Developer - Solutions Architect.

"You are so young; you stand before beginnings. I would like to beg of you,

dear friend, as well as I can, to have patience with everything that remains unsolved in your heart. Try to love the questions themselves, like locked rooms and like books written in foreign languages. Do not now look for the answers. They cannot now be given to you because you could not live them. At present you need to live the question."

• Rainer Maria Rilke, Letters to a Young Poet

I learned programming in high school from a Fortran IV manual, which was like learning how to drive a car from the owner's manual—unexciting. Later, after taking an operating systems course at MIT, I gave up entirely on programming as a profession. I did not want to spend my life doing the same thing over and over again.

What made me change my mind and become a professional programmer? In large part it was Gerald M. Weinberg's The Psychology of Computer Programming, which I first read in 1982. Weinberg not only demonstrated that programming was more than technology, it is a social activity, but he showed how the social element related to the technical. In essence, he identified and addressed the types of fundamental questions that Rilke advised the young poet Franz Kappus to study. Such as:

• “What does it mean when we say a program is good?” I learned from Weinberg that a good program is as much a matter of cultural fit as technological merit. A designer has to understand that tradeoffs are made not only among technical factors, but among technical, social and economic constraints. Too often, I have seen engineers try to build the “perfect” product while ignoring ease of use or budget constraints.

• “How do you get programmers to work together as a team?” One programmer cannot do it all. Different people have different skills, different skills are needed at different parts of the project.

• “What is leadership all about?” How do you manage change and performance? Why do many managers manipulate programmers and tread them poorly and then wonder why they get poor results?
• “How do you find good programmers?” And just what does it mean to be a good programmer? Weinberg was one of the first to point out the stupidity of aptitude testing for programmers, and the importance of understanding individual psychology in dealing with programmers.

Weinberg confirmed my own intuition that software could have an enormous impact on society, and his discussion of programming as a social activity helped explain much of the “strange behavior” I saw around me as I began working on my first programs. For example, during one of my first projects, I saw that the inability of certain people to work together had more impact on the project’s outcome than the technological issues being debated.

Weinberg examined what a naïve programmer would consider just technical topics and demonstrated how the elements of human personality and interactions between people had just as much, if not more influence over the outcome of a computer programming project than the technical issues and debates.

By understanding the importance of questions such as these, even if not every question can be answered in every situation, my value as a programmer and designer transcends whatever today’s technology du jour happens to be. I would have to say that Weinberg’s book took years off my apprenticeship, and saved me much aggravation.

To this day, I view programming primarily as a human activity, with the technical merits secondary. This does not mean you can ignore the technical merits. What makes a programmer really great is not technical genius, but an understanding of the human context of what he or she is doing. Any programmer who creates a truly revolutionary and world-changing program understands this. Others did not, or did not care to, and their contributions are hidden behind or overwhelmed by others’ accomplishments.

It is incredible that a twenty-five year old programming text containing examples illustrated with technologies that many programmers today cannot even conceive of—I recently taught a programming class where not one of the students had any idea what a punched card or paper tape was—is still a great book.

Here is my final dnrTV session: http://dnrtv.com/default.aspx?showID=24. It covers advanced topics in Windows Workflow Foundation such as synchronization, transactions, and compensation.

I have signed the petition at: http://www.mwdadvisors.com/resources/stop-the-madness.php. SOA does not need another buzz word. I think SOA 2.0 ranks belong even ESB on the buzz word list.

This is also an experiment. We have heard of viral marketing. Let us see if we can have viral common sense.

How do workflow and service oriented architecture relate?

The real question is how service oriented architecture (SOA) and business processes relate.

Service orientation is about how to organize and utilize distributed capabilities that could be under the control of different owners.¹ Business Process Management (BPM) is about modeling, designing, deploying and managing business processes.²  Business processes are the capabilities, or the users of those capabilities. Workflow is a technology that builds the automated part of a business process. It integrates human decision with synchronous and asynchronous software systems. Of course this is somewhat recursive because a workflow could use other services in its implementation.

For me, SOA and BPM are not in conflict. People talk about layering BPM on top of SOA. Or that SOA is for IT folks, and BPM is for business people. In today's world, business cannot afford to have people who just think IT, or just think business. Given the way the human mind works, multiple models are often needed to think about certain problems.³ SOA and BPM are two different ways to think about the same problem: how organizations can best accomplish their missions. Thinking about business process will transform how you architect your services. Architecting your services will impact how you model your business processes.

¹ For more information about service oriented architecture take a look at the Reference Model that the OASIS TC that I am a member of has produced:  http://www.oasis-open.org/committees/download.php/18486/pr-2changes.pdf

² See http://ww6.infoworld.com/products/print_friendly.jsp?link=/article/06/02/20/75095_08FEbpmmap_1.html

³ See "Mental Models" by P.N. Johnson-Laird in Foundations of Cognitive Science edited by Michael I. Posner

I was interviewed by Carl Franklin and Richard Campbell on .NET Rocks: http://dotnetrocks.com/default.aspx?showID=183. Yes we talked about Workflow and SOA. But we touched on other topics such as the failure of technology to really make foreign language learning any better.

Here is part three of the Workflow Webcast series: http://dnrtv.com/default.aspx?showID=23

Here is the second talk on Workflow Foundation on Carl Franklin's dnrTV:

http://dnrtv.com/default.aspx?showID=22

Here is the first of four talks on Microsoft Windows Workflow Foundation that are appearing on Carl Franklin's dnrTV. This one was broadcast on June 2. Each of the following ones should appear in subsequent weeks.

http://dnrtv.com/default.aspx?showID=21

(Apologies to Christopher Alexander)

Christopher Alexander, the architect who inspired the Design Patterns movement, wrote a two part article that appeared in the April and May 1965 issues of Architectural Forum entitled “The City is Not a Tree.” The tree in the title is not a biological tree, but refers to a hierarchy being used as a way to organize how modern cities are built.

We all try to organize the world into neat categories. It helps us make sense of the world. Unfortunately, those categories and subcategories force us to view the world as a set of hierarchical categories. Alexander argued that architects who think that way produce buildings and cities that are sterile and unlivable. For example, zoning that refuses to mix residential, industrial and commercial use has some very severe drawbacks in transportation, living conditions, and tax policy.

The world has too many interrelationships to be viewed as a hierarchy, it is really a semi-lattice. Now there are parts of the world that are hierarchies. But a hierarchy is a semi-lattice, but the reverse is not true. The point is that if you view the world as a hierarchy you miss the true picture.

Software often has to model some part of the world. The World Wide Web is a semi-lattice. Image what the Web would be like if it could only be structured as a hierarchical directory such as Yahoo. Don’t get me wrong; neat categories are often useful. But Search has become such an important part of the Web because it allows you to capture the relationships in a semi-lattice.

Take the classic example I used to give my software engineering students when teaching them about abstraction and object-oriented systems: How do you define a chair? Of course they start out with a standard definition. A chair has a back, a seat, and four legs. But what about a bean bag? Or even a table? In the end, what emerges is that a chair is about a relationship between a piece of anatomy and surface that can support it.It is a relationship, not an object with constraints.

This is what led Alexander to focus on patterns and not components. Of course, some patterns could become components. But components (software or otherwise) are packaging artifacts, not fundamental abstractions. This is why the authors of Design Patterns have the principle of "Favor object composition over class inheritance." Class inheritance is a hierarchy. Object composition allows you to build a semi-lattice if that is appropriate.

Focusing on relationships means you focus on behavior, on what happens in the real world. Systems built on behavior are more flexible and more scalable than those based on constrained objects. Of course not all systems have to be flexible and scalable. Flexible and scalable often conflict with other desired goals such as performance.

Service orientation is based on focusing on the relationships or behaviors between the capabilities of distributed services because ultimately, a service performs some action in the real world. In service oriented systems you do not focus on constrained objects. You try to model the world as the semi-lattice it really is. 1

[1] Look at http://polaris.gseis.ucla.edu/pagre/simon.html for another interesting perspective.

One of the dogmas of messaging technology is that the "truth is always on the wire." In the context of interoperability that is certainly true. The message, not the platform object model that generated the message, is all that really exists between a service provider and consumer.

Like all principles it has its limits. The statement the "truth is on the wire" only means that using an agreed upon message format is equivalent to a using a common syntax for a language such as English. It does not matter how you define the message format. XML Schema, RelaxNG, or just "ask Alice" are all equivalent. Humans are better at handling ambiguity than machines, hence English syntax can be a lot looser than a message format. Nonetheless, the point remains valid.

Syntax tells you nothing about the semantics of the message. For those of you who abhor fancy terminology, semantics means nothing more or less than the real world actions that arise from processing the message.

Just like you can misunderstand an English sentence, you can "misunderstand" a SOAP message. This misunderstanding may be a programming error, or a misunderstood or mismatched policy.

For example, I send to my bank a correctly formatted message that says transfer $1000 from my cash reserve to my checking account. If the bank transfers the money from savings to checking, that is a programming error. The "wire truth" however was not violated.

Now suppose that the bank made the correct transfer, but the bank's policy (which I did not know of at the time) was to report such transfers to a credit bureau. My altered credit score resulted in a higher interest rate on the loan I was applying for. Understanding a service's policy is as important as understanding the message format.

Truth is not on the wire, truth is the real world effect of what happens when a SOAP message is processed. Truth is semantics.

Agile based software development methodologies often remind me of the story about the person who jumps off a 100 story building, and passing the 45th floor yells out "No problems yet!"

Agile based software methods have many good ideas. Their critique of the waterfall method has great merit. The best documentation is the code itself. Document based solutions do not work. But it does no good to demolish one myth only to have it be replaced by another.

To imagine that because the attempt to completely design everything up front is futile, the idea that you can iterate every few weeks and wind up with an adequate design is often wrong. That might work for a project that is strongly user interface or end-user driven. I doubt it would work for designing an air traffic control system, or system software such as Microsoft's Windows Communication Foundation. These kinds of projects have strong lifecycle requirements about safety, security, performance, or scalability. Often they require individuals to acquire new areas of knowledge or expertise.

Barry Boehm's spiral model of software development is a much better approach.¹ The idea behind the spiral model is that at each choice point in the software development process one assess the risk that the project could fail to meet its goals. Based on that analysis the next step is to mitigate that risk. It might mean doing a prototype, refining the requirements, or doing more testing. Some of these tasks may be done concurrently. Analyzing the results of these steps might cause the development process to backtrack. In all cases, the views of all the project stakeholders (customers, developers, marketing, etc.) are considered at each analysis point.

Given this approach, the classic view (from Boehm's original paper) looks like a spiral:



Since the spiral model is a risk driven process, some circumstances might dictate an agile methodology. Other cases would require other approaches. By making risk the focus, rather than a manifesto of principles there is a higher probability of making the correct choices.

Let risk mitigation guide your development process.

1. Boehm's original paper appeared in IEEE Computer 21(5) 61-72 in 1988. In 2000 he updated the model at the "Spiral Development: Experience, Principles, and Refinements Spiral Development Workshop".

One of the benefits of service oriented systems is that they are loosely coupled.

David Orchard analyzes what loose coupling means from the perspective of the Web services stack. A human being can recognize that a field in a form is misplaced, software cannot. So for a particular message invocation, early binding is necessary. This is certainly true for standards. There needs to be a defined place for addresses and security tokens.

Orchard asks us to imagine Purchase Order system. A particular piece of information in a particular message must be bound to the appropriate programming types. If you need to know the name of the purchaser, you must early bind to the format of that name. Or to use fancy language, the service must understand its semantics. But it is only necessary for those programming types that the service needs to understand. Here is where building service interactions as messages rather than as remote procedure calls (RPC) is important.

If a service interaction is defined in terms of RPC, then if you change the semantics, you must change the service interface. As long as one type of the method call changes, the whole interface is broken. If you send messages (concretely XML messages), so long as the service can find the information it needs, the service is not bound to a particular message format. Other information can change, but the service does not care.

For example, if a service processing a message does not care about security, they can ignore the WS-Security SOAP headers. Those headers can change and the service can ignore all the security possibilities. The inventory service does not care if the credit information changes.

True, if XPath is used you are dependent on a certain structure to find information, but if you mark your documents with its version, or associated XML Schema, you could use the appropriate location path for the document. Or if you want to bind everything to type you can use the appropriate XML Schema instance to serialize the message to the appropriate programming types.

Loose coupling at the application level is about inserting levels of indirection to handle versioning (so what else is new?). But a message can do this because at the service interface the message is opaque. A RPC is not opaque.

At the application level loose coupling is how easy is to make a change that does not impact other parts of the system. With opaque messaging, a new version can be added without impacting other clients. If a service wants to reject a version it no longer supports, or does not yet support, it can do so without impacting other clients. In this restricted, but vitally important sense, semantic meaning in a Web service can be late bound.

Sarbanes-Oxley mandates that public companies should be able to produce all materially relevant transactions during an audit.

In the world of service oriented architecture, huge volumes of business documents flow freely as messages between services. These services are orchestrated (or choreographed if you wish) to produced business processes. To give you some idea of the volume, some people fear that the volume of XML is starting to take larger and larger fractions of network bandwidth. This is why some are starting to push the use of Binary XML for SOA messages.

In this world of huge stores of electronic messages and documents, how in the world do you find all the relevant ones? This is where XML Schema comes to the rescue. Your XML documents should be defined with schema, and hence subject to validation. Performance considerations may dictate that you do not validate your documents during message processing. Nonetheless, with schema definitions you should be able to query your messages to search and find the relevant documents.

For example, if you need to find all transactions with a given company worth over a certain threshold, you have to the tools to find it.

Microsoft's Indigo platform will unify all the divergent transport technologies (ASMX, WSE, COM+, MSMQ, Remoting) that are in use today. For building a service on the .NET platform this is the technology you will use.

What technology should you use today?

The ASMX platform's programming model is the same as Indigo's. Attributes, indicating what technologies (security, reliability, etc.) you want the infrastructure to use are applied to methods. Hence, a converter will be provided to convert ASMX code to Indigo code.

Does this mean ASMX should be the technology