Thoughts on Software Development
Michael Stiefel's thoughts and opinions on software development.
Send comments to
Friday, March 17, 2006
My blog now uses DasBlog at http://www.reliablesoftware.com/DasBlog/default.aspx.
Please point your RSS Aggregators there.
I will keep the existing blog files available so all existing links still work.
Tuesday, February 07, 2006
A Software System is Not a Tree and Why Service Orientation Can Be a Good Thing
(Apologies to Christopher Alexander)
Christopher Alexander, the architect who inspired the Design Patterns movement) wrote a two part article that appeared in the April and May 1965 issues of Architectural Forum entitled “The City is Not a Tree.” The tree in the title is not a biological tree, but refers to a hierarchy being used as a way to organize how modern cities are built.
We all try to organize the world into neat categories. It helps us make sense of the world. Unfortunately, those categories and subcategories force us to view the world as a set of hierarchical categories. Alexander argued that architects who think that way produce buildings and cities that are sterile and unlivable. For example, zoning that refuses to mix residential, industrial and commercial use has some very severe drawbacks in transportation, living conditions, and tax policy.
The world has too many interrelationships to be viewed as a hierarchy, it is really a semi-lattice. Now there are parts of the world that are hierarchies. But a hierarchy is a semi-lattice, but the reverse is not true. The point is that if you view the world as a hierarchy you miss the true picture.
Software often has to model some part of the world. The World Wide Web is semi-lattice. Image what the Web would be like if it could only be structured as a hierarchical directory such as Yahoo. Don’t get me wrong neat categories are often useful. But Search has become such an important part of the Web because it allows you to capture the relationships in a semi-lattice.
Take the classic example I used to give my software engineering students when teaching them about abstraction and object-oriented systems: How do you define a chair? Of course they start out with a standard definition. A chair has a back, a seat, and four legs. But what about a bean bag? Or even a table? In the end, what emerges is that a chair is about a relationship between a piece of anatomy and surface that can support it. It is a relationship, not an object with constraints.
This is what led Alexander to focus on patterns and not components. Of course, some patterns could become components. But components (software or otherwise) are packaging artifacts, not fundamental abstractions. This is why the authors of Design Patterns have the principle of “Favor object composition over class inheritance.” Class inheritance is a hierarchy. Object composition allows you to build a semi-lattice if that is appropriate.
Focusing on relationships means you focus on behavior, on what happens in the real world. Systems built on behavior are more flexible and more scalable than those based on constrained objects. Of course not all systems have to be flexible and scalable. Flexible and scalable often conflict with other desired goals such as performance.
Service orientation is based on focusing on the relationships or behaviors between the capabilities of distributed services because ultimately, a service performs some action in the real world. In service oriented systems you do not focus on constrained objects. You try to model the world as the semi-lattice it really is. 1
Saturday, November 19, 2005
One of the dogmas of messaging technology is that the "truth is always on the wire." In the context of interoperability that is certainly true. The message, not the platform object model that generated the message, is all that really exists between a service provider and consumer.
Like all principles it has its limits. The statement the "truth is on the wire" only means that using an agreed upon message format is equivalent to a using a common syntax for a language such as English. It does not matter how you define the message format. XML Schema, RelaxNG, or just "ask Alice" are all equivalent. Humans are better at handling ambiguity than machines, hence English syntax can be a lot looser than a message format. Nonetheless, the point remains valid.
Syntax tells you nothing about the semantics of the message. For those of you who abhor fancy terminology, semantics means nothing more or less than the real world actions that arise from processing the message.
Just like you can misunderstand an English sentence, you can "misunderstand" a SOAP message. This misunderstanding may be a programming error, or a misunderstood or mismatched policy.
For example, I send to my bank a correctly formatted message that says transfer $1000 from my cash reserve to my checking account. If the bank transfers the money from savings to checking, that is a programming error. The "wire truth" however was not violated.
Now suppose that the bank made the correct transfer, but the bank's policy (which I did not know of at the time) was to report such transfers to a credit bureau. My altered credit score resulted in a higher interest rate on the loan I was applying for. Understanding a service's policy is as important as understanding the message format.
Truth is not on the wire, truth is the real world effect of what happens when a SOAP message is processed. Truth is semantics.
Monday, October 24, 2005
Risk Based Software Development
Agile based software development methodologies often remind me of the story about the person who jumps off a 100 story building, and passing the 45th floor yells out "No problems yet!"
Agile based software methods have many good ideas. Their critique of the waterfall method has great merit. The best documentation is the code itself. Document based solutions do not work. But it does no good to demolish one myth only to have it be replaced by another.
To imagine that because the attempt to completely design everything up front is futile, the idea that you can iterate every few weeks and wind up with an adequate design is often wrong. That might work for a project that is strongly user interface or end-user driven. I doubt it would work for designing an air traffic control system, or system software such as Microsoft's Windows Communication Foundation. These kinds of projects have strong lifecycle requirements about safety, security, performance, or scalability. Often they require individuals to acquire new areas of knowledge or expertise.
Barry Boehm's spiral model of software development is a much better approach.1 The idea behind the spiral model is that at each choice point in the software development process one assess the risk that the project could fail to meet its goals. Based on that analysis the next step is to mitigate that risk. It might mean doing a prototype, refining the requirements, or doing more testing. Some of these tasks may be done concurrently. Analyzing the results of these steps might cause the development process to backtrack. In all cases, the views of all the project stakeholders (customers, developers, marketing, etc.) are considered at each analysis point.
Given this approach, the classic view (from Boehm's original paper) looks like a spiral:
Since the spiral model is a risk driven process, some circumstances might dictate an agile methodology. Other cases would require other approaches. By making risk the focus, rather than a manifesto of principles there is a higher probability of making the correct choices.
Let risk mitigation guide your development process.
1. Boehm's original paper appeared in IEEE Computer 21(5) 61-72 in 1988. In 2000 he updated the model at the "Spiral Development: Experience, Principles, and Refinements Spiral Development Workshop".
Friday, August 05, 2005
One of the benefits of service oriented systems is that they are loosely coupled.
David Orchard analyzes what loose coupling means from the perspective of the Web services stack. A human being can recognize that a field in a form is misplaced, software cannot. So for a particular message invocation, early binding is necessary. This is certainly true for standards. There needs to be a defined place for addresses and security tokens.
Orchard asks us to imagine Purchase Order system. A particular piece of information in a particular message must be bound to the appropriate programming types. If you need to know the name of the purchaser, you must early bind to the format of that name. Or to use fancy language, the service must understand its semantics. But it is only necessary for those programming types that the service needs to understand. Here is where building service interactions as messages rather than as remote procedure calls (RPC) is important.
If a service interaction is defined in terms of RPC, then if you change the semantics, you must change the service interface. As long as one type of the method call changes, the whole interface is broken. If you send messages (concretely XML messages), so long as the service can find the information it needs, the service is not bound to a particular message format. Other information can change, but the service does not care.
For example, if a service processing a message does not care about security, they can ignore the WS-Security SOAP headers. Those headers can change and the service can ignore all the security possibilities. The inventory service does not care if the credit information changes.
True, if XPath is used you are dependent on a certain structure to find information, but if you mark your documents with its version, or associated XML Schema, you could use the appropriate location path for the document. Or if you want to bind everything to type you can use the appropriate XML Schema instance to serialize the message to the appropriate programming types.
Loose coupling at the application level is about inserting levels of indirection to handle versioning (so what else is new?). But a message can do this because at the service interface the message is opaque. A RPC is not opaque.
At the application level loose coupling is how easy is to make a change that does not impact other parts of the system. With opaque messaging, a new version can be added without impacting other clients. If a service wants to reject a version it no longer supports, or does not yet support, it can do so without impacting other clients. In this restricted, but vitally important sense, semantic meaning in a Web service can be late bound.
Tuesday, March 29, 2005
Sarbanes Oxley and XML Schema
Sarbanes-Oxley mandates that public companies should be able to produce all materially relevant transactions during an audit.
In the world of service oriented architecture, huge volumes of business documents flow freely as messages between services. These services are orchestrated (or choreographed if you wish) to produced business processes. To give you some idea of the volume, some people fear that the volume of XML is starting to take larger and larger fractions of network bandwidth. This is why some are starting to push the use of Binary XML for SOA messages.
In this world of huge stores of electronic messages and documents, how in the world do you find all the relevant ones? This is where XML Schema comes to the rescue. Your XML documents should be defined with schema, and hence subject to validation. Performance considerations may dictate that you do not validate your documents during message processing. Nonetheless, with schema definitions you should be able to query your messages to search and find the relevant documents.
For example, if you need to find all transactions with a given company worth over a certain threshold, you have to the tools to find it.
Tuesday, March 08, 2005
Microsoft's Indigo platform will unify all the divergent transport technologies (ASMX, WSE, COM+, MSMQ, Remoting) that are in use today. For building a service on the .NET platform this is the technology you will use.
What technology should you use today?
The ASMX platform's programming model is the same as Indigo's. Attributes, indicating what technologies (security, reliability, etc.) you want the infrastructure to use are applied to methods. Hence, a converter will be provided to convert ASMX code to Indigo code.
Does this mean ASMX should be the technology of choice? I would argue that WSE is the better technology to use. WSE's programming model is not that of Indigo. Classes and inheritance are used to interact with the WSE infrastructure. WSE will interoperate with Indigo. Nonetheless, the conceptual model of WSE is identical to that of Indigo.
ASMX is tied to the HTTP transport and its request / response protocol. It encourages programmers to think of a service call as a remote procedure call with programming types, not as an interoperable, versioned XML document message validated by XML Schema.
Service developers need to think of request / response as one of several possible message exchange patterns (MEP). The most fundamental MEP, the one all MEPs are built from, as the WS-Addressing spec makes clear, is the one-way asynchronous message. Business services tend to be asynchronous; you apply for a loan and you do not hear back for days.
Service messages can go through intermediaries before reaching the ultimate recipient. Each message segment may go over transports other than HTTP.
WSE's transport classes allow you to build services that use different MEPs over various transports. The SOAP envelope classes make it easy to build the SOAP message body as XML, or serialized XML objects. You learn to think in terms of XML documents and messages, not execution environment dependent types.
Using this conceptual model your services will last longer, and be easier to evolve in a business environment. That will be of more use to your business than using a technology that has a better upgrade path, but will have to be rewritten sooner because it is poorly designed and implemented.
02/29/2004 - 03/06/2004 03/14/2004 - 03/20/2004 06/13/2004 - 06/19/2004 06/27/2004 - 07/03/2004 08/08/2004 - 08/14/2004 10/31/2004 - 11/06/2004 12/12/2004 - 12/18/2004 12/19/2004 - 12/25/2004 03/06/2005 - 03/12/2005 03/27/2005 - 04/02/2005 07/31/2005 - 08/06/2005 10/23/2005 - 10/29/2005 11/13/2005 - 11/19/2005 02/05/2006 - 02/11/2006 03/12/2006 - 03/18/2006
|All Content (c) 2000 - 2004 Reliable Software, Inc. All rights reserved.|