Sergio Del Piccolo

Solution Architecture Overview
Bob Maggio, Architect, SourceCode

Bob went through a K2 [blackpearl] solution showing much of the detailed architecture. A couple of noteworthy items to share here are:

1. When working with a Smart Objects, you can add a process level reference to the Smart Object in the solution. By doing this, each activity or event will point to the Smart Object in the reference. Not using the referenced Smart Object results in each call to the Smart Object adding to the operating cost of your solution and thus negatively affecting performance.

2. K2.Net 2003 and K2 [blackpearl] differ regarding activity instances. In K2.net 2003, there would be an activity instance for each person assigned to a task. In K2 [blackpearl] there is now a single activity instance regardless of the number of people assigned to a task.

K2 [blackpearl] Architecture Overview
Koos du Preez, CTO, SourceCode

Koos went through the some of K2 [blackpearl] architecture. The first thing he spoke about though was performance. Although not an architecture item, the product architecture is a major consideration in achieving optimal performance. Performance will be much improved with the upcoming 803 release. While the main goals of this release are 64 bit support and improving the K2 [server] installer, it includes significant performance enhancements. The workflow server performance is much improved; primarily process execution. The worklist and worklist item performance has been improved significantly as well.  

Koos also touched on the following items:

1. Workflow Authoring

K2 allows for authoring of processes using any designer, viz. Microsoft Visual Studio 2005, K2 Studio and Microsoft Visio 2007. It has a common authoring API which generates a KPRX file used by all designers. It also contains custom configuration data and customizable pre-built workflow schedules – which don’t require code generation. The schedules are created using XAML. One important point to note is that the extender files should not be checked into source control, as it will result in the extender files becoming corrupted.

When the solution is built, the result is compiled executable artifact which are contained in a MSBuild package ready for deployment. Deployment executes the MSBuild package which in turn results in the following steps being taken:
i – Obtaining environment setting
ii – Deploying artifacts using build tasks
    a – creates MOSS artifacts
    b – uploads MOSS artifacts
    c – deploys the workflow
    d – deploys the events
    e – deploys Smart Objects

One thing to remember is that Service Objects need to be manually deployed to other environments. Lastly, a great tip regarding deployment is that when deploying using the command line tool, detailed information is returned as well as error information, e.g. an error that might be returned using this method could be “Error creating MOSS site”.

2. Runtime

At runtime, a .Net AppDomain is created on the K2 [server]. All process definitions and behaviour is loaded from the K2 databases. Isolated and versioned execution of processes start.

3. K2 Layered Architecture

i – K2 [blackpearl] comes with a K2 SSO (Single Sign-On) – see below for more.
ii – Communications are pluggable by design. Currently the K2 API to the server uses TCP on port 5555. This can be replaced by WCF or an alternative protocol if needed.
iii – Roles can be authenticated using AD (Active Directory), LDAP or Microsoft SQL Server amongst others.
iv – The K2 [server] can listen for K2.Server 2003 and K2 Studio 2003 using TCP over port 5252.

4. K2 Single Sign-On (SSO)

The K2 SSO is based on a primary ID. Line of business (LOB) IDs are attached to the primary ID. It supports multiple backend LOB credential caching using pluggable authentication providers. These can have multiple instances of the authentication providers e.g. multiple SAP instances can be attached. The K2 SSO supports Windows SSPI (NTLM and Kerberos).

The K2 SSO ships with the following authentication providers: AD; SQL; Siebel; SAP; SalesForce. The K2 SSO is pluggable and an SDK exists for creation of custom authentication providers. It is highly secure, making use of SQL Server symmetric keys in turn protected by system credentials. Caching can be set to expire or turned off. Automatic prompting, validation and caching of credentials occur the first time an event results in a LOB being hit.

K2 SSO supports seamless Smart Object integration. The way it works is as follows:
i – The API requests the Smart Object method call
ii – Marshalling calls the Smart Object Exec with broker package
iii – The Service Instance requests the LOB credentials
iv – The Security Framework accesses the SSO cache
v – If the credentials are not cached then the Security Framework prompts the user for them

When a prompt is requested, it is the result of an error being thrown. If not handled, then the process will be in error state. This needs to be taken into account when using Microsoft InfoPath as a user interface.

5. K2 Databases

There can be up to 14 K2 databases. They can be configured for performance by adjusting the threads for the processing database and the log database. In the k2server.setup file, the <K2ThreadPool> tag is set at 20 thread by default for both databases. These numbers can be adjusted e.g. processing uses 25 or 30 threads and the log 15 or 10 threads. It is also possible to rename the databases as long as the names are updated in the configuration manager.

6. Common Logging Framework

The logging framework allows for logging events to be sent to multiple targets, including:
i – Microsoft SQL Server
ii – MSMQ
iii – File
iv – Windows Event Log
v – Console

The verbosity and severity levels for each target can be set. The message queue used to send logging events can be bypassed, resulting in synchronous real time logging of messages. This does affect overall performance though. This is done by opening the HostServerLogging.config and altering the <AsyncQueueEnabled> tag. When making this alteration, the service needs to be restarted.

It is also possible to create predefined messages taking parameters using text, etc.

7. Monitoring (Windows PerfMon)

The following are tracked using PerfMon:
i – Processes Started, per second
ii – Worklist Opened, per second
iii – Worklist items finished, per second
iv – Process Memory Usage
v – Process Thread Count
vi – Process Modules Loaded
vii – TCP Bytes Retrieved Total, per second
viii – TCP Bytes Sent Total, per second
ix – TCP Concurrent Connections
vii – TCP Connections Opened, per second

Business Process Management Today
Maureen Flemming, Program Director, Business Process, Integration and Deployment Software, IDC

The afternoon session had Maureen give some insight into the market today and Business Process Management. The session was highly informative, although I did not take too many notes.

As is clearly evident, the second day was busy and managed to live up to the high standards of the day 1. Here’s to day 3, but before that there are a few bets to win at the bowling alley this evening.

SourceCode, K2, K2[blackpearl], Process Management, K2 SSO, K2 Databases, Monitoring K2, K2 Logging Framework, K2 Workflow Authoring

The inaugural K2 Insiders Conference started on April 8th, in Denver Colorado. I’ve been looking forward to it for a long time, waiting to meet fellow Insiders and to have a chance to find out more about the inner workings of BlackPearl and new offerings from SourceCode.

Key Note, Vision and Roadmap
Adriaan Van Wyk, CEO, SourceCode & Olaf Wagner, Senior VP – Product Development, SourceCode

Adriaan started with welcoming everyone and speaking to the history of SourceCode and the product and it’s evolution to the current version of BlackPearl. One phrase that stood out to me was his description of BlackPearl being software for creating Process Driven Applications.

The Year in Review

Adriaan spoke about 2007 which turned out to be a challenging year for SourceCode with the release of BlackPearl; some of the issues that the firm faced and what they were planning to do to resolve them. The release of Service Pack 1 helped SourceCode esolve many issues that had existed before and the product team was working hard to get the next two releases (803 and 805)out. They will contain the following enhancements:

803
- Full 64 bit support
- A new and improved installer for deployment
- Internationalization  support
- 30+ patches including
   – Visual Studio stability around InfoPath projects
   – Enhanced workflow performance
   – Enhanced reporting performance
   – Enhanced Smart Object performance
   – Smart Object lifecycle improvements
   – Sales tools for partners

805
- Team Foundation Server integration
- References bug fixes
- Additional Smart Object and Reporting performance enhancements
- K2 [connect] for SAP RTM 
- Additional bug fixes

Strategic Components for K2

The next item on the agenda were the strategic components for K2, what they meant and proposed time frames for delivery:

1 – K2 Studio

K2 are planning to release a new version K2 Studio. K2 Studio is a development environment that can be used for creating process driven applications. It is felt that K2 Studio will be better suited for non-developers as many feel intimidated by having to use Visual Studio. K2 Studio re-hosts the Visual Studio designers used to develop applications. It will support the following: process design; Smart Object design; the Artifact Library (when available) and any future designers released.

The K2 Studio can be installed without the need for Visual Studio to be installed and solutions can be opened and edited in either designer. There is support for K2.net 2003 processes and artifacts. The K2 Studio does make use of MS Build to create the deployment packages (similar to Visual Studio). One limitation of the K2 Studio is the fact the user will not be able to get to .Net code – the solution will have to be opened in Visual Studio.

The design of the K2 Studio is similar to that of Office 12 and 14, making use of the ribbon menu. It maintains the relevant context, allowing for Activity and Event options to be edited as well as Deployment options.

2 – Process Management

The plan is to release a Process Management API and tools, with the primary focus being for inclusion in K2 Studio and Visual Studio. A web tool is planned for a later release. Processes will be able to be managed at process and instance levels. It will allow for in-flight process instances to be altered. The changes can then be applied to a single instance, all instances of a specified definition or all processes.

The demo showed a process being opened from a report, the instance being drilled-down to and being selected. It opened the instance within Visual Studio although no solution had been opened. The View Flow was displayed and it was mentioned that a new zoom-able version of the View Flow was planned for a future release. Access to rules and policies are available for alteration on activities that had not been reached. Activity wizards can be re-run and altered. Once complete the instance was redeployed to the server, showing some analysis of the two versions and highlighting their differences.

Needless to say, this is a very powerful feature allowing for an incredible amount of flexibility. The impact can be significant and can effect reports and policies and rules in unexpected ways. Some guidance will be required around this and there will no doubt need to be internal policies managing changes within your firm.

3 – Process Portals

These are portals allowing for management of processes through WSS sites. They are not dependant on MOSS. They may either be created or existing can sites can be extended to become Process Portals. They contain web parts allowing for the management of the processes and the templates used for creation of the sites is customizable.

The available web parts are: Process Overview; a My Task List with all my Worklist items and a Task List for all Worklist items (a managerial view). All process instances can be managed including running, error and stopped instances. There is access to reports, administration, and security management. The web parts are fully extendable as well.

Now users won’t need to leave SharePoint to manage their processes or work on their Worklist items.

4 – K2 [connect]

K2 [connect] is SourceCode’s integration solution for process driven applications. It envisaged that it will remove many of the integration pains experienced when there is a requirement for data integration. The ease of use and low cost is seen as the value proposition for K2 [connect]. It’s based on WCF adapters and can be hosted without the need for Microsoft BizTalk. 

What it does:
- Allows for is a synchronous access to data as logical composite service objects
- Allows for easy binding of data to forms
- Declarative mapping between internal SAP fields and external fields
- Supports composite mapping and transformation
- Comes with a Microsoft BizTalk license

What it is not / does not do:
- An Enterprise Service Bus (ESB)
- Provide persistence points for data i.e. a message box.

Currently there is a version for SAP in beta which is expected for release as RTM by the end of April 2008 (yes this month). It adds nearly no load to the SAP system, which will make many SAP administrators that I deal with happy. It allows for processes to be triggered by SAP events, and creates a two-way connection allowing for reads and writes. It can also be used in reporting.

The next version of K2 [connect] which is expected in October 2008, will include:
- Support for Microsoft BizTalk WCF LOB SDK
- Support for Siebel, Oracle, SQL Server, and complex services
- Updated designers
- Integration with WF

5 – End-Point Publishing

End-Point publishing allows for the publishing of any Smart Object with a WCF endpoint or to the UDDI Catalog. It automatically versions the endpoint to the Smart Object. Workflow processes can be published as endpoints as well. The release plan for End-Point publishing is for a beta in the next of month, with a release to RTM expected between 1 and 3 months after that depending on the feedback received.

6 – Silverlight Workflow Designer

The Silverlight Workflow Designer is planned to replace the current AJAX Workflow Designer. It is built on Silverlight 2.0 and can be hosted in any web application. It will be backwardly compatible with the current version, although there is a change in the design concept: it will focus on people and results rather than the normal K2-style event steps. It is built on a new extensibility model and won’t be specific to MOSS/WSS.

7 – BlackPearl Simulation

The BlackPearl Simulation engine will allow customers to answer “what if” reports to assist in intelligent decision making around their business processes. It is built on Silverlight 2.0 and is architected to have an extensible resource and reporting model. It has a “video editing” mode, and can use actual historical data in its models. It can measure KPIs across process versions. Currently there is no release timeframe.

8 – BlackPearl Workspace 2.0

That’s right – the current BlackPearl Workspace is pencilled for an upgrade. The new version will have a new look similar to that of the K2.net 2003 Workspace. It will be a more granular AJAX implementation with improved performance. Web parts will be available for inclusion on SharePoint sites. Currently there is no rush to get this out and it is thought that it will be ready for release by the end of 2008.

9 – Out of Office

The Out of Office feature is planned to allow for external events to effect Out of Office status, e.g. Out of Office in Microsoft Exchange will change the users Out of Office status in BlackPearl. Another feature is task-type delegation, e.g. delegating leave approvals to one person and expense approvals to another. It will allow for sharing based on action rights and for confirguration to be saved. Once saved, Out of Office can be turned on or off. 

10 – Release 806

This release has no time frame but it is planned to include the following enhancements:
- Visual Studio 2008 support
- Full K2.net 2003 Interop support and migration wizards
- K2 Studio RTM
- Process Portals RTM
- K2 Endpoint RTM
- Out of Office RTM
- Dynamic SQL Service (tables, views and stored procedures)
- Web Service / WCF Workflow templates RTM (low priority)
- K2 BlackPearl templates for Microsoft BizTalk 2006 R2 (low priority)

In Conclusion

What a day! There were a lot of exciting new features and enhancements, as well as problems shared with the Insiders. Much discussion and feedback around the vision and product was generated. It proved to be very informative and gave us a chance to meet and interact with some of the people behind the scenes. Hopefully this results in a better product for your process driven application J needs.

SourceCode, BlackPearl, K2, K2 [connect], K2 Studio, Process Management, Process Portals, Endpoint Publishing, Silverlight Workflow Designer, BlackPearl Simulation, BlackPearl Workspace, Out of Office

On March 4th of this year the Heroes Happen {Here} passed through Vancouver. The day proved to be quite interesting from a technical point of view with different tracks for different audiences. A few items I wanted to bring to your attention were:

1. Mohammed Akif led a great presentation on Implementing Service Oriented Architecture in your organization. He mentioned that he would post his slides when he got a chance. Let’s hope he does it soon. In the meantime, he has a recording about Service Orientation on his blog (No Spin Architecture) that is worth viewing.
2. It was good to see Richard Baumet as a SQL Server Expert at the event. The experts were answering questions posed to them by some of the attendees. Richard is the Vancouver PASS chapter leader. VanPASS have arranged for Donald Farmer to present this month (March 26th) on Predictive Analytics with SQL Server Analysis Services and Excel 2007 Data Mining Add-ins. Make sure that you attend the event if you can or view their live meeting.
3. Finally, I was amused to find a link to a comic section on the Heroes Happen {Here} site.
   - http://www.microsoft.com/canada/heroeshappenhere/cool-stuff/default.mspx
   - http://www.itdevcomics.com/
   Talk about stereotyping people working in the IT industry! Ok, I do like comics, but still – the nerve! My personal favourite reaction to this is a post by Justice Gray entitled Microsoft: Heroes Happen Here…well, fat and ugly heroes anyway.

Heroes Happen Here, VanPASS, Mohammed Akif, SOA, Service Orientation

Integrating Legacy Assets with BizTalk 2006 R2
William Henry, IONA

William spoke a little about Iona and their history before going into their product – IONA WCF Adapter used to consuming CORBA & JMS in .Net. He spent most of the session demonstrating how to use the software. He also mentioned the Atrix Connect for WCF which is to be released in Q1 of 2008.

Advanced Orchestration Concepts & Best Practices
Stephen Thomas

Stephen started off speaking about his guiding principles regarding orchestrations:

• Design easily understandable business processes
• Build for the future
• Build business processes that can be easily unit tested (e.g. with BizUnit)
• Use all available features of BizTalk to simplify development
• Keep “abilities” in mind
• Reusability
• Build shared orchestrations to accomplish specific tasks
• Return generic responses and use the Business Rule Engine to evaluate
• Maintainability
• Build clean and simple process flow
• Select and follow a naming convention
• Supportability
• Detailed error messaging for easy administration
• Avoided suspended non-resumable at all costs

Stephen next spoke about some useful tips for orchestration development:

• Leverage publish and subscribe pattern
• Promoted properties as MessageContextPropertiesBase
• Multi-Message Mapping
• Only available in orchestrations
• Watch for namespace issues – use code instead
• Direct Port Binding
• Watch out for endless loops
• Helper .Net Components
• Passing Xlang messages as streams
• Doing simple tasks using static methods
• Robust schema validation returning multiple errors
• Define messages using Multi-Port Messages when possible

Next Stephen spoke about the following advanced orchestration topics:

Untyped Messages

• What is it?  
• Using messages a XmlDoc rather than defined schema
• How does it help?
• Accepts multiple different types of messages in the same process
• Reduces redundant orchestrations
• Easily routable using direct binding and correlation sets
• Pitfalls?
• Typically routing done by Receive Port filters
• No access to Distinguished Fields or MessageDataPropertyBase and MessageContextPropertyBase properties

Dynamic Transformations

• What is it?
• Selecting maps at runtime and dynamically creating the transform object without a Transform shape
• How does it help?
• Allows for schemaless orchestrations to support multiple types of messages
• Allows for vendor specific mapping
• There is no need to redeploy maps when changes occur
• Pitfalls?
• Best to map on Send  Receive Ports
• It is possible to yield no output message – a null will be returned
• Possible performance impacts under high load – use a caching service

Configured Ports  & Self Correlating

• What is it?
• Used to start orchestration shape to send messages back into parent orchestration without correlation sets
• How does it help?
• Separate common logic into easily called processes
• No need to worry about binding
• done using correlation token
• no need for a business connection between messages
• Pitfalls?
• Untyped messages not supported – the message must be serialized

Convoys

• What is it?
• Parallel and sequential message processing support
• Routing is independent of running orchestration instances
• How does it help?
• Allows you to handle race conditions on the message box
• Pitfalls?
• Zombies – completed with discarded messages
• Difficult to reprocess message
• limit to 3 convoy properties

For more information regarding convoys see Stephen’s BizTalk Server Convoy Deep Dive whitepaper.

Links: Stephen W. Thomas BizTalk Blog

Building Solutions with the Microsoft ESB Guidance
Marty Waznicky, Principle Program Manager, Connected Systems Division

This presentation seemed like it was part two of the previous presentation that I saw from Marty – see SOA Conference: Day 3 for more. Marty then demonstrated the following:

• Exception Management process
• Management Portal
• Dynamic Routing
• Uddi services
• Dynamic Resolution
• Dynamic Transformations
• Itinerary Processing
• Scatter Gather pattern

Finally he mentioned that the new release should be posted to the MSDN patterns and practices site next week.

Links:

ESB Guidance Toolkit
Marty Wasznicky – Regional BizTalk Rants

Robust Error Handling for BizTalk Solutions
Matt Meleski, MVP / Senior Consultant, Objectsharp Consulting

Matt’s presentation covered planning and designing to handle exceptions that occur in pipelines and orchestrations. He covered the following techniques:

• Central Exception Handling using the Exception Management Framework in the ESB Toolkit
• Using Compensation in Orchestrations
• Pipeline and Orchestration Retry Patterns leveraging built in Message Box persistence

Matt broke exceptions into the following two types (with their origins):

1. .Net Exceptions
   • Receive Adapter
   • Receive Pipeline
   • Maps
   • Orchestration
   • Send Pipeline
   • Send Adapter
2. Messaging Failures
   • Publish to Message Box
   • Subscribe from Message Box
   • Orchestration not enlisted/started

BizTalk exception can then be handled in the following two ways:

1. Rely on BTS MessageBox & BizTalk framework
   • Suspend/retry using console
   • Handle errors in pipe comps (Try/Catch)
   • Failed message routing
   • Use MOM or custom technique to be alerted and resend the error messages
2. Route to file share, database, SharePoint
   • Custom processing for failed messages
   • Failed messages not in MessageBox

Matt did a couple of demos covering the following topics:

• Looking at MOM for notification of errors
• Making use of “Enabled Routing” flag for Failed Messages
• Dead letter drops – resume pattern
• ESB Exception Management Framework
• Compensation shape
• Distributed transaction (System.Transactions with Atomic Scope shapes)
• ESB Exception Management Framework
• Resume pattern

The presentation was very interesting especially the part around using the Exception Management Framework in the ESB Toolkit which can be installed separately from the rest of the ESB Toolkit – a great idea for firms not interested in the developing an ESB but wanting to take advantage of a centralized error handling process.

Link:
The ABCs of .Net – Matt Meleski
ESB Guidance Toolkit

BizTalk Server 2006 R2 for SOA & BPM
Ofer Ashkenazi, Senior Technical Product Manager, Connected Systems Division

Ofer’s presented a structured tour of the BizTalk capabilities and their benefits in implementing complex SOA and BPM solutions. He used the BizTalk Capabilities Categorization poster as the starting point for the discussion.

Links to the posters:
BizTalk Server 2006 R2 Capabilities Poster
BizTalk Server 2006 R2 Runtime Architecture Poster
BizTalk Server 2006 Legacy Modernization with HIS 2006 Poster

It was a good high level overview of all of BizTalk’s capabilities. His slide deck would be very useful in some of the pre-sales presentations that I have had to make in the past. I may have to get my hands on them for future use – watch this space.

Architecture of the Microsoft Enterprise Service Bus Guidance
Marty Waznicky, Principle Program Manager, Connected Systems

Marty led the presentation on the ESB Guidance toolkit. He started by giving his preferred defintion of SOA:
SOA enables loose coupling interoperability, management of change, operation of business services, in a governable environment. Business Services operating in a well run SOA can be complesed into business process that align IT with business.

He then summarized a few of the capabilities of the toolkit:

1. Messaging Infrastructure
   • Supports WS*
   • Supports existing applications
   • Supports partner 3rd party vendors
   • Supports B2B
   • Pub-Sub infrastructure
2. End point management
   • Decouple consumer from service
   • Registry/repository integration
   • Integration with SOA governance solutions
3. Operation support
   • End-to-end visibility
   • Exception mediations
   • Quality of service
4. Invocation patterns
• Itinerary process
• Common Messaging patterns
• Route, transform, validate, route and mediate on exception

Benefits of the ESB Guidance Tookit include:

• Loosely coupled message environment
• Policy driven level routing and mediation
• Itinerary based routing
• Code reuse

Top 5 Features of the ESB Guidance:

1. Dynamic Service Invocation
2. Unified Exception Management
3. ESB Management Portal
4. Itinerary Processing
5. SOA Governance Integration

Following the introduction there was a demo of the AmberPoint SOA Governance product which seemed impressive to say the least.

The final part of the presentation was Marty installing the ESB Guidance toolkit and showing what was coming in the new release slated for later this week.

Links:
ESB Guidance Toolkit
Marty Wasznicky – Regional BizTalk Rants

Microsoft, Microsoft SOA Conference, BizTalk,  Enterprise Service Bus Guidance, ESB Guidance Toolkit

BizTalkAdapters for WCF: Deep Dive
Aaron Skonnard, Pluralsight

This session was very good, with Aaron having only a couple of slides on the WCF adapters that come with BizTalk Server 2006 R2. The first slide showing the WCF Runtime Architecture as shown below:
 

Aaron spoke about the runtime architecture and then went on to speak about the WCF Adapters, as shown in the following diagram:

Following the short PowerPoint presentation Aaron showed a few code demos of the adapters being used in BizTalk calling a WCF service. Most of the demos covered the whitepaper written him and available from MSDN – WCF Adapters in BizTalk Server 2006 R2.

In my view this was by far the best presentation that I’ve seen so far in this years conference.

Links: Service Station – Aaron Skonnard

Mission Critical BizTalk in Telecoms
Bruno Rodrigues, Oi/Telemar – Brazil

Bruno spoke about the integration challenges faced when integrating multiple Billing, CRM and network platforms while trying to provision services like land lines, mobile phones and data products. From a technical standpoint there wasn’t much in terms of design and code, but from a business value view, the presentation clearly identified the benefits of the solution – reducing operating costs and improving productivity.

Healthcare Enterprise Integration: SOA Solutions
Simon Chester & Jason Shantz, Senior Consultants, Knowledge Tech

I was looking forward to seeing Simon and Jason speak about this topic for a while. Both are past colleagues of mine while working at Sunaptic Solutions and then Visiphor. Their presentation covered a Healthcare BizTalk – HL7 implementation in British Columbia while working at Sunaptic Solutions. Simon led the first half of the presentation speaking about the implementation and issues that they ran into during the implementation, viz. issues with ordered delivery, using the canonical model and the number of artifacts created by the model, the challenges of the dependencies between these, deployment, scalability. He then came up with a wishlist of capabilities that would have made the solution design simpler and the implementation more manageable. Jason (a BizTalk Server Virtual Technical Specialist) then stepped up and addressed Simon’s wishlist and explained how with BizTalk Server 2006 R2 would help to meet these challenges.

Links: Simon Chester

Connections in the Cloud – BizTalk Services and WCF
Justin Smith, Technical Evangelist, Microsoft

The presentation covered the Internet Service Bus hosted by Microsoft which has the following benefits:

• Firewall friendly messaging
• Scalability pub/sub engine for internet
• Standards based security
• Http/REST transformations
• Discoverability

Currently it is available in CTP (Community Technology Preview) at http://labs.biztalk.net. To get started download and install BizTalk Services SDK. Justin went on and spoke about getting started with samples.

The Internet Service Bus is integral to the “Oslo” vision of having composite applications spanning systems, evolving into Software + Services.

Microsoft, Microsoft SOA Conference, BizTalk,  Oslo, Software + Services, Internet Service Bus,  Windows Communication Foundation, WCF

Keynote: Microsoft’s Vision for the Next Generation Application Platform
Robert Wahbe, Corporate Vice President, Connected Systems Division
Don Ferguson, Technical Fellow
Steve Martin, Director, Connected Systems Division

During the keynote speech Robert Wahbe announced Microsoft’s vision, roadmap and next wave of products regarding SOA and the ability to make it available to firms of all sizes, codenamed “Oslo”. For more information see Microsoft’s press release. It comprises of a group of products and services that Microsoft intends to deliver over the next few years. This forms part of the vision of Software + Services that has been a Microsoft focus recently.

As part of reaching the goals set out by “Oslo” Microsoft will be enhancing the current technology available today, focusing on the following five areas:

• Server – Microsoft BizTalk Server “6” will provide a core foundation for distributed and highly scalable SOA and BPM solutions, and deliver the capability to develop, manage and deploy composite applications.
• Services - BizTalk Services “1” will offer a commercially supported release of Web-based services enabling hosted composite applications that cross organizational boundaries. Of note is that this release will include advanced messaging, identity and workflow capabilities.
• Framework - The Microsoft .NET Framework “4” release will further enable model-driven development with Windows Communication Foundation (WCF) and Windows Workflow Foundation (WF).
• Tools - New technology planned for Visual Studio “10” will make significant strides in end-to-end application life-cycle management through new tools for model-driven design of distributed applications.
• Repository - There will also be investments in aligning the metadata repositories across the Server and Tools product sets. Microsoft System Center “5,” Visual Studio “10” and BizTalk Server “6” will utilize a repository technology for managing, versioning and deploying models.

“Oslo in a nutshell” will comprise of Services which will extended from the client to cloud and hosted by Microsoft, (e.g. www.biztalk.net) and Models and making them a mainstream part of development.

The keynote also delved further into these two concepts.

• Model Driven Development (Models) – Microsoft aims to create new tools and a model that takes models across domains (Business Analysts, Architects, IT Professionals, and Developers).
• Internet Service Bus (Services ) – The services will be hosted on the cloud, allowing for the ability to take small enterprises and allow them to build customizable and simple connected Business Processes. The goal being to make it simple to connect people to applications they need. See the Architecture Journal – Journal 13 for a great article on this topic by Donald Ferguson, Dennis Pilarinos and John Schwchuk.

The final thing that stood out to me was the announcement of a SOA & BP Pack – which is a discounted software package including BizTalk Server 2006 R2, SharePoint Server, Visual Studio Team System and SQL Server 2005. I’m not sure how much this will cost, but it will need to be a reasonable cost if the aim to have firms of all sizes have the ability to deal with Software + Services.

Links:
www.microsoft.com/soa

People_Ready Processes with SP Workflows & Forms Services
Christian Stark, Senior Product Manager, Microsoft

This presentation was an introductory level presentation about using Windows Workflow Foundation (WF) with Microsoft Office SharePoint Server and Office 2007. There were a few demos around the way Office products have built-in support for workflows and around creating simple worklfows with Visual Studio and SharePoint Designer. It touched on the following Microsoft products:

• Visio 2007 – the modeling tool, used to design and analyze processes.
• InfoPath 2007 – used to create forms for UI.
• Outlook 2007 – used to receive notifications.
• SharePoint Designer 2007 – used to create/customize simple to intermediate workflows.
• Word 2007, PowerPoint 2007, Excel 2007 – used to interact with workflows processes.
• Visual Studio 2005 – used to create workflows, extending templates, etc.

Also mentioned was the creation of the Microsoft Business Process Alliance which provides a technological alliance of business process partner, including Ascentn, K2.net, Global 360 and Metastorm. Finally, the final point made was that Microsoft would continue to provide the Business Process platform and that it would be relying on the partners to provide the business (product) solutions required by organizations.

Flexible Governance Infrastructure
Frank Martinez, Executive Vice President of Product Strategy, SOA

Frank presented a great vision of creating a governance infrastructure for SOA in the enterprise. He touched on the main goals of SOA governance to reduce cost through reuse, increase agility to better align IT and the business and to reduce the risk, fragility and complexity of integration by improving interoperability through standards.

He covered the top 5 fallacies regarding SOA governance:

1. We have good IT and application lifecycle governance, thus we have good SOA governance
2. We don’t have an SOA program, thus we don’t need SOA governance
3. We don’t have any services, thus we don’t need SOA governance
4. We already have run-time SOA management capabilities, thus we already have SOA governance
5. We already have an SOA registry/repository, thus we already have SOA governance

He covered a couple of SOA products, viz. Workbench SOA Governance (lifecycle related) and Service Manager SOA Management and Security (operations related).

He ended with the following take away’s

• There is No “one size fits all” governance model
• Effective SOA has to address people, policy, process and technology… in that order
• Governance automating delivers economies of scale …when it supports your governance model and structures
• Early cycle governance model (that is collaborative) can act as an accelerator for enterprise SOA goals and objectives
• Closed-loop policy definition, enforcement, auditing, compliance reporting is a must have for effective governance automation

For more information regarding governance and SOA see The Business Benefits of Shared Services in an SOA by Frank.

Panel Discussion

The final session of the day was a Microsoft panel discussion. The most notable piece of news there for me was that Visual Studio 2008 will ship by the end of this year.

The evening ended with an Ask the Experts Reception and Sponsor Expo where I managed to run into a few partners from K2.Net. It was a full first day for me with more to come tomorrow.

Microsoft, Microsoft SOA Conference, BizTalk, Oslo, Software + Services, Internet Service Bus, Windows Workflow Foundation, WF

This year I will be attending the Microsoft SOA & Business Process Conference 2007 conference in Redmond again. I went to the conference last year and found it to be informative and a great place to network with some of the Microsoft MVP’s, partners, and customers. I am particularly looking forward to seeing a couple of friends (and ex-colleagues, but I won’t hold that against them) Simon Chester and Jason Shantz present on Healthcare Enterprise Integration: SOA Solutions. For those interested, their presentation will be on Wednesday, October 31st, from 2.30-3.30pm in the Hood room.

Let me know if you will be attending and we can arrange to meet!

Yesterday the Unify 2007 Tour presented by Microsoft Canada came to Vancouver. There were four sessions divided into two tracks – developers and IT Pros. The presentations that I attended (developers track) were informative and interesting. I have summarized the points from the presentations that I found interesting.

Session 1: Designing Architecture
John Bristowe & Damir Bersinic

The initial part of this session was an introduction to the main theme of the day – the different challenges facing developers and IT Pros in the development and deployment of enterprise applications. The main point being that developers are eager to use new technological advances to improve productivity and meet difficult deadlines while IT Pros are more conservative. Their concern is having more uptime, requiring testing and controlled releases on new technological products.

There was a demo on using the System Definition Model (SDM) within Visual Studio for Architects where the application dependencies were compared to the environment’s logical design and differences were brought to the front, e.g. the application being dependent on a service pack which was not present on the environment.

Session 2: Building, Deploying and Maintaining the ApplicationDesigning Architecture
John Bristowe & Ruth Morton

This session focused on the deployment and monitoring of applications.

• Documentation – developers hate writing it and IT Pros expect it, especially when an application is developed with multiple dependencies.
• System Access – developers require it in data-centric applications and IT Pros hate giving access. The rule of thumb is to code with least privileged accounts to get the required access.
• Monitoring Tools – developers are always keen to code new tools to monitor their applications, IT Pros prefer using existing tools thereby not requiring new applications being installed and supported. While I could agree with the first two points, I find this one a little hard to believe – who wants to code a new application that monitors your latest application? It sounds a little like rework to me.

Session 3: Developing Web Applications on the Microsoft Application Platform
John Bristowe

The main topic in this session was IIS 7.0. It was my first look at IIS 7.0 and I was pretty impressed. My favourite enhancement shown was the ability to view the web.config from IIS in the form of a console. There are icon for all web.config tags, e.g. Application Settings, Connection Strings, Sessions State, SMTP Email, etc. When changing the settings in IIS the underlying web.config is updated and should you alter the web.config the console updates.

IIS 7.0 has two modes for ASP.Net integration. The first mode, Classic, is the same as the current version of IIS. The second mode, Integrated, removes the calls to the ISAPI filter and IIS controls the entire process.

Another new feature of IIS 7.0 was the support for PHP 5 through Fast CGI for IIS. Fast CGI increases performance by reusing existing Common Gateway Interface (CGI) processes. PHP 5 can be supported on older versions of IIS (5.x and 6) by downloading the latest Technical Preview. For more information see Fast CGI for IIS.

Session 4: Building a Strong Data Foundation to Your Application
John Bristowe

The two main points of value from this session for me were:

• Visual Studio 2005 for Database Professionals – this is a great tool for database development in any application. The demo showed some of it’s useful features including
• Integration with Team Foundation Server
• Rename Refactoring – previewing changes prior to execution
• Schema and Data comparison tools
• Leverage of the Test Project Infrastructure
• Data generation functionality
• Language INtegrated Query (LINQ)
• The ability to query and transform relational and XML objects
• Provides SQL and XQuery like functionality in C# and VB.Net
• Supports type checking, IntelliSense and query refactoring
• It has an extensible model for languages and APIs

The LINQ functionality for SQL and XML will form part of C# 3.0 and VB 9.0 and will be released in .Net 3.5. Why version 3.5? I have no idea…

All in all the day was informative, although my overall memory will be about the multiple rants I heard regarding the pricing structure of Visual Studio Team Suite and its versions. Nice tools although most of the people I spoke to would not be able to use any of the features shown in the demos as they had Visual Studio 2005 Professional and their employers could not afford the full blown version.

Links:
Unify 2007 Tour for Developers
Unify 2007 Tour for IT Pros
IIS
The LINQ Project

Microsoft, Unify 2007 Tour, IIS 7.0, LINQ, Visual Studio 2005

Day 4 at the conference and there are some bleary eyed attendees. While the conference has been great so far, I think people are tired and ready to go home. Today’s presentations were all good in my opinion:

Aaron Skonnard, Cofounder, Pluralsight
Gruia Pitigoi-Aron, Microsoft
BizTalk Web Services: The Next Generation

Aaron pointed out that Service Orientation defines an architectural paradigm for software federation, with federation being the unification of self-governing entities.

• SOA is focused on unifying autonomous services, providing architectural governance, reducing dependencies and minimizing assumptions.
• SOA tenets help achieve loose coupling.
• Benefits of SOA are :
• Cost reductions, increased agility
• Business & software alignment
• Technology freedom
• Independent evolution
• Longer-term reuse
• Web Services provide a way to implement the SO design principles:
• XML reduces format complexities
• WS-* reduces transport complexities
• Web Services is not a silver bullet
• Not everything can use XML, SOAP, WS-*
• Can’t throw away existing investments
• Many web service stacks only support HTTP
• SOA must plan for disagreement on the following fronts
• Transport/application adapters
• Message format translators
• Message schema transformations
• Combining BizTalk Server 2006 with web services provides the complete solution:
• BTS fills various messaging integration gaps
• BTS embraces we technologies
• BTS 2006 R2 ship with WCF adapters
• New adapters for each mainstream binding
• A WCF-Custom adapter for extensibility
• WCF Adapters use cases
• Transaction message send/receive
• Using WS-* headers for routing/processing
• Using custom bindings (binding elements)
• Numerous security scenarios
• In-proc hosting of non-HTTP endpoints

Gruia then demonstrated using the WCF adapters to enable transactions and showed some security features using BizTalk Server, including single sign-on capability.

Aaron’s final point was that BTS 2006 R2 will provide full WS-* support, custom bindings, flexible hosting & communications.

Aaron Skonnard, Cofounder, Pluralsight
Gruia Pitigoi-Aron, Microsoft
BizTalk WCF Adapters: In-Depth

In this session Aaron and Gruia continued showing demos of the new WCF Adapters. There were no slides only the a dive into some real code (I found this refreshing).

The first demonstration showed a client sending a message to BTS through a web service with option to select separate XML versions for the message structure. BizTalk then routed the message to separate versions of the backend application. Version 1 used transport security (which resulted in a smaller message) and version 2 used message security. The BizTalk receive location was a WCF-NetTcp adapter using a custom pipeline to add a custom header to the message header that contains the version (it can also look for differences in actions, namespace, etc). The pipeline promotes the value (needs a property schema), the send port filters on the custom operation and then routes it to the service depending on the version number. It provided a model for versioning using web services.

The second demonstration showed how to build a custom WCF adapter. The binding tab on the adapter allows to pick different binding types (you can install other bindings or even create a custom binding element).

Don Smith, Product Manager, Patterns & Practices, Microsoft
Web Services Software Factory

Don started by speaking about how software factories help you build a specific kind of application (smart client, service, etc). He then went on to speak about the Software Factory that can be downloaded from the Patterns and Practices website. He mentioned that the software factory incorporates a variety of guidance content types and form factors:

• Architecture & design guidance, patterns, and how-to’s (readable/printable)
• Reference implementations (executable) – sample applications
• Application blocks (reusable)
• Guidance packages (actionable)

Guidance packages provided are:

• Open – documentation describes manual and automated tasks, the result of the automation is exposed
• Configurable – key configuration settings can be captured within Visual Studios, automated guidance is defined using XML
• Extensible – guidance can be tailored to meet specific requirements
• Verifiable – generated code can be verified for compliance within a standard

Don demonstrated the Software Factory by setting up a wine rating service for a winery.

The benefits of using the factory are:

• Increased quality
• Better consistency of projects/services
• Better performance of developer

According to Don the next version of the Software Factory with WCF support is set to be released in December 2006. It will include:

• Versioning
• ASMX/WCF Interop
• WF
• Message validation
• Security

Links:
Blog: Don Smith
Patterns & Practices
Web Service Software Factory
Service Factory Community Workspace
The LINQ Project

Microsoft, Microsoft SOA Conference, BizTalk, WCF, Windows Communication Foundation, Web Services Software Factory