Agile Approach to Hybrid and Multi-Cloud Integration – Part 3

Continuing from my last blog in this series, I am going to focus on identifying and mapping patterns for each integration flow requirements, identify a logical integration platform, and lastly then map and select cloud services and/or on-premise software platforms.

As highlighted in previous blogs, I started by selecting an example use case. The use case in question is a SaaS to SaaS use case. The need for a SaaS to SaaS can be borne out of an enterprise’s shadow IT or just simple business pressure for agility without any future thought of any interaction and integration needs. There is also a very good chance that the SaaS applications in question will be from different cloud vendors.

Up to this point in the workshop an enterprise has decided what Cloud principles and integration principles are important to them. (See a previous blog for more details).  I want to go cover some of the characteristics of one of the principles I highlighted in that earlier blog as this assists an enterprise selecting an integration approach. Specifically the principle of:

Pre-Built Direct over Platform over Custom Integration Flows.

  • Pre-Built Direct
    • Primarily Single Vendor SaaS Focused
    • Vendor Supplied and Maintained
    • Commonly Utilized with Mature Information Flows
    • Configuration Based
    • Adapter Not Required
    • Reduced Maintenance
    • Quicker Time to Deployment
    • Limited Integration Flexibility
  • Platform Based
    • Increase Flexibility
    • Commonly Makes Use of Adapters
    • Can Assist with Exposing Existing System Functionality as Services
    • Be-aware of Business Logic Bleed
    • Be-aware of Business Process vs Technical Orchestration
  • Custom Direct
    • Enterprise Maintained
    • Increasing Complexity
    • Technical Debt Builds Up Over Time
    • Fragmented Intelligence and Monitoring
    • Quicker Initial Time to Market
    • Adapter Not Commonly Utilized

In addition, the enterprise at this stage fully understands the pros/cons of each platform deployment option and has selected a preferred option.

Platform Deployment Options

Just because an enterprise has selected a preferred deployment option does not mean they are can not utilize any of the other deployment options. If they can justify the use of one or more of the other deployment options then they should see the deployment options as not mutually exclusive.

  • An enterprise may select a different approach for different integrations
  • Various deployment options may be seen as a transitional architecture state
  • Use of two or more options can be combined
  • Different approaches might be utilized for different platforms

So far I have not talked about the type of platforms used for application integration. I liked to view application integration platforms as logical platforms with a collection of integration capabilities. I do this as vendor and/or open source integration platforms tend to have integration capabilities that span or overlap with other integration platforms. We can address this situation when we map a physical integration software/service with the logical platforms.

Example Logical Platforms

Below are the patterns available for SaaS to SaaS integration. Very quickly we can discount a number of these patterns depending on which platform deployment option was preferred, selection of principles agreed to adhere to, and what type of integration is required.

In the future I intend to publish all of these patterns. In the mean time I will cover just a couple of the most common in this blog. The most common pattern I have seen when enterprises want to integrate SaaS cloud services from different vendors is unfortunately on-premises.

Application Integration – On-Premises Platform Deployment

Here is the conceptual view:

Integration of a Cloud-based application with an on-premises or Cloud-based application using integration capabilities only on premises.
ProblemHow do I integrate Cloud-based and on-premises applications with minimal Cloud footprint?
ContextThis pattern is applicable when there is a desire to integrate a Cloud-based application and an on-premises application or other Cloud-based application. There may be multiple Cloud-based applications and multiple on-premises applications.
Forces
  • Need to handle on premises to on premises as well as on premises to Cloud.
  • Limited need for Cloud to Cloud application integration.
  • Want to extend an established on-premises Application Integration Platform to Cloud-based applications.
  • Quickly attach to existing on-premises integration flows.
SolutionThis on-premises Integration pattern employs an Application Integration Platform only on premises. This is likely the current situation in most IT environments. On-premises applications are integrated using the on-premises Application Integration Platform and there is a desire to add Cloud-based applications to this existing integration approach.
As SaaS to SaaS integration needs arise, this pattern could likely evolve into the Hybrid Integration pattern.
ConsequencesPros:
Leverages existing investment in on-premises Application Integration Platform.
Easiest way to add Cloud-based applications to exiting integrations.
Best support for on-premises to on-premises integrations.

Cons:
Sub-optimal approach for SaaS to SaaS integrations.
Does not take advantage of what public Cloud has to offer with respect to integration.
Related PatternsApp Integration – Hybrid Integration (INAI04)
App Integration – Cloud Integration (INAI01)
ExamplesSalesForce to Oracle Netsuite ERP

This is just the conceptual view. For each conceptual view there are a number of interaction views that highlights details of the possible interaction aspects of the integration flow depending on:

  • Synchronous Communication
  • Asynchronous Request-Response
  • Long Running Orchestration

After an on-premise platform deployment, the next natural step that I have seen with many enterprises is to transition to a hybrid platform deployment. Lets quickly go over this pattern.

Application Integration – Hybrid Platform Deployment

Here is the conceptual view:

Integration of a Cloud-based application with an on-premises or Cloud-based application using integration capabilities both in the Cloud and on premises.
ProblemHow do I integrate Cloud-based and on-premises applications with maximum flexibility?
ContextThis pattern is applicable when there is a desire to integrate a Cloud-based application with an on-premises or Cloud-based application. There may be multiple Cloud-based applications and multiple on-premises applications.
Forces
  • Need a strategic and comprehensive platform
  • Want to balance quick delivery projects with a systematic approach to integration
  • Need to handle combinations of Cloud to Cloud, Cloud to on premises, and On-premise to on premises
  • Want to extend an established on-premises Application Integration Platform
  • Quickly attach to existing on-premises integration flows while leveraging Cloud-based integration
SolutionThe Hybrid Integration pattern employs an Application Integration Platform both in the Cloud and on premises. While this increases the cost and complexity of the overall solution it also provides the greatest flexibility. On-premises applications can be integrated using the on-premises Application Integration Platform while SaaS to SaaS integration can be done using the Cloud-based Integration Platform. Both Application Integration Platforms come into play when the integration is between Cloud-based applications and on-premises applications.

For performance reasons it may be desirable to have communications use only the on-premises Application Integration Platform or only the Cloud-based Application Integration Platform. When this is the case, it is an application of one of the other Cloud to on-premises application patterns
ConsequencesPros:
Provides the greatest flexibility.
Encapsulates on-premises integrations and Cloud-based integrations. Cloud to on-premises communication only required when integrating on premises to Cloud.
Cons:
Increased complexity compared to having only one Integration Platform.
Potential performance impact due to integration going through two Integration Platforms.
May require different skill sets for on-premises and Cloud-based Integration Platforms.
Related PatternsApp Integration – Cloud Integration (INAI01)
App Integration – On-Premises Integration (INAI05)

Ancillary Patterns

I know that reading patterns can be a bit of a dry read. But I want to cover at a high level some ancillary patterns. These are patterns which can be applied to one or more of the patterns that you may have already identified. These are more important in a hybrid and multi-cloud environment.

  • Interface Formalization – This is applicable when there is a desire to create and expose a formal API for internally and/or externally use. The consumption of this API may be on premise or Cloud based.
  • Cloud Query Result Caching – This is applicable when the response requires considerable time or compute resources to create, for example a report. It is also applicable if the source for the response has limited throughput or is difficult to scale to support new initiatives e.g. mobile. This pattern is also applicable if the response is a large data set. The source could be on-premises or in another Cloud.
  • Cloud Orchestration – This is applicable when there is a desire to create an orchestration incorporating Cloud-based applications and on-premises applications using Cloud-based infrastructure.
  • On-Premises Orchestration – This pattern is applicable when there is a desire to create an orchestration incorporating Cloud-based applications and on-premises applications using only on-premises infrastructure.

Identify Logical Platforms

After all the integration flows have been mapped to a number of patterns, the next step is now possible which is to map the logical integration platforms on the whiteboard.

  • Using sticky notes place logical platform in appropriate deployment location based on the identified pattern
  • Re-draw integration flows lines if necessary
Identify Logical Platforms

Map On-Premise Platform and/or Cloud Service

The next step is to decide on one or more cloud services and/or integration products. This should be done considering the preferred deployment, integration capabilities that need to be supported, the TIME investment decisions, and the selected patterns.

Again, using the sticky notes, I get the enterprise to place the name of the cloud services and/or products that will address each platform required.

I think that will do for this week. The next blog will be the last in this series and I will be tackling prioritization of the integrations,, while considering the benefits, costs, and risks.

Good Luck, Now Go Architect Have a Happy Thanksgiving