SOA should not be seen as purely as an approach to integrate existing systems together. It also has applicability in building new discrete business functions when building new custom “composite” applications. More on this later. But I thought first I would actually address Service Oriented Integration. (SOI).
The primary goal of service-oriented integration is to better leverage existing systems within the IT environment by applying service-oriented principles.
Service-oriented integration is the mechanism to expose existing sources of data, functionality, and process so that those sources can be readily consumed by a composite application.
Services
Creating a SOA Service from existing assets generally requires a good deal more than just adding a standards based interface, i.e. simply service enabling existing assets is insufficient.
The SOA Service needs to expose process, functionality, and data that is usable in a broader context than the source of the capability was designed to meet. Therefore, creating a SOA Service usually entails some amount of aggregation, transformation, or expansion of existing capabilities provided by the source systems. This requires a SOA Services layer between the existing assets and the consumers.
The very essence of service-oriented integration is the building of a catalog of SOA Services that expose, in a business-enabling way, the data, functionality, and process contained in existing systems. In order to reap the benefits of service-oriented integration, the SOA Services must decouple the consumers from the source systems without creating tight coupling between the consumer and the SOA Service itself. Additionally, the SOA Services must expose functionality that is aligned with the business needs of the service consumers, not just provide an API or data-level pass through to the source systems.
There are three types of capabilities that existing source systems can provide to composite application construction: process, functionality, and data. Although the approach for creating SOA Services based on existing process, functionality, or data is similar, these different capabilities also provide unique challenges when creating SOA Services.
Process Integration
Frequently the existing source systems include business process or workflow built into the system. From the perspective of the source system, this is a complete business process. However, from the broader context of the organization, the built-in business process is actually only a portion of a larger business process that spans multiple back-end systems. Therefore, when creating the catalog of SOA Services these built-in business processes should be viewed as sub-processes that are part of the larger enterprise business process. This relationship is illustrated
Therefore, when creating the catalog of SOA Services these built-in business processes should be viewed as sub-processes that are part of the larger enterprise business process.
Functional Integration
After our quick look at process integration, lets now look at functional integration.
The functional capabilities contained in existing source systems is usually too fine grained and domain specific to be useful, as is, for creating composite applications.
The application hosting the functionality was usually constructed to provide users fine-grained control. Frequently the users of the system attend special training to become proficient with the application. Additionally, the application invariably contains far more functionality than is necessary in a broader context. Most of the functionality is specific to the domain (e.g. HR, Finance, Shipping) and has little relevance in an “enterprise” context.
Therefore, exposing existing functionality as SOA Services usually requires abstracting the functionality to a higher level so that it has meaning (and is more useful) in a broader context. Frequently this entails combining several fine-grained operations into a more course-grained operation.
Sometimes the fine-grained operations are associated with a built-in workflow exposed through the user interface, i.e. a set of prescribed operations that the end user performs in sequence.
In this instance, when creating a SOA Service it is usually desirable to have a single operation on the SOA Service that encapsulates and hides the built-in workflow by performing the individual steps automatically.
The orchestration may even span two or more existing applications. This orchestration of fine-grained operations to create a coarser-grained
operation is a common technique to abstract functionality.
The input and output data specifications for operations on existing applications also tend to be application specific and not particularly amenable to a broader audience. Therefore, the SOA Service frequently exposes an interface that is significantly different than the underlying application operations expose. This data format issue is covered more in the next section.
Data Integration
Each existing application contains its own data model and data formats. This proliferation of data models and data formats is exacerbated by the fact that a single enterprise entity (e.g. customer, product, order) frequently has data elements stored in multiple existing applications. To be successful at exposing existing data via SOA Services, the integration approach must manage this complexity.
Master Data Management (MDM) is one approach to manage this complexity. However, MDM is complementary to service-oriented integration, and if MDM has been deployed to manage data complexity, then it can be easily used by the SOA Services. MDM is beneficial to, but not required for, successful service-oriented integration.
The primary goal of a SOA Service that exposes an enterprise data entity is to make it easier to work with the entity and hide the complexity of how that entity is stored in existing source systems.
To achieve this goal, the SOA Service needs to incorporate the following capabilities:
- Consumer Representation – The SOA Service should present an interface to the entity than is appropriate for the consumer of the data. A single SOA Service might provide multiple representations of a single entity, each representation appropriate for a different type of consumer.
- Aggregation – The SOA Service may need to pull data elements from multiple source systems to construct a representation for a particular consumer.
- Synchronization – An update to an entity may require updating data elements in more than one source system. The SOA Service needs to ensure that all updates are done properly so the entity maintains consistency
Providing consumer representations and reading from and writing to multiple source systems leads to the issue of data format transformations. For a very small number of source systems, point-to-point transformations can be used by the SOA Services. However, this approach becomes untenable as the number of source systems increases. Thus, a better approach is to create a normalized format for the data entities and then provide transformations to and from the normalized format for each source system
It should be noted that a single canonical data model for the entire enterprise is not required to successfully employ normalized data formats. Rather a federated approach to normalization can be used. For example, in a large enterprise each functional domain could create a normalized format. Transformations between the domain formats would then be created for SOA Services that span domains
Another benefit of creating SOA Services that expose enterprise data entities is that the storage of enterprise data can usually be significantly simplified once the SOA Services are deployed. Providing a single source of enterprise data simplifies access and eliminates the need for copying of data into operational data stores. Since access to the enterprise data is via the SOA Services and consumers are insulated from changes to the underlying source systems, a storage rationalization effort can be undertaken to reduce and simplify the number of source systems.
That is all for now. In my next blog I think I will round off this topic by explaining how SOI differs from more traditional integration techniques.
Good Luck, Now Go Architect…
An Enterprise Architect's Musings 