One of the most exciting “new” approaches for software development and deployment is service oriented architecture (SOA). SOA is a component-based, standards-driven architecture that primarily utilizes the Web as a vehicle for delivering services to users. A service is work that’s done on request by software or hardware and that achieves a particular result while being reusable by the same, or other, service requestors. SOA turns application functionality into the software equivalent of Lego® bricks. Combining a common software component (such as identity verification), with a well-defined interface, results in a “service” that other applications use as needed. With a “toy chest” (library) of such mix-and-match parts, you can build composite applications to meet almost any business need, the same way Lego’s can morph from castle to crocodile to catapult when the same components are rearranged.
Services, unlike traditional applications, can be on widely distributed servers, ready to be executed only when needed. How services are built, on which operating systems they run or within which applications they reside aren’t important as long as they support standard connection interfaces.
Employed properly, SOA can help achieve the perennial goals of IT:
- Rapid development and deployment of quality systems that meet enterprise needs
- Quick, flexible and appropriate response to change
- Re-use and reduced redundancy
Achieving these goals will result in significant cost savings while providing new capabilities.
Even though implementing software in a service oriented architecture (SOA) is very new, building systems from reusable and sharable components is not new. Software developers began building computer applications from reusable components years ago. In the early 1960s, when computer programs were developed and executed using punched paper cards, it made sense to reuse modules and subroutines in as many applications as possible. The components had already been tested, and nobody wanted to redesign and re-keypunch routines that worked. In large organizations, there were filing cabinets full of programs and subroutines (components) that were reused and shared by all developers.
Unfortunately, with the advent of digital storage for source code (copy libraries, etc.) and the ability to modify the code using automated tools, developers stopped sharing their components and often didn’t even reuse them. Among the most common reasons for not sharing or reusing components is the ease of “tweaking” component code. This has resulted in different versions of similar code being implemented in many different applications. Another common reason for not sharing service components is programmer ego. “My code is better than your code” is a typical attitude among programmers. This attitude, coupled with the false notion that job security is somehow enhanced by not sharing, creates an environment that does not allow for cooperation.
Recently, object-oriented (OO) analysis, design, and development techniques and modeling notations such as the Unified Modeling Language (UML) have become increasingly popular as a means to develop components, yet these techniques and the new, graphical object development tools have only made it easier to proliferate redundant and duplicate components.
The supposed benefits of an OO approach to component development – reusability, modularity, avoidance of duplication, reliability, and flexibility for expansion – are exactly what is necessary for effective service components. Yet very few organizations have been successful in managing object development to realize these benefits. This is primarily because objects are easy to develop, but extremely difficult to manage. Most organizations that have adopted OO techniques and tools for component development have not established effective component management practices. In fact, the typical software development environment can be characterized as barely controlled chaos.
While it may be possible to develop useable service components in this type of environment, it is not possible to ensure the consistent development of quality service components necessary for effective services oriented architecture (SOA). True quality service component development, reuse and sharing require proper management of service component development. This means establishing and enforcing standards. It also means using standard techniques and best practices such as those described in this paper. Most in the IT industry agree that consistently developing quality software requires an engineering-based methodology. This is even truer when the software being developed are services and service components which must be both reusable and shareable in order to be effective.
Enterprise Service Engineering
Enterprise Service Engineering (ESE) is a unified method that consistently allows quality services to be developed from quality components. Quality components and services are more than just error-free code. They also have the following characteristics:
- Support Enterprise Strategic Objectives
- Meet Business Area Requirements
- Are Reliable, Flexible, and Scalable
- Share Corporate Data and Standards
- Built from Reusable Components
- Delivered On Time and On Budget
Enterprise Service Engineering (ESE) uses Enterprise Architecture (Enterprise Metadata) as the framework for developing and managing quality services from quality components.
ESE embodies the concept of Model Driven Architecture (MDA). Linking service components with business requirements ensures that components are developed only to meet specific business needs.
Architecture models not only help bridge the gap between business requirements and services, but they also allow improved communication between business experts (information consumers) and system designers (information system providers). The artifacts (models) that comprise enterprise architecture reflect the business goals and critical success factors of the enterprise. The “business models” are technology independent, and can be revised, amended, and refined independently from the implementation environment of physical objects. Logical data and process models provide a firm base from which service component designs are developed.
“Logical model” components represent business objects composed of related clusters of information. These logical data objects are combined as needed for specific functions or applications. The data objects, including metadata, are then combined with logical control flow models that represent object behavior. The logical definition of an entity, accompanied by its available processes (behaviors), forms an encapsulated unit that describes the functionality (logical component model), or OO object class definitions, and are used as the basis for multiple physical component designs. The designs can then be implemented on any platform using any application development tools. This means that logical models that accurately reflect business rules and requirements can be used to effectively manage component development — quality requirements become quality models; quality models become quality designs; quality designs become quality components
that make up quality information systems (see figure 1).
Figure 1: Quality Software [Component] Engineering
Quality is designed in – not tested in!
The “business architecture” of an enterprise is made up of its strategic business rules (goals, objectives, strategies, measures, etc.), conceptual information model and high-level function model. Its “information architecture” is represented as logical data models and logical component models. The “service component architecture” is the combination of processes, database and component designs. These are implemented on the enterprises “technology architecture” as databases and code. Together, all the architectures, linked and managed as a continuum of architectures in a single repository, make up the “Enterprise Architecture.” Enterprise architecture also establishes guidelines, standards, and operational services that define the enterprise’s software engineering environment. All of these artifacts, appropriately integrated, provide a knowledgebase for enterprise management at all levels.
Logical model components include the links with the documented business requirements for the components. These links – requirement to logical model element to physical design object – provide the best structure for component management and make change management both easier and faster.
Using the ESE approach for forward engineering quality service components, implemented code and databases exactly reflect physical designs, which are, in turn, derived from and linked to the logical data and component models, which are detailed representations of the information and functions models, which accurately depict the strategic business requirements of the enterprise. This provides complete traceability, allows technology independence.
The characteristics of Enterprise Service Engineering are:
- Staff skills and productivity improve so that better software is developed faster.
- Standards are established and enforced.
- Metrics are identified and utilized.
- Appropriate tool sets are acquired and used.
- Service component development quality becomes consistent and repeatable.
- Changing requirements are met rapidly and well.
- Maintenance costs (typically, 70 to 80 percent of software development budgets) are reduced significantly, often as much as 75%.
Enterprise Service Engineering Life Cycle
The phases of the ESE life cycle are illustrated in Figure 2.
Although the ESE phases are depicted as sequential, the ESE life cycle is actually a controlled iteration of techniques, methods, and practices – many of which can and should be executed concurrently. ESE is neither a “waterfall” nor a “spiral.” The ESE life cycle involves a multi-phased approach that coordinates strategic, operational, and organizational demands.
Figure 2: Enterprise Service Engineering Phases
The first step of the ESE method is defining business requirements in real-world terms: e.g., goals, strategies, objectives, tasks, critical success factors, performance metrics, etc.
As business requirements are being defined, the enterprise architecture is developed using models that are both unambiguous to developers and understandable to business experts. The enterprise architecture is made up of business definitions and logical data and activity models that represent the information and business rules necessary to support the defined business requirements. This “intellectual capital” represents great value to the enterprise and ensures that all enterprise systems actually meet enterprise needs.
In the enterprise architecture, business rules are captured in business language. Business rules are also captured as metadata in logical component models to ensure that components will be easy to manage and change. ESE extends the basic theoretical relational modeling approach to facilitate capturing business rules. Uniquely, hierarchical and structured instances of an entity are documented within the entity itself. Metadata is actually “hard coded” in the entities as static values; e.g., the entity CUSTOMER TYPE could contain the following values which represent business rules about the specific types of customers for this enterprise:
An entity containing metadata instances is used to create a table design with the same instances, which is then used to generate the Data Definition Language (DDL) or XML (eXtensible Markup Language) that will create a database table that actually contains the metadata. Not only does this allow databases that are guaranteed to accurately reflect business requirements and rules, but service components can be designed and developed that hardly ever change. All decision elements, domain instances, and even called component names can be stored in tables, not code so only the content of the “rule” and metadata tables change – never the code.
ESE provides an environment in quality services built from quality components can be successfully developed and managed. This means that enterprises that already have a system development life cycle (SDLC) that allows them to consistently produce quality software can continue to use the same tools and processes. The only potential changes will be acquiring new language and infrastructure skills to take advantage of new web-focused service technologies. Enterprises that do not have software engineering processes that consistently produce quality applications will need to improve their processes. In a services environment, because of the need for rapid development and change coupled with the need for “plug and play” components, the tolerance for suboptimal software is very small.
Like any other large information systems project, implementing SOA and building quality services can get bogged down if the scope is too broad, and the number of people involved is too large. A clear purpose and scope is necessary to manage the application of service development resources, as well as the expectations of potential service users. The simplest way to limit scope is to build an enterprise SOA one business capability at a time. Each iteration supports a single organizational element, enterprise function or business master data object (e.g., customer, product, account, etc.), and the scope of development is limited by the defined requirements. At the same time, the relationship of the limited scope implementation to the enterprise models and goals is maintained, thus service component engineers are able to “think globally and work locally.”
For initial service implementation, which usually provides proof-of-concept or prototype, the scope must be sufficient to provide real, immediate, and high profile benefits. After the first set of services is developed and implemented, additional services can be developed and integrated over time as enterprise needs dictate and as resources are available. The iterative nature of ESE allows for rapid prototyping and agile implementation of services to meet business needs quickly. It also allows rapid response to changing needs.
Re-engineering Business Processes
The primary underpinning of any IT initiative, particularly ESE, should be business process reengineering. It doesn’t make sense to develop enterprise services in support of business processes that are sub-optimal or worse, unnecessary. Good software in support of bad processes will never be considered quality systems. Hammer and Champy (“Reengineering the Corporation” – 1993), say it another way, “automating a mess yields an automated mess.”
Business process reengineering (BPR) involves a change in the way an organization conducts its business. BPR is the redesign of the organization, culture, and business processes using technology as an enabler to achieve quantum improvements in cost, time, service, and quality. Information technology is not the driver of BPR. Rather, it is the organization’s desire to improve its processes and the use of technology to enable increased productivity and quality improvements.
BPR is not just about continuous, incremental and evolutionary productivity-enhancements. It also utilizes an approach which suggests scrapping a dysfunctional process and starting from scratch to obtain larger benefits.
The US Department of Defense has determined that organizations successful in BPR have a number of common traits:
- They are strongly supported by the CEO
- They break reengineering into small or medium-sized elements
- Most have a willingness to tolerate change and to withstand the uncertainties that change can generate
- Many have systems, processes, or strategies that are worth hiding from competitors.
In a publication for the National Academy of Public Administration, Dr. Sharon L. Caudle identified six critical success factors that ensure government reengineering initiatives achieve the desired results:
Understand reengineering.
- Understand business process fundamentals.
- Know what reengineering is.
- Differentiate and integrate process improvement approaches.
Build a business and political case.
- Have necessary and sufficient business (mission delivery) reasons for reengineering.
- Have the organizational commitment and capacity to initiate and sustain reengineering.
- Secure and sustain political support for reengineering projects.
Adopt a process management approach.
- Understand the organizational mandate and set mission-strategic directions and goals cascading to process-specific goals and decision-making across and down the organization.
- Define, model, and prioritize business processes important for mission performance.
- Practice hands-on senior management ownership of process improvement through personal involvement, responsibility, and decision-making.
- Adjust organizational structure to better support process management initiatives.
- Create an assessment program to evaluate process management.
Measure and track performance continuously.
- Create organizational understanding of the value of measurement and how it will be used.
- Tie performance management to customer and stakeholder current and future expectations.
Practice change management and provide central support.
- Develop human resource management strategies to support reengineering.
- Build information resources management strategies and a technology framework to support process change.
- Create a central support group to assist and integrate reengineering efforts and other improvement efforts across the organization.
- Create an overarching and project-specific internal and external communication and education program.
Manage reengineering projects for results.
- Have clear criteria to select what should be reengineered.
- Place the project at the right level with a defined reengineering team purpose and goals.
- Use a well-trained, diversified, expert team to ensure optimum project performance.
- Follow a structured, disciplined approach for reengineering.
Another truly critical success factor is that enterprise architecture must be used to govern and scope BPR. Just as blueprints are used to build and improve a building, the models and diagrams that depict enterprise architecture serve as the framework and blueprint for BPR. EA-based BPR allows enterprises to shape processes as efficient and effective ways to meet strategic goals and objectives. EA-based BPR concentrates on what the enterprise actually needs so that efforts result in both long-term and short-term accomplishments.
The relationship between EA and BPR is symbiotic. EA drives process change and as the enterprise changes, the enterprise architecture is updated to reflect the changes. This ensures that BPR is always properly focused. EA and BPR, along with SOA and strategic and performance planning are all cross-organization, cross-program, cross-project, and cross-function. As illustrated in the graphic that follows, they are overlapping and tightly coupled. It is important to understand that while they can exist alone, they can only thrive together.
Re-engineering Legacy Systems and Components
In today’s software development environments, software engineers almost never engineer new systems without having to either reverse engineer or integrate existing (legacy) applications and components. Often the only reason a new system is developed is to replace an aging system that is failing to meet current enterprise needs.
It is extremely difficult to reverse engineer and integrate legacy systems in service oriented environment using traditional techniques. A combination of forward and reverse engineering using ESE overcomes the problems inherent in traditional; labor-intensive approaches. Combining the reverse-engineered physical data and process models of the legacy systems with forward-engineered logical models representing analysis of current business requirements forms a solid basis for creating new physical, component designs that can be implemented to replace aging legacy applications. This is also the basis for effective management of existing components.
Enterprise Service Engineering Tool Suite
Automated tools are the enabling technology that allows data sharing, service component reuse, and rapid change management to be realized. Service component development tools are available from multiple sources, but these tools, by themselves, are only the means to create components. What’s also needed is the means for managing the components. An ESE tool suite also includes an “industrial strength” repository that allows enterprise needs and measures to be linked directly to a strategic information model, enterprise data dictionary, legacy database models, data integration and transformation process models, and data warehouse and data mart database design models. This linkage is the key consistently successful development and implementation of high-quality components. It also allows quick reaction to changes in environment, policy, or customer requirements. The ultimate purpose of the ESE repository is managing all of the enterprise metadata reflected in the multi-dimensional architecture models. This metadata is the glue that holds everything together. Metadata as enabler – an engineering discipline is only as effective and the accuracy of the information (metadata) at the interfaces — metadata is the principal communications medium between individuals and technologies involved in every step of the ESE life cycle.
An effective ESE tool suite provides the capability to document an enterprise’s multi-level, strategic plans, and link them to appropriate performance measures. More importantly, it also allows business requirements to be translated into service component models that become quality services and service components. These models are also the building blocks of database, data mart, data warehouse, decision support and executive information system designs. These designs can then be developed and implemented by integrating quality service components, ensuring that the enterprise’s SOA meets its business needs.
Summary
SOA has the potential to allow agile IT support for current and changing business needs. It promises improvements in quality, productivity and flexibility. However SOA isn’t a cure-all. Nor will it (or should it) replace every tightly integrated legacy application in your enterprise.
In order to achieve the benefits of service oriented architecture (rapid development, reuse, easier maintenance, and extensibility), component development and management must be part of a rigorous and repeatable methodology that is architecture-based and has an enterprise-wide scope. Enterprise Service Engineering (ESE) is a practical and agile approach that has all of these characteristics.
SOA provides the infrastructure, quality services and service components provide the means, and ESE provides the method.
ESE is also model-driven. The logical models resulting from an ESE approach are technology independent and reflect business goals and requirements as well as “intellectual capital” of an enterprise. They can be subsequently revised, amended, and refined independently of the implementation environment of any physical systems.
The ESE models provide a firm base (logical component model) from which superior component designs and truly useable (and reusable) components can be developed.
With ESE, the extra time spent getting requirements right and designing logical components pays significant dividends in reducing the time spent designing and building service components. It also ensures that the enterprise can consistently develop quality information systems.
ESE is a unified methodology that incorporates the best practices of business and information management. This methodology involves all the activities that organizations perform to improve productivity, gain and maintain competitive advantage, optimize resources, deliver quality products and services, and meet internal and external customer expectations and demand.
ESE provides the means to achieve rapid development of quality service components, little or no maintenance, and delighted customers.
The integration of ESE with other Enterprise Engineering best practices is described in “Getting Your Acts Together – a Consolidated, Strategic Approach to Federal Enterprise Transformation” – also by the author.
Bibliography – Sources and further reading:
- Object-Oriented Modeling and Design, Rumbaugh, Blaha, Premerlani, Eddy & Lorensen, Prentice Hall, 1991
- Object-Oriented Analysis, Coad & Yourdon, Yourdon Press, 1991
- Object-Oriented Design, Coad & Yourdon, Yourdon Press, 1991
- Object-Oriented Analysis and Design, James Martin & James Odell, Prentice Hall, 1992
- Distributed Object-Oriented Data-Systems Design, Andleigh & Gretzinger, Prentice Hall, 1992
- Information Engineering: Strategic Systems Development, Clive Finkelstein, Addison-Wesley, 1992
- Object-Oriented Software Engineering, Jacobson, Christerson, Jonsson & Overgaard, Addison-Wesley, 1994
- Object-Oriented Information Engineering, Stephen L. Montgomery, Ap Professional, 1994
- Object-Oriented Analysis and Design with Applications, Grady Booch, Addison-Wesley, 1994
- Object-Oriented Software Engineering : A Use Case Driven Approach, Ivar Jacobson, Addison-Wesley, 1994
- Data Reverse Engineering: Slaying the Legacy Dragon, Peter H. Aiken, McGraw-Hill, 1996
- Business Process Reengineering Assessment Guide, U.S. Government Accountability Office, May 1997
- Unified Modeling Language Reference Manual, Rumbaugh, Jacobson, Booch, Addison-Wesley, 1998
- Unified Modeling Language Users Guide, Booch, Addison-Wesley, 1998
- Building Corporate Portals with XML, Clive Finkelstein & Peter Aiken, McGraw-Hill, 2000
- Enterprise Architecture for Integration: Rapid Delivery Methods and Technologies, Clive Finkelstein, Artech House, 2006
Author’s Note: The following papers are available from the author:
Enterprise Architecture Engineering
Enterprise Architecture Engineering Critical Success Factors
High-Performance Enterprise Architecture Engineering – Implementing the Zachman Framework for Enterprise Architecture
Enterprise Change Management – An Architected Approach
Getting Your Acts Together – An Architected Solution for Government Transformation
A Strategic Approach to Data Warehouse Engineering
Data Warehouse Architecture – A Blueprint for Success
Critical Success Factors for Data Warehouse Engineering
How to Succeed in the 21st Century – Critical Information Management Success Factors
XML Metadata Management – Controlling XML Chaos
Business Rules are Meta-Data
Enterprise Application Modernization – Solving IT’s Biggest Problem
Strategic Enterprise Application Integration
e-Engineering – A Unified Method for Enterprise Transformation
Enterprise Portal Engineering
Quality Software Engineering
Software Engineering Process Improvement
Enterprise Service Engineering – An Architected Approach for SOA Component Development, Deployment, and Management
