Today’s Metadata Management conundrum exceeds the capabilities of any one tool, subject area, business domain, capability, competency, or person. Instead, it requires a holistic, all-encompassing consideration of Metadata Management, which mandates inclusion of competing perspectives and technologies to collaborate harmoniously on growing advanced analytics.
Realistically, it is more a People Problem than anything else. The people leading Metadata Management initiatives must possess extraordinary knowledge and skills as they apply to Data Literacy, Governance, Provenance, and Organization. This elite talent requires empowerment, resources, and training.
Also, it is a Business Problem – think of Information Technology as a Business Unit sorely in need of realignment with other business units. This alignment happens through active communication and collaboration, augmented with concise key performance indicators and metrics.
Furthermore, it is a Process Problem, where agility, gamification, and orchestration play essential roles, especially as they concern data management. Rethinking some, if not all, of the value stream processes and domains may be required. Certainly rethinking the data lifecycle is in order.
It is NOT solely a Technology Problem. Obviously the bane of any Metadata Management program lies in myopic thinking and the notion that it is just a technology issue. Some Metadata Management tools provide a fairly complete set of services, while others work only with a specific vendor’s tooling. Stretching further requires aiming for Data and Metadata as Services.
It is NOT only a Data Problem. Metadata permeates the corporate DNA. Proper Metadata Management spans all functions and processes across the corporate digital nervous system. The proper Metadata structures, vis-à-vis taxonomies and classification schemes, allow the disparate competing business perspectives to coexist.
Scope
The recommended systematic approach for establishing the legitimacy of a Metadata Management program requires a succinct scope so as to establish immediate value (e.g., case for change) that directly addresses pressing business drivers, taxonomies and classification schemes, and supporting technologies.
- Collect Metadata Management requirements from key stakeholders through interviews and validation sessions
- Review current documentation
- Assess gaps with respect to current-state and desired-state
- Evolve a governance model
- Organization with roles and responsibilities for key positions
- Define the foundational components for Metadata Management governance – framework, policy, processes, identification criteria for critical data assets, standard operating procedures
- Investigate primary Use Cases
- Corporate lexicon
- Organizational perspectives
- Codification utilization
- Classification schemes
- Regulatory compliance
- Provenance
- Identify the existing Metadata Management tools available and, as part of a Proof of Concept, conduct tool fitment analysis and recommend additional features, packages, and applications that can be leveraged advantageously
- Interleave Metadata instantiation and governance processes into requisite placements within the Data Lifecycle, as well as Software Development Lifecycle and Integrated Data Repository
- Establish best practices for Metadata Management
- Assess the Metadata Management training needs and develop training recommendations for identified governors, stewards, and data owners
- Suggest governance augmentation opportunities
Approach
Employ an Agile – iterative and incremental – approach to initiating the Metadata Management program. Be sure to build and refine foundational components, especially as they pertain to organization, people, processes, and enabling technologies. Build Metadata Governance Services to help measure the program’s progress using specific KPIs and metrics. Assess periodically against maturity model measures.
Clearly communicate goals, successes, and failures to ensure management buy-in. Hold Lessons Learned Sessions to ascertain knowledge gained. Review strategic alignment and goals to gain and keep management commitment. Work with operational constituents to enable appropriate cataloging and use of metadata throughout the organization. Hold project-level scrums to align goals for metadata capture and deployment. Share knowledge gain, lessons learned, and insights frequently.
Keep in mind some key rollout strategy tips:
- Performance – Remember – Culture!
- Always have timelines
- Crawl, Walk and Run Methodology
- “Build while Performing” and “Refine while Performing”
- Evolve vision to stay competitive
- Don’t try to boil the ocean
Continually improve Metadata Management services and performance management by measuring, interpreting, executing, reporting, and communicating all results. Most of all, optimize the Metadata Management processes.
Collaboration Model
Agility
Employ a business-oriented, best practice-based Metadata Management plan from which to initiate robust capabilities and demonstrate the inherent value associated with the program.
- Prepare
- Assemble Team
- Deploy Governance
- Assign Roles and Responsibilities
- Establish Milestones and Deliverables
- Elicit business participation
- Determine KPIs and Metrics
- Discover
- Select Use Case(s) (Confirm, delineate, tag)
- Design
- Test hypotheses and Use Cases in well-outfitted Sandbox
- Construct Metadata Framework and Architecture
- Select appropriate tool Complement
- Elicit Vendor participation
- Setup prototypical enabling technology environment
- Develop
- Initiate or augment Metadata Management capabilities
- Profile Data – extract and identify Metadata
- Prime Tool (profile, quality, integration)
- Set Standards and Procedures
- Clearly articulate Business Definitions
- Structure Taxonomies
- Iterate
- Remain agile
- Iterate Use Case 1…n
- Confirm results with business community
- Demonstrate analytics
- Adjust (Learn)
- Document Lessons Learned and Best Practices
- Repeat
Use Case Approach
For each Use Case, start by identifying, defining, and analyzing potentially problematic Metadata using a profiling tool. Then, audit the Metadata to ensure accuracy, validity, timeliness, and completeness. Use this work product to establish or augment the foundation for all Metadata instantiation initiatives to follow. For example, demonstrate substantial business advantage and value that clearly improves overall reliability of the chosen data set for making decisions, performing analyses, and gaining insights. Identify and document pertinent and useful business rules required to improve Metadata quality.
Metadata Management Use Case Approach
Starting Point for the Metadata Repository
Effective Metadata Management relies more on a culture of knowledge sharing across the enterprise than on a complex technical solution.
- Create a Registry
- Define the taxonomies for metadata types
- Set up dependencies
- Establish structures and dependencies in a standard format
- Prime Metadata Repository with data elements, business definitions, usage definitions, entity relationships, business rules, patterns, and so on
- Scan, locate and collect Metadata from structured, unstructured, and complex data sources (internal and external)
- Other ways to capture business metadata
- Profile
- Internal (Intranet) and External (Internet)
- Indexing and mining unstructured text data
- Create manually
- Build custom APIs to interface with other applications and extract
- Standardize the business metadata as a snapshot and maintain changes and versions historically
- Automate governance, process management, event management, and notifications
- Other ways to capture business metadata
Filling the Metadata Repository
Each step augments and improves upon the Metadata Repository with newly discovered artifacts, glossary entries, and other metadata elements that illuminate the way in which metadata aids growing advanced analytics.
Expectations
Envisioning the requisite changes to the Enterprise Data Management program in order to handle increased information management complexities demands keen focus on building a robust Metadata Management capability. The goal of such a transformation emphasizes business-driven, service-based delivery of information from which to reap actionable insights…
- Metadata Management Framework supports Enterprise Data Management program
- Integration among elements
- Synergies among tools
- Evolving governance leads to enhanced levels of Metadata Management maturity
- Augmentation of trust in data
- Greater reliance on automation and orchestration (reduce manual intervention and complexity, improved compliance)
- Elevate Risk Management capabilities
- Augmentation of Metadata Repository demonstrating additional value
- Clearly articulated Business Glossary entries, taxonomies, catalogs, classification schemes, rules, etc.
- Comprehensive, universal sets of Allowed Values, Patterns, Rules
- Catalog of Anatomies for Metrics (Baseline)
- Standardization of Metadata Management architecture
- Governance policies, procedures, and Impact Analyses
- Metadata Management Dashboards (measure)
- Lessons Learned and Best Practices
- Advanced Analytics successes
Best Practices
Metadata is potentially infinite… and new categories are raising every day. Agility of approach counteracts rapid changes in enabling technologies’ Metadata Management capabilities
- Best solutions leverage existing resources (e.g., ASG Rochade, Informatica, Collibra, Hadoop, Cloudera)
- Leverage available processes
- Leverage existing repositories
- Leverage vendor resources
- Gain commitment for on-going support
- Organizational pain funds solutions
- Emphasize the long-term nature of the problem, solution
- Implement a supportable solution
- Scope breadth and depth accurately
- Scope user community properly
- Do not try to “boil the ocean”
- Prioritize value-add metadata, implement gradually, and stop when costs exceed incremental benefit
- Metadata must be Governed, Controlled, and Managed to maintain highest levels of integrity
- Initial efforts to store and integrate Metadata prepare for delivering real value through industrialized metadata collection and enterprise-wide consumption in project life-cycles and business processes
- Addressing Metadata Management together with Master Data Management, Data Governance and Data Quality may prove more daunting, but it can reveal advantageous synergies
- Think about it from the start…
Team Goals
The primary driver for Metadata-related projects focuses on ensuring the highest quality metadata. Metadata Management programs foster clear, managed competencies around metadata. To do this, the team must achieve clearly defined business accountabilities and responsibilities associated with the management of metadata. This requires recognized expertise, whether acquired or homegrown. Obviously, no program succeeds without defining measurable KPIs and metrics. In doing so, the Metadata Management team raises the confidence about accurate and timely metadata.
