In Part 1, we discussed the front end of the lifecycle – developing requirements for, and selecting data sources. In Part 2, we addressed the acquisition and management lifecycle framework, including:
– Harnessing the power of a data catalog for internal source comparison
– Selection of external data sources
– Comparing the content and condition of external data products
– Licensing and pricing considerations.
Now you’re ready to finalize your product decision, work with procurement to evaluate vendors, negotiate and analyze contract terms, develop a comprehensive Service Level Agreement, and onboard the data product. For Part 3, we’ll discuss:
- Coordinating with procurement— Pricing, Purchasing and Onboarding
- Developing contract terms
- Service level and operating agreements
- Managing service level agreements
- Metrics and monitoring data providers
- Data provider communication.
Coordinating with Purchasing / Procurement
A Purchasing (aka, Procurement) group supports operational requirements by ensuring that business requirements and specifications are met, through working with internal customers to purchase products and services from a reliable source, at a favorable price, with reasonable contract terms.
This part of the process can be complex and incur delays, or smooth and straightforward. The degree of collaboration that you establish, and the process steps that your organization has put in place, are factors affecting your experience.
On the part of Purchasing, it is very helpful to have a governing policy in place. Terms of the policy can include those depicted in the diagram below.
Becoming familiar with the policy will save you a lot of time and prevent missteps. Since your external data source is undoubtedly not the first one they’ve helped bring into the organization, make them part of your data product acquisition effort early in the selection process. Here are some tips:
- As soon as you determine that your data set must be obtained from an external source, identify your point of contact and reach out to them to communicate your objectives.
- If your organization is large, individuals or teams may be assigned to work with specific organizational units, or may specialize in certain vendor segments or types of data products, such as product surveys, clickstream data, and so on.
- Consider your point of contact as your trusted advisor. They’ll be able to provide perspective on your requirements, pricing, licensing, contracts, and service level agreements, and they will apply knowledge from past experiences with data vendors, which can inform your decision.
- As mentioned, familiarize yourself with available policies, processes and procedures that apply to your product selection and contracting effort. Your Purchasing organization has likely published a policy establishing standard methods and procedures that must be followed. It typically addresses a broad range of mandated policy terms, such as:
- Initiating a purchasing request – Addressing requisitions, providing a business case
- Approval authority – Who can commit the company to the purchase, delegation of authority, and maximum purchase limits by position level
- Contract execution –Wwho can officially sign a contract
- Purchasing process policy – What activities must be completed, and who is responsible
- Vendor selection – Identifying potential vendors, and informational requirements
- Due diligence – Standards for qualifying a vendor and the vendor process
- Vendor onboarding – What information is required to set up a vendor, and who has the authorization to onboard
- Competitive bidding – Criteria for purchases that must be offered to multiple vendors, versus to a single source
- Legal review – Requirements for legal review of contracts
- Contract monitoring – Responsibilities of the purchaser to monitor contract fulfillment and performance.
Your Purchasing group may also provide you with process descriptions detailing activity steps and responsibilities that apply to your purchase, as well as information about specific procedures, for example, a user guide for an automated requisition system.
Again, providing notice in advance is in your best interest. If you contact your procurement group at the last minute, you won’t be able to leverage their experience and knowledge. They’ll have less time to understand your objectives, and will be less effective in helping you to negotiate pricing and terms, arranging internal contract reviews, and so on. (And they may be annoyed as well. You can’t really blame them for that— remember the old saying “Lack of planning on your part does not constitute an emergency on my part.”)
Without advance collaboration, you may find that you haven’t followed a mandatory process step, or you may have to contact multiple individuals to chase down information and justify approvals. Approvers will need to ensure that you complete documentation and policy requirements, and then Purchasing will have to conduct due diligence to qualify the data provider, in the areas of financial status, corporate history, qualifications, reputation, delivery record, pending litigation, and other aspects of corporate health and integrity that protect your company’s interests.
Your organization may offer standard work products, depicted in the box above, that help you partner effectively with procurement, such as:
- Procurement process with defined activity steps (hip bone, thigh bone)
- Procurement procedures and systems (Form ABC, System DEF)
- A purchasing authorization scheme (Levels of approval, designated individuals)
- And, a procurement activity checklist (helping you to discharge your responsibilities).
When you’ve successfully advanced your joint initiative to purchase the selected data product, the next step is to review and negotiate the contract.
Developing Contract Terms
Well-defined contracts are legally binding documents that define the relationship between the purchaser and the vendor, and comprehensively describe the rights and responsibilities of each party.
You have an important role in ensuring that the data product will meet your requirements for content and service. A contract is the single most important control you have over product delivery, and is essential in avoiding unanticipated costs, legal disputes, and operational risks.
Although vendors will typically provide you with a standard contract for purchase of the product, you should assume that you have the ability to negotiate terms. Actively engage in reviewing contract terms, to understand and evaluate how they may impact content, pricing, delivery, and other factors.
As you review, note your questions and concerns, and discuss them with your Purchasing and Legal points of contact. You may be involved in meetings to negotiate terms. Take time to get answers to your questions, and ensure that your concerns are addressed; this mitigates the likelihood of potential negative outcomes.
Contract provisions that directly relate to the product purchase may include the following:
- Allowable usage of data – For instance, can you expose the data externally, and if so, are there any stipulations that may apply? The contract may specify limits on usage, such as the number of storage locations, the ability to share data internally, which organizational units have the right to access, and so on.
- The contract may contain limitations on re-distribution, either of the data product as a whole, or aggregations utilizing the product. The contract may also state what controls you must have in place.
- Length of service – Purchasing data for a longer period may help achieve favorable pricing, but commits your organization to pay for the product over that period. Do you anticipate that you’ll continue to need this data long-term? If not, you can specify a shorter timeframe.
- Licensing terms – As mentioned, vendors may license by platform, by purchaser, by storage location, or by number of end-users.
- Pricing or subscription terms – The contract may specify an annual fee or a monthly fee. Any price increases, or decreases, that may pertain over time should be clearly spelled out.
- Renewal – The agreement may be automatically renewable at a specified price, or require manual renewal periodically.
- Cancellation – The contract should specifically state how much advance notice is required, and what happens if insufficient notice is provided.
- Termination conditions – What conditions may allow termination before the agreed contract period elapses, for example, major cost increases, failure to meet service levels, or bankruptcy.
- Data disposition – How the purchaser must handle data that has been delivered from the product once the contract is cancelled or terminated. This consideration can be important, as some vendors require the data to be deleted.
- Protection of data – How the provider assures that the data is available, how they recover the product in the event of disaster, and any contingency plans that may affect delivery.
- Historical data period – The contract should specify the time period of data delivered through the product, for example, a rolling 30 years of stock market transactions.
- Method of transmission – The contract should state the primary method of transmission or means of delivery, and any backup methods, if offered.
- Notifications – Under what conditions, and how frequently, the vendor notifies the purchaser of changes to the product, any pending lapse of service, or other significant events.
- Warranties – The contract should state relevant warranties, such as that the product is free from infringement, fit to use for the purpose, and free from defects. The contract’s definition of a “defect” in the data product deserves your close attention; it may be useful to expand this definition with specific expectations related to data quality, as mentioned earlier.
- Compliance – The contract should address penalties for non-compliance. For example, if you’re depending on daily data and the vendor misses two days, what remediation measure will you accept? Determination of non-compliance is typically metrics-based, as specified in the Service Level Agreement.
- Other contract terms – Contracts typically contain many other terms, such as dispute resolution, applicable laws, insurance requirements, indemnification, and limitations of liability. For these and similar contract terms, review by an attorney is highly advised.
Careful review of contracts is critical. Fortunately, you’ll have assistance from your Purchasing group and appointed Legal advisor to help you finalize a well-defined agreement.
Your organization may have standard work
products available that can assist you in the contracts review and negotiation process:
- Standard contract requirements
- A contract review checklist
- Sample contracts for other data products
- And, published legal guidelines for contracts.
Developing Service Level Agreements
We previously mentioned Service Level Agreements (SLAs) in the context of governing internal data sources. They specify the activities that will be performed by the data supplier for the benefit of the data consumer, and define performance and quality requirements for the product.
There is no purchase contract required for internal data sourcing. Therefore, the SLA is the single control document that defines the services provided by the supplier to support the sourced data, and codifies expectations; setting parameters for cooperation, responsibility, and internal guarantees with respect to the data. SLAs are an important factor in assuring that the data will be reliable, fostering the consumer’s trust.
In some situations, internal data provisioning agreements may be less rigorous, for example, describing support services for a one-time data load, or providing a data set requested for a local end-user application, versus for a highly shared data repository. These operational agreements may be judged to be sufficient for the purpose.
For external data products, the SLA may be included in the contract, but it is typically a separate formal document— negotiated, approved, signed, and executed at the same time as the contract. The SLA references contract terms and may affect pricing, as the vendor may offer a menu of services to choose from.
Most vendors will provide you with a standard Service Level Agreement, which is a good starting point. Review the standard SLA carefully in the light of your requirements and performance expectations— including the ‘fine print.’ In collaboration with consumer stakeholders, procurement, and legal counsel, you can negotiate appropriate modifications.
SLAs should define and specify both services and management components. Some key elements that address services include:
- Services description – Summary of services that will be provided to ensure reliable delivery, performance, and quality.
- Availability – Clearly defined commitments regarding uptime and data availability.
- Delivery procedures – How the consumer will access the data product.
- Currency and time periods – Expectations for the currency of the data, and that the agreed upon time period is as specified in the contract.
- Responsibilities of each party – Commitments of the vendor or internal supplier to ensure that the product is delivered as specified in the contract and SLA, and to announce service changes. In addition the SLA defines consumer responsibilities for initiating change requests and reporting issues.
- Issue reporting – How the consumer notifies the supplier or vendor of a failure or defect, for example, an incident report.
- Issue response – What the supplier or vendor must do when a consumer makes them aware of an issue, commitment to a specified response time, classification of issue severity, and response prioritization by severity.
- Issue escalation – Designation of the responsible staff level, and associated procedures for responding to a reported issue that has not been satisfactorily resolved by the specified time frame.
- Penalty parameters – For an external data product, you can negotiate penalties for failure in service availability, such as data availability or completeness, and service quality, such as the number of defects allowed in the product.
- Penalty form – For example, payment to the consumer for damages, according to a tiered severity level, a service credit, or license extensions to offset future product expenditure.
Managing and Monitoring Service Level Agreements
Management and monitoring are also important components of a Service Level Agreement. Key elements that should be addressed include:
- Performance monitoring plan – How contract and SLA performance will be monitored, what the supplier or vendor provides to enable monitoring, for example, daily and aggregate statistics, log records, et cetera, and how they are made available.
- Monitoring frequency – How often the consumer reviews product performance and quality, and how results are made available to the supplier or vendor.
- Definitions for measurement standards – How the consumer’s data quality expectations and specifications are verified; we’ll address this on the next screen.
- Documentation of data content – The supplier or vendor should commit to supplying a data dictionary or data model with all associated metadata describing the product.
- Communication plan – The types and frequency of communications between the supplier or vendor and the purchasing consumer, such as notices of changes, monitoring reports, service interruption notifications, and so on.
- Dispute resolution process – In the event that a vendor breaches the contract, for example, discontinues a data product before the end of the contract period, what legal steps will be taken to remediate the breach, for example, negotiation, arbitration, or litigation.
- SLA updates – A schedule for the review of SLA terms and updates, and a list of events which may cause an unscheduled review.
As you can see, developing an effective SLA is not a trivial task. However, with input from your stakeholders, and support from Purchasing and Legal, you are equipped to create a comprehensive document that will serve your interests. Your organization may offer standard work products to simplify your efforts in developing or modifying SLAs, depicted in the box below.
Data Provider Metrics and Monitoring
Over time, you will need to ensure that the internal supplier or data vendor performs to the levels defined in the SLA. While the supplier or vendor may generate metrics associated with the product and its delivery, it’s also advisable to develop a set of your own.
- Metrics should be meaningful, reasonable, and attainable. For instance, some events are not under the supplier or vendor’s control, such as a regional power outage.
- Automated capture of operational metadata when data is received is a standard feature of most ingestion platforms, for example, time to access, time to load, and how many records were loaded, as well as failures and restarts. If a manual load was initiated, metadata may be written to a log, and statistics can be generated from it.
- Other metrics addressing the content and condition of the data may need to be obtained through queries executed in the destination data store, such as the minimum and maximum date range received in today’s load of historical data. When you first begin to receive the data, test it to ensure that it satisfies requirements and is in acceptable condition.
- Apply or develop data quality checks to monitor characteristics determined to be important for the product data set. This allows you to verify that the required condition of the data is met, as established in requirements, and specified in the contract and SLA. For example:
- Completeness – Did you receive the agreed upon scope of the data product— all of the tables and columns specified? What is the number (measure) and percentage (metric) of records that have null values?
- Conformity – Does the data received conform to the documentation provided by the supplier or vendor, in terms of its format, values, ranges, and precision?
- Currency – Does the data reflect the required time period?
- Uniqueness – Are there duplicate records? How many, and what percentage?
- Data set selection – If you specified a defined selection of data, does the data received correspond to your selection? For example, an epidemiological researcher may want to limit a population data set to only one disease of interest.
Management metrics you will want to monitor, for the purpose of verifying compliance with the contract and the SLA, include tracking issues with the product, for example:
- How many issues were reported,
- How many were resolved in the timeframe specified,
- The number of escalations and their resolution.
Compliance metrics not only serve to shape the supplier or vendor’s behavior through regular communication, but also constitute evidence in the event that defined service levels are not met, remediation measures are not successful, and penalties are sought according to contract guidelines.
Your organization may have standard work products available to help you develop appropriate metrics and conduct performance monitoring, as illustrated in the diagram above.
Data Provider Communication
Now you’re receiving data, you’ve tested it, and you’re using metrics to monitor performance and quality. You have expanded the scope of data for business use, fulfilling an important role.
As you wrap up your data acquisition effort for external data, be sure to populate metadata for the product in the Data Catalog. If you’re consuming internal data, remind the internal supplier to add your data store as a consuming destination.
Establishing a productive relationship with your supplier or data vendor requires regular communication. You developed a communications plan in the SLA, and here are some tips to help you maintain a positive relationship, enhance customer service, and encourage stellar performance.
Some communication channels should be formal, as reflected in the SLA communication plan. They are the foundation for accountability, such as:
- Contract and SLA renewal negotiations – At the stipulated time prior to contract expiration, an external vendor should provide advance notification about any planned changes to the data product, alterations in contract terms, or pricing. You should review any requested modifications carefully, and enlist your support team to ensure that you are fully informed about the implications. In turn, your experience with the product may lead you to request modifications to the contract or services.
- Contract performance reviews – Meetings with the supplier or vendor should take place on a predetermined regular schedule, for example, monthly or quarterly. Metrics produced by both parties should be distributed in advance, and any concerns or anticipated changes addressed. Meeting notes and action items should be recorded.
- Service Level Agreement reviews – Meetings with the supplier or vendor should take place periodically, for the purpose of reviewing SLA terms, enhancing the SLA with additional performance or quality requirements, and discussing modification suggestions based on systematic monitoring.
- Issue escalation – When resolution of an issue has been escalated, the consumer should document the issue, the steps taken toward resolution, the level of escalation that was required, and if the time to resolution was exceeded. Maintaining a log of escalated issues may support remediation measures, or provide evidence for levying penalties.
Some communication types lend themselves to more informal treatment, such as an email or phone call to your point of contact. For example:
- You have a question about how a data element is obtained, produced, or calculated
- You want to provide specific positive or negative feedback about an instance of customer service.
- Or, you want to explore another product, or ask about future versions.
Your supplier or vendor is providing a product and services to you, and it’s important to communicate effectively, in a consistent and reasonable manner. The tone you adopt can help you ensure that you set clear expectations and receive satisfactory service. Let them know your communication preferences and the responsiveness you’re expecting, and when possible, offer positive feedback, as well as expressing any complaints. This is especially important when interacting with internal suppliers, as you should assume they’re just as busy as you are, and you want to foster a positive relationship and successful outcomes for the good of the organization.
Standard work products that can help you be successful in maintaining a productive relationship include:
- Vendor communication policies or guidelines,
- Internal communication guidelines for suppliers and consumers,
- A contract evaluation template or scorecard,
- Meeting notes and action item tracking.
Data Provider Management – Conclusion
At this point, you’ve accomplished everything you set out to do. You’re now equipped with approaches and methods that you can incorporate into an action plan for conducting a data sourcing effort and managing data providers effectively.
Your organization, for its part, now has a list of suggested work products that can be developed along with specific acquisition efforts, streamlining the data acquisition lifecycle and gaining efficiency with each iteration. This results in many benefits to data consumers:
- Future data requestors will be able to leverage templates for data selection, vendor comparisons, license evaluations, and other key activities,
- Quality expectations will be included in SLAs and operating agreements,
- Purchasing can develop checklists that assist those acquiring data to move through the process effectively,
- Legal can publish guidelines to communicate the requestor’s responsibilities and ensure a productive effort.
Buttoned up, thorough, effective, efficient! Good job!
