20 February 1997
Friends of the Database Management Industry,
The Information Technology Laboratory at the U.S. National Institute for Standards and Technology (NIST) recently announced a drastic reduction in its level of participation in database management
standardization activities, including termination of SQL validation test development. In response to this withdrawal of support, representatives from IBM, Informix, Oracle and Sybase prepared the
attached position paper on The Role of NIST in SQL Standardization.
Representing over 85% of a market estimated to exceed $20 Billion world-wide for SQL relational database management systems and related software, the undersigned strongly believe that NIST’s
continued participation in SQL standardization activities is required to best serve taxpayers, Federal agencies and U.S. domiciled producers of products conforming to SQL standards.
We encourage you to review the attached paper and solicit your support in reinstating the SQL Standards Program at NIST.
Respectfully,
Lawrence L. Wills
IBM Director of Standards
International Business Machines Corp.
Michael Saranga
Senior Vice President
Informix Software, Inc.
Ken Jacobs
Vice President
Oracle Corporation
Bob Epstein
Executive Vice President & CIO
Sybase, Inc.
Abstract
This paper examines the current state of SQL standardization, considering the technical aspects of that standardization as well as the economic and political aspects. It especially addresses the
role played by NIST in reaching the present state and discusses the impacts expected from NIST’s announced change in participation. The paper concludes by identifying areas in which NIST’s
ongoing participation is urgently recommended by the database industry of the United States.
Executive Summary
SQL is a computer application that permits a user at a computer terminal to request in a simple, easy-to-understand and use manner, virtually any information from a database. When the request is
entered, SQL searches the database for the desired information and returns it to the user in a matter of seconds. SQL is used globally and is a very successful product, with versions available from
a large number of vendors.
SQL’s success is largely due to the voluntary national and international standards that have been developed to define it. In fact, SQL arguably represents one of the most successful standards in
existence.
These standards are the foundation of a multi-billion dollar industry employing thousands of people in the United States. The success of these standards and the industry that implements them is due
in no small part to the contributions of the U.S. Federal Government. The Government’s participation in the development of the standard and its support of independent testing and verification of
products claiming conformance to the SQL standard have been major factors in that success.
Recently, NIST (National Institute of Standards and Technology), the Government agency most directly involved in supporting the SQL standard, announced its intent to drastically reduce its level of
participation in SQL standardization efforts. By this change, NIST leaves the largest consumer of information technology, the United States Federal Government, without an effective voice in the
development of standards for database languages. Worse, it leaves the industry without an independent agent for the development of SQL validation testing capabilities. It is NIST’s participation
as a neutral, vendor-independent player that has been a major factor in the standard’s worldwide acceptance, and has permitted the industry to grow and prosper. Without NIST’s continued
participation in the development of new extensions to SQL standards, its development of SQL conformance tests, and its acting to verify conformance of SQL products to the standard, the global
acceptances of SQL as the database language of choice could be in serious jeopardy.
The database industry in the U.S. strongly believes that NIST’s on-going participation is urgently needed to protect taxpayer’s interests and to provide the independent verification of
conformance that is needed worldwide.
Status of SQL Standards
SQL is arguably among the most successful standards ever published. In late 1986, ANSI (the American National Standards Institute) published the first edition of a formal standard for SQL as ANSI
X3.135-1986; a couple of months later, ISO (the International Organization for Standardization) published an identical document as ISO 9075-1987; the document was informally known as SQL-86 (or
SQL-87). This document was replaced in 1989 by an enhanced specification that added a small number of features, principally the specification for basic referential integrity; this revised standard
was known as ANSI X3.135-1989 and ISO 9075:1989, informally known as SQL-89. NIST (the National Institute of Standards and Technology, part of the U.S. Federal Government’s Commerce Department)
was concerned that both SQL-86 and SQL-89 specified the ways in which SQL was embedded into a host programming language in an annex to the standard and that the annex was not even normative
(required) in nature; as a result, NIST proposed and edited a second standard in the United States, ANSI X3.168- 1989, Information Technology – Database Language Embedded SQL, that made the host
language embeddings of SQL normative. SQL-86 and SQL-89 were rightfully criticized for being “minimal”, or “least common denominator”, languages that standardized only those SQL features
already implemented in a common manner by most or all vendors; it was not feasible to build entire applications in standard SQL without depending on a plethora of vendor extensions to the standard.
Even before SQL-89 was published, ANSI Technical Committee X3H2, Database, and the corresponding ISO group, ISO/IEC JTC1/SC21/WG3, had begun work on a major revision of the SQL standard. Published
in late 1992 as ANSI X3.135-1992 and ISO/IEC 9075:1992, and commonly called SQL-92, this standard has proved to be a major event in database standardization history. For the first time, it became
possible to write significant applications entirely in a standard language, although it was and is true that no vendor implemented the entire language. The focus of SQL-92 was just this: to
standardize enough language to allow significant applications to be built without depending on vendor extensions to SQL.
By early 1991, the two groups had agreed on the content of SQL-92 and were going through the final stages of standardization. However, they had already begun work on another major revision of the
language, commonly known by the nickname “SQL3” (because it is the third generation of the language). The “theme” of SQL3 was to be the addition of object technology to the language. This was
not a little controversial and there were expectations that it would be difficult to accomplish, but input both from vendors of SQL products and from users of those products was clear – this was
urgently required for continued productivity in business. Indeed, this major enhancement proved to be complex and difficult indeed; several false starts occupied many months of the committees’
time and energy and it was not until late in 1995 that a broadly acceptable approach was found. Today, the two groups have begun the first formal stages of processing (called CD, or Committee
Draft) that will result in a standard being published in a couple of years. It would be a mistake to say that this work is complete, however. Many difficult questions remain in determining the most
effective alternatives among many in this technology. Perhaps most importantly, little is known about methodologies for testing systems with object-oriented capabilities.
Importance of SQL Standards
While SQL has its basis in the relational model formulated by Dr. E.F. Codd and others in the late 1970’s and early 1980’s, its success is due more to three factors unrelated to a purely relational
approach to data management: First, it was adopted by IBM, then the largest computer systems vendor and a company with enough market presence to drive an entire industry in certain directions.
Second, it was the subject of a formal, de jure standards effort that gave both competing vendors and concerned users reassurance that the language would not be “owned” by a single vendor.
Finally, it quickly became the subject of a Federal Information Processing Standard, FIPS 127, which demonstrated to vendors and users alike that the U.S. Federal Government had an interest in the
language, its stability, and its utility. The FIPS made it possible for vendors to bid on government agency procurements of database systems without depending on the expensive, time-consuming, and
often biased procedures that dominated in the past. It also helped reassure the database user community in the private sector that there was a common point of validation that did not require them
to band together – in ways that might be difficult to do among competitors – just to present a sufficiently large presence to insist on standard implementations of products.
The result of this success is an industry that grew from infancy to gross revenues exceeding $15 billion in less than 15 years. While reasonable observers may disagree over some of the causes of
that phenomenal success, it is obvious that one factor is the stability that the standard provided for vendors and application developers.
NIST’s Historical Role
A standard is truly relevant only if those who depend on it have some way to verify that products claiming conformance do, in fact, provide that conformance. There are many ways, of course, to
accomplish this verification. In order to have maximum effect, the way (or ways) chosen must provide several assurances.
First, the standard must be interpreted in ways that are useful – for example, while the actual SQL standard does not require that an implementation support a specific minimum number of tables, all
agree that a product supporting only one table is not useful; consequently, a meaningful interpretation of the standard should specify some minimum number of tables before conformance can be
claimed. Second, questions always arise about the specific meaning of the words of a standard; if those depending on the standard know that the questions will be formulated by a neutral body and
submitted to the standards organizations for answers and possible amendments to the standard, they will have greater confidence in the standard than if questions are formulated principally by
(possibly self-serving) organizations responsible for implementing the standard. Third, so that those depending on the standard do not have to rely on (again, possibly self-serving) vendor opinions
about whether their products actually conform to the standard, there must be ways to test claims of conformance and to make the evaluations of those claims readily available. The conformance tests
have vastly greater credibility if they are developed by an organization or organizations seen as not having an agenda that makes them beholden to those benefiting from the conformance claims.
NIST, in its participation in the SQL standards effort over the years has served several crucial roles. First, it has provided representation that has consistently made significant technical
contributions to the standard as it evolved. In this role, it served the greater database community’s interests by sharing its expertise in many areas with those seeking to make the language
useful to the largest possible audience. Second, it provided invaluable support to the process itself by supporting its representatives’ ability to participate widely in the standards activities –
to the extent of representing the United States in the international arena and even chairing the ISO SQL group. By having a single individual that represented the USA and chaired the international
group, the United States had significant leverage in the evolution of the international standard for SQL, both from the vantage point of management of the process and from the moral authority that
derived from such commitment. Third, the existence of the FIPS (eventually FIPS 127-2, specifying conformance to SQL-92) – developed, maintained, interpreted, and tested by an independent member of
the user community, rather than a member of the vendor community or a consortium owned by the vendor community – gave to the SQL standard a validity that encourages sincere efforts at genuine
conformance by the vendors and trust by the consumers in those claims. Fourth, NIST represented what is probably the largest consumer of information technology products in the world – the United
States Federal Government – in the development and maintenance of the SQL standard. Fulfilling the role assigned it by Congress (among other ways, through the now-withdrawn Brook’s Bill), NIST
provided the taxpayers of the USA assurance that their hard-earned tax dollars were not being spent on proprietary database products that locked an agency into a single vendor, but on products that
met a standard and for which meaningful competition existed.
Critics of “big government” are concerned that over-regulation costs consumers money, stifle competition, overwhelm businesses with paperwork, and generally damage the economy. Re-examination of
the role of government’s effects on the private sector easily identifies many areas where those concerns are valid. However, the database industry in the United States – indeed, the information
technology industry – believes that the roles played by NIST do not fall into those areas. Indeed, NIST’s participation has been invaluable in the evolution of the database industry into the
significant contributor to the country’s economy that it is.
Other observers of government/industry relationships question the desirability of having the government “pick winners”, either specific companies or specific technologies. Again, the database
industry does not believe that NIST’s roles comprise picking winners, even in the area of technology. Instead, we believe that NIST’s participation, in both style and substance, has shown
recognition that relational technology emerged in the forefront of commercial database technology and has gone on to provide stability for consumers of that technology. By contrast, when NIST
declares that its contributions to SQL are being eliminated in favor of areas where standards have not yet emerged, NIST to taking the position of choosing new winners in untested areas! We are
surprised that, given today’s climate of government, that NIST believes it appropriate to decide that it knows better than the industry which areas of technology need its contributions. While we
have no doubts that NIST’s participation in evolving standards and testing methodologies in new areas of technologies will be welcomed by participants in those technologies, we wonder what the
benefits will be to the taxpayer and to the government agencies that depend on NIST.
Revolution in SQL Standards and SQL-Based Products
While it is true that SQL standards and SQL-based products have reached a level of maturity, we believe it is incorrect to assume that they no longer need NIST’s participation. If the intense
activity that is taking place in the information technology industry is any indication, something of a revolution is sweeping the SQL industry.
With the emergence of Internet-based information access and the World Wide Web, there is a strong interest in extending database technology to deal with the management of unstructured data along
with the traditional record-oriented structured data. There is a strong feeling in the market place that new technologies are required to address these new kinds of data. A new breed of database
systems, called Object-Relational systems, are emerging to solve some of these new market requirements. These systems attempt to integrate the management of multimedia data such as text, image,
video, sound, etc. with the management of alphanumeric data. With all these developments, it is clear that the database industry is experiencing a renaissance. It is probably not much of an
exaggeration to say that conditions somewhat similar to those that prevailed in the late 1970’s (when the work on relational database systems started) are again upon us.
It is also important to note that the SQL standard is going through significant changes to keep up with the developments that are taking place in the database industry. The ongoing work on SQL3 is
an indication of these changes. The scope of SQL3 is no longer just the management of tabular data structures and the associated query sub-language; SQL3 now encompasses new capabilities such as
object extensions, procedural extensions, and multimedia data type and query specifications.
Unlike SQL-92, SQL3 is no longer a monolithic, single document. SQL3 is divided into 8 parts, each proceeding at its own pace for final adoption:
- Call-level Interface – SQL/CLI: (CLI-95 published as a standard in 1995; SQL3/CLI is a now Working Draft)
- Procedural extensions – SQL/PSM: (PSM-96 published as a standard in 1996; SQL3/PSM is now a Working Draft)
- Object-Relational extensions – SQL/Foundation: (currently a Committee Draft)
- Extended Object extensions – SQL/Object: (currently a Committee Draft)
- Host Language Bindings – SQL/Bindings: (currently a Committee Draft)
- Distributed Transactions – SQL/Transactions: (currently a Working Draft)
- Temporal Extensions – SQL/Temporal: (currently a Working Draft)
- Virtual Tables – SQL/VirtualTable: (currently a Working Draft)
In addition, a companion project is working on standardizing multimedia data type extensions under the umbrella of SQL/MM. Currently parts dealing with Spatial and Text data types have progressed
to Committee Draft stage. In addition, a new part on Still Image data types is getting started.
The database industry strongly believes that it is very important for NIST to continue its participation in development of the SQL standard in ways similar to its historical involvement. It was
appropriate for NIST to get involved in SQL standards in the late 1970’s; NIST’s continued participation is even more appropriate today.
The Effect of NIST’s Announcement
With NIST’s recent changes in the style, substance, and depth of its participation in SQL standards activities, several changes are imminent in the industry. First, the standards development
effort loses the largest consumer of database technology as a significant contributor to technical progression and support of its activities. Second, the development of tests for higher levels of
SQL (as well as the important new technology being developed by the on-going standards work, such as SQL/CLI, SQL/PSM, abstract data types, and complex data – all underpinnings of the emerging
National Information Infrastructure) currently done by NIST must find a new home; it is unlikely that the new home will be viewed by vendors and consumers alike as free of unfair influences and be
seen to have long-term commitments to that development independent of the fortunes of any specific corporate entity. Third, procurements of SQL products by U.S. Federal Government agencies are less
able to depend on an independent body representing the government to guide them, and are consequently more likely to be filled by products based on short-term perceptions of needs and less by
long-term benefits related to application portability.
The database industry does not believe that any of NIST’s participation activities were solely responsible for the incredible growth of that industry, but it does uniformly believe that the
credibility offered by having NIST represent the user community and the Federal Government, and by having NIST serve as an independent testing agency, was – and continues to be – invaluable in
supporting the industry and the thousands of jobs that it has created. The database industry does not believe that it is the role of government to build an industry or to control it; it does
believe, however, that government has a proper role in supporting a U.S. industry that has no peer anywhere in the world. NIST is the only Federal Government agency that has the expertise and
authority to represent that community and simultaneously develop the conformance tests that give the standard the credibility that it has earned. Without that authority, the global leadership in
SQL that U.S. vendors have attained becomes much easier to threaten. If vendors in other technological countries, such as Japan, find themselves able to claim ownership of “the standard” without
independent verification of that fact, then it becomes increasingly difficult for U.S. producers to compete in those countries’ markets and in markets targeted by those countries.
Conclusions
The database industry is one of the most important U.S. software industries. It is an area where U.S. vendors completely dominate the international marketplace and is the source of thousands of
jobs that benefit many communities in the country. The existence of a respected and trusted standard underlying the products of that industry is a prime driving force behind that success. This
preeminence cannot be reliably maintained without the on-going advancement and maintenance of that standard – and of the trust evolving from the independent testing and verification of claims of
conformance to it. As an independent, vendor-neutral participant in the SQL standardization efforts, NIST not only provides the perspective of a very large consumer of SQL products, but is in the
unique position to bring balance to the USA’s representation in the international standards arena and provide the neutral perspective of a user to SQL’s development.
NIST represents a large portion of the user community in the U.S. Federal Government. By participating in SQL standards development, NIST influences the standards to reflect Federal Government
needs. This, in turn, represents millions of tax dollar savings by ensuring that procuring agencies can reliably determine which products will meet their needs without locking them into proprietary
languages from which escape would be prohibitively expensive.
Finally, NIST’s active participation in the development and maintenance of conformance tests ensures that those tests will be vendor-neutral and have worldwide recognition. This inherently
provides consumer confidence, while encouraging the growth and development of the producing industry. It permits vendors of SQL products to continue to focus on those areas where competition is
most beneficial, such as performance and management, without having to struggle to convince large users that their language variants are sufficiently beneficial to justify changing from one vendor
to another.
The database industry is concerned with the manner in which U.S. Federal Government agencies will specify procurements of database systems in the future. Without a viable standards program –
reliably tested using independently-developed tests – there is inevitably temptation (and pressure) for procurements to use a single vendor’s literature to formulate Requests for Proposal (RFPs).
This leads to increased vendor lock-in and decreased competition; that in turn leads to higher costs and decreased flexibility. Worse, vendors whose literature is not used must then resort to
formal protests and even legal action to eliminate the unfairness inherent in the situation. This benefits no one, least of all the taxpayer.
Summary
We believe that NIST’s continued participation in the SQL standardization process is extremely desirable. While NIST has indicated its intent to continue participation at reduced levels and with a
somewhat different focus, we believe that the following two contributions are urgent and invaluable:
- Provision of at least one individual contributor who is an expert in relational and object database technology, with the personal stature to effectively represent the interests of the user
community in the USA and to represent USA interests internationally - Provision of an on-going program to develop and maintain SQL tests
In addition, we believe that the choice of mechanisms for actually testing and certifying database products for conformance to the SQL standard is of great concern to the database industry. We
believe that the choice must be made with input from throughout the database community, including not only NIST, but also database system vendors, the user community, and other Federal Government
agencies.
Submitted by Michael M. Gorman
Whitemarsh Information Systems Corporation
2008 Althea Lane
Bowie, Maryland 20716-1518 USA
