For most of us practicing the Data Management profession, it is frequently an enigma knowing who those in the same line of work are. We often wonder: How many people are in this field? How long do they stay in this practice? How old are they when they first join? How old are they now? The only way we may be able to get a glimpse is asking a peer or attending a conference.
The Data Management profession started sometime in the late 1960s and early 1970s as the focus on the use of computers began to shift from “data processing” to “information management;” the change was triggered by the deployment of a new technology, the Database Management Systems or DBMSs. The need for this professional increased as the second generation of the DBMSs, the Relational DBMSs, appeared in 1980. Since, almost all organizations have embraced these platforms and they are now ubiquitous; it is hard to imagine what the world looked like before Ted Codd’s relational model.
In the 1970s DCI published a paper on the Data Management practice, to my knowledge the first effort of its kind; it provided much needed insight at the time. In 1982, Mark L. Gillenson published a paper on “The State of Practice of Data Administration[1]” through the IBM Systems Research Institute, a second one in 1985 called “Trends in Data Administration[2]” and a third one in 1991 titled the “Database Administration at the Crossroads[3].” In 2002, PricewaterhouseCoopers published the “Global Data Management Survey[4]”, followed by the publication in 2005 of the “Global Data Management Survey 2004.[5]” These studies are focused on the Data Management profession or practice, and even though more needs to be written on Data Management, little is known about those who practice it. Today we have many conferences in Data and Information Management, Enterprise Data and Information Architecture, Business Intelligence, and more. In addition, we have magazines such as DM Review, The Data Administration News Letter and others. Yet, the mystery continues.
“Speak with Data (as stated by Ishikawa)”
This year, the DAMA International[6] association decided that now is the time to change this state of affairs. At the recent DAMA International Symposium & Wilshire Meta-Data Conference (April 2006 in Denver Colorado), the association conducted the first ever survey of Data Management Professionals. The conference was attended by more than 1,000 professionals and about one third of the attendees participated in the survey.
The purpose of the survey was to learn more about the professional and its work environment; we chose the 2006 Conference because it is the foremost event in the industry, it is attended by practitioners in all disciplines, and the attendees come from all over the world. The respondents are geographically distributed as depicted in Figure 1.
Figure1: Geographic Distribution of Respondents
The reader is reminded that the sample suffers from some limitations imposed by the environment where it was taken; for example, the representation from non-US residents is small, about 13%; the representation is even less significant for countries outside North America and Europe (about 1%). In addition, some practitioners such as DBAs have a lesser representation due to the focus of the conference. Despite these shortcomings, we are able to reach some initial conclusions on this group. We will continue to analyze the geographic and demographic implications and will continue to study this data to provide additional insight in the future.
The Overall Picture
The Data Management professional works in roles such as Data Architect, Data Administrator, Database Administrator, Data Modeler, Database Designer, Information/Data Quality Specialist, Metadata/Repository Management Specialist, Data Movement Specialist, etc.
This individual is on average over 45 years old, has over 20 years of work experience and over 12 years of data management experience. This person typically begins a professional career in the mid-twenties, and joins the Data Management field about 8 years later. He or she typically has a college degree and quite often a post-graduate. This professional is likely to attend 3 to 5 Data Management related conferences during his or her career.
About 65% of the conference attendees are active members of DAMA International. The typical member has over 4 years of membership. This indicates that a Data Management Professional joins the association nearly 8 years after starting the Data Management practice.
There are no formal estimates of the number of Data Management professionals in the world; we know that in the USA small organizations usually employ between one and ten, medium size organizations between 10 and 50 and large organizations at least 10 and sometimes more than 200. The evidence indicates that this employment distribution is very similar outside the US. This is reinforced by the fact that DAMA International has chapters in many countries and has many non-US members in its Global Chapter (a world-wide, virtual chapter). We can only speculate that the number is in the hundreds of thousands.
Age & Experience
Although the typical Data Management professional is on average over 45 years old, about 35% of them are under 44 years of age, 43% are 45 to 54 years old, and 22% are 55 or over. The complete breakdown is shown in Figure 2.
Figure 2: Distribution by Age Group
This individual is typically a mature professional that begins Data Management as a second career and usually has 10 or more years of experience (91%) and frequently over 20 years of experience (61%). Figure 3 shows the overall experience of the Data Management Professional.
Figure 3: Overall Professional Experience
There are very few people with less than 5 years of total experience. Once this person joins the Data Management profession, usually about 8 years after starting a professional career, he or she seems to embrace the practice for the long run; 54% of these professionals have 10 or more years in the field; 77.6% have 5 years or more. Figure 4 shows that their experience does not have a bell-shaped distribution; the cause for this is under research.
Figure 4: Data Management Experience
Formal & Continuous Education
This professional for the most part has solid formal education; about 83% have a college degree and, of those, over 36% have post-graduate education. Figure 5 shows the proportions of professionals by their highest level of formal education; from high school to prost-graduate programs.
Figure 5: Highest Level of Formal Education
The primary source of Data Management fundamentals for these practitioners is professional practice and self-development accounting for 76%. These two categories include on the job learning, reading trade articles and books and seeking couching from experienced professionals. Figure 6 shows the complete set of most significant sources of data management fundamentals.
Figure 6: Most Significant Source of Data Management Fundamentals
For continuing education, this person is very likely to attend several Data Management related conferences and, in some cases, pursue a Professional Certification. Over one third are likely to pursue the Certified Data Management Professional or CDMP, provided by DAMA International and the ICCP[7].
Lack of employer support is the most significant impediment for a professional to pursue continuous education; especially a certification program. It is discouraging to find out that one third of the respondents indicated their employers do not have an interest in certification and over half did not know if their employer offered support (financial or otherwise) for pursuing certification. Correspondingly, a significant number, over two thirds, are not interested or are not sure if the would like to pursue certification as a means for continued education and self-development.
The Work Environment
This individual works in all industries and in organizations of all sizes. We found out that one in ten professionals who responded to the survey work in a small organization (100 or fewer employees), while 4 work in a medium sized organization (101 to 5,000 employees) and 5 in a large organization (5,001 or more employees).
These organizations fall in all sectors (the definition of sector is based on the US North American Industry Classification System or NAICS[8]); the responses indicated representation for all sectors except the ‘Administrative and Support and Waste Management and Remediation Services’ sector. However, I personally had the opportunity to work in this sector and I know there are many data management professionals in these companies.
For every 10 attendees to this event, about 2 came from the ‘Finance and Insurance’ sector; one came from each of the ‘Manufacturing,’ ‘Information,’ ‘Professional,’ ‘Scientific and Technical Services,’ and ‘Health Care and Social Assistance’ sectors; about 2 came from the combined ‘Utilities,’ ‘Educational Services,’ ‘Public Administration,’ ‘Retail Trade,’ and ‘Transportation and Warehousing’ sectors; and the last attendee came from all the remaining sectors combined.
The number of Data Management Professionals in an organization, and their proportion to the total number of employees, varies greatly from one organization to the next; our initial analysis was unable to determine if there are any correlations to size, location or sector. We will continue to analyze the data to determine if there are any significant correlations or insights and will keep you posted.
Conclusion
Collecting and analyzing information is arduous and time consuming, but the results are rewarding. DAMA International feels that now we are better prepared to serve the Data Management professional because we know this group a little better. The market analysis team is working on subsequent surveys to find out more about the jobs these people fulfill for their employers and about the state of the practice. Who knows, maybe next time we are seeking some insight, we may reach out and touch you.
The association is always seeking opportunities to collaborate with other organizations to further our understanding of the profession and those who practice it. We aim to provide all practitioners, their employers and scholars in the field with insight into this elusive group and their profession. What do you think about this article? Please let us know at vp_marketing@dama.org.
[1] The State of Practice of Data Administration – 1981, Mark L. Gillenson, Article in the Communications of the ACM, October 1982, Volume 25, Number 10.
[2] Trends in Data Administration, Mark L. Gillenson, MIS Quarterly, December 1985.
[3] Database Administration at a Crossroads: The Era of End-User-Oriented, Decentralized Data Processing, Mark L. Gillenson, Journal of Database Administration, Fall 1991.
[4] Global Data Management Survey – The New Economy is the Data Economy, PriceWaterhouseCoopers, authored by Henry Kenyon, John Benge, Richard Lambert and Willie Jordaan.
[5] Global Data Management Survey 2004 – Data Quality Management, PriceWaterhouseCoopers, authored by John Benge, George Marinos, Willie Jordaan, Paul Smith and Henry Kenyon.
[6] For more information visit www.dama.org.
[7] The Institute for Certification of Computing Professionals (www.iccp.org)
[8] For a complete listing of the NAICS categorization, visit http://www.census.gov/epcd/naics02/naicod02.htm.
