Zen and the Art of Data Maintenance: People Silos Cause Data Silos

COL02x - feature image silverstonWith the exponential growth of data from so many different sources, data silos (or in other words, separate unintegrated data stores) are more prevalent than ever.

However, what is the root cause of data silos? People silos! The more that people view themselves as separated, the more that data is separated.

But aren’t people and various organizations, in truth, actually separate entities? Or are they? If they are actually separate entities, what issues does this cause?

Imagine two groups in an organization: a data science group and a data management group. The data science group is focused on creating and implementing useful models for predictive analytics. The data management group is focused on creating quality integrated information. However, the data science group does not want to talk to the data management group as it does not want to get involved in bureaucratic standards and common vocabulary issues or matters that seem to be less important than their deliverables.

The data management group prioritizes their important task of data integration and sees it as the foundation of successful systems. What I have seen in this scenario is a lack of true cohesive, unified efforts between these two groups. Hence, the data silos between the data management group and the data science group. Research suggests that data scientists are spending about 80% of their data on data quality issues. Often, they will use a data engineer (not from the data management group) to cleanse their data. Since one of the key interests of the data management group is to create data quality, doesn’t it make sense for these groups to collaborate to create a much better outcome? And yet, in my experience and the experience of many others, this does not seem to happen.

There are many other examples of various people and groups not collaborating and acting independently. For example, I have seen the human resource department of an organization say that their data is extremely sensitive (which of course it is), so therefore this type of data cannot be shared or integrated with other types of data and must be kept in a separate system. The irony is that human resource data is one of the most interrelated types of information since so much data is connected with people-oriented information.

Another example is that the financial area of an organization may keep their information in a separate accounting system and be very cautious of data sharing since their information is also extremely sensitive. I have been on data integration efforts where the financial organization will say that they are glad to share and help integrate their information, and yet when we have asked for their data, there are often significant stalls (sometimes indefinite) in getting this data.

Finally, I have been involved in many customer relationship management integration efforts, and when I ask salespeople to share and integrate their customer information so that it can be a useful enterprise resource, this is often a challenge. The underlying thought here (even though not expressed outwardly in this manner) is often, “Really, you want me to share intimidate details that I have collected from my dear customers? First of all, this information is confidential. Secondly, this information really represents power that I have.”

Does data represent power? If you look at the examples of Amazon, Google, and Facebook, it seems that those who have more data have more power. So why would people and organizations share and integrate data and give away their power?

The key operating premises in the above scenarios is that people and organizations are separate. But are they really? Or are all people and organizations already integrated and connected and we just pretend that we are not?

What Does This Have to Do with Zen?

Zen is a Japanese word for ‘awareness’.  It also represents a school of Buddhism with philosophical principles. One premise of Zen (and many other philosophical teachings in other traditions) is that the more that we think we are separate, the more difficult things become. Another way to say this is that the greater our ego (our self-referencing), the more suffering we experience.

It seems to me that we can extend this thinking to the premise that the more that we think we are separate within an organization/company, the less integrated our systems are, which makes the organization weaker, less productive, more chaotic, and prone to disintegration (which literally means to ‘separate into parts’ or ‘to crumble’).

So when we think of ourselves or our teams or organizational units as separate, we tend to build separate, disintegrated systems. When we see ourselves or our projects as separate, we tend to create our own vocabulary/language for each system, develop separate hardware and software infrastructure and implementations, develop independent data stores, and this leads to ‘data silos’.

So, if we can think of ourselves as non-separated and connected, then this will lead to systems and data integration.

Although this is not the norm, I have witnessed great examples of this type of thinking in several large organizations that work extremely well as a large integrated team without much ego. For example, in one large financial institution, they adopted a standard business language and standards across the entire organization, were all working towards a common vision that they were all excited about and were able to develop extremely efficient and effective systems to support their investors. The investors themselves were an integrated part of the development process!

Many organizations incent people by pinning them against each other in competitions. In our culture and society, we tend to reward individual successes and compare our accomplishments with each other. In organizations, people are often competing for promotions, raises, and power. Thus, the idea of not acting as if we are separate is difficult to realize.

What Can We Do?

This is a difficult issue and not easy to address, yet if we are to better integrate data, we must address it.

Three possibilities for moving towards more integrated people and thus more integrated systems and data are:

  • Think of ourselves and our teams as connected. Continue to ask the question, ‘How am I connected to various people, organizations and things?’ Realize that even though we experience some separation, we are also all connected to everything.
  • Establish a shared and compelling purpose for your area of the business and connect it to the overall purpose of the entire organization. When the vision and purpose of an organization is clear, commonly accepted, compelling and concise (I call this the four C’s), then it is more likely that people will see themselves as integrated if they are drawn to the shared purpose and vision.
  • Establish a common vocabulary. Quite often, miscommunication leads to separateness. Interestingly, the etymology of the word ‘communicate’, comes directly from the Latin ‘communicationem’ which literally means “a making common”. Luckily, the need for establishing common business glossaries has been recognized more lately. To help these types of efforts, we have developed a tool called the Universal Business Glossary, to help establish common terms and definitions by re-using constructs that have worked well in many organizations (many of these standard terms and definitions are based upon decades of research and based upon many of the concepts in the Data Model Resource Books). If you would like to learn more about this, please see Universal Business Glossary Webinar.

If we were to answer the question, to be separate or not to be separate, let’s consider that both are appropriate. While it seems that to some degree, we are separate, let’s make sure to acknowledge the opposite: that we are also not separate at all. At a deeper level, we are interrelated with all other people, all aspects of our organizations and systems, and for that matter, with everything else!


Share this post

Len Silverston

Len Silverston

Len Silverston is a best-selling author, consultant, speaker and internationally acclaimed expert and thought leader in the fields of data governance, data modeling, data management, and in the human dynamics of integrating information data. He is the author of The Data Model Resource Book series (Volumes 1, 2, and 3), which describe hundreds of reusable data models. The volume 1 book was rated #12 on the Computer Literacy Best Seller List and his volume 1 and 2 books have been translated into Chinese and in 2009, he co-authored “The Data Model Resource Book, Volume 3, Universal Patterns for Data Modeling”, which has been translated into Korean. Mr. Silverston has published many articles and has been a keynote speaker at many international data conferences. He is the winner of the DAMA (Data Administration Management Association) International Professional Achievement Award and the DAMA International Community Award. He has given many keynotes and has received the highest speaker rating at several international conferences. Mr. Silverston's company, Universal Data Models, LLC, http://www.universaldatamodels.com provides consulting, training, publications, and software to enable information integration as well as people integration. - He is also a personal and corporate mindfulness coach, trainer, and teacher and has studied and taught many forms of spirituality and life development skills for over thirty years. He has attended, staffed and/or led hundreds of days of silent, intensive retreats as well as dozens of life development workshops. After intensive practice in Zen, he was ordained as a Zen Priest in 2011. ‘Kensho’ Len Silverston provides ongoing ‘Zen With Len’ (http://www.zenwithlen.com) individual and corporate coaching, seminars, meditation gatherings, and retreats.

scroll to top