Over the years, I’ve learned that for agile teams to succeed, everyone – from team members to those who interact with the team – must work together in an agile manner. It’s a fact borne out by not just my own experience, but that of thousands of other agile adherents. Although developers have quickly taken to agile software development techniques, other communities within the IT industry have not – in particular, data professionals. Many agile techniques are applicable to data-oriented development –visit www.agiledata.org for details– and in this newsletter, I’ll explore an important one: agile data modeling.
Please note: Scott Ambler is currently running the 2018 edition of his Data Quality Survey at:
https://www.surveymonkey.com/r/dataQuality2018.
Results from previous data quality studies are openly and freely shared at http://Ambysoft.com/surveys. Scott will also be writing up his analysis of the study for a future TDAN.com article. The survey is 11 questions long and will take you about 5 minutes to complete. It would be a huge help to Scott and to the industry, if you could invest a few minutes of your valuable time filling out the survey. Thank you very much in advance.
Data modeling is the act of exploring data-oriented structures, often to create some form of diagram and supporting documentation describing those structures. Evolutionary data modeling is data modeling performed in an iterative and incremental manner, and agile data modeling is evolutionary data modeling done in a collaborative manner.
In a more than 10-year-old issue of Software Development, I show how to create a physical data model in an agile manner, explaining how the model evolves throughout the project. In those columns, I identified several important lessons that all data professionals should take to heart:
- Be agile, yet still support the needs of the enterprise. Agility and fulfilling enterprise concerns aren’t contradictory goals. Data professionals often lament that developers don’t take the bigger picture into account, and often, that’s a fair assessment. However, that doesn’t mean you need to model everything to the -nth degree before you start coding. Instead, seek the sweet spot somewhere between the extremes. Invest some time thinking about, and then acting on, the bigger picture, but don’t hold up development. Better yet, work collaboratively with developers to pass on your “big picture” skills – and pick up some “small picture” skills in return.
- Travel light. Agile data modelers write just enough documentation to get the job done and no more. Gone are the days when modelers took weeks or even months to write a comprehensive data model. Now, you typically have minutes to capture the essential information before you must move on to other tasks.
- Agile data models are just barely good enough. Agile developers solve today’s problem today and trust that they can solve tomorrow’s problem tomorrow. The same can be said of agile data modelers: You know that you don’t have to model all of the potential data attributes an entity might need; instead, you model only the information required today and recognize that you can evolve your model when new requirements demand it.
- Agile data models can and should follow your corporate standards. Agile Modeling (AM) includes a specific practice for adopting and following modeling standards and guidelines. Critical data modeling standards focus on naming conventions for tables and their columns, as well as conventions for writing stored procedures and triggers. These standards should be straightforward, simple, and adequately described so that the team can learn and then follow them.
- Trying to define all the requirements up front is a risky proposition. One of software development’s bitter truths is that, like it or not, requirements change. People change their minds, don’t fully understand what they want at first, and sometimes external events occur that necessitate changes. If you try to baseline requirements too early in the life of a software development project, you’ll simply guarantee that you will build the wrong thing. Embrace change and adopt techniques that allow you to react effectively to it.
- It isn’t enough to specialize in one aspect of technology. I shouldn’t be talking about just data professionals and programmers – I should widen my focus to include all IT professionals, because the industry still embraces the antiquated Taylorist concept of specialists. The most effective developers are generalizing specialists, people with one or more specialties (such as data modeling or programming), a general knowledge of software process, and, better yet, a good understanding of the domain in which they work. Expand your horizons and pick up some non-data skills. When you do, you’ll be more effective because you can relate better to your coworkers. Remember, agile practitioners work in a highly collaborative manner, so getting good at collaborating with others is a critical skill.
- Some tasks that traditionalists avoid, agile practitioners choose to embrace. Agile practitioners realize that refactoring is a critical skill. Agile database administrators (DBAs) realize that database refactoring is a critical skill they need to evolve database schemas. Structural database refactorings, such as moving a column from one table to another, are quite common and often require data migration. Traditionalists believe that data migration is hard – they’re right about that – and often try to avoid it whenever possible. Thus, they’re not very good at migrating data. Agile practitioners, on the other hand, choose to get good at critical activities like these, and become more effective as a result.
Evolutionary, and more often agile, approaches are the norm for modern development projects. Data professionals can also work in an agile manner – if they choose to.
Again, please respond to Scott’s survey at https://www.surveymonkey.com/r/dataQuality2018.
