My late grandfather liked westerns. He liked that the bad guy was always easy to spot. The bad guy was the fella wearing the black hat. But what about the guy in the white hat who shot all those people? Was he not a bad guy as well?
In information management we are at a crossroads where, as a profession, we need to make some serious grown-up choices about the color of hat we are wearing. The problem is that we can’t see our hats. They are on our heads, and the only feedback we will get about whether we are a white hat or a black hat is our impact on the outcomes that people experience as a result of the information and process outcomes we enable.
The problem is, because we can’t see what color hats we are wearing, we often think we are the good guys. Facebook wants to connect people, and Google wants to index the sum total of human knowledge. But to get there, we will have to wade through a morass of fake news, targeted advertising, and analytics that are invasive to the very nature of our individual being.
Take Facebook (a Rodney Dangerfield joke beckons, but I’ll ignore it) ….
In 2015, a Facebook engineer identified unusual trends in the sources of shared links on the platform. Particular US-based sites that were essentially aggregators for alt-right content were among the most shared content on the platform. This was shared with fellow engineers in an internal chat group. Nothing happened.
Around the same time, an executive in Twitter became concerned about the patterns in retweet data and link sharing data through that platform as well. They raised their concerns. Nothing happened.
Well, that’s not entirely true. A US President got elected and the UK voted to leave the European Union. “Fake news,” combined with an algorithmic filter bubble and psychometric targeting of adverts and link sharing, is clearly an increasingly causal factor in both of these outcomes (I will not discuss the validity of either electoral decision, other than to say, “What the flaming hell were you thinking?” to either US or UK based readers.)
To put it another way: the technology we have developed to enable us to better sell fast cars to middle aged men and insurance policies to prudent women have been co-opted to do the exact things they were designed to do: improve targeting of adverts, increase ROI, encourage people to buy the product. What a wonderful and fantastic data driven nirvana we have created.
But we have been here before. Technology is essentially ethically neutral. The same technologies are used to create a scissors for a left-handed person as a right-handed person. But if you are a right-handed person trying to use a left-handed scissors all I can say is: welcome to my world (I’m left handed). While the technologies are themselves neutral, the process outcome experienced because of the combination of those technologies is biased against approximately 10% of the population.
Now, imagine you are dealing with an AI process that has ingested court rulings for the last 100 years to identify the likelihood of guilt or innocence of a defendant in a criminal prosecution. Would you be surprised to learn that AI sentencing support systems are inherently biased towards harsher sentences for persons of certain ethnicities? Would it help address your surprise if you learned that the training data used to teach the algorithm what to think used historical case transcripts and sentencing data from periods when people of certain ethnicities were more likely to receive a harsher sentence than defendants of a different ethnicity charged with the same or similar offences?
One of my favorite movies is Scent of a Woman starring Al Pacino. The climax of the movie sees Lt. Colonel Frank Slade (retired) taking to his feet in defence of his friend, Charlie Simms. In his speech, Slade advises the collected disciplinary committee of Baird School:
“Makers of men. Creators of leaders. Be careful what kind of leaders you’re producin’ here”
Data is risky business. Often the risk is an impact on others. Often the root cause of that risk is the ethical frame or perspective of the people in the organization. Data geeks, data leaders, be careful what kind of leaders we’re producin’ here.
The emerging weeping and moaning about how Facebook’s quantitative metrics for sharing which drive their (advertising) business model have been co-opted, captured, and gamed by extremist factions and extreme opinions, or the discussions about how Twitter might have amplified the reach of controversial figures promoting less than savoury views through their “verified” program and associated algorithms is, frankly, not dissimilar to the tears of a person who filled their house with gasoline and dry kindling and then had a cigar to celebrate.
However, these are the most public manifestations of a broader analytics malaise that has affected society, and a broader problem in data and data analytics. We have increasingly lost touch with the objectives of the people whose data we are processing. Organizations have embraced analytics and what to emulate in their widget and potato manufacturing facility the same type of analytics capabilities that Facebook and Google boast about, regardless of the operational, technical, or cultural fit of such technology. AI processes that carry in their DNA the inherent biases of their creators increasingly make decisions that have significant impacts on the rights and freedoms of people.
Often, these are unintended consequences, such as accidentally creating the media bubble and echo chamber that allows controversial election decisions in political processes to arise. Nothing to see here. But really, we just wanted to connect everyone. And share this cool cat gif with everybody…
Are we the bad guys?
We are not. But history may not remember us as such. That’s why we now need to urgently address the challenges of Ethical Information Management. If we don’t act now, Moore’s law suggests it will be too late.
Makers of data professionals. Creators of Leaders. Be careful what kind of data leaders we’re producin’ here.
