Data Governance vs. Model Governance and AI Ethics
It's one thing having data governance, but we need governance across the whole data value chain - from data capturing to data use and presentation. I won't delve too much into the ethics of data presentation and visualisation. I will instead point you in the direction of some fantastic books on the topic, such as How Charts Lie: Getting Smarter about Visual Information by Alberto Cairo. It was one of my top Data Analytics Book Recommendations.
I will, however, speak about data analytics model governance, as this is an important topic in a nascent analytics environment (and in a lot more detail than in my previous post on AI Governance).
A few areas should be considered in great detail when it comes to analytics governance. These are covered in this blog post on AI Ethics, Data Science Ethics and Model Governance.
What is Data Science and AI Ethics?
What data is used for model building and use? How was this data collected? Was it collected in accordance with regulations? Is it used in accordance with the purpose for which it was collected? If not, is there a way to make the data appropriate by analysing the data on an aggregated level or anonymised?
Model ethics is a broad topic that covers many areas, from data bias to understanding model decision making. We have seen institutions globally develop frameworks for model governance and ethics, and UNESCO member states have adopted a global agreement on the Ethics of Artificial Intelligence.
Singapore has also adopted an AI Governance framework that is well worth a read.
Data Should be Aggregated and Anonymised
In some instances, individual data is far more critical for both models and decision making. For example, a company assessing the effectiveness of medication in a trial may require a person's age, weight, gender, nationality and previous medical history. In this case, personal information (deemed personally identifiable information under some regulations) needs to be respected and processed not only in accordance with the law and data privacy legislation but also in a way such that the client's trust is maintained.
Model Reliability
Data scientists should ensure their models are reliable. Reliability means that models can provide an accurate and repeatable output for a particular input. While models are never 100% correct (there is always an element of error, especially for predictive models), one should receive the same output from a model given the same inputs.
Model Auditability
Are models auditable? Coming from a financial modelling background, I am very used to the models I build being audited by a team of professionals. However, it is becoming more standard practice to audit data science models before they go into production. This means that it should be easy to audit models with clear methodologies, documentation and model commentary.
Model audits will go a long way to ensuring ethical AI.
Transparent Decision Making
Is decision making transparent? This is becoming increasingly complex with the advent of AutoML and black-box models. Furthermore, with the proliferation of AI solutions across corporations, model transparency is becoming increasingly important. Breaking transparency down, it means that we can interpret and explain the results of a model (it may also refer to model fairness and reproducibility).
Do We Understand Data and Model Bias?
Are we implementing models with appropriate judgement and care, with a careful note of how decisions may prejudice certain companies, individuals or groups of the population? Data scientists need to be aware of the data they are using to train their models, as data with a human element in the decision-making process may contain bias. Data scientists should be auditing data used for model building as the first step in reducing model bias. In addition, model outputs should be carefully monitored in the Model-Ops process. We need to ensure there is no unintended bias in the models we create. We also need to ensure that those developing models and working with data fully understand the technology.
Alignment
Are the models developed being used in accordance with their development goals and objectives? It is essential to note the use cases for which a model was designed and be very careful when using models for unintended use cases. For example, the model developer may have considered certain assumptions or made methodology decisions that no longer apply to the new use case.
AI Ethics Governance, Committees and Privacy by Design
An excellent methodology explained by Dr Rumman Chowdhury, Accenture's lead for responsible AI, explains AI ethics and governance in terms of fire wardens versus firefighters. Whereas firefighters arrive to put out a blaze and sometimes aren't even successful, fire wardens instead ensure fire precautions are in place and immediately raise the alarm when a fire starts. Similarly, we need to empower people working with data to become AI ethics fire wardens who raise issues as soon as they are noticed - preferably before models are put into production and cause unnecessary harm.
Another concept for model designers to be aware of is Privacy by Design. Privacy by Design means that privacy is considered in the model design and development process rather than being implemented as an afterthought.
I believe if we are conscious of all of these elements, our models are likely to be better suited to the task at hand while ensuring they are as ethical as possible.