Chapter 27 — Data Science Ethics, Privacy, and Responsible AI

Technical skill without ethical guardrails causes harm. This final chapter covers fairness and bias, privacy, transparency and accountability, and the responsible-AI practices every data scientist should apply — with runnable demonstrations of bias measurement and a simple privacy technique.

Learning Objectives

Prerequisites / Imports

In [1]:
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

1 Why Ethics Matters in Data Science

Models make decisions about people — loans, hiring, sentencing, healthcare. Bias in data or models can entrench inequality. Privacy breaches harm individuals. Responsible AI is the practice of building systems that are fair, private, transparent, and accountable.

2 A Synthetic Biased Hiring Model

We simulate hiring data where one group is systematically disadvantaged, then train a model that inherits that bias.

In [1]:
rng = np.random.default_rng(1)
n = 800
group = rng.choice(['A','B'], n, p=[0.6,0.4])
# Base qualification, but group B is unfairly downgraded in the training labels
qualified = (rng.random(n) < 0.5 + 0.3*(group=='A') - 0.1*(group=='B')).astype(int)
X = pd.DataFrame({'score': rng.normal(70, 10, n).round(1), 'group': group})
X['group'] = (X['group']=='A').astype(int)  # encode A=1, B=0
y = qualified
clf = LogisticRegression().fit(X, y)
pred = clf.predict(X)
print('overall accuracy:', round(accuracy_score(y, pred), 3))

overall accuracy: 0.728

3 Measuring Fairness

Demographic parity: selection rates are equal across groups. Equal opportunity: true positive rates are equal across groups. We compute both gaps.

In [1]:
Xdf = X.copy()
Xdf['group_name'] = np.where(Xdf['group']==1, 'A', 'B')
Xdf['pred'] = pred
Xdf['qualified'] = y
selection = Xdf.groupby('group_name')['pred'].mean()
tpr = Xdf[Xdf['qualified']==1].groupby('group_name')['pred'].mean()
print('Selection rate (demographic parity):'); print(selection.round(3))
print('\nTPR (equal opportunity):'); print(tpr.round(3))
print('\nDemographic parity difference:', round(float(selection.max()-selection.min()), 3))
print('Equal opportunity difference:', round(float(tpr.max()-tpr.min()), 3))

Selection rate (demographic parity):
group_name
A    1.0
B    0.0
Name: pred, dtype: float64

TPR (equal opportunity):
group_name
A    1.0
B    0.0
Name: pred, dtype: float64

Demographic parity difference: 1.0
Equal opportunity difference: 1.0

4 Mitigating Bias

A simple remedy is to use group-aware thresholds that equalize selection rates.

In [1]:
# Choose per-group thresholds to equalize selection rates
thr_a, thr_b = 0.5, 0.35
Xdf['fair_pred'] = ((Xdf['group']==1) & (Xdf['score']>=68) |
                     (Xdf['group']==0) & (Xdf['score']>=60)).astype(int)
fair_sel = Xdf.groupby('group_name')['fair_pred'].mean()
print('Selection rate after mitigation:'); print(fair_sel.round(3))
print('New parity difference:', round(float(fair_sel.max()-fair_sel.min()), 3))

Selection rate after mitigation:
group_name
A    0.574
B    0.854
Name: fair_pred, dtype: float64
New parity difference: 0.28

5 Privacy and Differential Privacy

Releasing summary statistics can leak individual data. Differential privacy adds calibrated noise so the result barely changes whether any one person is in the dataset.

In [1]:
rng = np.random.default_rng(0)
true_mean = 72.4
# Noisy release: add Laplace noise scaled to sensitivity/epsilon
epsilon = 1.0
sensitivity = 1.0  # conceptual
noise = rng.laplace(0, sensitivity/epsilon)
released = round(float(true_mean + noise), 2)
print('true mean:', true_mean)
print('differentially private release:', released)
print('error:', round(abs(released-true_mean), 2))

true mean: 72.4
differentially private release: 72.72
error: 0.32

6 Transparency and Accountability

A model card documents a model's intended use, data, performance, and ethical limitations. Accountability means someone is responsible for outcomes. Explainability helps users trust and audit predictions.

7 Regulation and Principles

Case Study: A Responsible-AI Checklist

Apply a short checklist to the biased hiring model from earlier.

In [1]:
checklist = [
    ('Representativeness', 'Are both groups represented in sufficient numbers?'),
    ('Fairness metrics', 'Are parity and equal-opportunity gaps measured and within tolerance?'),
    ('Privacy', 'Is personal data minimized and access controlled?'),
    ('Transparency', 'Is there a model card and explanation for decisions?'),
    ('Accountability', 'Is there a named owner and a remediation process?'),
    ('Human oversight', 'Is a human reviewing high-impact outcomes?'),
]
for item, question in checklist:
    print(f'- {item}: {question}')

- Representativeness: Are both groups represented in sufficient numbers?
- Fairness metrics: Are parity and equal-opportunity gaps measured and within tolerance?
- Privacy: Is personal data minimized and access controlled?
- Transparency: Is there a model card and explanation for decisions?
- Accountability: Is there a named owner and a remediation process?
- Human oversight: Is a human reviewing high-impact outcomes?

Exercises

  1. Define bias and give one example in a data-science context.
  2. Train a model on biased data and compute its demographic parity difference.
  3. Compute equal-opportunity TPR for two groups.
  4. Apply per-group thresholds to reduce a parity gap.
  5. Explain differential privacy in one sentence.
  6. Add Laplace noise to a mean and report the released value.
  7. Draft a short model card for a churn model.
  8. List three data-minimization practices.
  9. Explain the difference between accountability and explainability.
  10. Apply the responsible-AI checklist to a project of your choice.

Python Data Science: From Foundations to Applications — Chapter 27