The Review Board Met on a Tuesday

The deck had forty slides and Ines had read all of them twice, once on the train and once at her desk with a coffee, and both times she'd stopped on slide 27 and stayed there. "Adverse impact ratio 0.84 across all protected categories," said Marcus, who had built the thing, and who was standing at the front of the room with the particular energy of someone who has not been told no in a while. "Above the four-fifths threshold. We're compliant." "We're compliant on the aggregate," Ines said. "Yes." "Slide 27." Marcus clicked back. Slide 27 was a table. It broke the ratio down by category and then, in a smaller table underneath that someone had clearly added late, by category intersected with age band. "Women over fifty," Ines said. "0.71." "That's a small cell. Forty-two applicants in the validation set. The confidence interval is enormous." "The confidence interval is enormous," Ines agreed. "Which means we don't know. Which is different from it being fine." The other three people in the room had said very little. Priyanka from Legal had a pen she kept capping and uncapping. Tom from the business side wanted to ship before the end of quarter and had said so, pleasantly, twice. Dr Okafor, the external member, was reading slide 27 on her own laptop and had not looked up in some time. "What would you need," Marcus said, and it was to his credit that he said it as a real question. "Another two hundred applicants in that cell. Or a hold-out that's stratified so the cell is big enough to say something." "That's six weeks." "Probably eight." Tom uncrossed and recrossed his legs. "Can we ship with a monitor? Flag the cell, review manually, pull it if it drifts?" "Who reviews manually?" "Recruiting." "Recruiting is the thing we're replacing," said Dr Okafor, without looking up. It was the first thing she'd said in twenty minutes. "That's the entire business case. You can't propose the people you're making redundant as the safety net." Silence. The good kind, Ines thought, the kind where people are actually hearing something.

Marcus sat down. "Okay. Say it's real. Say the model's doing something with age. What is it doing? The features are anonymised, there's no date of birth, we stripped graduation years." "Gaps," said Priyanka. Everyone looked at her. She capped the pen. "Career gaps. My sister took four years out for her kids and went back at forty-six. Her CV has a hole in it you could drive a bus through. If the model learned that gaps predict something, it learned age, and it learned motherhood, and it did it without ever being told either." Marcus didn't say anything for a while. Then he said, "That would show up in the SHAP values." "Would it? Or would it show up as 'tenure at most recent role', which is right there in your top ten?" He pulled up the feature importances. Tenure at most recent role was number four. Ines let it sit. She'd learned, on this board, that the worst thing she could do at this moment was say anything that sounded like winning. "Eight weeks," Marcus said eventually. Not to Tom. To the table. "We stratify the hold-out, we add a gap-normalised tenure feature, we re-run. If the cell comes back above 0.8 with a real interval, we ship." "And if it doesn't?" "Then it doesn't." Tom looked like he wanted to say something about quarter. He didn't. Dr Okafor closed her laptop. Afterwards, in the corridor, Marcus caught up with Ines by the lifts. "For what it's worth," he said, "I did see 27. I put the second table in myself at midnight. I think I was hoping someone else would say it." "Someone else did." "Yeah." He looked at the lift doors. "Next time I'll say it." The lift came. They rode down in silence, which was fine.