My co-researcher and I built a framework to try to break that cycle. The idea comes from fairness research in healthcare AI, where a very similar problem shows up: models trained on years of biased treatment data end up repeating the same mistakes instead of correcting them.
The method is called adversarial debiasing, which sounds more complicated than it actually is. Picture two students working through the same problem set. One student is trying to figure out how urgently a district needs aid. The other student is only shown that first student's reasoning, nothing else, and has to guess which district it's about. If the second student can guess correctly too often, that tells you the first student was leaning on location as a shortcut instead of actually judging need. So during training, we penalize the model every time that shortcut shows up, over and over, until it starts paying attention to things that actually matter: rainfall, poverty rate, population density, how much damage there really was.
We tested this against real numbers from Bangladesh''s official Post-Disaster Needs Assessment, covering 87 upazilas in 11 districts hit by the 2022 floods. The gap between how well-off and how poor districts scored dropped by 41.6 per cent. Regional fairness gaps fell by 43.2 per cent. And once we corrected for bias, 70.6 per cent of upazilas ended up with a different priority ranking than before.