Data alone should never drive policy.
After you, our visitors, participate in our educational games, we offer you the opportunity to complete voluntary surveys that aim to elicit your perspective on “unfair fairness” and what to do about it. We give visitors this opportunity because we believe that your voices matter. We want to know what you think, and we think that these perspectives can be a valuable datapoint to reference as part of efforts to redress unfair fairness. With that said, we also believe that this data can never be the whole story. While survey research data can help us begin to think about how to approach technology-related harms or moral dilemmas, it can never responsibly be cited as the sole justification for important moral or policy decision-making. We strongly believe that we must see this data for what it is — data, from a particular subset of the population — as opposed to a definite “ground truth” that ought to guide society’s relationship with technology.
There are multiple applications to consider.
We know that “unfair fairness” can manifest in different contexts, whether in the provision of healthcare, education delivery, the justice system, the labor market, and more. Even within these spaces, “unfair fairness” can perpetuate varying degrees of harm, with differing effects for different populations. We firmly believe that this granularity and these nuances matter. Exploring both the breadth and depth of harm is key. That’s why our educational games explore two very different settings with different consequences; moreover, for us, these two games are a starting point. In the coming months, we hope to release additional games that highlight the myriad of ways that “unfair fairness” can inadvertently enable harm.
Education before experimentation.
Finally, we believe that user education must come before user experimentation. That is, as much as we want to learn from you, we want you to learn from us first. That’s why our games are educational tools first and foremost, with the option to participate in surveys after viewing educational content. We firmly believe that we have a responsibility to educate the public, policymakers, and civil society groups as to the realities of “unfair fairness”; more broadly, we believe that this education is key, particularly as calls for more fair, equitable, and safe algorithms grow. Indeed, the first step to tackling the harms of leveling down is ensuring that more people know about it. That, above all, is the mission of our tool.