New APPS

By Gordon Hull

As a criterion for algorithmic assessment, “fairness” has encountered numerous problems.  Many of these emerged in the wake of ProPublica’s argument that Broward County’s pretrial detention system, COMPAS, was unfair to black suspects.  To recall: In 2016, ProPublica published an investigation piece criticizing Broward County, Florida’s use of a software program called COMPAS in its pretrial detention system.  COMPAS produced a recidivism risk score for each suspect, which could then be used in deciding whether someone should be detained prior to their trial.  ProPublica’s investigation found that, among suspects that did not have a rearrest prior to their trial, black suspects were much more likely to have been rated as “high risk” for rearrest than white suspects.  Conversely, among suspects who were arrested a second time, white suspects were more likely to have been labeled “low risk” than black ones.  The system thus appeared to be discriminating against black suspects.  The story led to an extensive debate (for an accessible summary with cites, see Ben Green’s discussion here) over how fairness should be understood in a machine learning context.

The debate basically showed that ProPublica focused on outcomes and demonstrated that the system failed to achieve separation fairness, which is met when all groups subject to the algorithm’s decisions receive the same false negative/positive rate.  The system failed because “high-risk” black suspects were much more likely than white to be false positives.  In response, the software vendor argued that the system made fair predictions because among those classified in the same way (high or low risk), both racial groups exhibited the predicted outcome at the same rate.  In other words, among those classified as “high risk,” there was no racial difference in how likely they were to actually be rearrested.  The algorithm thus satisfied the criterion of sufficiency fairness.  In the ensuing debate, computer scientists arrived at a proof that, except in very limited cases, it was impossible to simultaneously satisfy both separation and sufficiency fairness criteria.

In the meantime, on the philosophy side, Brian Hedden has argued that a provably fair algorithm could nonetheless be shown to potentially violate 11 of 12 possible fairness conditions.  In a response piece, Benjamin Eva showed the limits of the twelfth with a different test and proposed a new criterion:

(more…)

People make snap judgments about those they see the first time – mentally categorizing someone as friendly, threatening, trustworthy, etc.  Most of us know that those impressions are idiosyncratic, and suffused with cultural biases along race, gender and other lines.  So obviously I know what you’re thinking… we need an AI that do that, right?  At least that’s what this new PNAS paper seems to think (h/t Nico Osaka for the link).  The authors start right in with the significance:

“We quickly and irresistibly form impressions of what other people are like based solely on how their faces look. These impressions have real-life consequences ranging from hiring decisions to sentencing decisions. We model and visualize the perceptual bases of facial impressions in the most comprehensive fashion to date, producing photorealistic models of 34 perceived social and physical attributes (e.g., trustworthiness and age). These models leverage and demonstrate the utility of deep learning in face evaluation, allowing for 1) generation of an infinite number of faces that vary along these perceived attribute dimensions, 2) manipulation of any face photograph along these dimensions, and 3) prediction of the impressions any face image may evoke in the general (mostly White, North American) population”

Let’s maybe think for a minute, yes?  Because we know that people make these impressions on unsound bases!

First, adversarial networks are already able to produce fake faces that are indistinguishable from real ones.  Those fake faces can now be manipulated to appear more or less trustworthy, hostile, friendly, etc.  When you make fake political ads, for example, that’s going to be useful.  Already 6 years ago, one “Melvin Redick of Harrisburg, Pa., a friendly-looking American with a backward baseball cap and a young daughter, posted on Facebook a link to a brand-new website,” saying on June 8, 2016 that “these guys show hidden truth about Hillary Clinton, George Soros and other leaders of the US. Visit #DCLeaks website. It’s really interesting!” Of course, both Melvin Redick and the site he pointed to were complete fabrications by the Russians.  Now we can make Melvin look trustworthy, and Clinton less so.

Second, the ability to manipulate existing face photos is a disaster-in-waiting.  Again, we saw crude efforts with this before – making Obama appear darker than he is, for example.  But here news photos could be altered to make Vladimir Putin appear trustworthy, or Mr. Rogers untrustworthy.  This goes nowhere good, especially when combined with deepfake technology that already takes people out of their contexts and puts them in other ones (disproportionately, so far, women are pasted into porn videos, but the Russians recently tried to produce a deepfake of Zelensky surrendering.  Fortunately that one was done sloppily).

Third, and I think this one is possibly the scariest, what about scanning images to see whether someone will be assessed as trustworthy?  AI-based hiring is already under-regulated!  Now employers will run your photo through the software and make customer-service hiring decisions based on who customers will perceive as trustworthy.  What could go wrong?

All of this of course assumes that this sort of software actually works.  The history of physiognomic AI, which uses all sorts of supposedly objective cues to determine personality and which is basically complete (usually racist) bunk suggests that the science is probably not as good as the article acts like.  So maybe we’re lucky and this algorithm does not actually work as advertised.  Of course, the fact that AI software is garbage doesn't preclude its being used to make people's lives miserable.  Just consider the bizarre case of VibraImage.

But don’t worry.  The PNAS authors are aware of ethics, noting that “the framework developed here adds significantly to the ethical concerns that already enshroud image manipulation software:”

“Our model can induce (perceived) changes within the individual’s face itself and may be difficult to detect when applied subtly enough. We argue that such methods (as well as their implementations and supporting data) should be made transparent from the start, such that the community can develop robust detection and defense protocols to accompany the technology, as they have done, for example, in developing highly accurate image forensics techniques to detect synthetic faces generated by SG2. More generally, to the extent that improper use of the image manipulation techniques described here is not covered by existing defamation law, it is appropriate to consider ways to limit use of these technologies through regulatory frameworks proposed in the broader context of face-recognition technologies.” 

Yes, the very effective American regulation of privacy does inspire confidence!  Also, “There is also potential for our data and models to perpetuate the biases they measure, which are first impressions of the population under study and have no necessary correspondence to the actual identities, attitudes, or competencies of people whom the images resemble or depict.”

Do you think?  As Luke Stark put it, facial recognition is the “plutonium of AI:” very dangerous and with very few legitimate uses.  This algorithm belongs in the same category, and should similarly be regulated like nuclear waste.  For example, as Ari Waldman and Mary Anne Franks have written, one of the problems with deepfakes is that the fake version gets out there on the internet, and it is nearly impossible to make it go away (if you even know about it).  Forensic software gets there too late, and those without resources aren’t going to be able to deploy it anyway.  Lawsuits are even less useful, since they're time-consuming and expensive to pursue, and lots of defendants won't be jurisdictionally available or have pockets deep enough to make the chase worth it.  In other words, not everybody is going to be able to defend themselves like Zelensky, who both warned about deepfakes and was able to produce video of himself not surrendering.  In the meantime, faked and shocking things generally get diffused faster and further than real news.  After all, “engagement” is the business model of social media.  Further, to the extent that people stay inside filter bubbles (like Fox News), they may never see the forensic corrections, and they probably won’t believe the real one is real, even if they do.

And as for reinforcing existing biases, Safiya Noble already wrote a whole book on how algorithms that guess what you’re probably thinking about someone can do just that.

By Gordon Hull

Machine Learning (ML) applications learn by repetition.  That is, they come to recognize what, say, a chair looks like, by seeing lots of images of chairs that have been correctly labeled as such.  Since the machine is trying to figure out a pattern or set of characteristics that distinguish chairs from other objects, the more chairs it sees, the better it will perform at its task.  This collection of labeled chairs is the algorithm’s “training data.”   There are notorious problems with bias in training data due to underrepresentation of certain groups.  For example, one study found that datasets designed to train ML to recognize objects performed poorly in developing countries, most likely due to underrepresentation of images from those places; when combined with pictures labeled in English, and the fact that standard Western commodity forms of household objects might look very different in the developing world, the ML was stumped.  Most famously in this country, facial recognition software performs best at identifying white men and worst at identifying Black women.  ImageNet, which is widely used for object recognition purposes, employees a hierarchy of labels that include calling a child wearing sunglasses a “failure, loser, non-starter, unsuccessful person” but also differentiates between assistant and associate professors.  Despite these and many other problems, massive datasets are essential for training ML applications.

For this reason, datasets have been called “infrastructural” for machine learning, defined as follows:

“Infrastructure is characterized, we argue, by a set of core features: it is embedded into, and acts as the foundation, for other tools and technologies; when working as intended for a particular community, it tends to seep into the background and become incorporated into routines; the invisibility of infrastructure, however, is situated – what is natural or taken for granted from one perspective may be highly visible or jarring from another; though frequently naturalized, infrastructure is built, and thus inherently contextual, situated, and shaped by specific aims.”

If AI is cars and buses and trains that do what we want, the datasets it trains on shape the roads and paths where those roads go, provide their material basis, become thereby incorporated into higher level routines like search, and tend to disappear into the background when not actively used.  But just like other examples of infrastructure – say, the bridges over the Long Island Freeway – infrastructure can embed priorities and affordances.  In this sense, dataset infrastructures have a politics, and serve as platforms on which applications are built.

(more…)