There is always bias (or, binary numbers are not the villain)

Twain Liu just wrote a piece on Quartz, entitled "Aristotle’s binary philosophies created today’s AI bias". This article is riddled with buzzword-laden arguments by feeling, such as summarizing "the very system on which all modern technology is built" as:

1 = true = rational = right = male
0 = false = emotional = left = female

The false dichotomies built into this asseveration rattle the brain. Indeed, the entire essay has this flavor, and it is not even false.

But since the piece has gotten a fair bit of attention, I feel the need to respond to the key claim of the piece. The entire argument rests on the dual assertion that the fact that computers use binary numbers (1s and 0s) as the basis for their operation is (a) based on Aristotle's (elitist, sexist) philosophy, and (b) the fundamental reason why algorithmic systems are biased. Hence, new computer systems not based on Aristotelian "binary" logic can be universal, unbiased pure goodness.

Well.

First off, the "computers are binary and essentially invented by Aristotle" claim is a load of argle-bargle and pure applesauce. (Clickbait headlines in the Atlantic notwithstanding.) When electronic computers were first being developed in the 40s and 50s, different systems were experimented with (including ternary (three-valued) logic), but binary was the most practical for a simple reason. With binary logic, you can represent a 0 by "voltage close to 0" and 1 by "voltage close to maximum". When you introduce more possible values, the system becomes more sensitive to noise, and hence less reliable. (There are other technical reasons for binary computing, and there are some other reasons to prefer ternary systems, but this is enough for my purposes.)

Now, to bias. Binary numbers have nothing whatsoever to do with algorithmic bias. The binary number system does not limit you to using only 1 or 0 for values you need to represent (after all, you could not specify an address to Google Maps just as a 1 or 0, say). Indeed, you can represent as many different values as you like by stringing bits together. You can have as many categories of whatever you like as you like. Any computer scientist would recognize this aspect of the claim to be laughable.

Algorithmic bias is due to the simple fact that all decision systems have biases. (Indeed, it is impossible to learn anything from experience without some sort of bias.) No real system has perfect information, and any decision made on the basis of imperfect information is biased in some way. The question is not "Can we create unbiased algorithms?" but "Do we know what our algorithm's biases are?" and "Can we mitigate the ones we do not like?"

Utopian visions like Ms. Liu's that if we just had the right philosophy, we could build computer systems that will be universal and unbiased, pure purveyors of algorithmic goodness, are false and actually dangerous. They promote the technocratic idea that there are unbiased algorithms out there, if we could just find them, and so keep our focus on algorithmic development.

However, bias is inevitable. The way to combat pernicious bias is through continuous monitoring to discover instances of problematic bias and exercise of good judgment to adjust systems (whether algorithms, training data, how the systems are used, etc.) to mitigate the bad effects while maintaining the good ones. The proper way to combat algorithmic bias (which some are working on) is to develop better ways of detecting and characterizing such bias, and the societal institutions and incentives that enable dealing with deleterious such biases. (And this leads into questions of value systems and politics, which cannot be avoided in this arena. There is no royal road.)

Visions of simple solutions derived from proper thinking are seductive. But the necessary condition for developing and maintaining diversity-enhancing technologies will be, I'm afraid, eternal vigilance.

Open Letter to PSU VP of Research Mark McLellan

Peter Boghossian is an assistant professor of philosophy at Portland State University. He recently, with two non-academic colleagues, published an account of an effort they made to probe peer-review methods within certain fields of inquiry that they term "grievance studies". Briefly, they wrote academic articles based on fanciful theories and hypotheses, matching as well as possible the style of writing and argumentation in the fields they addressed, and managed to get several articles accepted at leading journals. After doing so, they published their account of their effort, revealing the deception. This, they argue, has implications regarding the reliability of peer-review in those fields and perhaps regarding the legitimacy of the fields' methods themselves. I express no opinion regarding their study or conclusions.

What I am writing about is the response of Boghossian's institution, which was to investigate him for research impropriety, and ultimately to determine that Boghossian's "efforts to conduct human subjects research at PSU without a submitted nor approved protocol is a clear violation of the policies of [his] employer."

Unless the facts are substantially different from what has been published, this case raises concern about academic freedom and freedom of inquiry. It is debatable at best whether Boghossian's work required IRB review at all, and even if it had, the situation is not one to rise to the level of research malfeasance. If any readers have more information about the case, please let me know.

Below is the letter I wrote about the case this morning to Prof. Mark McLellan, Vice President for Research and Graduate Studies at Portland State University. Obviously, I speak only for myself, not for my institution.


Dear Prof. McLellan,

I have read with some concern of the investigation and conviction of Prof. Peter Boghossian of unethical research practices. This is a serious charge and as such warrants proper due process and full consideration of all relevant facts and circumstances. For the reasons I will detail below, I believe this not to have been the case here, and I urge reconsideration of this case, for the sake not only of Prof. Boghossian, but rather of the reputation of Portland State and the institution of the IRB.

I also note that, generally speaking, a first accusation or offense of this kind (lack of IRB review for research that did not result in proven tangible harm) will result in a warning and discussion with the faculty member, before proclaiming a determination that they have unambiguously violated ethical norms and university policy. Consider, for example, the very long time and repeated discoveries of egregious and intentional research malfeasance (far beyond anything that Prof. Boghossian is accused of) that were necessary before Dr. Wansink was finally censured by Cornell.

In the case of Prof. Boghossian, there are three essential questions whose answers would determine whether the project was subject to IRB review, and whether the project as conducted was unethical in any way.

First, was the project "research"? I believe the answer here is indeed "yes", since the project was undertaken to develop knowledge and disseminate it, in this case about the peer-review practices in certain fields of inquiry.

Second, did the project involve "human subjects"? Clearly, the fabricated research studies used as experimental probes did not. The reviewers of these articles, while part of the phenomenon under study (the "peer-review system") also were not human subjects per PSU's Human Subjects Research Review Committee Policy, which states:

A human subject is a living individual about whom an investigator obtains data, either from intervention or interaction with the individual, or through records which contain identifiable private information.

Since the peer reviewers were entirely anonymous and not identifiable, the investigators cannot be considered to have been obtaining data about them - no private information whatsoever was gathered, and the reviewers were performing their usual professional function. Thus they cannot have been considered human subjects by this definition, and the research was not subject to IRB review.

Furthermore, even if the project had been reviewed, it would have been exempt under 45 CFR 46.101, as "Research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behavior" with anonymous subjects. As such, at worst Prof. Boghossian should be admonished to seek IRB review for such research in the future.

Third was the accusation of "fabricating data" due to one of the fabricated research articles containing made-up statistics about canine sexual activity. Clearly, since the article was not intended to remain a part of the research literature, but to be unmasked as false, there was no intent to deceive the research community. As such, this was not fabrication or falsification of research data or results.

Taken together, the facts seem clear that Prof. Boghossian's project never warranted IRB review at all, or if it did, would have been exempt. In any case, the only potential consequence should be a discussion with him regarding the importance of undergoing IRB review for future such projects. I urge that Portland State rescind its determination that he violated university policy, and restore his professional and academic standing within the university to the status quo ante.

I would be happy of course to discuss this matter further if it would be of use.

Sincerely,

Shlomo Engelson Argamon
Professor of Computer Science
Director, Master of Data Science
Illinois Institute of Technology