Evaluating ChatGPT as a Bullshit Machine

The structure of the paper is as follows: in the first section, we outline how ChatGPT and similar LLMs operate. Next, we consider the view that when they make factual errors, they are lying or hallucinating: that is, deliberately uttering falsehoods, or blamelessly uttering them on the basis of misleading input information. We argue that neither of these ways of thinking are accurate, insofar as both lying and hallucinating require some concern with the truth of their statements, whereas LLM...

 ChatGPT is a bullshit machine

The idea of ChatGPT as a bullshit machine is a helpful one when combined with the distinction between hard and soft bullshit. Reaching again for the example of the dodgy student paper: we’ve all, I take it, marked papers where it was obvious that a dictionary or thesaurus had been deployed with a crushing lack of subtlety; where fifty-dollar words are used not because they’re the best choice, nor even because they serve to obfuscate the truth, but simply because the author wants to convey...

 Jone's Dilemma

1-48. This principle is named after Reginald Victor Jones, a British professor heavily involved in solving science and technology intelligence challenges. In this deception, the target receives information through multiple means and methods, from many angles, throughout an operational environment. Deception generally becomes more difficult as the number of conduits available to the deception target to confirm the real situation increases. However, the greater the number of conduits that are d...

 Multiple Forms of Surprise

1-46. A strong correlation exists between deception and surprise. The more forms of surprise built into the deception plan, the more likely it will overwhelm the target. These forms of surprise include size, activity, location, unit, time, equipment, intent, and style. One effect of surprise is the cry-wolf syndrome in which repeated false alarms have the potential to desensitize an enemy. A pattern of behavior lulls an opponent into a sense of normal behavior to allow a friendly action to oc...

 Magruder’s Principle

1-42. Magruder’s principle states that it is generally easier to induce the deception target to maintain a preexisting belief than to deceive the deception target for the purpose of changing that belief. Magruder’s principle exploits target biases and the human tendency to confirm exiting beliefs. Magruder’s principle alludes to two paths. A path of the deceiver changing the belief of a target and a path of maintaining a present belief. The principle then advises the better of the two p...

 Counterinsurgency

At its heart, a counterinsurgency is an armed struggle for the support of the population. This support can be achieved or lost through information engagement, strong representative government, access to goods and services, fear, or violence. This armed struggle also involves eliminating insurgents who threaten the safety and security of the population. However, military units alone cannot defeat an insurgency. Most of the work involves discovering and solving the population’s underlying iss...

 Plasticity and Lock-In/Entrenchment of Values

Entrenchment of values creates multiple equilibria because there is a significant element of chance in which value system becomes most powerful at a particular place and time, and because, once a value system has become sufficiently powerful, it can stay that way by suppressing the competition. Moreover, the theory of cultural evolution helps to explain why the predominant cultures in society tend to entrench themselves. Simply: those cultures that do not entrench themselves in this way are, ...

 The Allegorithm

A Sim in The Sims is a simple animated character, with few facial features or expressions. In The Sims 2 they seem a little more lifelike, but the improvement of the representation in some particular ways only raises the standards by which it appears to fall short in others. From the point of view of allegorithm, it all seems more the other way around. Everyday life in gamespace seems an imperfect version of the game. The gamespace of everyday life may be more complex and variegated, but it s...

 Abstractions Turned Obfuscations

There is an old saying in Silicon Valley, “There is the first 80% and the second 80%.” While it is really hard to create new technologies, it is also really hard to implement them for any measurable advantage. This has always been true: The steam engine didn’t matter until it was put into a ship and locomotive; the Wright brothers’ flight didn’t matter until it moved people; electricity needed to be delivered to the home; and telephony didn’t matter un...

 The AI Business Model is Wrong

AI decision support is potentially valuable to practitioners. Accountants might value an AI tool’s ability to draft a tax return. Radiologists might value the AI’s guess about whether an X-ray suggests a cancerous mass. But with AIs’ tendency to “hallucinate” and confabulate, there’s an increasing recognition that these AI judgments require a “human in the loop” to carefully review their judgments. In other words, an AI-supported radiologist...