For projects I deeply care about, choosing to drive instead of letting the model drive is one way to intervene in every single decision. It’s much slower to type things out, but it does force me to engage with every decision and gives me an opportunity to override it. In fact, for this post I’m using a Codex session…
When did you realize Wikipedia had a problem with AI-generated material? Ellia Lebleu: It was around December 2023. A few other editors and I were chatting on Wikipedia’s Discord server, and we had all seen very strange articles with the same odd characteristics. They repeated phrases like “stands as a testament” or…
Imagine a building, let's say a small office tower. When this building was built, they used a type of pipe that will degrade over time and eventually need to be replaced. This is standard infrastructural debt; basic wear and tear, as they say. Then, let's say that when it comes time to replace the pipes, you need to…
‘Do you suppose the potter is content with mental creation? Do you suppose the idea is enough? That there is nothing in the feel of the clay itself, in watching the thing grow as hand and mind work together? Do you suppose the actual growth doesn't act as a feedback to modify and improve the idea?' 'You are not a…
The common public perception of prompt engineering is purely mechanical: better inputs yield clearer outputs. However, a deeper look reveals that human emotion and tone have become algorithmic variables. LLMs do not “think” in facts; they predict the statistically most probable next word based on the mathematical…
- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all. - Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an…
The idea of a permanent underclass has recently been embraced in part as an online joke and in part out of a sincere fear about how A.I. automation will upend the labor market and create a new norm of inequality. In an A.I.-dominated future, those with capital will buy “compute” (the tech term for A.I. horsepower) and…
En la época de Sócrates, los textos escritos aún no eran una herramienta habitual y todavía despertaban recelos. Los consideraban un sucedáneo de la palabra oral — liviana, alada, sagrada—. Aunque la Atenas del siglo V a. C. ya contaba con un incipiente comercio de libros, no sería hasta un siglo después, en tiempos…
The structure of the paper is as follows: in the first section, we outline how ChatGPT and similar LLMs operate. Next, we consider the view that when they make factual errors, they are lying or hallucinating: that is, deliberately uttering falsehoods, or blamelessly uttering them on the basis of misleading input…
The idea of ChatGPT as a bullshit machine is a helpful one when combined with the distinction between hard and soft bullshit. Reaching again for the example of the dodgy student paper: we’ve all, I take it, marked papers where it was obvious that a dictionary or thesaurus had been deployed with a crushing lack of…
The current state-of-the-art Gemini model can fit roughly 1.5 million words in its context. That’s enough for me to upload the full text of all fourteen of my books, plus every article, blog post, or interview I’ve ever published—and the entirety of my collection of research notes that I’ve compiled over the years…
Of the platforms where we know flesh and blood humans do spend enough time, 92% of all content is created by 10% of all users, and engagement with different posts can vary from 0.03% to 0.1% of all viewers. In other words, of the 52.6% of internet traffic which is human-driven, about 9 in 10 users stuck to messaging…
Every bubble pops eventually. When this one goes, what will be left behind? Well, there will be little models – Hugging Face, Llama, etc – that run on commodity hardware. The people who are learning to “prompt engineer” these “toy models” have gotten far more out of them than even their makers imagined possible. They…
To grasp the proposed relationship between compression and understanding, imagine that you have a text file containing a million examples of addition, subtraction, multiplication, and division. Although any compression algorithm could reduce the size of this file, the way to achieve the greatest compression ratio…
Before you continue, pause and consider: How would you prove you're not a language model generating predictive text? What special human tricks can you do that a language model can't? 1. Triangulate objective reality [...] This leaves us with some low-hanging fruit for humanness. We can tell richly detailed stories…
The survival of man depends on the early construction of an ultraintelligent machine. Let an ultraintelligent machine be defined as a. machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent…
Let’s consider Arabic numerals as compared with Roman numerals. With a positional notation system, such as the one created by Arabic numerals, it’s easier to perform multiplication and division; if you’re competing in a multiplication contest, Arabic numerals provide you with an advantage. But I wouldn’t say that…
One of the problems with playing against computers is how quickly and how often they change. Grandmasters are used to preparing very deeply for our opponents, researching all of their latest games and looking for weaknesses. Mostly this preparation focuses on openings, the established sequences of moves that start the…
AI researchers have typically tried to get around the issues posed by by Montezuma’s Revenge and Pitfall! by instructing reinforcement-learning algorithms to explore randomly at times, while adding rewards for exploration—what’s known as “intrinsic motivation.” But the Uber researchers believe this fails to capture an…
The philosophers’ fascination with propositions was mirrored in good old-fashioned AI, the AI of John McCarthy, early Marvin Minsky, and Allen Newell, Herbert Simon, and Cliff Shaw. It was the idea that the way to make an intelligent agent was from the top down. You have a set of propositions in some proprietary…