Are AI systems like Claude and ChatGPT conscious beings or unfeeling calculators?
Before answering that, remember that humans are agency detection machines. A rockslide can kill you if you’re in the wrong place at the wrong time. A being with agency can kill you with intention by stalking your movements and planning an ambush. The latter requires significantly more cognitive resources to guard against, up to and including a “theory of mind” — a mental model of yourself and other agents in the world as having distinct beliefs, desires, and motivations. But just as we may infer a predator from rustling leaves, we tend to infer even complex forms of agency according to simple heuristics. Human cognition is thus vulnerable to apophenia and pareidolia, or the tendency to over-interpret nebulous stimuli in ways that project meaning and agency where there is none.
This basic fact about human psychology is important to keep in mind when interacting with AI systems like Claude and ChatGPT. As a practical matter, evolution clearly primed us to ascribe subjective states to beings based on superficial cues, like a face grimacing in pain. These mental short-cuts naturally extend to AI models with human-like capabilities, including language competency.
