Eliezer Yudkowsky

Researcher

1979-09-11

Eliezer Yudkowsky is an American AI safety researcher and writer. He co-founded the Machine Intelligence Research Institute and is known for work on rationality and AI alignment. He has published essays and technical writing on long-term AI risk.

Quotes by Eliezer Yudkowsky

Wow, the empty air finally said. Wow. That puts a pretty different perspective on things, I have to say. I'm going to remember this the next time I feel an impulse to blame myself for something. Neville, the term in the literature for this is 'egocentric bias', it means that you experience everything about your own life but you don't get to experience everything else that happens in the world. There was way, way more going on than you running in front of me. You're going to spend weeks remembering that thing you did there for six seconds, I can tell, but nobody else is going to bother thinking about it. Other people spend a lot less time thinking about your past mistakes than you do, just because you're not the center of their worlds. I guarantee to you that nobody except you has even considered blaming Neville Longbottom for what happened to Hermione. Not for a fraction of a second. You are being, if you will pardon the phrase, a silly-dilly. Now shut up and say goodbye.
Read quote
People form close friendships by knowing private things about each other, and the reason most people don't make close friends is because they're too embarrassed to share anything really important about themselves.
Read quote
Because the way people are built, Hermione, the way people are built to feel inside - Harry put a hand over his own heart, in the anatomically correct position, then paused and moved his hand up to point toward his head at around the ear level, - is that they hurt when they see their friends hurting. Someone inside their circle of concern, a member of their own tribe. That feeling has an off-switch, an off-switch labeled 'enemy' or 'foreigner' or sometimes just 'stranger'. That's how people are, if they don't learn otherwise.
Read quote
Since the beginningnot one unusual thinghas ever happened.
Read quote
Your strength as a rationalist is your ability to be more confused by fiction than by reality. If you are equally good at explaining any outcome, you have zero knowledge.
Read quote
Without thinking about it at all, Harry stepped in front of Hermione.There was an intake of breath from behind him, and then a moment later Hermione brushed past and stepped in front of him. Run, Harry! she said. Boys shouldn't have to be in danger.
Read quote
But it is cute. It's such a boy thing to do.Drop dead.Aw, you say the most romantic things.
Read quote
And then Harry Potter had launched in to a speech that was inspiring, yet vague. A speech to the effect that Fred and George and Lee had tremendous potential if they could just learn to be weirder. To make people's live surreal, instead of just surprising them with the equivalents of buckets of water propped above doors. (Fred and George had exchanged interested looks, they'd never thought of that one.) Harry Potter had invoked a picture of the prank they'd pulled on Neville - which, Harry had mentioned with some remorse, the Sorting Hat had chewed him out on - but which must have made Neville doubt his own sanity. For Neville it would have felt like being suddendly transported into an alternate universe. The same way everyone else had felt when they'd seen Snape apologize. That was the true power of pranking.
Read quote
So the next time you doubt the strangeness of the future, remember how you were born in a hunter-gatherer tribe ten thousand years ago, when no one knew of Science at all. Remember how you were shocked, to the depths of your being, when Science explained the great and terrible sacred mysteries that you once revered so highly. Remember how you once believed that you could fly by eating the right mushrooms, and then you accepted with disappointment that you would never fly, and then you flew. Remember how you had always thought that slavery was right and proper, and then you changed your mind. Don't imagine how you could have predicted the change, for that is amnesia. Remember that, in fact, you did not guess. Remember how, century after century, the world changed in ways you did not guess.Maybe then you will be less shocked by what happens next.
Read quote
I only want power so I can get books.
Read quote
Most Muggles lived in a world defined by the limits of what you could do with cars and telephones. Even though Muggle physics explicitly permitted possibilities like molecular nanotechnology or the Penrose process for extracting energy from black holes, most people filed that away in the same section of their brain that stored fairy tales and history books, well away from their personal realities: Long ago and far away, ever so long ago.
Read quote
Then you get the wrong answer and you can't go to the Moon that way! Nature isn't a person, you can't trick them into believing something else, if you try to tell the Moon it's made of cheese you can argue for days and it won't change the Moon! What you're talking about is rationalization, like starting with a sheet of paper, moving straight down to the bottom line, using ink to write 'and therefore, the Moon is made of cheese', and then moving back up to write all sorts of clever arguments above. But either the Moon is made of cheese or it isn't. The moment you wrote the bottom line, it was already true or already false. Whether or not the whole sheet of paper ends up with the right conclusion or the wrong conclusion is fixed the instant you write down the bottom line.
Read quote
It's a most peculiar psychology— this business of 'Science is based on faith too, so there!' Typically this is said by people who claim that faith is a good thing. Then why do they say 'Science is based on faith too!' in that angry-triumphal tone, rather than as a compliment? And a rather dangerous compliment to give, one would think, from their perspective. If science is based on 'faith', then science is of the same kind as religion— directly comparable. If science is a religion, it is the religion that heals the sick and reveals the secrets of the stars. It would make sense to say, 'The priests of science can blatantly, publicly, verifiably walk on the Moon as a faith-based miracle, and your priests' faith can't do the same.' Are you sure you wish to go there, oh faithist? Perhaps, on further reflection, you would prefer to retract this whole business of 'Science is a religion too!
Read quote
By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it.
Read quote
And someday when the descendants of humanity have spread from star to star they won't tell the children about the history of Ancient Earth until they're old enough to bear it and when they learn they'll weep to hear that such a thing as Death had ever once existed
Read quote
If you once tell a lie, the truth is ever after your enemy.
Read quote
One of chief pieces of advice I give to aspiring rationalists is Don't try to be clever. And, Listen to those quiet, nagging doubts. If you don't know, you don't know what you don't know, you don't know how much you don't know, and you don't know how much you needed to know.
Read quote
There is no justice in the laws of nature, no term for fairness in the equations of motion. The Universe is neither evil, nor good, it simply does not care. The stars don't care, or the Sun, or the sky. But they don't have to! WE care! There IS light in the world, and it is US!
Read quote
Then you get the wrong answer and you can't go to the Moon that way! Nature isn't a person, you can't trick them into believing something else, if you try to tell the Moon it's made of cheese you can argue for days and it won't change the Moon! What you're talking about is rationalization, like starting with a sheet of paper, moving straight down to the bottom line, using ink to write 'and therefore, the Moon is made of cheese', and then moving back up to write all sorts of clever arguments above. But either the Moon is made of cheese or it isn't. The moment you wrote the bottom line, it was already true or already false. Whether or not the whole sheet of paper ends up with the right conclusion or the wrong conclusion is fixed the instant you write down the bottom line.
Read quote
By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it.
Read quote