周末阅读：AI 引发的几个大问题

AI is really forcing some big questions into the open. Here are some weekend reads on them.

The biggest idea

The Economist address the effective altruism (EA) belief system, which it calls “this century’s biggest idea”:

[P]lenty of EAs in Silicon Valley think that only they can be trusted with the technology. That is an alarming idea. Effective altruists’ argument for worrying about really powerful AIs is that the machines could optimise for a single and fatally wrong objective. Everyone should worry, too, about what happens when a small group of people fall into the same trap

The full analysis is worth reading. Also read the brief response by Alexander Berger, the CEO of Coefficient Giving, an EA-aligned philanthropist.

The religious angle

The New York Times reports that Anthropic had been consulting with religious scholars and theologians on the question of machine consciousness. Anthropic has prioritised questions of whether its systems might imbue consciousness and personhood more than other AI companies. It goes so far as Anthropic’s team proposed to withdraw from a Vatican event before the Pope’s encyclical Magnifica Humanitas argued that AIs “do not undergo experiences, do not possess a body, do not feel joy or pain… nor do they have a moral conscience.”

EV reader, Anil Seth’s X thread and the pushback he received are worth scanning.

These are huge questions. We’ll touch on consciousness in tomorrow’s newsletter.
Meanwhile, happy to hear your reasoned thoughts below.