The purpose (Wolfram)

It's not going to be the case, as I thought, that there's us that is intelligent, and there's everything else in the world that's not. It's not going to be some big abstract difference between us and the clouds and the cellular automata. It's not an abstract difference. It's not something where we can say, look, this brain-like neural network is just qualitatively different than this cellular automaton thing. Rather, it's a detailed difference that this brain-like thing was produced by this long history of civilization, et cetera, whereas this cellular automaton was just created by my computer in the last microsecond.

...

Here's one of my scenarios that I'm curious about. Let's say there's a time when human consciousness is readily uploadable into digital form, virtualized and so on, and pretty soon we have a box of a trillion souls. There are a trillion souls in a box, all virtualized. We look at this box. In the box, there will be hopefully nice molecular computing, maybe it'll be derived from biology in some sense, but maybe not, but there will be all kinds of molecules doing things, electrons doing things. The box is doing all kinds of elaborate stuff.

Then we look at the rock sitting next to the box. Inside the rock, there's all kinds of elaborate stuff going on, all kinds of electrons doing all kinds of things. We say, "What's the difference between the rock and the box of a trillion souls?" The answer will be that the box of trillion souls has this long history. The details of what's happening there were derived from the history of civilization and people watching videos made in 2015 or whatever. Whereas the rock came from its geological history, but it's not the particular history of our civilization.

This question of realizing that there isn't this distinction between intelligence and mere computation leads you to imagine the future of civilization ends up being the box of trillion souls, and then what is the purpose of that?

Stephen Wolfram, Edge