This semester I took a course in Deep Learning. My friend—a former TA, and now a PhD candidate at Yale—had told me that deep learning was state-of-the-art, and that if I was only going to pick one AI course, it should be deep learning.
What I learned was that intelligence is, at best, some sort of long, context-based autocorrect. You can replicate this at a high level with neurons, numbers and bits, which is just to say that there isn't some inherent structure to the human brain that can't be modeled well enough by a computer.
My takeaway from this undergraduate course is not nuanced. I'm not going to hold myself to it five years from now. The rest of this is an emotionally-driven argument, borne out of my frustration with AI and ML and CS in general.
There just doesn't seem to be any real limit to what AI can do. And by real I mean logically, mathematically, necessarily and absolutely true, not some sort of physical or computational constraint imposed by the number of GPUs you have or whatnot.
Like, there are some things some programs can't do. A single perception can't model the XOR function. It just can't. But deep neural networks? It can do everything we can dream of doing, which is to say that it can write essays and spit out code based on nothing but intuition and pattern matching.
Okay, cool. We know that. Anyone interested in technology in the year 2024 has some idea of what AI can and cannot do. You can model any complicated function with a neural network—some variant of this has already been proved with the universal approximation theorem.1
I-
Now, this is no complaint about the economy or the society or the Valley or anything but-
What is so interesting about this?
So the model can extrapolate from data. It might even be able to extrapolate from very little data, or no data. But it's not Ultron, it's not Skynet, it's not even JARVIS yet. It's just a program that let's us do more of what we've already been doing, at a faster rate.
From a sort of sociological perspective, AI (the technology) is interesting. But compared to everything else in mathematical and computer science theory, AI hardly pushes the boundaries of what is possible. We still don't know a thing about the nature of mathematics,2 or the nature of computation,3 much less the universe itself.4 Anything else—biology, chemistry, linguistics—forget it!