Four numbers in a box does not sound like much of an idea. But those four numbers say where two arrows land — and once you know that, you know where every point in the plane lands, because everything else is built out of those two. Drag the arrows below and watch the entire grid come with them.
The purple arrow is where (1, 0) ends up; the amber one is where (0, 1) ends up. Drag either tip. The faint grid is where the plane started, the bright one is where it is now, and the shape rides along. Notice what never changes: the origin stays put, straight lines stay straight, and parallel lines stay parallel. That is the whole meaning of the word linear.
The unit square has area 1. After the transformation it is a parallelogram, and its area is the determinant. Every other area in the plane scales by exactly the same factor. Make the arrows cross over each other and the determinant goes negative — space has been flipped over. Line them up exactly and it hits zero: the plane has been squashed onto a line, and there is no way back.
Almost every arrow gets knocked off its line by the transformation. A few do not — they just get longer or shorter. Those are the eigenvectors, and how much they stretch is the eigenvalue. The faint arrows are a fan of test directions with their destinations; the bright ones are the directions that stayed put. A rotation has none at all, because nothing keeps its heading when you turn everything.
Do two transformations one after the other and you get a third, which is what multiplying matrices means. But swapping the order gives a different answer: shear-then-rotate is not rotate-then-shear. Both panels use the same two operations on the same shape.
A layer of a neural network is a matrix multiplication followed by one bend — that is genuinely all it is. When Machine Learning trains, it is adjusting the numbers in boxes like this one. When How LLMs Work talks about a token becoming a vector and attention mixing them, both operations are matrices. When Image Generation denoises, the same. The reason a graphics card can run any of it is that a GPU is, in essence, a machine built to do this one operation to millions of vectors at once.
The version here is two-dimensional so you can see it. A language model's vectors have several thousand dimensions, and the transformations are matrices with millions of entries. Nothing about the idea changes — you simply lose the ability to picture it, which is precisely why it is worth getting the picture straight while it still fits on a screen.