And yet another way to think of it...
A stateful system is focused, at least partially, on saving to and retrieving from storage. It is a "data at rest" system.
A stateless system is focused, essentially entirely, on data "in flight". Processing of data moving from one point to another.