To put it simply, a StatefulSet should be used to govern one or more related pods that need to track state in some way. A StatefulSet would be what you use instead of a Deployment.
StatefulSets are a particularly cool implementation detail among Kubernetes workload APIs. It works very similarly