Always prime
Bare drywall (it drinks finish paint unevenly), dramatic color changes (especially light-over-dark), stains and smoke damage (stain-blocking primer), and glossy surfaces (adhesion). These four account for nearly all legitimate primer use — each prevents a specific failure mode you would otherwise paint twice.
Skip it
Same-color or similar-color repaints on clean walls (scuff-sand and go), and previously painted walls in good condition getting a mid-grade two-coat finish. Modern paint-over-paint adhesion is excellent; primer here is mostly ritual and budget.
The tinted-primer trick
Have the primer tinted halfway to your finish color — it buys back most of a coat on color changes and costs nothing extra at the counter. Paint store staff do this automatically when asked; it is the single best-kept open secret in DIY painting.