The list of possible differential diagnoses for complaints of back and neck pain is enormously long. Many proposed diagnostic workups trying to avoid the challenge of considering all possible etiologies by early commitment to a specific area of the spine (i.e., neck or lower back). Others limit the differential diagnoses by pursuing a rational but limited goal, like ruling out conditions that may result in disability or death in the setting of an emergency department. We have briefly described a long list of medical conditions, each of which may present as back or neck pain, and whose prevalence ranges from common to very rare. We then showed that they all can be assigned to one of 7 groups according to judicious history taking, physical examination and simple imaging and laboratory tests: (1) pain that radiates from the cranium, chest or abdomen; (2) pain with signs of infection; (3) pain with signs of nerve root compression; (4) pain with signs of cord or cauda compression; (5) mechanical pain; (6) rheumatic pain; (7) pain with other characteristics, with or without local tenderness. Further referral to diagnostic tests and specialist consultations after initial assignment to one of these seven categories would save the patient time and unnecessary tests. We believe that this review and the proposed diagnostic algorithm may be valuable for medical education and for application in the primary care setting for the purpose of conducting a diagnostic workup of any type of back or neck pain in all patient groups and may be possibly used for development of diagnostic software and machine learning.