Brain imaging characteristics of MOG antibody disease are largely unknown and it is unclear whether they differ from those of multiple sclerosis and AQP4 antibody disease. The aim of this study was to identify brain imaging discriminators between those three inflammatory central nervous system diseases in adults and children to support diagnostic decisions, drive antibody testing and generate disease mechanism hypotheses. Clinical brain scans of 83 patients with brain lesions (67 in the training and 16 in the validation cohort, 65 adults and 18 children) with MOG antibody (n = 26), AQP4 antibody disease (n = 26) and multiple sclerosis (n = 31) recruited from Oxford neuromyelitis optica and multiple sclerosis clinical services were retrospectively and anonymously scored on a set of 29 predefined magnetic resonance imaging features by two independent raters. Principal component analysis was used to perform an overview of patients without a priori knowledge of the diagnosis. Orthogonal partial least squares discriminant analysis was used to build models separating diagnostic groups and identify best classifiers, which were then tested on an independent cohort set. Adults and children with MOG antibody disease frequently had fluffy brainstem lesions, often located in pons and/or adjacent to fourth ventricle. Children across all conditions showed more frequent bilateral, large, brainstem and deep grey matter lesions. MOG antibody disease spontaneously separated from multiple sclerosis but overlapped with AQP4 antibody disease. Multiple sclerosis was discriminated from MOG antibody disease and from AQP4 antibody disease with high predictive values, while MOG antibody disease could not be accurately discriminated from AQP4 antibody disease. Best classifiers between MOG antibody disease and multiple sclerosis were similar in adults and children, and included ovoid lesions adjacent to the body of lateral ventricles, Dawson’s fingers, T1 hypointense lesions (multiple sclerosis), fluffy lesions and three lesions or less (MOG antibody). In the validation cohort patients with antibody-mediated conditions were differentiated from multiple sclerosis with high accuracy. Both antibody-mediated conditions can be clearly separated from multiple sclerosis on conventional brain imaging, both in adults and children. The overlap between MOG antibody oligodendrocytopathy and AQP4 antibody astrocytopathy suggests that the primary immune target is not the main substrate for brain lesion characteristics. This is also supported by the clear distinction between multiple sclerosis and MOG antibody disease both considered primary demyelinating conditions. We identify discriminatory features, which may be useful in classifying atypical multiple sclerosis, seronegative neuromyelitis optica spectrum disorders and relapsing acute disseminated encephalomyelitis, and characterizing cohorts for antibody discovery.