We used resting-state fMRI and diffusion-weighted imaging to investigate changes in attention-related and salience networks in mild cognitive impairment and Alzheimer’s disease. Resting-state fMRI data of 37 patients with Alzheimer’s disease, 50 patients with mild cognitive impairment and 34 healthy older controls, and diffusion-weighted imaging data of 33 patients with Alzheimer’s disease, 31 patients with mild cognitive impairment and 16 healthy older controls from the Alzheimer’s disease Neuroimaging Initiative were analyzed. Compared with healthy older controls, patients with Alzheimer’s disease mainly showed (a) decreased resting-state functional connectivity in attentional networks, between the anterior cingulate cortex and attentional networks, and between the anterior cingulate cortex and the amygdala, (b) increased resting-state functional connectivity between parts of the orbitofrontal cortex, (c) decreased structural connectivity within the parietal, occipital and frontal lobes, between the limbic areas (anterior cingulate cortex and insula) and the frontal lobe, and between the amygdala and the insula. In contrast, patients with mild cognitive impairment showed decreased resting-state functional connectivity in the attentional networks exclusively, and increased structural connectivity (as reflected by fractional anisotropy) in limbic and frontal areas.