1550s, "one given to change," from change (n.) + diminutive suffix -ling. Meaning "person or thing left in place of one secretly taken" is from 1560s; specific reference to an infant or young child (usually stupid, strange, or ugly) superstitiously believed to have been left by the faeries in place of a beautiful or charming one they have stolen away is from 1580s. An earlier word for it was oaf or auf.
1620s, auf, oph (modern form from 1630s; oafish is from 1610s), "a changeling; a foolish or otherwise defective child left by the fairies in place of another carried off," from a Scandinavian source such as Norwegian alfr "silly person," in Old Norse "elf" (see elf). Hence, "a misbegotten, deformed idiot, a simpleton" (17c.). Until recently, some dictionaries still gave the plural as oaves.