c. 1300, dismaien, "become or be alarmed, upset, or frightened; to confound, break down the courage of by danger or difficulty or fear of calamity, fill with despairing apprehension;" perhaps formed in Anglo-French or Middle English from dis-, here probably intensive (see dis-), + amaien, esmaien, from Old French esmaier "to trouble, disturb."
This is from Vulgar Latin *exmagare "divest of power or ability" (source of Italian smagare "to weaken, dismay, discourage"), from ex- (see ex-) + Proto-Germanic *magan "to be able" (source also of Old High German magen "to be powerful or able"), from PIE root *magh- "to be able, have power."
There also was an Old French *desmaier (attested only in past participle dismaye), from de-, intensive prefix, + Old French esmaier, which also might be the source of the Middle English word. Spanish desmayer "to be dispirited" is a loan word from Old French. Related: Dismayed; dismaying.