Old English æfre "ever, at any time, always;" of uncertain origin, no cognates in any other Germanic language; perhaps a contraction of a in feore, literally "ever in life" (the expression a to fore is common in Old English writings). First element is almost certainly related to Old English a "always, ever," from Proto-Germanic *aiwi-, extended form of PIE root *aiw- "vital force, life; long life, eternity." Liberman suggests second element is comparative adjectival suffix -re.
Sometimes contracted to e'er in dialect and poetry. Ever began to be used in late Old English as a way to generalize or intensify when, what, where, etc. The sense evolution was from "at any time at all, in any way" to "at any particular time; at some time or another; under any circumstances." Ever so "to whatever extent" is recorded by 1680s. Expression did you ever? (implying "see/do/hear of such a thing") attested by 1840.