1540s, originally "person hired to remove refuse from streets," from Middle English scawageour (late 14c.), London official in charge of collecting tax on goods sold by foreign merchants, from Anglo-French scawager, from scawage "toll or duty on goods offered for sale in one's precinct" (c. 1400), from Old North French escauwage "inspection," from a Germanic source (compare Old High German scouwon, Old English sceawian "to look at, inspect;" see show (v.)).
It has come to be regarded as an agent noun in -er, but the verb is a late back-formation from the noun. With unetymological -n- (c. 1500) as in harbinger, passenger, messenger, etc. Extended to animals 1590s. Scavenger hunt is attested from 1937.