Old English sponge, spunge, from Latin spongia "a sponge," also "sea animal from which a sponge comes," from Greek spongia, related to spongos "sponge," of unknown origin. "Probably a loanword from a non-IE language, borrowed independently into Greek, Latin and Armenian in a form *sphong-" [de Vaan]. The Latin word is the source of Old Saxon spunsia, Middle Dutch spongie, Old French esponge, Spanish esponja, Italian spugna.
In English in reference to the marine animal from 1530s. To throw in the sponge "quit, submit" (1860) is from prizefighting, in reference to the sponges used to cleanse the faces of combatants between rounds (compare later throw in the towel). Sponge-cake is attested from 1808.
late 14c., "to soak up with a sponge," also (transitive) "to cleanse or wipe with a sponge," from sponge (n.). The slang sense of "to live in a parasitic manner, live at the expense of others" is attested from 1670s; sponger (n.) in this sense is from 1670s. Originally it was the victim who was the sponge (c. 1600), because he or she was being "squeezed." Intransitive sense "dive for sponges" is from 1881. Related: Sponged; sponging.