Old English sponge, spunge, from Latin spongia "a sponge," also "sea animal from which a sponge comes," from Greek spongia, related to spongos "sponge," of unknown origin. "Probably a loanword from a non-IE language, borrowed independently into Greek, Latin and Armenian in a form *sphong-" [de Vaan]. The Latin word is the source of Old Saxon spunsia, Middle Dutch spongie, Old French esponge, Spanish esponja, Italian spugna.
In English in reference to the marine animal from 1530s. To throw in the sponge "quit, submit" (1860) is from prizefighting, in reference to the sponges used to cleanse the faces of combatants between rounds (compare later throw in the towel). Sponge-cake is attested from 1808.