1660s, "willful destroyer of what is beautiful or venerable," from Vandals, name of the Germanic tribe that sacked Rome in 455 under Genseric, from Latin Vandalus (plural Vandali), from the tribe's name for itself (Old English Wendlas), perhaps from Proto-Germanic *wandljaz "wanderer." The literal historical sense in English is recorded from 1550s.
There does not seem to be in the story of the capture of Rome by the Vandals any justification for the charge of willful and objectless destruction of public buildings which is implied in the word 'vandalism.' It is probable that this charge grew out of the fierce persecution which was carried on by [the Vandal king] Gaiseric and his son against the Catholic Christians, and which is the darkest stain on their characters. [Encyclopaedia Britannica, 13th ed., 1926]