children's game, from plural of marble (n.); the game is recorded by that name by 1709 but is probably older (it was known in 13c. German as tribekugeln). It originally was played with small balls of polished marble or alabaster, later of clay. Glass marbles with the colored swirl date from the 1840s.
Meaning "mental faculties, common sense" (as in to lose or not have all one's marbles) is by 1927, American English slang, perhaps [OED] from earlier slang marbles "furniture, personal effects, 'the goods' " (1864, Hotten), a corrupt translation of French meubles (plural) "furniture" (see furniture).