1530s, "scullion, kitchen knave," of uncertain origin. Perhaps in reference to military units or attendants so called for the color of their dress or their character; more likely originally a mock-military reference to scullions and kitchen-knaves of noble households, of black-liveried personal guards, and of shoeblacks. See black (adj.) + guard (n.). By 1736, sense had emerged of "one of the idle criminal class; man of coarse and offensive manners." Hence the adjectival use (1784), "of low or worthless character."