c. 1600, in music, "involving tones foreign to the normal tonality of the scale, not diatonic," from Latin chromaticus, from Greek khromatikos "relating to color, suited for color" (also used in reference to music), from khroma (genitive khromatos) "color, complexion, character" (but chiefly used metaphorically of embellishments in music), originally "skin, surface" (see chroma).
Greek also used khroma for certain modifications of the usual diatonic music scale. The reason the Greeks used this word in music is not now entirely clear. Perhaps the connection is the extended sense of khroma, "ornaments, make-up, embellishments," via the notion of "characteristic" of a musical scale or speech.
In English, the musical sense of "progressing by half-tones, involving the sharps and flats of the staff" is by 1881. Meaning "of or pertaining to color" is from 1829.