1858, "defective or inferior because not of a natural or proper color," from off (prep.) + color (n.); originally used of gems; figurative extension to "not of the proper character, of questionable taste, risqué" is American English, 1867.
Others are reading
Definitions of off-colour from WordNet
in violation of good taste even verging on the indecent;