1690s, originally an art criticism term, "assemblage of figures or objects forming a harmonious whole in a painting or design," from French groupe "cluster, group" (17c.), from Italian gruppo "group, knot," which probably is, with Spanish grupo, from a Germanic source, from Proto-Germanic *kruppaz "round mass, lump," part of the general group of Germanic kr- words with the sense "rounded mass" (such as crop (n.).
Extended to "any assemblage, a number of individuals related in some way" by 1736. Meaning "pop music combo" is from 1958. As it was borrowed after the Great Vowel Shift in English, the pronunciation of the -ou- follows French rather than English models.