"southern Appalachian person," by 1892, from hill (n.) + Billy/Billie, popular or pet form of William. In reference to a type of U.S. folk music, first attested 1924.
I would hate to see some old railroad man come here and take my job, and then, I don't think it is right to hire some Hill Billy and give him the same right as I just because he was hired the same time I was. [The Railroad Trainmen's Journal, vol. ix, July 1892]
In short, a Hill-Billie is a free and untrammelled white citizen of Alabama, who lives in the hills, has no means to speak of, dresses as he can, talks as he pleases, drinks whiskey when he gets it, and fires of his revolver as the fancy takes him. [New York Journal, April 23, 1900]