1824, originally in African-American vernacular in the South.
The slaves themselves entertain the very highest contempt for white servants, whom they designate as 'poor white trash.' [Fanny Kemble, journal, Jan. 6, 1833]
[T]he term [poor white] is rather loosely applied by Northern writers even to mountaineers and to small farmers who live on a precarious footing. But in the Southern conception, not everyone who is both poor and white is a "poor white." To the Southerner, the "poor white" in the strictest sense is a being beyond the pale of even the most generous democratic recognition; in the negro's term, "po' white trash," or so much social débris. [Robert Penn Warren, "The Briar Patch," 1930, footnote]
"person or thing that people hope will be very successful in the near future," 1911, originally in U.S. sporting use in reference to the quest for a white man capable of beating champion pugilist Jack Johnson.
The white collar men are your clerks; they are your bookkeepers, your cashiers, your office men. We call them the 'white collar men' in order to distinguish them from the men who work with uniform and overalls and carry the dinner pails. The boys over on the West side got that name for them. It was supposed to be something a little better than they were. [Malcolm McDowell, quoted in Chicago Commerce, June 12, 1914]
White-collar crime attested by 1957 (there is a white-collar criminaloids from 1934).