"cracker," attested 1830 in a specialized sense ("This may be ascribed to the Red Necks, a name bestowed upon the Presbyterians in Fayetteville" -- Ann Royall, "Southern Tour I," p.148), from red (adj.1) + neck (n.). According to various theories, red perhaps from anger, or from pellagra, but most likely from mule farmers' outdoors labor in the sun, wearing a shirt and straw hat, with the neck exposed. Compare redshanks, old derogatory name for Scots Highlanders and Celtic Irish (1540s), from their going bare-legged.
It turns up again in an American context in 1904, again from Fayetteville, in a list of dialect words, meaning this time "an uncouth countryman" ["Dialect Notes," American Dialect Society, vol. ii, part vi, 1904], but seems not to have been in widespread use in the U.S. before c. 1915. In the meantime, it was used from c. 1894 in South Africa (translating Dutch Roinek) as an insulting Boer name for "an Englishman."
Another common Boer name for an Englishman is "redneck," drawn from the fact that the back of an Englishman's neck is often burnt red by the sun. This does not happen to the Boer, who always wears a broad-brimmed hat. [James Bryce, "Impressions of South Africa," London, 1899]