also set-back, 1670s, "reversal, check to progress," from the verbal phrase, attested mid-15c. as "withhold;" see set (v.) + back (adv.). Backset (1721) is used in the same sense. The meaning "space between a building and a property line or roadway" is from 1916. To set (someone) back "cost" (a certain sum of money) is from 1900.
BACKWOODSMEN. ... This word is commonly used as a term of reproach (and that, only in a familiar style,) to designate those people, who, being at a distance from the sea and entirely agricultural, are considered as either hostile or indifferent to the interests of the commercial states. [John Pickering, "A Vocabulary, or Collection of Words and Phrases Which Have Been Supposed to be Peculiar to the United States of America," Boston, 1816]
c. 1200, "toward the rear," a contraction of Old English on bæc "backward, behind, at or on the back;" see see a- (1) + back (n.). Now surviving mainly in taken aback, which originally was a nautical expression in reference to a vessel's square sails when a sudden change of wind flattens them back against the masts and stops the forward motion (1754). The figurative sense from this, "suddenly or unexpectedly checked or disappointed," is by 1792.