1650s, "public department in charge of letter-carrying," from post (n.3) + office. Meaning "building where postal business is carried on, office or place where letters are received for transmission," is from 1650s. In slang or euphemistic sense of "a sexual game" it refers to an actual parlor game first attested early 1850s in which pretend "letters" were paid for by kisses.
slang, "perform male masturbation," by 1896, from jerk (v.) denoting rapid pulling motion + off (adv.). Compare come off "experience orgasm" (17c.). Farmer and Henley ("Slang and Its Analogues") also lists as synonyms jerk (one's) jelly and jerk (one's) juice. The noun jerk off or jerkoff as an emphatic form of jerk (n.2) is attested by 1968. As an adjective from 1957.