c. 1300, "what one has to do, ordinary business," from Anglo-French afere, Old French afaire "business, event; rank, estate" (12c., Modern French affaire), from the infinitive phrase à faire "to do," from Latin ad "to" (see ad-) + facere "to do, make" (from PIE root *dhe- "to set, put").
According to OED a Northern word originally, brought into general use and given a French spelling by Caxton (15c.). General sense of "vague proceedings" (in romance, war, etc.) first attested 1702. Meaning "an affair of the heart; a passionate episode" is from French affaire de coeur (itself attested as a French phrase in English from 1809); to have an affair with someone in this sense is by 1726, earlier have an affair of love:
'Tis manifeſtly contrary to the Law of Nature, that one Woman ſhould cohabit or have an Affair of Love with more than one Man at the ſame time. ["Pufendorf's Law of Nature and Nations," transl. J. Spavan, London, 1716]