c. 1300, leisir, "free time, time at one's disposal," also (early 14c.) "opportunity to do something, chance, occasion, an opportune time," also "lack of hurry," from Old French leisir, variant of loisir "capacity, ability, freedom (to do something); permission; spare time; free will; idleness, inactivity," noun use of infinitive leisir "be permitted," from Latin licere "to be allowed" (see licence (n.)).
Especially "opportunity afforded by freedom from necessary occupations" (late 14c.). "In Fr. the word has undergone much the same development of sense as in Eng." [OED]. The -u- appeared 16c., probably on analogy of pleasure (n.), etc. To do something at leisure "without haste, with deliberation" (late 14c.) preserves the older sense. To do something at (one's) leisure "when one has time" is from mid-15c.