late 13c., anoien, annuien, "to harm, hurt, injure; be troublesome or vexatious to, disquiet, upset," from Anglo-French anuier, Old French enoiier "to weary, vex, anger," anuier "be troublesome or irksome to;" according to French sources both from Late Latin inodiare "make loathsome," from Latin (esse) in odio "(it is to me) hateful," from ablative of odium "hatred," from PIE root *od- (2) "to hate" (see odium).
Also in Middle English as a noun, "feeling of irritation, displeasure, distaste" (c. 1200, still in Shakespeare), from Old French enoi, anoi "annoyance;" the same French word was borrowed into English later in a different sense as ennui. And compare Spanish enojo "offense, injury, anger;" enojar "to molest, trouble, vex." Middle English also had annoyful and annoyous (both late 14c.).