c. 1200, materie, "subject of thought, speech, or expression," from Anglo-French matere, Old French matere "subject, theme, topic; substance, content, material; character, education" (12c., Modern French matière), from Latin materia "substance from which something is made," also "hard inner wood of a tree" (source also of Portuguese madeira "wood"), from mater "origin, source, mother" (see mother (n.1)). Or, on another theory, it represents *dmateria, from PIE root *dem-/*dom- (source of Latin domus "house," English timber). With sense development in Latin influenced by Greek hyle (see hylo-), of which it was the equivalent in philosophy.
Meaning "physical substance generally, matter, material" is early 14c.; that of "substance of which some specific object is made or consists of" is attested from late 14c. That of "piece of business, affair, activity, situation, circumstance" is from late 14c. From mid-14c. as "subject of a literary work, content of what is written, main theme." Also in Middle English as "cause, reasons, ground; essential character; field of investigation."
Matter of course "something expected" attested from 1739. For that matter attested from 1670s. What is the matter "what concerns (someone), the cause of the difficulty" is attested from mid-15c. To make no matter "be no difference to" also is mid-15c.