1580s, "act of referring" (some matter, to someone for consideration), from refer + -ance, or else from French référence, from Medieval Latin *referentia, from Latin referentem (nominative referens), present participle of referre.
Meaning "direction to a book or passage" where certain information may be found is recorded from 1610s. By 1837 as "one who or that which may be referred to." The meaning "testimonial" is from 1895. Reference book , a dictionary, encyclopedia, or similar book intended to be consulted as occasion requires, dates from 1808; reference library is by 1834. Phrase in reference to is attested from 1590s. "By slipshod extension, the word is often now made to mean a person to whom r[eference] is permitted as a witness to character, & even a written testimonial" [Fowler, 1926]. The earlier word for "one who gives characters for people seeking employment" was referee (1862) but this word had a bad savor, of literate accomplices of professional beggars and thieves.