as a conventional title of courtesy before a man's Christian name, mid-15c., unaccented variant of master (n.), but without its meaning. As a form of address when the man's name is unknown (often with a tinge of rudeness), from 1760.
The disappearance of master and mister, and the restricted and obsolescent use of sir, as an unaccompanied term of address, and the like facts with regard to mistress, Mrs., and madam, tend to deprive the English language of polite terms of address to strangers. Sir and madam or ma'am as direct terms of address are old-fashioned and obsolescent in ordinary speech, and mister and lady in this use are confined almost entirely to the lower classes. [Century Dictionary, 1895]