c. 1400, "distrust, want of confidence, doubt of the ability or disposition of others," from Latin diffidentia "mistrust, distrust, want of confidence," from diffidere "to mistrust, lack confidence," from dis- "away" (see dis-) + fidere "to trust" (from PIE root *bheidh- "to trust, confide, persuade"). The opposite of confidence. Original sense (distrust of others) is obsolete; the modern sense is of "distrust of oneself, want of confidence in one's ability, worth, or fitness" (1650s), hence "retiring disposition, modest reserve."
Diffidence is a defect: it is an undue distrust of self, with fear of being censured for failure, tending to unfit one for duty. [Century Dictionary]