"small piece," c. 1200; related Old English bite "act of biting," and bita "piece bitten off," which probably are the source of the modern words meaning "boring-piece of a drill" (the "biting" part, 1590s), "mouthpiece of a horse's bridle" (mid-14c.), and "a piece (of food) bitten off, morsel" (c. 1000). All from Proto-Germanic *biton (source also of Old Saxon biti, Old Norse bit, Old Frisian bite, Middle Dutch bete, Old High German bizzo "biting," German Bissen "a bite, morsel"), from PIE root *bheid- "to split."
Meaning "small piece, fragment" of anything is from c. 1600. Sense of "short space of time" is 1650s. Theatrical bit part is from 1909. Money sense "small coin" in two bits, etc. is originally from the U.S. South and the West Indies, in reference to silver wedges cut or stamped from Spanish dollars (later Mexican reals); transferred to "eighth of a dollar."