Old English distæf "long, cleft stick that holds flax for spinning," from dis- "bunch of flax" (cognates: Middle Low German dise, Low German diesse "a bunch of flax on a distaff;" compare bedizen) + stæf "stick, staff" (see staff(n.)).
Figurative of "women's work" from late 14c.; a synonym in English for "a woman, the female sex, female authority in the family," at least since late 15c., presumably because spinning was typically done by women of all ranks. Hence distaff side (1848) a 19c. collective name (affecting to be older) for the female members of a family, especially with reference to relationship and descent (opposed to the spear side).
St. Distaff's Day (1640s) was Jan. 7, when "women resumed their spinning and other ordinary employments after the holidays" [OED].