mid-14c., pencel, "an artist's small, fine brush of camel hair," used for painting, manuscript illustration, etc., from Old French pincel "artist's paintbrush" (13c., Modern French pinceau) and directly from Medieval Latin pincellus, from Latin penicillus "painter's brush, hair-pencil," literally "little tail," diminutive of peniculus "brush," itself a diminutive of penis "tail" (see penis).
Small brushes formerly were used for writing before modern lead or chalk pencils. Sticks of pure graphite (commonly known as black lead) were used for marking things in England from the mid-16c., and the wooden enclosure for them was developed in the same century on the Continent. This seems to have been the time the word pencil was transferred from a type of brush to "graphite writing implement." The modern clay-graphite mix was developed early 19c., and pencils of this sort were mass-produced from mid-19c. Hymen L. Lipman of Philadelphia obtained a patent for the pencil with an attached eraser in 1858.
Derogatory slang pencil-pusher "office worker" is from 1881 (pen-driver "clerk, writer" is from 1878); pencil neck "weak person" first recorded 1973. Pencil-sharpener as a mechanical device for putting the point on a lead pencil is by 1875.