billy-goat (n.)

masc. proper name, from Old North French Willaume, Norman form of French Guillaume, of Germanic origin (cognates: Old High German Willahelm, German Wilhelm), from willio "will" (see will (n.)) + helma "helmet," from Proto-Germanic *helmaz "protective covering" (from PIE root *kel- (1) "to cover, conceal, save;" compare helm (n.2)). After the Conquest, the most popular given name in England until supplanted by John.

Old English gat "she-goat," from Proto-Germanic *gaito (source also of Old Saxon get, Old Norse geit, Danish gjed, Middle Dutch gheet, Dutch geit, Old High German geiz, German Geiß, Gothic gaits "goat"), from PIE *ghaid-o- "young goat," also forming words for "to play" (source also of Latin hædus "kid").

They are sprightly, capricious, and wanton, and their strong odor (technically called hircine) is proverbial. [Century Dictionary]

The word for "male goat" in Old English was bucca or gatbucca (see buck (n.)) until late 1300s shift to he-goat, she-goat (Nanny goat is 18c., billy goat 19c.). Meaning "licentious man" is attested from 1670s (hence goat-milker, name of a bird formerly believed to suck the milk from goats at night, but also old slang for "a prostitute," also "the female pudendum"). To get (someone's) goat is by 1908, American English, the source of many fanciful explanation stories; perhaps from French prendre sa chèvre "take one's source of milk," or more likely it is "to steal a goat mascot" from a racehorse, warship, fire company, military unit, etc.

... to become separated from your goat is a thing no soldierman is willing to contemplate. ["Letitia, Nursery Corps, U.S.A.," in American Magazine, vol. lxiv, June 1907]

"club," 1848, American English, originally burglars' slang for "crowbar." The meaning "policeman's club" is recorded by 1856, probably from nickname of William, applied to various objects (compare jack, jimmy, jenny). But compare French bille "a short, stout stick" (see billet (n.1)).