early 15c., bouget, "leather pouch, small bag or sack," from Middle French bougette, diminutive of Old French bouge "leather bag, wallet, pouch," from Latin bulga "leather bag," a word of Gaulish origin (compare Old Irish bolg "bag," Breton bolc'h "flax pod"), from PIE *bhelgh- "to swell," extended form of root *bhel- (2) "to blow, swell."
The modern financial meaning "statement of probable expenditures and revenues" (1733) is from the notion of the treasury minister keeping his fiscal plans in a wallet. Also used from late 16c. in a general sense of "a stock, store, or collection of miscellaneous items," which led to 18c. transferred sense "bundle of news," hence the use of the word as the title of some newspapers.