"small sack," c. 1200, bagge, probably from Old Norse baggi "pack, bundle," or a similar Scandinavian source. OED rejects connection to other Germanic words for "bellows, belly" as without evidence and finds a Celtic origin untenable. In some senses perhaps from Old French bague, which is also from Germanic.
As disparaging slang for "woman" it dates from 1924 in modern use (but various specialized senses of this are much older, and compare baggage). Meaning "person's area of interest or expertise" is 1964, from African-American vernacular, from jazz sense of "category," probably via notion of putting something in a bag. Meaning "fold of loose skin under the eye" is by 1867. Related: bags.
To be left holding the bag (and presumably nothing else), "cheated, swindled" is attested by 1793. Many figurative senses, such as the verb meaning "to kill game" (1814) and its colloquial extension to "catch, seize, steal" (1818) are from the notion of the game bag (late 15c.) into which the product of the hunt was placed. This also probably explains modern slang in the bag "assured, certain" (1922, American English).
To let the cat out of the bag "reveal the secret" is from 1760. The source is probably the French expression Acheter chat en poche "buy a cat in a bag," which is attested in 18c. French and explained in Bailey's "Universal Etymological English Dictionary" (1736), under the entry for To buy a pig in a poke as "to buy a Thing without looking at it, or enquiring into the Value of it." (Similar expressions are found in Italian and German.) Thus to let the cat out of the bag would be to inadvertently reveal the hidden truth of a matter one is attempting to pass off as something better or different, which is in line with the earliest uses in English.
Sir Joseph letteth the cat out of the bag, and sheweth principles inimical to the cause of true philosophy, by wishing to make great men Fellows, instead of wise men ["Peter Pindar," "Peter's Prophecy," 1788]