1650s, powny, from Scottish, apparently from obsolete French poulenet "little foal" (mid-15c.), diminutive of Old French poulain "foal," from Late Latin pullanus "young of an animal," from Latin pullus "young of a horse, fowl, etc." (from PIE root *pau- (1) "few, little") [Skeat's suggestion, still accepted]. Compare, from the same source, foal, filly, Sanskrit potah "a young animal," Greek polos "foal," Latin pullus "young animal," Lithuanian putytis "young animal, young bird."
German, sensibly, indicates this animal by attaching a diminutive suffix to its word for "horse," which might yield Modern English *horslet. Modern French poney is a 19c. borrowing from English. Meaning "crib of a text as a cheating aid" (1827) and "small liquor glass" (1849) both are from notion of "smallness" (the former also "something one rides," a translation being something that enables a student to "get along fast"). As the name of a popular dance, it dates from 1963. The U.S. Pony Express began 1860 (and operated about 18 months before being superseded by the transcontinental telegraph). The figurative one-trick pony is 1897, American English, in reference to circus acts.