Old English *steortian, *stiertan, Kentish variants of styrtan "to leap up" (attested only in Northumbrian past participle sturtende), from Proto-Germanic *stert- (source also of Old Frisian stirta "to fall, tumble," Middle Dutch sterten, Dutch storten "to rush, fall," Old High German sturzen, German stürzen "to hurl, throw, plunge"). According to Watkins, the notion is "move briskly, move swiftly," and the Proto-Germanic word is from PIE root *ster- (1) "stiff."
From "move or spring suddenly," sense evolved by c. 1300 to "awaken suddenly, flinch or recoil in alarm," and by 1660s to "cause to begin acting or operating." Meaning "begin to move, leave, depart" (without implication of suddenness) is from 1821. The connection probably is from sporting senses ("to force an animal from its lair," late 14c.). Transitive sense of "set in motion or action" is from 1670s; specifically as "to set (machinery) in action" from 1841.
Related: Started; starting. To start something "cause trouble" is 1915, American English colloquial. To start over "begin again" is from 1912. Starting-line in running is from 1855; starting-block in running first recorded 1937.