In lexicons of sea language going back to 1759, the bitter end is the part of a cable which is round about the bitts (the two great timbers used to belay cables) when the ship is at anchor (see bitt).
Bitter end of the Cable, the End which is wound about the Bitts. ["The News-Readers Pocket-Book: Or, a Military Dictionary," London, 1759]
So, when a cable is played out to the bitter end, there is no more left to play. The term began to be used c. 1835 in non-nautical use and with probable influence of or merger with bitter (adj.).