Having prepared his army for a renewed attack on Antioch, John amused himself by hunting wild boar on Mount Taurus in Cilicia, where he accidentally cut himself on the hand with a poisoned arrow. John initially ignored the wound and it became infected. He died a number of days after the accident, on 8 April 1143, probably of septicaemia. John's final action as emperor was to choose Manuel, the younger of his surviving sons, to be his successor. John is recorded as citing two main reasons for choosing Manuel over his older brother Isaac: Isaac's irascibility, and the courage that Manuel had shown on campaign at Neocaesarea. Another theory alleges that the reason for this choice was the AIMA prophecy, which foretold that John's successor should be one whose name began with an "M". Fittingly, John's close friend John Axouch, although he is recorded as having tried hard to persuade the dying emperor that Isaac was the better candidate to succeed, was instrumental in ensuring that Manuel's assumption of power was free from any overt opposition.

Overall, John II Komnenos left the empire a great deal better off than he had found it. Substantial territories had been recovered, and his successes against the invading Petchenegs, Serbs and Seljuk Turks, along with his attempts to establish Byzantine suzerainty over the Crusader States in Antioch and Edessa, did much to restore the reputation of his empire. His careful, methodical approach to warfare had protected the empire from the risk of sudden defeats, while his determination and skill had allowed him to rack up a long list of successful sieges and assaults against enemy strongholds. By the time of his death, he had earned near universal respect, even from the Crusaders, for his courage, dedication and piety.