Taking advantage of the disintegration of the Byzantine Empire, Kaloyan captured former Byzantine territories in Thrace. Initially he attempted to secure a peaceful division of the lands with the crusaders (or "Latins"). He asked Innocent III to prevent them from attacking Bulgaria. However, the crusaders wanted to implement their treaty which divided the Byzantine territories between them, including lands that Kaloyan claimed.

Kaloyan gave shelter to Byzantine refugees and persuaded them to stir up riots in Thrace and Macedonia against the Latins. The refugees, according to Robert of Clari's account, also pledged they would elect him emperor if he invaded the Latin Empire. The Greek burghers of Adrianople (now Edirne in Turkey) and nearby towns rose up against the Latins in early 1205. Kaloyan promised that he would send them reinforcements before Easter. Considering Kaloyan's cooperation with the rebels a dangerous alliance, Emperor Baldwin decided to launch a counter-attack and ordered the withdrawal of his troops from Asia Minor. He laid siege to Adrianople before he could muster all his troops. Kaloyan hurried to the town at the head of an army of more than 14,000 Bulgarian, Vlach and Cuman warriors. A feigned retreat by the Cumans drew the heavy cavalry of the crusaders into an ambush in the marshes north of Adrianople, enabling Kaloyan to inflict a crushing defeat on them on 14 April 1205.

Despite everything, the battle is hard and fought until late in the evening. The main part of the Latin army is eliminated, the knights are defeated and their emperor, Baldwin I, is taken prisoner in Veliko Tarnovo, where he is locked at the top of a tower in the Tsarevets fortress.

Word quickly spread around Europe of the defeat of the knights in the battle of Adrianople. Without a doubt, it was a great shock for the world at the time, due to the fact that the glory of the undefeatable knight army was known to everyone from those in rags to those in riches. Hearing that the knights, whose fame travelled far and wide, who had taken one of the largest cities at the time, Constantinople, the capital whose walls were rumoured to be unbreakable, was devastating for the Catholic world.