The French right, left, and centre had all now failed. The last cohesive French force consisted of two battalions of the Old Guard stationed around La Belle Alliance; they had been so placed to act as a final reserve and to protect Napoleon in the event of a French retreat. He hoped to rally the French army behind them, but as retreat turned into rout, they too were forced to withdraw, one on either side of La Belle Alliance, in square as protection against Coalition cavalry. Until persuaded that the battle was lost and he should leave, Napoleon commanded the square to the left of the inn. Adam's Brigade charged and forced back this square, while the Prussians engaged the other.

As dusk fell, both squares withdrew in relatively good order, but the French artillery and everything else fell into the hands of the Prussian and Anglo-allied armies. The retreating Guards were surrounded by thousands of fleeing, broken French troops. Coalition cavalry harried the fugitives until about 23:00, with Gneisenau pursuing them as far as Genappe before ordering a halt. There, Napoleon's abandoned carriage was captured, still containing an annotated copy of Machiavelli's The Prince, and diamonds left behind in the rush to escape. These diamonds became part of King Friedrich Wilhelm of Prussia's crown jewels; one Major Keller of the F/15th received the Pour le Mérite with oak leaves for the feat. By this time 78 guns and 2,000 prisoners had also been taken, including more generals.