It was a teeny weeny debt and the service cost very little. Maybe under current conditions, it was worth retiring finally. Osborne is just having fun with his pride comment. It's about finance.
Think we may still have some outstanding debt from the Napoleonic wars that it makes little sense to retire. Last I looked we had, anyway.
Shows how in the long run, the cost of debt principle erodes in a way that suits the debtor.