Some £2.1 million would have been spent on the new scheme for London Assembly members and himself, but Mr Khan believed this could not be justified while heavy cuts were being made in elsewhere.
The decision over the four-year plan comes after the government refused local councillors and mayors to gain access to the Local Government Pension Scheme.
But ministers did grant City Hall and the Mayor of London permission to set up their own scheme as they are full-time paid members of staff and not volunteers.
A source told getwestlondon that Mr Khan does not want to appear to be making politicians richer whilst Londoners bear the brunt of it through their taxes.
The anonymous Assembly member said: "He does share the concerns of members, but ultimately cannot justify spending the money."