Admittedly, they should have left at the end of The God Complex because that was a fantastic goodbye. But considering they were the Doctor's family, he could never leave them, they could never leave him - there really was no other way to separate them but through death.
They didn't die (at least not on screen and not at the age we knew them), they did leave. They got to live a long and happy life together in the 'past'.
Shout by DeletedBlockedParentSpoilers2014-01-22T15:17:32Z
amy and rory's death was so unnecessary, why couldn't they just leave