It doesn't really work like that in my opinion.
He saved us in those years because he overperformed, as in his performances helped cover up a shambolic defense in that time period.
De Gea is currently saving the expected amount of saves for an average keeper, but for what we expect from a keeper like De Gea we call that underperforming. Still, I would say it's harsh to say he's the one costing us if he's putting in an average performance.
Rarely can one part of the team be highlighted to be costing us when it's a collective lack of effort.
What if the defense performed out of their mind and barely let an attacker through? Then the defense would be saving us. Or if our attackers scored a crazy amount of goals each match so that it didn't matter what happened at the back, then our attackers would be saving us.
We need to up our game across the board to be able to challenge. We can't be a team that only does well when our keeper is outperforming expectations.