I don't really blame Potter for not fancying Ajax. Yes they are a big name but they have fallen from grace quite spectacularly in recent years.
They are looking for their 5th manager since Ten Hag left, they have been through the Overmars scandal and then his replacement Mislintat was a disaster, Van der Sar stepped down as CEO, they have a fairly poor squad and are currently 5th with the prospect of no European football next season.
So they aren't exactly an attractive proposition at the moment and to be honest Potter will more than likely be offered a better job.