Ole wasn't as bad as some make out, but he also wasn't good enough to be a success longer term. It wasn't just the transfers in and out of the club that were the issue, it was also the tactics and coaching. His one big strength seemed to be man-management, but even that seemed to only be a short-term lift rather than a longer-term development of a good mentality.
The thing I hold against Ole more than other is that when he joined we still had quite a lot of younger players with high potential, exactly the type of players that need the right type of coaching and man-management to take the next step up. By the time Ole left, those players were now in the middle or later parts of their career and hadn't really improved at all. Most of them had had their best ever performances in their first season under Ole when he came in and cut out the toxic atmosphere that Mourinho had left behind, but then instead of improving further they just got worse and worse as time wore on. That five and a half years at the most important stage of their career of Mourinho followed by Ole just did a number on them and gave them little chance of proper development. A number of them also became quite injury prone during that period, and it could be argued that Ole relying on them to play while they had injuries contributed to their long-term physical decline.