It's opinions, but I don't see both, nor our whole defense as the problem at all and I don't think we should invest much in it. Better to invest more in the offensive areas which are quite poor. Smalling, Jones or Bailly, we have showed strong defensive unit and being very solid, but going forward all our problems arise.
This might be better in another thread but I actually think our defense is our main problem, specifically our lack our full backs/wing backs. CB wise I think we are competitive but player for player our defense is some way behind City, Chelsea and Spurs IMO.
To be as concise as possible, we have a converted RW at RB and no quality LB (Darmian average on left, Young is converted RW, Blind average and slow). Defensively we are ok and offensively we are pretty poor. I see it like this:
A) we have minimal attacking threat from LB (leaving our LW isolated and making him very easy to mark)
B) we have limited attacking threat from RB (Valencia is good getting forward but his final pass/delivery has been poor for years - this problem is compounded when Mata plays RW because Valencia can't commit too far forward given Mata's lack of pace should we lose the ball)
C) we are very vulnerable down the LB channel - when Stoke & Huddersfield target you there and come away with points you know something is wrong. Martial/Rashford are exciting attacking players but they're playing almost like defensive wingers because Mou can't trust our LB.
D) we are ok defensively with Valencia, he sometimes gets caught out of position and isn't the strongest in the air but normally can correct mistakes with his pace (although now he is 32 that won't carry on forever)
Our weakness at fullback basically has the knock on effect of making it so much harder for us to attack because we have less numbers going forward and our RW/LW players are usually so far back having helped our our fullbacks they have to cover an unrealistic amount of ground to catch up with a counter attack. Basically leaving Lukaku alone to try and hold off 2 or 3 players when we hoof it up to him.