Rim defense is so important that if you were only allowed to defend one spot on the court, the rim should be it.
And in reality, that's kind of what's happening

The number 1 defensive team in the NBA is the Milwaukee Bucks. The Milwaukee Bucks are also #1 in the NBA at rim-protection. They're 4th in the league in blocks. They allow only 29% of opponent's FGAs to come within 6 feet of the hoop, which leads the league (by comparison, we are 21st in the NBA, allowing 36.4% of opponent FGAs to come within 6 feet of the rim).
And on those at-rim attempts, the Bucks hold opponents to 9.4%
below their averages; the runner-up for best OFG% at-rim is Toronto, who only hold their opponents 4.7% below their averages. So, exactly half. And you fall off to 3.2% below when you go to the 3rd ranked team, the Lakers. For comparison, we allow opponents to shoot 1.6%
above their averages on at-rim attempts.
By contrast, the Bucks let guys shoot threes. Opposing teams shoot 2.2% better than their averages from behind the arc against the Bucks. Yet they're the league's best defense. How?
Simple. Gotta protect the rim. Do that well enough, and everything else becomes secondary. If you can only take away one shot, make it the at-rim attempt. The lay-up/dunk is the league's most efficient shot. Remove it as an option for your opponent and your defense soars even if you aren't great on the perimeter.
Just for comparison, since Favors has come back we've held opponents to 4.9%
below their averages at the rim, which is 5th among all teams during that span. So yes, Favors coming back and us returning to a drop-scheme does seem to be helping. Of course the usual caveats about Houston missing their big 3 and Indiana missing Brogdon and stuff all apply.