It's just clearly so disruptive to his rhythm, and the team's rhythm overall. He needs to get an actual stretch of minutes, preferably with shooters around: JJ would be nice.
And then yanking him and replacing him makes everyone else get disrupted too.