We only have one above average perimeter one on one defender out there in Aminu. Vasquez lacks the foot speed and constantly gets beat by quicker guards, and Rivers is a little in over his head right now until he gets used to things. Not to mention that Ellis is hard for anyone to guard. And he doesn't guard people running off screens so good.
It'll take time.