I think that a lot of scouts, professional and non-professional, consistently show a bunch of really dodgy signs. The best guys aren't flawless, but they're usually aware of the flaws they tend to make and often will be upfront about them.
Aside from the issue you describe (which is basically just a mix between moving the goal posts and special pleading), these are some pretty common ones:
- Aestheticism: A lot of people, whether scouts or just people interested, tend to make the mistake of confusing their aesthetic preference for efficacy. It's fairly common to see people love mind-blowing handles, even if they don't produce much of an impact on the defender, or for people to love super deep shooting prospects even if they're not that good at it: they just enjoy the visual of people taking that shot.
- Highlight Clip Syndrome: For me, it usually takes a good 5 games to get what I call a ''decent'' grasp of a prospect, and I usually try to wait until 8 or 9 before making strong judgements. Some people watch a 12 minute highlight clip on YouTube and consider themselves an expert: obviously this is more of a casual fan issue than a professional scout one.
- Stubbornness: A lot of people come to views early in the draft process, or even pre-draft sometimes, and just refuse to change their mind as evidence mounts up. It's healthy and important to accept new information and change your mind. For example, early in the season I had Nico Mannion in the teens or even late lottery for a variety of reasons, but as the season wore on it became clear that his finishing problems were worse than I had thought and his shot didn't quite show up off the dribble how I liked. So I've dropped him into the mid 20s. Gotta be willing to change your mind.
- Body Shape Judgments: So many people look at a player who looks good and make judgements from there, ignoring the game. You see people saying ''ah but he's 6'8, 220lbs, and a ball handler!'' and it's like yeah but is he any good at that ball handling or is he just on a team that's letting him do whatever?
- Team considerations: Some teams in college make players look better at some things than they are, and other teams make them look worse. Kentucky tends to suppress almost everyone truly great, whereas FSU tends to make people look like better defenders than they are. Think: are you evaluating a prospect's abilities, or are you evaluating the prospect's individual performance within this singular team context?
These are just some examples, there are more.