The problem comes when they have enough power to send a couple polite enough officers who will insist you come in to clear up some confusion by answering a few questions on the record, you are forced one way or another into their vehicle, and you're just never seen again hide or hair.
In one permutation or another that's how it happens, and you'd be surprised how quickly it can happen here now that an unaccountable network of secret police and gulags has been constructed and normalized in American society.
It's easy to say from a state of calm that you'd theoretically "not comply", whether that means you would physically struggle with them or run away or retrieve a weapon from your home or whatever. It's another thing entirely to be actually confronted with this scenario and to make that dire decision with very short notice. You may only have seconds to both change mindsets to fight or flight and come up with a plan of action. Frankly, many people are not capable of this, and many more can't come up with actions that would get them far.
The other thing is that people don't know that that's the outcome. They bargain with themselves because it's an easier option than the above. "Maybe they do just want me to answer some questions?" "What's the worst these unassuming guys could do?" People tend to normalcy and civility, not fighting authority.
All that to say, if you don't want to comply you need to make tangible plans and preparations and not just self-assuring thoughts, because they will have certainly prepared for the possibility that you will struggle, or run, or retrieve a weapon.