The adversarial system is designed to allow these kind of attacks.

"You don't know what you're talking about" is easily countered by "I invented these crypto systems; these have had no serious challenges in the X years since invented; I've defeated these crypto systems; I've published these papers in these peer reviewed journals; I have these honorary awards from these universities." etc etc.

It's a lot more troubling in other cases.