How do teams find and vet domain experts for eval and preference data?

Question from the other side of this. I'm a senior C++ dev, 20+ years, mostly on big legacy codebase. Lately I keep seeing eval and RLHF projects that want people like me, or doctors, lawyers, etc, not general annotators. Made me curious how it looks from the team side. If you run these pipelines, where do you actually find those experts? Vendors, freelance networks, or you recruit them yourself? And how do you know their work is any good? With general labeling you can check agreement between annotators. But if two senior engineers disagree on which answer is better, who decides? submitted by