I'm not having much luck with putting a password back on:
Code:
Cannot require sv_password when server is already reserved or clients connected!
Another non-passwordy idea that would fix my issue would a cvar be to kick players that join when the game is live and there are enough players on.