Please consider offering answers and suggestions to help other students!
And if you fix a problem by following a suggestion here,
it would be great if other interested students could see a short
"Great, fixed it!" followup message.
Hello, my agent was getting rank 1 consistently, but sometimes it got the second rank; when I investigated the cause, I found this on the leaderboard.
LEADERBOARD AFTER 1000 GAMES
Resistance Wins: 268, Spy Wins: 732, Resistance Win Rate: 0.2680
1: SatisfactoryAgent | win_rate=0.4847 res_win_rate=0.2744 spy_win_rate=0.8027 | errors=0 games=1859 wins=901 losses=958 res=1119 spy=740 res_wins=307 res_losses=812 spy_wins=594 spy_losses=146
2: DumbAgent | win_rate=0.4762 res_win_rate=0.2809 spy_win_rate=0.8253 | errors=0 games=1915 wins=912 losses=1003 res=1228 spy=687 res_wins=345 res_losses=883 spy_wins=567 spy_losses=120
3rd and 4rth are irrelevant.
If you look at the res_win_rate and spy_win_rate, my agent's individual rates are higher than the SatisfactoryAgent's win rates. I still lost because my agent played 1915 games while SatisfactoryAgent played only 1859.
I would like to argue that my agent still beat SatisfactoryAgent because if you play more games, you will obviously lose more.
Just in case, I'll add this leaderboard for the next tournament
LEADERBOARD AFTER 1000 GAMES
Resistance Wins: 272, Spy Wins: 728, Resistance Win Rate: 0.2720
1: DumbAgent | win_rate=0.5095 res_win_rate=0.3138 spy_win_rate=0.8441 | errors=0 games=1843 wins=939 losses=904 res=1163 spy=680 res_wins=365 res_losses=798 spy_wins=574 spy_losses=106
2: SatisfactoryAgent | win_rate=0.4739 res_win_rate=0.2693 spy_win_rate=0.7958 | errors=0 games=1840 wins=872 losses=968 res=1125 spy=715 res_wins=303 res_losses=822 spy_wins=569 spy_losses=146
In this leaderboard, there is only a difference of 3 games between the agents, and my agent still won by 3%.
I would like Mr. Andrew's view on this. Will you consider this difference between the number of games a significant factor while evaluating?