Fun with Rating Lists

Part One - What if we remove all draws from games and only calculate a rating list based on won and lost games, what would change and which engine profit the most.


Part Two - What are the best killer engines. The best killer engine is the engine that has won its games in the shortest number of moves. The minimum number of won games must be at least 100.


Part One

We start removing all draws from the mainbase of the Gambit Rating List, then run a new fictional rating list and last create a sorted list which engines profit the most....and those who don't profit at all! And so we get 3 lists.

The normal official Gambit Rating List. For eye-pleasing reason we only list the top-10, view the full list here.

 # PLAYER                :  RATING  ERROR  POINTS  PLAYED   (%)  LOS    W     D    L  Draw%
 1 Stockfish 13          :  3677.1   17.1  1555.5    2000    78  100  1157   797   46   40
 2 Stockfish 21-05-18    :  3651.2   17.5   795.0    1000    80  100   610   370   20   37
 3 Stockfish 12          :  3617.6   28.2  1130.5    1500    75   99   817   627   56   42
 4 Komodo-Dragon 1       :  3586.5   26.1  1021.0    1500    68   62   680   682  138   45
 5 Komodo-Dragon 2       :  3581.3   23.4  1771.5    2400    74  100  1276   991  133   41
 6 Lc0 v27               :  3532.0   20.2   501.0     800    63   97   307   388  105   49
 7 Stockfish 11          :  3501.6   35.6   647.5    1000    65   89   436   423  141   42
 8 Stockfish 10          :  3481.9   16.9   834.0    1300    64  100   561   546  193   42
 9 Komodo 14             :  3433.6   20.4   443.5     900    49   75   235   417  248   46
10 SlowChess 2.6         :  3425.2   19.7   284.0     600    47   98   143   282  175   47

__________________________________________________________________________________________________

The new fictional rating list based on won and lost games only, a massive elo increase at the top. View the full list here.

 # PLAYER                :  RATING  ERROR  POINTS  PLAYED   (%)  LOS   W     D   L  Draw%
 1 Stockfish 13          :  4098.1   50.7  1157.0    1203    96   89  1157   0   46   0
 2 Stockfish 21-05-18    :  4058.9   82.7   610.0     630    97   99   610   0   20   0
 3 Stockfish 12          :  3934.3   69.2   817.0     873    94  100   817   0   56   0
 4 Komodo-Dragon 2       :  3842.2   36.8  1276.0    1409    91   82  1276   0  133   0
 5 Komodo-Dragon         :  3821.2   48.6   680.0     818    83  100   680   0  138   0
 6 Lc0 v27               :  3680.9   53.1   307.0     412    75   85   307   0  105   0
 7 Stockfish 11          :  3648.7   50.5   436.0     577    76   98   436   0  141   0
 8 Stockfish 10          :  3601.3   56.8   561.0     754    74  100   561   0  193   0
 9 Komodo 14             :  3470.9   50.6   235.0     483    49   60   235   0  248   0
10 SlowChess 2.6         :  3462.0   43.5   143.0     318    45   99   143   0  175   0

__________________________________________________________________________________________________

So now, does every engine profit? Not really and that is softly speaking, view the full list here .

 No. Engine                   Old-elo  New-elo    Diff
  1. Stockfish 13            : 3677.1 - 4098.1 = 421.0
  2. Stockfish 21-05-18      : 3651.2 - 4058.9 = 407.7
  3. Stockfish 12            : 3617.6 - 3934.3 = 316.7
  4. Komodo-Dragon 2         : 3581.3 - 3842.2 = 260.9
  5. Komodo-Dragon 1         : 3586.5 - 3821.2 = 234.7
  6. Lc0 v27                 : 3532.0 - 3680.9 = 148.9
  7. Stockfish 11            : 3501.6 - 3648.7 = 147.1
  8. Stockfish 10            : 3481.9 - 3601.3 = 119.4
  9. BitGenie 6.h64          : 2594.8 - 2689.8 = 95.0
 10. Jumbo 0.7.0             : 2579.5 - 2666.7 = 87.2
 
103. Pedone 3.0              : 3265.2 - 3231.3 = -33.9
104. Ethereal 12.00          : 3305.1 - 3269.0 = -36.1
105. Ethereal 12.75          : 3354.3 - 3313.1 = -41.2
106. Nemorino 6.00           : 3306.3 - 3264.4 = -41.9
107. SlowChess 2.4           : 3317.4 - 3274.7 = -42.7
108. Booot 6.5               : 3253.5 - 3185.8 = -67.7
109. Igel 3.0.0              : 3344.3 - 3275.9 = -68.4
110. Komodo 10               : 3278.3 - 3204.7 = -73.6
111. rofChade 2.3            : 3237.3 - 3160.5 = -76.8
112. Arasan 2.22             : 3098.8 - 3009.2 = -89.6
113. RubiChess 2.0           : 3149.5 - 2975.7 = -173.8 

Conclusion, how boring comp-comp draws at times are, let's keep them, else we get nonsense elo rating lists.

__________________________________________________________________________________________________

Part Two

in search for the best killer engine

The best killer engine is the engine that has won its games in the shortest number of moves. The minimum number of won games must be at least 100. For the Gambit Rating List we get:

PGN database      : pgn\mainbase-40-2.pgn
Minimal won games : 100
Longest game      : 398 moves  Booot 6.5 - Ethereal 12.00 1/2-1/2
Shortest game     :  11 moves  Foxsee 7.8 - BitGenie 5 1-0
Longest won game  : 249 moves  Lc0 v27 - Igel 3.0.0 1-0
Shortest won game :  11 moves  Foxsee 7.8 - BitGenie 5 1-0

 No. Engine               Games  Moves
  1. Marvin 5.0             579   49.8
  2. Lzero v27              591   51.4
  3. Benjamin               672   52.0
  4. Seer 2.0.0             425   52.4
  5. Monolith 0.3           660   52.4
  6. Seer 2.0.1             931   52.5
  7. Berserk 3.3.0          406   52.8
  8. Arasan 12.2            392   52.9
  9. Cheng 4.41             648   53.2
 10. Pedone 3.0             265   54.0
 
110. RubiChess 2.1          463   66.4
111. Olithink 5.9.5         199   66.8
112. Igel 3.0.0             111   67.6
113. Olithink 5.9.3         230   68.6
114. Nemorino 6.00          588   70.0
115. Olithink 5.9.9         205   71.9
116. Lc0 v27                307   88.9 

The oddity we see here that the CPU version of Lc0 lists second while the GPU version is at the bottom of the list. The full list plus the 4 PGN's can be viewed here.

__________________________________________________________________________________________________

CCRL

in search for the best killer engine

The number of engines of the CCRL list is too big to get reasonable results hence with split them into 8 elo sections and each elo section has its own best killer engine list. The links are:


/html/ccrl-3200-9999.htm
/html/ccrl-3000-3200.htm
/html/ccrl-2800-3000.htm
/html/ccrl-2600-2800.htm
/html/ccrl-2400-2600.htm
/html/ccrl-2200-2400.htm
/html/ccrl-2000-2200.htm
/html/ccrl-0000-2000.htm


Following the GRL Marvin 5.0 is the best killer engine and while the engine also tops the CCRL 3000-3200 elo pool we declare that Marvin 5.0 is the best killer engine.


For what's it worth of course