Ratings of chess engines are well established on the known rating lists. Will anything change when we feed engines with only (playable) gambit positions? We carefully created 100 gambit opening positions to find out.
Conditions
1. Only active engines, those who are in development.
2. Time control CCRL style, 40/2. Hash table size 128Mb.
3. Cutechess-cli concurrency compatible.
Excluded Engines
1. Inactive engines, those engines who haven't been updated
for a long time.
2. Known derivatives.
Gambit Rating List
2021-12-15
Gambit Rating List
2021-09-30
on 20 cores and 4Gb Hash
Whenever a new engine manages to enter the top-5 of the Gambit Rating List it wil be included in the 20 core rating list. The newest entry is Koivisto 6.16 The latest update update is Komodo-Dragon 2.5 which gained 63 elo and is now only 13 elo points away from Stockfish 14.
# PLAYER : RATING POINTS PLAYED (%) 1 Stockfish 14 : 3608.5 408.5 600 68 2 Komodo-Dragon 2.5 : 3595.6 338.5 500 68 3 Komodo-Dragon 2 : 3533.5 299.5 500 60 4 Ethereal 13.25-NNUE : 3437.4 250.0 600 42 5 Koivisto 6.16 : 3425.6 239.0 600 40 6 SlowChess 2.7 : 3421.2 235.0 600 39 7 RubiChess 2.2 : 3415.2 229.5 600 38
___________________________________________________________________________________________________
CCRL vs GRL
a comparison
With the start of the Gambit Rating List we modeled the ratings to be compatible with CCRL. Now (day of writing July 9, 2021) with over 90,000 games played we can make a comparison and conclude it does. Consider:
# PLAYER : RATING PLAYED CCRL Difference 1 Stockfish 13 : 3667.5 3500 3506 161 2 Komodo-Dragon 1 : 3581.3 3000 3469 112 3 Lc0 v27 : 3529.8 800 ---- not listed on the CCRL 40/15 list. 4 SlowChess 2.6 : 3421.9 2400 3379 42 5 RubiChess 2.1 : 3380.5 2900 3338 42 6 Pedone 3.1 : 3361.7 2900 3334 27 7 Igel 3.0.5 : 3355.0 2900 3342 13 8 Ethereal 12.75 : 3353.8 2700 3320 33
Our first conclusion is that the top engines profit the most, see the full list. A few observations:
Observation 1 - Looking at the top-3
Engine | CCRL elo | Difference | GRL elo | Difference |
Stockfish 13 | 3506 | 3667 | ||
Komodo Dragon | 3469 | -37 | 3581 | -86 |
SlowChess 2.6 | 3379 | -127 | 3421 | -246 |
On CCRL the difference between Stockfish and Komodo is 37 elo, on GRL 86.
On CCRL the difference between Stockfish and SlowChess is 127 elo, on GRL 246.
Stockfish and Komodo massively profited from the gambit openings.
Observation 2 - if we look further from rank 4-20 (with the exception of Nemorini) we see that these engines still profit from the gambit openings but less and less. Engines after rank 20 even start to lose considerable elo. The biggest exceptions are : Tucano 9.0 +45 | Benjamin +41 and Supernova 2.3 +35. While we are not familiar with the playing style of Tucano and Supernova we are with Benjamin because it's the gambit version of ProDeo.
Observation 3 - Benjamin -- because of its playing style -- greatly profits (+87 elo) when we compare the ratings with ProDeo 3.0 and ProDeo 3.1 while 3.0 and 3.1 definitely are stronger, not only in self-play but also when we compare the ratings of CCRL and GRL.
Engine | CCRL elo | GRL elo |
ProDeo 3.1 | 2798 | 2791 |
ProDeo 3.0 | 2776 | 2729 |
Benjamin 1.0 | 2699 | 2739 |
On CCRL ProDeo 3.0 scores 77 elo more than Benjamin while on the GRL Benjamin rates 10 points more than ProDeo 3.0, a difference of 87 elo!
Conclusions
1. The GRL makes sense.
2. Making a special gambit version makes even more sense.
___________________________________________________________________________________________________
Notes
Anchore Engines are inactive rock solid engines with well established elo ratings that are needed to stabilize the ORDO elo calculation. Anchore Engines will disappear in time when more games are played, they are only needed at the start of a rating list. However games played by Anchore Engines are included in the PGN download.