We use the STS test suite that was developed during the years 2009-2010 by Dann Corbit and Swaminathan Natarajan containing 1500 theme based STS positions for the purpose of tuning engines and in particular the evaluation function. And with Fruit 2.1 we are going to explore its mobility. With MEA (pre-installed) we run STS at 5 different fast time controls and produce and overview.
Click picture to enlarge
The first table shows the sorted individual results of each setting for each time control used. The second table shows a sorted list of the overall rating by adding up the 5 gained ratings divided by 5.
We are now ready for step-2, the eng-eng testing part. And we are especially interested [mob=105] candidate improvement version and if it scores better against the official Fruit 2.1. Also if [mob=125] will do worse against the official Fruit 2.1 but let's test them all anyway.
Keep track of your eng-eng match results. As one can see from the above picture PET offers you 2 time controls. Just store your match resullts in the tc-one.txt and tc-two.txt files, see pre-installed example. A more advanced (stand alone) system for match control based on eng-eng results we like to introduce now.
LOS.EXE is a simple command line tool that will read the PGN of your matches and stores the overall information in a text-based database called LOS.TXT whereafter your browser will start and show you the results. When using the 6 Fruit and 2 Stockfish matches of this page we get:
Will add test.pgn to the LOS.TXT database and show you the updated web-page.
LOS test.pgn 100
A third paramter will override the default setting of 1000 and produce the development of the match in steps of 100 games instead of 1000. It can very insightful to study the obstinacy how matches (which engines are so close in strength) may sometimes develop and why programmers play so many games. Consider an old ProDeo match as an example. It's only after 4500 games ProDeo X is starting to see the light.
Format of the LOS.TXT database
The most easy way the maintain the database is after adding a match to open LOS.TXT with a text editor and modify the first [Match] and second [Parameter] items to your wishes.
Fruit [mob=95] - Fruit 2.1,[Piece Activity = 95],1000,49.9%,46.8%,40/10,2018.06.20,mob-95.pgn
Just stick to the comma driven format.
While the LOS program is included in the PET download below you can download it seperately. Source code included for further development.