I found a new test suite called «Stockfish Test Suite 2021» (STS) in this German forum. The test is a compilation/extract of 20+ more or less well know test suites and contains 238 positions which were not solved by cfish on the author’s older computer. The epd-file is here: Stockfish-Test-Suite-2021
I run the test on my M1 with the fresh compiled Stockfish family binaries (see this page). Configuration: 5 seconds for each position, 1 thread, 1024 MB hash, tests executed with epdtest.
The results – or better: the differences – are quite astonishing, at least for me. Also published are the results of the ERET. You can download the log files for both tests here: log.zip
| Engine | ERET 111 pos. | STS 238 pos. |
|---|---|---|
| Black-Diamond-v13 | 81 | 40 |
| Bluefish-v13 | 82 | 21 |
| cfish-1103202 | 81 | 1 |
| corchess-1.3 | 79 | 8 |
| crystal-3.1 | 81 | 73 |
| Harmon-v13 | 72 | 2 |
| Honey-v13 | 81 | 3 |
| Oki-Maguro-v13 | 80 | 0 |
| Stockfish-13 | 82 | 1 |
| Sugar-AI-150 | 82 | 5 |
| Sugar-AI-ICCF-150 | 84 | 64 |