TCEC Cup 5

Abstract

TCEC Cup 5, the knockout event of TCEC Season 17, involved just sixteen of the top engines with slightly varied rules. Tie-breaks kicked in after four games and the initial tempo of 30′+5″/move quickened with every two games after the first eight.1

¹
Precisely, games 9-10 @ 16′+4″, g11-12 @ 8′+3″, g13-14 @ 4′+2″, g15-16 @ 2′+1″; remainder, ‘Bullet’ @ 1′+1″.

The ‘acceleration formula’ soon tested the competitors, the GUI, the kibitzers and the audience’s concentration. Would the shorter format, a smaller sample size of ability, lead to more surprises? The play was a strong contrast to the extended marathon of the TCEC17 Superfinal but an equally exciting sprint celebration of the game.

Some of the ‘top 16’ engine authors took a break at this point. Fire, KomodoMCTS, Houdini, Andscacs and Laser were not entered in the lists, making way for Arasan, Vajolet2, Pedone, Nemorino and Winter. Engine versions and nominal Elo figures did not change between rounds. Engine details are available elsewhere (CPW, 2020; Haworth and Hernandez, 2020a) but their logos in seed order are listed here in Fig. 1.

The ‘standard pairing’ was again used, with seed s playing seed 2^5-r-s+1 in round r if all wins went to the higher seed. Thus, seed s1 (the Cup holder, Stockfish) would have played s16, s8, s4 and s2 if all had survived long enough – which did not quite happen. The higher seed, with its expected and actual performance, is listed first in Table 1. As in TCEC Cup 4 (Haworth and Hernandez, 2019), the matches were played out only until the result was decided.

Fig. 1.

Logos for the TCEC Cup engines in seeded order (Stockfish → Leela Chess Zero $\to \dots \to$ Winter).

The usual ‘TCEC opening’ team, the second author here and Jeroen Noomen, randomly chose from three books without repetition. Nelson’s League 1 book was used in round 1. His Premier Division book was used for the other rounds. Tie-breaker games started with positions from Jeroen’s ‘TCEC Seasons 9-17’ book which now contains 405 positions.

As in previous TCEC Cup events, interest focused on actual performance ‘%P’ compared with expected performance ‘E%P’ implied by TCEC Elo difference ‘Elo Δ’. The accuracy of the TCEC Elos especially for the newer engines, the upgrades to over half the field and the influence of the random openings were again the main uncertainties.

1. Rounds 1 and 2

Stockfish opened round 1 by despatching Winter 3-0. The latter would be at super-GM level even if its true Elo were 400 less so it is remarkable to see an engine dismissed in this way. The second match always features the two ‘middle seeds’ and it took eighteen games to determine Xiphos as the winner. After the first eight, the games accelerated memorably to the frantic Bullet tempo of 1′+1″/m at which point rofChade fell off the pace twice in a ‘D43’ QGD Semi-Slav – a highlight contest and a useful test of the TCEC infrastructure. As the inferior seed had previously been edged out of the TCEC17 League 1 play-off (TCEC, 2020) on the third ‘number of wins’ tie-break, we might say this was a dish of revenge served cold with a dash of anthropological sauce. Stoofvlees and Lc0 also progressed 3-0, surprisingly easily. Both AllieStein and Ethereal conceded one loss.

²
In these tables, the first-named engine played White first except where indicated by a ‘*’. Alongside the ‘%P’ column, ‘+’ (‘–’) indicates an unexpected excess (shortfall) of 5% in the their performance. ‘#g’ ≡ number of games.

In round 2, Stockfish continued unbeaten with only a moment’s hesitation against Xiphos. It seems better than its Elo in Rapid tempo. As is intrinsically likely, the second match provided close action but Komodo edged ahead in extra time and held with Black. Leela followed Stockfish’s scoring exactly and crushed ScorpioNN. AllieStein delivered the first comeback, bouncing back from a loss and like Komodo, moving ahead in extra time and holding the last. These engines have no nerves.

Table 1

TCEC Cup 5: round one results from the winner’s perspective.2

Table 2

TCEC Cup 5: round two, quarter-final results from the winner’s perspective

2. The semi-finals, Bronze play off and final

The top four seeds had come through the early rounds, which served up the same engine quartet as TCEC Cup 4 (Haworth and Hernandez, 2019) but with an adjusted pairing. Stockfish continued its demolition run, this time giving Komodo little hope of advantage over the board: 2½-½. Leela’s match with AllieStein was a completely different story. These two neural network engines are closely related and nominally only 25 Elo points apart. Toe to toe they stood, drawing in various ways until Leela conquered in game 7. Here, with the game edging away, AllieStein suddenly realised that it was lost on move 39 while Leela had to be persuaded by the board itself some fifteen moves later. GM Matthew Sadler: “I found the Allie-Leela match of a really high level. I thought the games where Leela was White against Allie (the Trompovsky, the Sämisch and the Benoni) were exceptionally interesting: encounters of the very highest quality.”

Both the Bronze final and the actual final were completed in the minimum four games. AllieStein took third place with a 3-1 win over Komodo. The latter had ignored contrary opinions to call their 294-move game as a draw on move 13. Stockfish encountered marginally more resistance from Leela Chess Zero after building on the helpful ‘D16’ QGD Slav opening to win. Matthew: “The final was also excellent of course. Stockfish made a great job of a very favourable White opening in the first game, and then just managed to keep Leela at bay with Black. These two engines are so close: Season 18 is going to be another thriller!” The evaluation graphs of the eight games as in Fig. 2 give some sense of the progress of these two matches.

Table 3

TCEC Cup 5: round three, semi-final results from the winner’s perspective

In conclusion, we congratulate Stockfish on retaining the TCEC Cup, this time unbeaten – something of a consolation after losing the 17^th TCEC championship to Leela Chess Zero. The battle for supremacy between ‘Shannon AB’ and ‘Neural Network’ engines continues to TCEC Season 18 and it may not be long before some hybrid engine combining both of their strengths triumphs over all.

Fig. 2.

The evaluation curves of games 1-4 of the Bronze Final (a-d) and the Final (e-h).

Table 4

TCEC Cup 5: round two, bronze-final and final results from the winner’s perspective

We also thank all those who joined us on the TCEC chat line with good humour and assorted, even deeply perceptive observations not only about the chess but also about the more serious matter of life under lockdown. If we may single out one contributor here, special thanks to ‘lmabacus’ (2020) for curating some diverting sidelights on the TCEC seasons. The extreme games are noted in Table 5 but the full detail of TCEC Cup 5 is only available online (Haworth and Hernandez, 2020b). All decisive games have been completed by the first author’s Fritz17 at depth 24 to mate for those who want to teach chess, benchmark their own technique or just see the king fall. The entries into the land of sub-8-man chess have also been flagged with depths noted (de Man et al., 2018; Lomonosov, 2012). Our thanks to Matthew Sadler, the leading chess commentator on the neural-network engines (Sadler and Regan, 2019), for his observations above on two of the key matches.

Table 5

The shortest and longest 1-0, drawn and 0-1 games in each phase of TCEC Cup 5: ‘16/13’ in row 1, column 8 for example means ‘game 16 in the pgn, game 13 in the match’

References

CPW (2020). https://tinyurl.com/icga046. Biographies of chess engines, authors and developers.

De Man, R., Fiekas, N. & Guo, B. (2018). https://tinyurl.com/icga007. Fiekas’ interface to ‘Syzygy formatted’ de Man sub-7-man and Guo 7-man

{DTZ}_{50}^{^{″}}

EGTs.

Haworth, G.McC. & Hernandez, N. (2019). TCEC Cup 4. ICGA Journal, 41(4), 259–263. For additional statistics and annotated pgn files, see http://centaur.reading.ac.uk/86831/.

Haworth, G.McC. & Hernandez, N. (2020a). The 17^th Top Chess Engine Championship: TCEC17. Submitted to the ICGA Journal, see http://centaur.reading.ac.uk/89999/ for additional statistics and annotated pgn files.

Haworth, G.McC. & Hernandez, N. (2020b). TCEC Cup 5. This note, submitted to the ICGA Journal, see http://centaur.reading.ac.uk/90606/ for additional statistics and annotated pgn files.

‘lmabacus’ (2020). https://tinyurl.com/lmabacus01. Various TCEC-related records.

Lomonosov (2012). http://tb7.chessok.com/. Query interface to sub-8-man DTM EGTs.

Sadler, M. & Regan, N. (2019). Game Changer: AlphaZero’s Groundbreaking Chess Strategies and the Promise of AI. New in Chess.

TCEC (2020). https://www.tcec-chess.com/archive.html. The archive of past TCEC seasons.

TCEC Cup 5

Abstract

1 Precisely, games 9-10 @ 16′+4″, g11-12 @ 8′+3″, g13-14 @ 4′+2″, g15-16 @ 2′+1″; remainder, ‘Bullet’ @ 1′+1″.

2 In these tables, the first-named engine played White first except where indicated by a ‘*’. Alongside the ‘%P’ column, ‘+’ (‘–’) indicates an unexpected excess (shortfall) of 5% in the their performance. ‘#g’ ≡ number of games.

References

¹
Precisely, games 9-10 @ 16′+4″, g11-12 @ 8′+3″, g13-14 @ 4′+2″, g15-16 @ 2′+1″; remainder, ‘Bullet’ @ 1′+1″.

²
In these tables, the first-named engine played White first except where indicated by a ‘*’. Alongside the ‘%P’ column, ‘+’ (‘–’) indicates an unexpected excess (shortfall) of 5% in the their performance. ‘#g’ ≡ number of games.