TCEC13: The 13 th top chess engine championship

As for TCEC12, each engine played both White and Black from 14 defined four-ply openings. The results are as in Table 2: ‘P%’ is the %-score and ‘ ELO ± ’ is the change to the engine’s nominal ELO based on its performance. ‘nSB’ is the Sonnerborn-Berger score, normalised as for one double round-robin.

Online interest naturally focused on the new ‘NN approach’ engines LC0 and Deus X (Silver, 2018), Deus X being powered and trained by LC0 software. It is however trained from human games rather than from zero, which is the convention for training AlphaZero and LC0. As seen in the results of Table 2, and surprising to those not actually involved, LC0 justified its ‘wild card’ invitation with a comfortable win and Deus X was the runner-up on debut and in its very first version.

2. Division 3: two DRR phases, 14 rounds, 112 games, tempo 30′+10″/m

Again, the eight engines involved played both sides of 14 prescribed four-ply openings. LC0 upgraded to a new version. 13 games were won ‘below the diagonal’ including Nemorino-LC0 g11.1/45.

Ethereal (Grant, 2018) was way out on its own. LC0 and Arasan were 6.5 points behind, Arasan progressing courtesy of the one win between them, g19.1/73. The key error was 29. … Bg4??, which missed 30. c6+ Ka7 31. Rd3 Rxd5 32. Nxd5 Qxb2 33. Ne3 Qc1+ 34. Nd1 Qc2 35. Nf2. LC0 and Deus X, the latter some 2.5 points behind LC0, were in fact frustrated by overheating in the GPU hardware, which therefore had to be throttled back. First impressions are that LC0 and Deus X play in a more human way, being relatively strong on strategy like AlphaZero but weaker on tactics. Nemorino rather than Hannibal was demoted as it crashed twice, one being against Arasan.

Only nine of the 40 wins are below the diagonal in the cross-table of Table 4. Ethereal and ChessBrainVB quickly distanced the rest of the field and finished three points clear of Xiphos, an engine that double-promoted from Division 4 in TCEC12. Ethereal won, adding another 28 games without defeat. In game 19.2/74, Ethereal reached an adjudicated KRKRPP mate in 42 moves against Xiphos. There is always a question of what contribution the sub-7-man EGTs make. Here, Xiphos was not using the Syzygy 6-man EGTs (de Man, 2018) while Ethereal was in the end consulting them more than 10 million times per move.

Vajolet cut back on threads and power after two disconnects. Arasan had one less win than Nirvana and so was this time on the wrong end of the third tie-break.

Eight wins went to engines relatively lower in the final ranking, see Table 5, most notably g7.2/26 Laser’s defeat of Chiron (a wild finale of 30 moves and two Q-sacrifices) and ChessBrainVB’s two wins over Fritz, g6.2/22 and g20.2/78.

Other notable games included g1.3/3 Chiron-Fritz, EGT-adjudicated as a 61m mate after 83 moves: in fact, it had been a 7-man 46m mate after 72 moves but the shortest route to goal is not usually the one most easily traversed. Booot sadly got off on the wrong foot with disconnects in games 2.2 and 4.3. Ethereal playing Black swiftly demolished Fritz in g7.3/27 and Jonny in g8.2/30. In g9.4/36, Ethereal demonstrated the value of the EGTs in beating ChessBrainVB after reaching a KRPKNP endgame with mate in 37 moves: Ethereal consulted the EGT over 100m times on move 50w.

Ethereal and Chiron had established their claims to the top spots on the podium with the first round-robin. They extended away with Fizbo a distant third. Ethereal has just one loss, to Chiron, in its last 84 games and has uniquely promoted three times this season. It won both sides of an opening against ChessbrainVB, Jonny and Fritz here. ChessbrainVB mysteriously worsened with each round-robin and returned to Division 2 after being third at the mid-point. The two early crashes by Booot led to its downfall and saved Laser from the same fate. Given that crashes are so disappointing for the online audience, TCEC could usefully pull together the known intelligence on how to avoid them.

Andscacs, Ethereal, Ginkgo, Komodo and Stockfish updated for this season whereas Chiron, Fire and Houdini did not. A key question was whether Chiron and Ethereal would stay in the top division after their promotions. The mandated openings from the second author here specified the first eight moves.

After the first round-robin, Stockfish led Komodo with Andscacs, Ethereal and Houdini contesting third place. After colour-switching the engines in the second round-robin to level the playing field, a clearer potential podium suggested itself: Stockfish, Houdini, Komodo, Fire in equidistant line astern with Ethereal just fifth. However, a presumably updated Ethereal might fare better in TCEC Cup 1 (Haworth and Hernandez, 2019c), an interlude following this division. After the first quarter, where one might claim to be half-informed statistically, Ginkgo and Chiron were occupying the relegation zone. The matches Stockfish–Andscacs and Komodo–Ginkgo were 2-0 wins for the first-named engine.

At the half-way point, Stockfish had pulled 3½ points clear of Houdini, courtesy of two relatively successful results, 4-0 v Andscacs and 3½-½ v Ethereal. Both leaders remained unbeaten and had scored 3-1 against Komodo, which was clear 3^rd. Fire was a lonely 4^th: the top half of Division P seems to be unchallenged and perhaps sequenced. Ethereal just edged 5^th on number of wins but was only 1½ points clear of tail-ender Ginkgo.

The third DRR saw Komodo wake up, breathing fire. It inflicted a first loss on Stockfish and its third on Fire and Ethereal: it sustained no losses itself. It finally overhauled the still unbeaten but win-shy Houdini with scores of 4½/7 in RR5 and 5½/7 in RR6. Would Komodo continue in this vein: would Houdini’s +2 against Komodo save it in a tiebreak? Who would ultimately join Stockfish in the Superfinal? In RR8, g50.4/200, Komodo beat Stockfish and two games later, Stockfish beat Houdini: the first game had plenty of play left after 73 moves but the second was a clearer and quicker win from an advantageous opening.

OPEN IN VIEWER

Table 7

The Premier Division head-to-head and per-round-robin results: four DRR phases, 56 rounds

The division was marked by relatively few wins for Black, the long g14/4.2 Fire-Komodo battle being of particular interest. Perhaps the only two notable ‘underdog wins’ below the cross-table diagonal of Table 6 were by the demoted engines against Ethereal (games 25.4/100 and 52.1/209), which was only three points above demotion itself.

This season, TCEC introduced a change of mode between Division P and the Superfinal. This was the ‘TCEC Cup’, a knockout tournament involving all the TCEC 13 engines. It was an excellent innovation, which will no doubt be repeated. The authors here report on its thirty-one matches separately (Haworth and Hernandez, 2019c).

Jeroen Noomen (2018) had adjusted his approach to choosing superfinal openings. His comments reveal how much thought goes into this aspect of TCEC. The 50 openings split across the ECO A/B/C/D/E range 13/12/12/6/7, the D/E lines being considered “too easy for top engines”. The openings aimed to leave a position with an advantage in the range [0.2, 0.55] and, despite the excellence of the engines, a win-rate of 20% was expected with 25% as target.

Once again, Jeroen made target. The win-rate was 22%, Stockfish winning 16 to Komodo’s 6. Stockfish had two wins with Black to Komodo’s one and there was only one game-pair, games 85-86, where both sides won. Thus, the final score was 55-45, see Table 8, a performance that would suggest an ELO difference of only 36. In fact, although Komodo lost the match, it did marginally better than might have been expected.

Wool (2018) provides an admirably generous and informative commentary on the games, covering the wins of course but also showing the struggle inherent in the many draws.

The two innovative engines exploiting neural-network architecture progressed to Division 3 with LC0 nearly promoting again to Division 2. Shall we see one of them passing through Division 2 next season?! TCEC are to be congratulated for taking on the cost, risk and controversy of including GPUs to facilitate these exciting NN developments. They are now being rewarded by positive momentum and results from these new engines. No doubt the overheating and reliability problems will be addressed and solved. Another highlight was Ethereal’s progression from Division 3 to Division P where it still gained ELO points despite shipping several losses.

The TCEC exploration of chess openings by the second author here and by Jeroen Noomen has been treated above. Terminations by the 50-move rule and ‘EGT wins’ are very rare as the engines anticipate these endings and evaluate accordingly.

Assaf Wool (2018), as mentioned above, continues to provide his usual statistics and perspective on the TCEC tournaments, picking out his own favourite games for each round robin. This is very much to be applauded. ‘GM Thechesspuzzler’ (2018) set up further playlists on YouTube. Kingscrusher (2018) is also commenting on TCEC and particularly LC0 in his comprehensive YouTube presence, 5000 videos and counting.

OPEN IN VIEWER

Table 9

The shortest and longest 1-0, drawn and 0-1 games in each phase of TCEC13

The pgn and logfiles for TCEC13, together with some chess and statistical analysis as in Table 10, are available (Haworth and Hernandez, 2019b) for further study. Some of the decisive games have had an exemplar playout added as a variation. Whether you are looking for opening novelties or subtle endgames, the longest, most balanced or the shortest, most dramatic battles, see Table 9, there is plenty of interest here, plenty of occasion for reflection. Feedback to the authors is most welcome.

OPEN IN VIEWER

Table 10

Generic statistics for each phase of TCEC13: results, terminations and average game-length