The 17 th Top Chess Engine Championship,TCEC17

New to TCEC were Counter, Fabchess, Gogobello, Ice and PeSTO. TCEC welcomed back Defenchess and Demolito. The second author here provided 8-ply openings for this league and 12-ply openings for the next.

At the quarter-way point, the leaders were Defenchess, PeSTO, Winter, Topple, Demolito and Pirarucu. With the sub-7-man endgame tables in RAM, now being consulted some 10 million times per second, forty times the previous rate, those engines not doing so were further disadvantaged: Minic lost a drawn KRRKRN endgame in game 5.

With all games played, the top six earning a place in the League 2 event were the unbeaten front-runner Defenchess, Demolito, Winter, Pirarucu, Igel and Ice. PeSTO had slipped to 8^th and Topple to 11^th. In fact, as chess22k and Fritz had failed stress-tests on the larger Linux/Wine platform and been rated non-starters, Minic and Pesto also stepped into League 2. Wool (2019), assiduous as ever, touched on some 50 games.1

OPEN IN VIEWER

Table 2

The TCEC17 Qualification League cross-table

With a quarter of the games played, the favourites for the top four places were, in order, Fire, RubiChess, Vajolet2 and Pedone with Defenchess knocking on the door.

Ultimately, the TCEC15 ‘Shannon AB’ Fire, Defenchess, RubiChess and Vajolet2 took the top four promotion spots. As Booot, ChessbaseVB, Chiron and Jonny were not ready to move to the new Linux platform, the next four – Pedone, Winter, Pesto and Nemorino joined them in moving to League 1. In this event, we saw Defenchess continue unbeaten to earn a second promotion and Pesto fill a second empty chair. Wasp (after three crashes) and five ‘Qualification League’ engines relegated.

Wool touched on a considerable number of mainly decisive games2

The exclusively ‘CPU’ engines in this League were competing for four places and the opportunity to fight with the best ‘League 1’ neural network engines for elevation to an enlarged Premier Division. The Elo strengths posted for this event indicated that KomodoMCTS, Xiphos, Ethereal and Laser were the leading favourites. With core-time up to 45′ and four round-robins rather than two, this event was to see the engines improving their game over an extended period. Nelson provided 16-ply openings, making a judgement as to how opening bias would affect the draw-rate.

From Round Robin 1, spectators (Chessdom, 2020a) picked out Ethereal–KomodoMCTS, their first meeting, game 18. In g46, KomodoMCTS failed to progress a 7-man win for 34 moves. Winter had a difficult win in game 53 but, still without EGTs, could not close out. The splendid 71-game unbeaten run by Defenchess in TCEC17 was ended in game 93 by no less than Ethereal. Wool picked out some RR1 games3

Tailing away from the leaders were Fizbo, hampered by comparatively fewer threads, PeSTO, Winter, Nemorino, Pedone and Vajolet2. Defenchess also lagged in the early going but rallied as the contest wore on.

Wool touched on quite a few RR2 games.4

Ethereal, Fire and KomodoMCTS were scarcely threatened in the last quarter of this event: their promotion was never in doubt. They had only one loss, game 138 as mentioned above. The residual excitement was that rofChade made a late move with five victories to move up from 7^th to 4^th, edging out Xiphos on ‘number of wins’ and making its inferior Sonneborn-Berger score irrelevant. Defenchess, having rejoined TCEC in the Qualification League, had very creditably been competing for 4^th place but slipped with a loss to Xiphos in round 58. Laser, having also gone undefeated in RR1-3, sustained four losses here and fell away to 9^th: fatigue cannot be considered a factor. Wool touched briefly on a number of games.6

Fizbo had not found a way to exploit the greatly increased number of cores and, maybe for that reason, was never in touch with the rest of the field. Nemorino, Winter and Pesto – only pulled off the bench as substitutes – were also clearly headed for relegation and duly returned to League 2. The remaining stand-in, Pedone, was safely in 12^th place and continued to compete for the 11^th spot.

OPEN IN VIEWER

Table 4

The TCEC17 League 1 cross-table

OPEN IN VIEWER

Table 5

The TCEC17 League 1 progress table

OPEN IN VIEWER

Fig. 2.

Net wins, round by round, for the top eight in the TCEC17 League 1.

At last, the neural network engines appeared on stage. Although several were expected, there were in fact only two that were not already in the Premier Division and so they automatically took the promotion places of what became a ‘null’ L1-NN event. After a complex conversion to Linux, ChessFighter (Lim, 2020) joined ScorpioNN and the four promotees above in a five-day play-off event. Nelson stayed with 16-ply openings but the time budget was extended once again to determine the four best engines. GPU firepower was increased by a factor of 2.38 with the use of four 2080Ti GPUs (Nvidia, 2018) not quite the factor of four on the CPU side.

Round Robin 1 kicked off with three wins. Ethereal made the most of its opening advantage and actually mated ChessFighter (Chessdom, 2020a). After their four draws in the previous event, Fire notably beat rofChade with a 7-man finale 30 moves closer to mate than both knew. In the first MCTS battle, KomodoMCTS mated ScorpioNN. If NNs are blind to mate, that is a fairly basic weakness but things turned up for them in games 5-6. ChessFighter was on the best end of a KQPPkq draw and ScorpioNN beat rofChade. The sting in the tail came when Ethereal lost to rofChade after incorrectly choosing to exchange rooks into a knight endgame thicket. A zugzwang at position 49b may have been its undoing: FinalGen (Romero, 2012) said ‘btm, 1-0 in 42 moves; wtm, draw’. Already, Fire and KomodoMCTS looked good for a return to the Premier Division while ChessFighter was alone in last place. Round Robin 2 only confirmed these standings but with Ethereal again losing, this time to Fire, the contest for 3^rd and 4^th remained close.

OPEN IN VIEWER

Fig. 3.

The logos for the engines originally in League 1-NN and in the Premier Division.

Round Robin 3 was ‘moving day’ for ScorpioNN and rofChade. There was good news (+2) for the former which beat KomodoMCTS and went favourite for 3^rd, bad news (-2) for the latter which sustained losses to the two leaders. The remaining battle for fourth was only resolved in the last game of the event where Ethereal was in fact never threatened by rofChade and remained in 4^th place. In the penultimate game, KomodoMCTS notably lost to Fire in what looked like a clear OCB draw. Fire was the only engine to go undefeated in this event: ChessFighter was the only engine not to win a game but even so, performed better than Elo forecast. Wool usefully picked out several games.7

OPEN IN VIEWER

Table 6

The TCEC17 League 1 play-off cross-table and net scores after each round robin

‘Aloril’ fortified inter-server comms with timestamping to combat the sometimes extreme delays imposed by internet firewalls. TCEC promoted all the CPU-CPU games to the top of the bill in order to defer the need for the GPUs that had only been used lightly in the previous event. The Premier Division, now 60+120 games, paralleled the FIDE Candidates Tournament in Yekaterinburg – both hoping to survive the Covid-19 crisis and with a grateful audience wishing them well.

Stockfish started hot with wins against Ethereal and Komodo. At the halfway point, the unbeaten Stockfish and Fire were the only engines in positive territory, +3 and +1 respectively. Wool (2019)8

With just one third of the event over, the second phase began with those strange, mystical beasts, the GPUs, present in every game. While it was difficult to predict the Superfinal line-up, this was best done by percentage score and Aloril’s (2020) Bayeselo calculations. Around game 74, TCEC became the only game in town as the FIDE Candidates tournament was halted by Covid-19 on March 26^th. Every thirty games, a round-robin belatedly completed. Table 7 and Fig. 4 give the net scores as they evolved and after each round-robin as reconstructed post hoc.

At the end of round-robin 1, AllieStein and Leela were unsurprisingly competing with Stockfish for a place in the Superfinal: Leela beat Stoofvlees, game 83. At the half-way point, AllieStein and Leela had lifted clear of their competition: Stoofvlees had succumbed in games 110 and 120 to both AllieStein and Stockfish. Round robin 3 saw an extraordinary string of wins by Leela while AllieStein went backwards with losses to Komodo and Ethereal. Round robin 4 saw Leela even pass Stockfish despite losing to the Komodo/Ethereal duo. Ethereal took the notional TCEC combativity award for the least draws. Sadler and Regan (2020) contributed a fine string of commentaries on games 79, 94, 102, 107, 132, 146, 149 and 169: others added their insights on some of these games and on games 83, 120, 139, 143, 164 and 168.

OPEN IN VIEWER

Table 7

The TCEC17 Premier League cross-table and net scores after each round robin

And so the Superfinalists were Stockfish and Leela Chess Zero, the former unbeaten once more in the Premier Division and the latter intriguingly and simultaneously sharper and more fallible with more wins but those two late losses. Game on!

OPEN IN VIEWER

Fig. 4.

Net wins, round by round, for the top five in the TCEC17 Premier Division.

For so many of the audience, this was the Superfinal they wanted. This is not to discount the clear excellence of the other engines, which are well respected, but many had, after all, been seriously investing in the ‘cloud evolution’ of one engine or the other – or both. The chat audience swelled in number9

It was not easy to sense a favourite but Leela had shown more cutting edge in the Premier Division – without which it would have taken third place to AllieStein again. GM Matthew Sadler (2020) thought Leela had shown enough to be rated the favourite: his ‘Perspective’ on the Superfinal is totally recommended and enables us to be relatively brief here.10

Figure 5 and Table 8 show the progress of the event. Jeroen Noomen (2020) infused the character of the mini-matches with his choice of openings, which certainly added to the excitement by reducing the draw-rate: ‘kudos’ to JN here. Seven times, White won both games of the pair while Black, which was admittedly not competing under its own colours, fared badly with only two wins – games 16 and 95. Leela eventually won ten mini-matches to Stockfish’s five though this was largely due to an unexplained final sprint in the last ten games. Until then, either engine could have triumphed and one statistic clearly shows how close it was. The average game-length was over 100 moves, thanks also to Leela’s cautious and often unnerving approach to pressing home its winning advantage.

Stockfish opened its account in game 3 but Leela replied immediately. Undeterred, Stockfish won game 7 – this time without reply – but Leela came back in games 12 and 14 to lead. Stockfish won with Black in game 16 to level the scores and after an exchange of handbags in games 26-29, went ahead again in game 33. However, Leela replied immediately by winning the same opening in game 34 and was never headed thereafter. Games 36 and 38 opened the lead to two for the first time but Stockfish stayed in touch until and with its win in game 87 when the lead was only 1. The suspense was palpable but the run of play changed at this point. Surely Leela had no idea that its time had come but it rattled off wins in games 88, 92, 94, 95 (with Black), 96 and 98 while Stockfish scored in only games 93 – and 99 by which time it was all over.

OPEN IN VIEWER

Fig. 5.

The TCEC17 superfinal: Leela Chess Zero’s lead as the match progressed.

OPEN IN VIEWER

Table 8

The TCEC17 superfinal statistics and decisive games

There were two winners in the TCEC17 Championship, Leela Chess Zero and chess itself. Has there been a more hard-fought match? Has chess ever been played at a higher level? Your authors here have to say ‘no’: these games were outstanding and everyone is encouraged to look past the result to the play itself. While it is difficult to see this Superfinal being bettered, Markov theory says ‘never say never’ so we have to assume that there will be more great engine contests. Both ‘AB’ engines and ‘NN’ engines are still improving even if it is difficult to see Stockfish being easily displaced by another pure AB-engine at the moment.

Presumably, there is an opportunity for an engine that combines the separate merits of ‘Shannon AB’ and ‘Neural’ engines while avoiding their weaknesses. Maybe such an engine cannot come through while TCEC’s CPU- and GPU-servers remain physically apart.

As ever, our thanks and congratulations to ‘TCEC operations’ and all the participating engine-authors (particularly given the platform changes they had to manage), to the members of ‘the cloud’ who are improving these engines, to the audience and to the leading chess commentators without whom much of the strategic genius in the play would go unnoticed. This championship is also reported in Wikipedia (2020).

For the first time, all the decisive games (Haworth and Hernandez, 2020) have been played out with varying difficulty using Fritz17 at a search-depth of 24 ply to provide reference solutions to potential training material. All the game entries to 7-man chess have been given ‘DTM’ Depth to Mate (Lomonosov, 2012) and ‘DTZ’ Depth to Zeroing of the ply count (de Man et al., 2018) depths. In Tables 9 and 10, we supply the usual statistics and data on the games.

OPEN IN VIEWER

Table 9

Generic statistics for each phase of TCEC17: results, terminations and average game-length

OPEN IN VIEWER

Table 10

The shortest and longest 1-0, drawn and 0-1 games in each phase of TCEC17