TCEC Season 17 started on January 1st, 2020 with a radically new structure: classic ‘CPU’ engines with ‘Shannon AB’ ancestry and ‘GPU, neural network’ engines had their separate parallel routes to an enlarged Premier Division and the Superfinal. Figures 1 and 3 and Table 1 provide the logos and details on the field of 40 engines.
Through the generous sponsorship of ‘noobpwnftw’, TCEC benefitted from a significant platform upgrade. On the CPU side, 4x Intel (2016) Xeon 4xE5-4669v4 processors enabled 176 threads rather than the previous 43 and the Syzygy ‘EGT’ endgame tables were promoted from SSD to 1TB RAM. The previous Windows Server 2012 R2 operating system was replaced by CentOS Linux release 7.7.1908 (Core) as the latter eased the administrators’ tasks and enabled more nodes/sec in the engine-searches. The move to Linux challenged a number of engine authors who we hope will be back in TCEC18.
The logos for the engines originally in the Qualification League and Leagues 1 and 2.
The qualification league: 16 engines, 1 DRR, 240 games @ 30′+5″/m
New to TCEC were Counter, Fabchess, Gogobello, Ice and PeSTO. TCEC welcomed back Defenchess and Demolito. The second author here provided 8-ply openings for this league and 12-ply openings for the next.
At the quarter-way point, the leaders were Defenchess, PeSTO, Winter, Topple, Demolito and Pirarucu. With the sub-7-man endgame tables in RAM, now being consulted some 10 million times per second, forty times the previous rate, those engines not doing so were further disadvantaged: Minic lost a drawn KRRKRN endgame in game 5.
With all games played, the top six earning a place in the League 2 event were the unbeaten front-runner Defenchess, Demolito, Winter, Pirarucu, Igel and Ice. PeSTO had slipped to 8th and Topple to 11th. In fact, as chess22k and Fritz had failed stress-tests on the larger Linux/Wine platform and been rated non-starters, Minic and Pesto also stepped into League 2. Wool (2019), assiduous as ever, touched on some 50 games.1
Chessdom (2020a) highlighted just two, g49 and g202.
The TCEC17 Qualification League cross-table
League 2: 16 engines, 1 DRR, 2RRs, 240 games @ 30′+5″/m
The TCEC17 League 2 cross-table of raw results, prior to Wasp’s disqualification
With a quarter of the games played, the favourites for the top four places were, in order, Fire, RubiChess, Vajolet2 and Pedone with Defenchess knocking on the door.
Ultimately, the TCEC15 ‘Shannon AB’ Fire, Defenchess, RubiChess and Vajolet2 took the top four promotion spots. As Booot, ChessbaseVB, Chiron and Jonny were not ready to move to the new Linux platform, the next four – Pedone, Winter, Pesto and Nemorino joined them in moving to League 1. In this event, we saw Defenchess continue unbeaten to earn a second promotion and Pesto fill a second empty chair. Wasp (after three crashes) and five ‘Qualification League’ engines relegated.
Wool touched on a considerable number of mainly decisive games2
while TCEC’s ‘Notable Games’ (Chessdom, 2020a) include games 29 and 97. In game 153, Minic missed 62. Bxh5 indicated as essential by the endgame tables. In game 206, Winter let the draw go with 75Kf7 and, without EGTs, surrendered the game to a grateful Defenchess, which had struggled to make any progress.
CPU league 1: 16 engines, 2 DRRs, 4 RRs, 480 games @ 45′+5″/m
The exclusively ‘CPU’ engines in this League were competing for four places and the opportunity to fight with the best ‘League 1’ neural network engines for elevation to an enlarged Premier Division. The Elo strengths posted for this event indicated that KomodoMCTS, Xiphos, Ethereal and Laser were the leading favourites. With core-time up to 45′ and four round-robins rather than two, this event was to see the engines improving their game over an extended period. Nelson provided 16-ply openings, making a judgement as to how opening bias would affect the draw-rate.
From Round Robin 1, spectators (Chessdom, 2020a) picked out Ethereal–KomodoMCTS, their first meeting, game 18. In g46, KomodoMCTS failed to progress a 7-man win for 34 moves. Winter had a difficult win in game 53 but, still without EGTs, could not close out. The splendid 71-game unbeaten run by Defenchess in TCEC17 was ended in game 93 by no less than Ethereal. Wool picked out some RR1 games3
L1 RR1 games 22, 44, 46, 50, 53, 64, 78, 86, 93, 108 and 116.
including ‘endgame’ games 46 and 53 above. After the 120 games, the top order was Ethereal, Xiphos, KMCTS, promotee rofChade, Laser, Fire and Andscacs.
Tailing away from the leaders were Fizbo, hampered by comparatively fewer threads, PeSTO, Winter, Nemorino, Pedone and Vajolet2. Defenchess also lagged in the early going but rallied as the contest wore on.
L1 RR2 games 133, 146, 159, 162, 173, 194, 197, 218, 22 and 227.
TCEC fans (Chessdom, 2020a) picked out game 138 in which KomodoMCTS notably beat Ethereal, and game 146 in which the slow-starting Fire beat Rubichess. In RR3, Winter beat rofChade in short game 285. Wool continued to note a few games5
L1 RR3 games 247, 263, 282, 284-5, 294, 301-2, 318, 339 and 340.
and fans picked out game 302 (Chessdom, 2020a) in which Laser beat rofChade. Defenchess continued to move up the standings.
Ethereal, Fire and KomodoMCTS were scarcely threatened in the last quarter of this event: their promotion was never in doubt. They had only one loss, game 138 as mentioned above. The residual excitement was that rofChade made a late move with five victories to move up from 7th to 4th, edging out Xiphos on ‘number of wins’ and making its inferior Sonneborn-Berger score irrelevant. Defenchess, having rejoined TCEC in the Qualification League, had very creditably been competing for 4th place but slipped with a loss to Xiphos in round 58. Laser, having also gone undefeated in RR1-3, sustained four losses here and fell away to 9th: fatigue cannot be considered a factor. Wool touched briefly on a number of games.6
L1 RR4 games 370, 380, 383, 392, 394, 426, 436-7, 445, 454 and 460.
Game 380 was picked out as notable by TCEC fans (Chessdom, 2020a).
Fizbo had not found a way to exploit the greatly increased number of cores and, maybe for that reason, was never in touch with the rest of the field. Nemorino, Winter and Pesto – only pulled off the bench as substitutes – were also clearly headed for relegation and duly returned to League 2. The remaining stand-in, Pedone, was safely in 12th place and continued to compete for the 11th spot.
The TCEC17 League 1 cross-table
The TCEC17 League 1 progress table
Net wins, round by round, for the top eight in the TCEC17 League 1.
The league 1 play-off: 6 engines, 2 DRRs, 4 RRs, 60 games @ 60′+5″/m
At last, the neural network engines appeared on stage. Although several were expected, there were in fact only two that were not already in the Premier Division and so they automatically took the promotion places of what became a ‘null’ L1-NN event. After a complex conversion to Linux, ChessFighter (Lim, 2020) joined ScorpioNN and the four promotees above in a five-day play-off event. Nelson stayed with 16-ply openings but the time budget was extended once again to determine the four best engines. GPU firepower was increased by a factor of 2.38 with the use of four 2080Ti GPUs (Nvidia, 2018) not quite the factor of four on the CPU side.
Round Robin 1 kicked off with three wins. Ethereal made the most of its opening advantage and actually mated ChessFighter (Chessdom, 2020a). After their four draws in the previous event, Fire notably beat rofChade with a 7-man finale 30 moves closer to mate than both knew. In the first MCTS battle, KomodoMCTS mated ScorpioNN. If NNs are blind to mate, that is a fairly basic weakness but things turned up for them in games 5-6. ChessFighter was on the best end of a KQPPkq draw and ScorpioNN beat rofChade. The sting in the tail came when Ethereal lost to rofChade after incorrectly choosing to exchange rooks into a knight endgame thicket. A zugzwang at position 49b may have been its undoing: FinalGen (Romero, 2012) said ‘btm, 1-0 in 42 moves; wtm, draw’. Already, Fire and KomodoMCTS looked good for a return to the Premier Division while ChessFighter was alone in last place. Round Robin 2 only confirmed these standings but with Ethereal again losing, this time to Fire, the contest for 3rd and 4th remained close.
The logos for the engines originally in League 1-NN and in the Premier Division.
Round Robin 3 was ‘moving day’ for ScorpioNN and rofChade. There was good news (+2) for the former which beat KomodoMCTS and went favourite for 3rd, bad news (-2) for the latter which sustained losses to the two leaders. The remaining battle for fourth was only resolved in the last game of the event where Ethereal was in fact never threatened by rofChade and remained in 4th place. In the penultimate game, KomodoMCTS notably lost to Fire in what looked like a clear OCB draw. Fire was the only engine to go undefeated in this event: ChessFighter was the only engine not to win a game but even so, performed better than Elo forecast. Wool usefully picked out several games.7
L1 playoff games: 3, 10, 15, 21.
The TCEC17 League 1 play-off cross-table and net scores after each round robin
The premier division: 10 engines, 2 DRRs, 4 RRs, 180 games @ 90′+5″/m
‘Aloril’ fortified inter-server comms with timestamping to combat the sometimes extreme delays imposed by internet firewalls. TCEC promoted all the CPU-CPU games to the top of the bill in order to defer the need for the GPUs that had only been used lightly in the previous event. The Premier Division, now 60+120 games, paralleled the FIDE Candidates Tournament in Yekaterinburg – both hoping to survive the Covid-19 crisis and with a grateful audience wishing them well.
Stockfish started hot with wins against Ethereal and Komodo. At the halfway point, the unbeaten Stockfish and Fire were the only engines in positive territory, +3 and +1 respectively. Wool (2019)8
Premier Phase 1 games 1, 6, 9, 12, 23, 31, 39-41, 53, 58.
like the fans (Chessdom, 2020a) focused on g9, Fire-KomodoMCTS. The last three had no wins and Houdini, running at a disadvantage via Wine on only 64 threads, assumed the role of principal donor with -2. The second double round-robin started in the same way: Stockfish 1, Ethereal 0. Just as the wins were drying up, three came along at once in games 39-41, KomodoMCTS taking a double hit and Stockfish beating its main rival, Fire. Round Robin 3 was akin to ‘moving day’ at a Golf Open with KomodoMCTS falling away to -3 while Stockfish went out to +5. Wool and ‘TCEC’ focussed on game 41, Stockfish–Fire which, played out, would have featured three pawn promotions and White winning despite not getting the first check in the QP-endgame. With phase 1 finished, the clear leader in the clubhouse was Stockfish with an impressive +6, so the question was “Will Fire’s +1 be enough?” Ethereal edged third with one more win than Komodo. The various video-commentaries (Chessdom, 2020b; ‘Gmthechesspuzzler’, 2020; ‘Jozarov’, 2020; ‘Kingscrusher’, 2020; Sadler and Regan, 2020) between them covered games 6, 12, 23, 31, 33, 39, 31, and 58. Haworth and Hernandez (2020) include a finer-grain index to the TCEC17 video-commentaries.
With just one third of the event over, the second phase began with those strange, mystical beasts, the GPUs, present in every game. While it was difficult to predict the Superfinal line-up, this was best done by percentage score and Aloril’s (2020) Bayeselo calculations. Around game 74, TCEC became the only game in town as the FIDE Candidates tournament was halted by Covid-19 on March 26th. Every thirty games, a round-robin belatedly completed. Table 7 and Fig. 4 give the net scores as they evolved and after each round-robin as reconstructed post hoc.
At the end of round-robin 1, AllieStein and Leela were unsurprisingly competing with Stockfish for a place in the Superfinal: Leela beat Stoofvlees, game 83. At the half-way point, AllieStein and Leela had lifted clear of their competition: Stoofvlees had succumbed in games 110 and 120 to both AllieStein and Stockfish. Round robin 3 saw an extraordinary string of wins by Leela while AllieStein went backwards with losses to Komodo and Ethereal. Round robin 4 saw Leela even pass Stockfish despite losing to the Komodo/Ethereal duo. Ethereal took the notional TCEC combativity award for the least draws. Sadler and Regan (2020) contributed a fine string of commentaries on games 79, 94, 102, 107, 132, 146, 149 and 169: others added their insights on some of these games and on games 83, 120, 139, 143, 164 and 168.
The TCEC17 Premier League cross-table and net scores after each round robin
And so the Superfinalists were Stockfish and Leela Chess Zero, the former unbeaten once more in the Premier Division and the latter intriguingly and simultaneously sharper and more fallible with more wins but those two late losses. Game on!
Net wins, round by round, for the top five in the TCEC17 Premier Division.
The superfinal: Stockfish vs Leela Chess Zero
For so many of the audience, this was the Superfinal they wanted. This is not to discount the clear excellence of the other engines, which are well respected, but many had, after all, been seriously investing in the ‘cloud evolution’ of one engine or the other – or both. The chat audience swelled in number9
The meter clicked over 3,000, which may not rival FIDE’s World Chess Championship numbers where there is live GM commentary but it was a record for TCEC.
and was extremely lively throughout, exchanging ideas in everything from the Queen’s Gambit Declined to the post-Covid-19 Shopping Gambit Accepted.
It was not easy to sense a favourite but Leela had shown more cutting edge in the Premier Division – without which it would have taken third place to AllieStein again. GM Matthew Sadler (2020) thought Leela had shown enough to be rated the favourite: his ‘Perspective’ on the Superfinal is totally recommended and enables us to be relatively brief here.10
Sadler covered wins and draws alike – games 1, 3-4, 6, 1-12, 14, 16, 18, 21-22, 25, 27, 33-38, 40, 50, 61, 66, 83-84, 89-90, 91-94 and 95-96. The Sadler/Regan videos covered games 6, 18, 21, 33-4, 40, 61, 64, 95 & 98.
Other commentators, including longtime TCEC supporters ‘Gmthechesspuzzler’ (2020), ‘Assaf Wool’ (2019) and ‘Kingscrusher’ (2020)11
‘Kingscrusher’ contributed at least 27 video-commentaries – on games 3, 4, 7, 12, 14, 16, 26-8, 33-4, 36, 38, 43-4, 66, 77, 83-4, 87-8, 92-6 and 98 – a huge contribution.
remained at their desks with minds keenly honed. The increased popularity of TCEC, probably combined with the constraints of Covid-19, saw other commentators in the stands (Chessdom, 2020a/b) including ‘agadmator’ (2020), ‘Jozarov’ (2020) and the Game Changer team of GM Sadler and WIM Regan (2019, 2020) who continue to produce their videos as we write. Our thanks to them all for their insightful commentaries on the chess itself, all of which complement this less chessic account of the match.
Figure 5 and Table 8 show the progress of the event. Jeroen Noomen (2020) infused the character of the mini-matches with his choice of openings, which certainly added to the excitement by reducing the draw-rate: ‘kudos’ to JN here. Seven times, White won both games of the pair while Black, which was admittedly not competing under its own colours, fared badly with only two wins – games 16 and 95. Leela eventually won ten mini-matches to Stockfish’s five though this was largely due to an unexplained final sprint in the last ten games. Until then, either engine could have triumphed and one statistic clearly shows how close it was. The average game-length was over 100 moves, thanks also to Leela’s cautious and often unnerving approach to pressing home its winning advantage.
Stockfish opened its account in game 3 but Leela replied immediately. Undeterred, Stockfish won game 7 – this time without reply – but Leela came back in games 12 and 14 to lead. Stockfish won with Black in game 16 to level the scores and after an exchange of handbags in games 26-29, went ahead again in game 33. However, Leela replied immediately by winning the same opening in game 34 and was never headed thereafter. Games 36 and 38 opened the lead to two for the first time but Stockfish stayed in touch until and with its win in game 87 when the lead was only 1. The suspense was palpable but the run of play changed at this point. Surely Leela had no idea that its time had come but it rattled off wins in games 88, 92, 94, 95 (with Black), 96 and 98 while Stockfish scored in only games 93 – and 99 by which time it was all over.
The TCEC17 superfinal: Leela Chess Zero’s lead as the match progressed.
The TCEC17 superfinal statistics and decisive games
Summary
There were two winners in the TCEC17 Championship, Leela Chess Zero and chess itself. Has there been a more hard-fought match? Has chess ever been played at a higher level? Your authors here have to say ‘no’: these games were outstanding and everyone is encouraged to look past the result to the play itself. While it is difficult to see this Superfinal being bettered, Markov theory says ‘never say never’ so we have to assume that there will be more great engine contests. Both ‘AB’ engines and ‘NN’ engines are still improving even if it is difficult to see Stockfish being easily displaced by another pure AB-engine at the moment.
Presumably, there is an opportunity for an engine that combines the separate merits of ‘Shannon AB’ and ‘Neural’ engines while avoiding their weaknesses. Maybe such an engine cannot come through while TCEC’s CPU- and GPU-servers remain physically apart.
As ever, our thanks and congratulations to ‘TCEC operations’ and all the participating engine-authors (particularly given the platform changes they had to manage), to the members of ‘the cloud’ who are improving these engines, to the audience and to the leading chess commentators without whom much of the strategic genius in the play would go unnoticed. This championship is also reported in Wikipedia (2020).
For the first time, all the decisive games (Haworth and Hernandez, 2020) have been played out with varying difficulty using Fritz17 at a search-depth of 24 ply to provide reference solutions to potential training material. All the game entries to 7-man chess have been given ‘DTM’ Depth to Mate (Lomonosov, 2012) and ‘DTZ’ Depth to Zeroing of the ply count (de Man et al., 2018) depths. In Tables 9 and 10, we supply the usual statistics and data on the games.
Generic statistics for each phase of TCEC17: results, terminations and average game-length
The shortest and longest 1-0, drawn and 0-1 games in each phase of TCEC17
References
1.
‘agadmator’ (2020). https://youtu.be/K0gBybDFcMg. Example commentary, Sufi game 7.
CPW (2020). https://tinyurl.com/icga046. Biographies of chess engines, authors and developers.
6.
de Man, R., Fiekas, N. & Guo, B. (2018). https://tinyurl.com/icga007. Fiekas’ interface to ‘Syzygy formatted’ de Man sub-7-man and Guo 7-man EGTs.
7.
‘Gmthechesspuzzler’ (2020). https://www.youtube.com/watch?v=vjp6k7MpBDg. Sufi game 90, an example Gmtcp video commentary.
8.
Haworth, G.McC. & Hernandez, N. (2020). The 17th TCEC event: TCEC 17. This report plus pgn files, data and statistics. ICGA Journal. Submitted to the http://centaur.reading.ac.uk/89999/.
9.
Intel (2016). https://tinyurl.com/icga059. Intel’s specification of the XEON®. E5-4669v4 processor.
10.
‘Jozarov’ (2020). https://youtu.be/1roczfJuteY. Example commentary, Sufi game 14.
11.
‘Kingscrusher’ (2020). https://youtu.be/yKvtigUSv88. Example Sufi commentary, game 3.
12.
Lim, A. (2020). http://www.talkchess.com/forum3/viewtopic.php?t=71163. On ChessFighter.
13.
Lomonosov (2012). http://tb7.chessok.com/. Facility for querying sub-8-man DTM EGTs.
14.
Noomen, J. (2020). http://tinyurl.com/icga0622. Discussion of TCEC17 Sufi openings.
15.
Nvidia (2018). https://www.nvidia.com/en-gb/geforce/graphics-cards/rtx-2080-ti/. GEFORCE GTX 2080 TI GPU specification and benchmark performance data.
16.
Romero, P.P. (2012). https://tinyurl.com/icga013. FinalGen: Tutorial, download and forum.
17.
Sadler, G.M.M. (2020). The TCEC17 Computer Chess Superfinal: a perspective. Submitted to the ICGA Journal.
18.
Sadler, G.M.M. & Regan, W.I.N. (2020). https://tinyurl.com/icga060. Some twenty ‘Chess Giant’ videos on TCEC17 Premier League and Superfinal games.
19.
Sadler, M. & Regan, N. (2019). Game Changer: AlphaZero’s Groundbreaking Chess Strategies and the Promise of AI. New in Chess.
20.
Wikipedia (2020). TCEC17 report. https://en.wikipedia.org/wiki/TCEC_Season_17.
21.
Wool, A. (2019). http://mytcecexperience.blogspot.co.uk/. AW’s ‘TCEC Experience’ blog.
