Coronavirus and sports leagues: obtaining a fair ranking when the season cannot resume

Abstract Many sports leagues are played in a tightly scheduled round-robin format, leaving a limited time window to postpone matches. If the season cannot resume due to an external shock such as the coronavirus pandemic in 2020, the ranking of the teams becomes non-trivial: it is necessary to account for schedule imbalances and possibly for the different number of matches played. First in the literature, we identify a set of desired axioms for ranking in these incomplete tournaments. It is verified that the generalized row sum, a parametric family of scoring rules, satisfies all of them. In particular, the well-established least-squares method maximizes the influence of the strength of opponents on the ranking. Our approach is applied for six major premier European soccer competitions, where the rankings are found to be robust concerning the weight of adjustment for the strength of the opponents. Some methodologically simpler alternative policies are also discussed.


Introduction
The pandemic of coronavirus disease 2019 stopped almost all sports leagues around the world. Some of them never resumed: the German premier men's handball league, the 2019/20 Handball-Bundesliga, was cancelled on 21 April 2020 (Web24 News, 2020); the Dutch soccer league, the 2019/20 Eredivisie ended with immediate effect on 24 April 2020 (Sports, 2020); and the French Prime Minister announced on 28 April 2020 that the 2019/20 sporting season is over (BBC, 2020). Therefore, several organizers have faced an unenviable dilemma: how to decide the final ranking in the 2019/20 season? Should all results be abandoned? Should the current standing be frozen? Should a reasonable subset of the matches be considered? Since the sports industry is a billion dollars business, the answer has huge financial consequences as promotion and relegation, qualification for international cups, and the allocation of broadcasting revenue (Bergantiños & Moreno-Ternero, 2020) all depend on the league ranking. 1 For instance, the Hungarian Handball Federation cancelled all results in the 2019/20 season (Stregspiller, 2020). The implications for the 2019/20 Hungarian women's handball league were quite dramatic. With eight matches left to play from the total of 26, the runner-up Siófok had an advantage of 3 However, none of the above works has used axiomatic arguments for their proposals. This inspired us to look for a simple solution that retains most properties of the usual ranking in round-robin tournaments as the majority of forecasting models do not guarantee these axioms. Our proposal is based exclusively on the number of points as well as the structure of the matches already played, and has the following advantages: • it takes into consideration all results from the unfinished season; • it accounts for the different number of matches played as well as for the fact that the teams played against different sets of opponents with varying strengths; • it does not depend on the form, home-away pattern, injuries or results in the preceding season(s); • it seldom requires further tie-breaking criteria (Csató, 2017); • it is suitable for any sport and does not call for the estimation of any parameter; and • it has a recursive formula where the first iteration still yields an adequate solution, consequently, it requires no specific software, and the ranking can be computed essentially by hand.
On the other hand, it should be acknowledged that no reasonable prediction is provided for the final number of points. In our opinion, that is not a serious problem as only the ranking of the teams is required to decide promotion, relegation, and qualification to other tournaments.
The rest of the article proceeds as follows. Section 2 studies some properties of the usual ranking in round-robin tournaments and discusses their relevance when all matches cannot be played. The generalized row sum method is introduced in Section 3 and shown to be compatible with the desired axioms. Section 4 contains its application for six major European soccer leagues. Section 5 summarizes policy implications, whereas Section 6 offers concluding remarks.

Axioms
As the first step, we present some conditions that are satisfied by the ranking obtained from the number of points in a round-robin tournament. This collection aims to assess whether they can be guaranteed if the league cannot resume after some matches are played. Precise mathematical formalization is avoided in order to be more accessible for the general audience.
Axiom 2.1. Independence of irrelevant matches: The relative ranking of two teams should be independent of any matches between other teams.
Obviously, the number of points scored by any team does not depend on the results of other teams. However, if head-to-head results are used for tie-breaking, this property does not necessarily hold. Berker (2014) provides an extensive analysis of tie-breaking in round-robin soccer tournaments.
Axiom 2.2. Self-consistency: Two teams should have the same rank if they achieved the same results against teams having the same strength. Furthermore, a team should be ranked strictly higher than another team if one of the following conditions hold: • it achieved better results against opponents having the same strength, • it achieved the same results against stronger opponents, and • it achieved better results against stronger opponents. 4 L. CSATÓ Self-consistency imposes some constraints on the ranking of the teams such that the constraints are determined by this ranking itself. Csató (2019, Section 3.2) offers a detailed explanation of the axiom.
The exact meaning of 'better results' against another team can be specified by the decision-maker; however, since the final ranking is usually determined by the number of points scored, a reasonable definition is more points scored. Then having better results implies a higher score, thus the ranking given by the number of points satisfies the first and the third requirements of self-consistency. Proving that the second condition also holds is somewhat more complicated. Consider two teams i and j. Assume for contradiction that team i has the same results against stronger opponents compared to team j but team i is ranked weakly below team j. In a round-robin tournament, their set of opponents almost coincide but team i is an opponent of team j and vice versa. Consequently, the initial assumption cannot hold because team i faced stronger opponents than team j.
If the tournament is not round-robin, then there exists no ranking method simultaneously satisfying independence of irrelevant matches and self-consistency (Csató, 2019, Theorem 1). The impossibility result holds under various domain restrictions. Self-consistency is a more important property than independence of irrelevant matches, the latter should be weakened to retain possibility (Csató, 2019). Axiom 2.3. Invariance to cycles: Assume that team i defeated team j, team j defeated team k, and team k defeated team j. The ranking should be independent of reversing these results such that team j wins against team i, team k wins against team j, and team i wins against team k.
Such cycles are responsible for the basic challenges of ranking. While they certainly cannot be deleted, it would be unjustified to modify the ranking after reversing their direction. Most statistical methods are unlikely to guarantee invariance to cycles.
Axiom 2.4. Home-away independence: The ranking of the teams should not change if the field of all matches is reversed: if the game was played by team i against team j at home, then it is considered as team i against team j away, whereas the outcome remains unchanged.
The number of points does not take into account whether a game is played at home or away (tiebreaking rules in some leagues do). It is a natural requirement in a round-robin tournament where the teams usually play once at home and once away; however, this home-away balance does not hold if the league is suspended. For example, Rennes, the third team in the 2019/20 French Ligue 1 after 28 rounds, played only at home against the dominating team Paris Saint-Germain, whereas the fourth Lille lost against Paris Saint-Germain both at home and away. On the other hand, Rennes would have to play five, and Lille would have to play six games away in the remaining 10 rounds.
There are at least two arguments for home-away independence even in incomplete roundrobin tournaments. First, according to our knowledge, this feature is not taken into account in any European soccer league, even though it would make sense in certain formats. In the 2019/20 season, the leagues in Hungary and Kosovo are organized with 12 teams playing a triple round-robin tournament,; hence, some teams play against others twice at home and once away, or vice versa. In addition, certain teams play one match fewer at home than other teams. The same format was used in Finland until 2018. In Northern Ireland and Scotland, the 12 teams play a triple round-robin tournament in the initial stage. After 33 games, the league is split into two sections of six teams each such that every team plays once more all the five teams in their section. Consequently, it is impossible to balance the home-away pattern by guaranteeing that each club plays twice at home and twice away against any other club in its section. While these unbalanced schedules generate interesting research problems, for example, concerning the measurement of competitive balance (Lenten, 2008), the number of points-and further tie-breaking rules-are not adjusted for the inherent inequality.
Second, accounting for home advantage requires a statistical estimation of at least one parameter, which will certainly result in debates around the exact methodology and the sample used. As the above examples illustrate, it is improbable that the decision-makers want to instigate such controversies.
Axiom 2.5. Consistency: As the number of rounds played increases, the ranking should converge to the ranking given by the number of points. In particular, the ranking should coincide with the ranking given by the number of points when the competition is finished.
Consistency is perhaps the most essential property of ranking the teams in a suspended league. The number of points has no competitive alternative in the real-world as the primary ranking criterion. It is also important to use a procedure that satisfies consistency due to its innate characteristics, not merely because it is defined separately for incomplete and complete round-robin tournaments. For example, this condition does not hold for the eigenvector-based method proposed in Lambers & Spieksma (2020a).
To summarize, five properties of the common ranking method in round-robin tournaments have been presented. Independence of irrelevant matches cannot be expected to hold if the league is suspended. Thus we are seeking a method satisfying self-consistency, invariance to cycles, home-away independence, and consistency.

A reasonable family of ranking methods
Our proposal for ranking in incomplete round-robin tournaments requires two inputs: the points vector p and the symmetric matches matrix M. In particular, p i is the number of points for team i, whereas m ij equals the number of matches between teams i and j.
The methods are based on the normalized points vector s with its entries summing up to zero. This is straightforward in sports where a win plus a loss is equivalent to two draws since wins can be awarded by α > 0, draws by zero, and losses by −α < 0. But some sports do not follow this symmetric setting, mainly because of the three points rule, which has become the standard in soccer: wins earn three points, draws earn one point, and losses earn zero points. It has many deficiencies from a theoretical point of view, for instance, reversing all results does not necessarily reverse the ranking and the number of total points allocated depends not only on the number of matches played. But teams should be definitely ranked even in these leagues.
Therefore, similarly to the quotient rule used in the unfinished 2019/20 Handball-Bundesliga to determine the final ranking, the average score per game is calculated for each team by dividing its number of points by its number of matches played. These values are normalized such that the average quotient is subtracted from the quotient of each team to obtain s i . Note that if two teams i, j have played the same number of matches and p i = p j , then their normalized points are also equal, s i = s j .
Example 3.1. Eintracht Frankfurt has played 24 matches and scored p i = 28 points in the 2019/20 German Bundesliga until 13 March 2020. Therefore, it has scored 28/24 ≈ 1.167 points per game. The sum of these quotients for all the 18 teams is 25, thus the normalized point of Eintracht Frankfurt is s i = 28/24 − 25/18 ≈ −0.222. Hertha BSC has also scored 28 points but over 25 matches, hence this club has a slightly lower normalized point of -0.269.
Although normalization of the scores considers the possibly different number of matches played by the teams, it still does not reflect the strength of the opponents.  Example 3.2. Table 1 compares the teams against which Borussia Dortmund and RB Leipzig should have played in the remaining nine rounds of the 2019/20 German Bundesliga. Dortmund had clearly a more difficult schedule ahead than Leipzig as the points scored by its future opponents in the previous rounds were higher by more than 11.7% (all of these teams played 25 matches). However, Dortmund had a lead of one point over Leipzig. Is it sufficient to rank Dortmund higher than Leipzig?
The strength of the schedule can be taken into account through the matches matrix M. The matches already played can be represented by an undirected graph, where the nodes correspond to the teams and the weight of any edge is determined by the number of matches played by the associated teams. The Laplacian matrix L = [ℓ ij ] of this graph is defined such that ℓ ij = −m ij for all i = j and ℓ ii is equal to the number of matches played by team i. Furthermore, let I be the matrix with ones in the diagonal and zeros otherwise, and e be the vector with e i = 1 for all i. The generalized row sum method was introduced in Chebotarev (1989) and Chebotarev (1994). It adjusts the normalized points by considering the performance of the opponents, the opponents of the opponents, and so on. Parameter ε quantifies the degree of this modification. The ranking induced by x(ε) converges to the ranking from the number of points as ε → 0; hence, the normalized point is a limit of generalized row sum. Uniqueness comes from the fact that the matrix I + εL is invertible. Chebotarev (1994, Section 4) demonstrates that x(ε) can be regarded as the prediction of the number of points at the end of the season after all omitted comparisons. The least-squares rating is unique if and only if any team can be compared with any other team at least indirectly, through other teams. This is a natural constraint-otherwise, there exist two subsets of the teams without any matches between the two sets. The condition certainly holds if half of the season is finished, that is, each team has played all other teams at least once. The method is called least squares since the above system of linear equations can be obtained as the optimal solution of a least-squared errors estimation (Csató, 2015;González-Díaz et al., 2014).
The least-squares rating vector q can be calculated recursively unless graph G is regular bipartite (Csató, 2015, Theorem 2 In formula (3.1), r is the maximal number of matches played by a team. Consequently, the entries of matrix rI − L contain the number of matches played by the associated teams, assuming that a team has played against itself if it has played fewer matches than other teams. Therefore, q (1) modifies the normalized points due to the strength of the opponents, q (2) accounts for the opponents of the opponents, and so on.
The ranking induced by x(ε) converges to the ranking from the least squares as ε → ∞, hence the least squares is the other extremum of generalized row sum (Chebotarev & Shamis, 1998, p. 326). The sum of points scored by the opponents is called the Buchholz point in chess and used as a tie-breaking criterion, justifying the alternative name recursive Buchholz (Brozos-Vázquez et al., 2010).
Proposition 3.1. The generalized row sum and least-squares rankings satisfy self-consistency, invariance to cycles, home-away independence and consistency.
Proof. Chebotarev & Shamis (1998, Theorem 5) proves self-consistency. Invariance to cycles holds because reversing all results along a cycle does not affect the points vector p. Home-away independence is guaranteed by disregarding the field of the game in the points vector p and the matches matrix M. Consistency is verified for the generalized row sum by Chebotarev (1994 In addition, these rankings are independent of the particular value α for wins in sports using a symmetric scoring rule. Analogously, the result is the same for the 3-1-0, 6-2-0, and 4-2-1 point systems, in other words, the rewards can be shifted and multiplied arbitrarily. The generalized row sum and especially the least-squares methods have several successful applications, including, among others: (a) international price comparisons by the Eurostat and OECD, where the least-squares procedure is called the EKS (Éltetoő-Köves-Szulc) method (European Union / OECD, 2012); (b) evaluating movies (Jiang et al., 2011); (c) deriving alternative quality of life indices (Petróczy, 2021); (d) ranking the participants of the Eurovision Song Contest (Čaklović & Kurdija, 2017); (e) ranking universities on the basis of applicants' preferences (Csató & Tóth, 2020); (f) ranking historical go (Chao et al., 2018) and tennis players (Bozóki et al., 2016); and (g) ranking teams in Swisssystem chess tournaments (Csató, 2017). Further details on their theoretical properties can be found in Chebotarev (1994), Shamis (1994) and González-Díaz et al. (2014).

Application: ranking in the 2019/20 season of six major European soccer leagues
The five major premier European soccer leagues have been chosen to illustrate how this parametric family of ranking methods work, together with the league in the Netherlands, which has been declared void on 25 April 2020. All of them have been suspended in March 2020 due to the coronavirus pandemic, when about 70-75% of the matches in the 2019/20 season have already been played: • the Premier League in England with 20 teams, 29 rounds finished except for four clubs, which have played only 28 games; • the Ligue 1 in France with 20 teams, 28 rounds finished except for the game Strasbourg vs. Paris Saint-Germain; • the Bundesliga in Germany with 18 teams, 25 rounds finished except for the game Werder Bremen vs. Eintracht Frankfurt; • the Seria A in Italy with 20 teams, 26 rounds finished except for eight clubs, which have played only 25 games; • the Eredivisie in the Netherlands with 18 teams, 26 rounds finished except for four clubs, which have played only 25 games; and • the La Liga in Spain with 20 teams, 27 rounds finished.
Tables 2 and 3 report the current number of points for each team, their rankings by the generalized row sum method with two values of ε, as well as by the least-squares method. In the case of the smallest parameter, ε = 0.001, the strength of opponents serves only to break the ties in the normalized points.
The first and most crucial observation is the robustness of the rankings. This stands in stark contrast to Swiss-system chess team tournaments, where accounting for the opponents can substantially affect the ranking, particularly for the middle teams (Csató, 2017). The probable reason is the large number of matches already played. It is a favourable finding, which indicates that freezing the current standingwith the tie-breaking rules according to Tables 2 and 3-or using the quotient rule (normalized points) would be a relatively fair solution.
The ideal case is represented by Spain: if the ties in the number of points are resolved through the strength of opponents (ε → 0), then the ranking does not depend on the weight of the adjustment at all. There is no need to consider the number of matches played in La Liga. Compared to the ranking induced by the number of points, the only change is on the fourth position as officially Real Sociedad overtakes Getafe due to more goals scored, but the latter team has faced stronger opponents. This difference has fundamental sporting effects since the fourth club automatically qualifies for the group stage of the UEFA Champions League, and the fifth goes only to the UEFA Europa League.
While some top clubs in the Netherlands have played fewer matches, this does not influence the usual ranking with goal difference as the tie-breaking rule. Crucially, the parameter responsible for adjusting the normalized points due to the strength of the opponents does not change the ranking except for certain middle teams, which has not much sporting effect. This is important because Ajax Amsterdam-the team having the most (56) points in a tie with AZ Alkmaar, favoured by goal difference-has not been declared the champion but it has obtained the only slot for the Netherlands in the group stage of the 2020/21 UEFA Champions League, whereas the second club has entered the second qualifying round in the league path; therefore, it should have won three clashes to reach the group stage. Interestingly, AZ Alkmaar is preferred to Ajax by the eigenvector-based method of Lambers & Spieksma (2020a). However, the remaining opponents of Ajax scored 321 points, whereas the remaining opponents of AZ  11  11  10  32  12  12  12  33  12  12  12  30  13  13  13  28  13  13  15  33  13  13  13  28  15  14  14  27  14  14  13  32  14  14  14  27  14  15  15  26  15  15  14  29  15  15  15  26  16  16  16  26  16  16  16  27  16  16  16  25  17  17  17  19  17  17  17  26  17  17  17  25  18  18  18  15  18  18  18  25  18  18  18  18  19  19  19  23  19  19  19  16  20  20  20  20  20  20  20 Alkmaar scored 335 points, which supports the conjecture that Ajax had a more difficult schedule in the matches already played. Similarly, the value of ε barely affects the ranking in Italy. Disregarding tie-breaking, the changes are on the 10th and the 14th positions, which are almost insignificant because they influence neither qualification for European cups nor relegation. On the other hand, the seventh place-which provides a slot in the second qualifying round of the UEFA Europa League-should be given to one of the teams that scored 35 points, Hellas Verona or Parma as the additional match played by Milan is seemingly responsible for its advantage of one point. Although Parma has a worse goal difference, this club has faced stronger opponents than Hellas Verona.
The situation in the Premier League is analogous to Serie A. While Arsenal has one point less (40) compared to Tottenham Hotspur, this is compensated by the fewer number of games played for small values of ε. The generalized row sum method also exchanges Brighton & Hove Albion with West Ham United on the insignificant 15th place.
Turning to the Bundesliga, the 13th place has minimal sporting effects. The seventh team qualifies for the second qualifying round of the UEFA Europa League because the cup winner, Bayern Munich, is one of the first six clubs. Furthermore, as Example 3.2 has explained, RB Leipzig can be the runnerup instead of Borussia Dortmund if the strength of the opponents receives a considerable role in the L. CSATÓ  12  12  12  35  13  13  13  37  12  12  11  28  13  14  14  34  14  14  14  34  15  15  15  27  14  13  13  29  15  16  16  34  14  14  14  26  15  15  15  27  16  15  15  30  16  16  16  22  16  16  16  27  17  17  17  30  17  17  17  18  17  17  17  27  18  18  18  27  18  18  18  16  18  18  18  25  19  19  19  23  19  19  19  21  20  20  20  13  20  20  20 ranking, that is, for large values of ε. Nonetheless, both the second-and the third-placed clubs enter the group stage of the UEFA Champions League. The ranking in Ligue 1-a league that has never been finished-is perhaps the most challenging. Since Paris Saint-Germain (first with 68 points) has won both the domestic cup and the French league cup competition and the UEFA Europa League titleholder, the Spanish Sevilla has qualified for the UEFA Champions League through its domestic league, the third-placed club has qualified for the Champions League group stage, the fourth-and the fifth-placed clubs have gone to the group stage of the Europa League, and the sixth-placed club has received a slot in the second qualifying round of the Europa League.
If the strength of the opponents takes a large weight, then Lille with 49 points is ranked above Rennes with 50 points and obtains the Champions League slot, whereas Montpellier with 40 points is ranked above Nice with 41 points and goes to the Europa League. The issue of Rennes and Lille has been highlighted in Guyon (2020a): Lille played and lost twice against the dominating Paris Saint-Germain, whereas Rennes faced it once, and the number of points scored so far by their future opponents is 347 for Lille but 379 for Rennes (and even 381 or 382 if accounting for the missing clash Strasbourg vs. Paris Saint-Germain from the 29th round).
Finally, recall that the least-squares rating vector can be computed recursively according to formula (3.1). Accounting for the strength of direct opponents, that is, vector q (1) almost always induces the same ranking as the final rating vector q, the only exception being that Lyon with 40 points in Ligue 1 is ranked above Angers having 39 points only after considering the opponents of opponents (in addition, the ranking according to q (2) is different from the ranking induced by q (1) , q (3) and q in the 2019/20 Premier League). In our examples, primarily the direct impact of the opponents count. Consequently, it is not necessary to calculate the inverse of any matrix, and one matrix multiplication can be sufficient to determine a fair ranking.

Policy implications
Ranking in a suspended round-robin league is a non-trivial problem because of the need to account for schedule imbalances and possibly for the different number of matches played by the teams. The quotient rule addresses the second issue but does not reflect the strength of opponents. Furthermore, if most teams have played the same number of matches in an unfinished season, which is the likely case, then tie-breaker criteria are required to be used almost as often as in any round-robin league to assign unique ranks to competitors that are equal on points.
However, both goal difference (winning margin) and head-to-head records are dubious rules in the presence of unplayed matches: the former favours teams that have played against weaker teams, and the latter is strongly influenced by the home-away pattern of the games played. For instance, see the tiebreaking decisions in the 2019/20 season of the French Ligue 1 (2020). Nice was fifth ahead of Reims due to better head-to-head results over their home and away matches, although Reims had a higher goal difference. Brest was ranked above Metz because of superior goal difference as they played only one match. Since the original tie-breaking rule at the end of the season would have been goal difference, the choice of the French top soccer league seems to be somewhat arbitrary.
Regarding the proposed generalized row sum method, even though it is difficult to argue for any particular value of ε, we think it is best to give the largest possible weight to the role of opponents. At least, there is some evidence for supporting the least-squares ranking in Swiss-system chess team tournaments (Csató, 2017). Since this procedure does not contain a parameter, it would be a better alternative than the generalized row sum, which may lead to such disputes.
Therefore it is recommended to choose from the following solutions, listed in decreasing order of complexity and preference: 1. least-squares method; 2. generalized row sum method with a small ε that breaks only the ties remaining in the normalized points; 3. quotient rule (points per game) with the tie-breaking criteria of goal difference, goals scored; 4. number of points with the tie-breaking criteria of fewer matches played, goal difference and goals scored.

Conclusions
The problem of ranking the teams in an incomplete round-robin tournament has been discussed. We have taken an axiomatic approach to identify the key properties of real-world rankings. A family of L. CSATÓ ranking methods has been shown to satisfy the desired requirements. It has been applied for the 2019/20 season of six major European soccer leagues that were suspended after the outbreak of coronavirus. Accounting for the strength of opponents turns out to be necessary for incomplete tournaments. This factor can be considered only by introducing an additional parameter. However, its value has only marginal influence in all leagues considered here as the set of clubs to be relegated is entirely independent of this choice, and qualification to the European cups is barely influenced.
The adjustment of the standard scores is carried out in two steps: the correction for the number of matches played (normalized points) is followed by accounting for the strength of the opponents. This provides a simple, robust, and scientifically well-established way to obtain a fair final standing in a round-robin league that cannot resume.
Naturally, ranking in a sports league is a zero-sum game, thus any solution will prefer certain teams compared to an alternative regime. Future rulebooks should explicitly define what happens if a league has to be finished without playing all matches. Our paper can contribute to single out a suitable policy for this purpose.