The problem with trying to analyze half a season’s worth of games is that it is a small sample that may be victim to a great deal of error. It is unclear what is a genuine, observable trend and what will ultimately wash out as the sample size increases. Given this, it is worth taking a look at performances in previous seasons to gain some perspective. This will also help identify what impact Favre has had on the team, and whether the problems are his doing, or whether he has simply been unable to fix ongoing issues.

This article will take a look at Dortmund’s performances in the last four season, using data taken from various sources, including FiveThirtyEight, WhoScored and FBRef. I will compare Dortmund’s points and win totals with expected totals, analyze over/underperformance in results, and test the causes of over or underperforming expectations.

Simulating Wins

Simulating win totals from 2016 to 2019 using FiveThirtyEight’s win probabilities help us understand whether Dortmund have really underperformed. Figure 1 presents the results.

In total, Dortmund have won 64 of 119 games since the 2016/17 season. Using the FiveThirtyEight win probabilities for all 119 games, I ran a total of 1000 simulations to calculate the expected number of wins since 2016. The average number of wins is about 69. They have underperformed the mean total by 5, but the number of wins falls well within a reasonable range based on the distribution of expected wins, as shown by the red dashed line in Figure 1.

Expected vs Observed Points Differences

In addition to analyzing the win total simulations, I calculated the expected points totals, again using FiveThirtyEight’s win probabilities. In order to do this, I multiplied Dortmund’s win probability by the total number of points they could win in each game (3), giving the expected points. Figure 2 plots the cumulative expected and observed points totals for each season in the dataset. The cumulative expected totals are very similar to the observed totals, which is, for the most part, to be expected.

On average, Dortmund’s total points were the same as their expected points in 16/17, and very slightly lower than their expected points in 17/18. In 2018/19, Dortmund performed much better than their expected points total, gaining a total of 19 points more than the probabilities suggest they should, which is pretty impressive. Finally, Dortmund have very slightly overperformed their expected totals so far this season as well.

These results are not as damning as we might expect, which is why taking a look beyond Favre’s tenure is so valuable.

Over/Underperformance

Having considered the difference between Dortmund’s expected and actual points over time, I considered positive and negative performance from a slightly different angle. I calculated the over/underperformance in each game by subtracting the expected points value from the observed points, producing a positive value when Dortmund perform better than expected and a negative value when they perform worse than expected.

Figure 4 plots Dortmund’s mean average over/underperformance against all of their Bundesliga opponents in the last three years.

On average, Dortmund have had the best results against Gladbach, while they have struggled against Union Berlin, Paderborn, and Darmstadt. The three teams that have produced the largest underperformances help shine a light on the limitations to this approach. The level of over/underperformance is entirely dependent on the prior probability. This means that some of the worst performances, like all games away to Bayern, are not classified as underperformances. As a result, Dortmund’s average against Bayern is actually positive, which feels like a poor representation of the games between the two in the last few seasons.

This doesn’t mean that this approach is incorrect, as it is accurately depicting the points above and below expectations. It just fails to capture the full breadth of over and underperformance.

Where Does It All Go Wrong?

So looking back over the course of the last few seasons helps demonstrate that last season was a real outlier, and that this season looks more like the two seasons prior to the last. Dortmund typically expect to win about 60 points, and they are on course to do exactly that this season. While this season feels like a real disappointment, it is, in reality, a regression to the mean.

That isn’t to say that fans should not be disappointed, but rather to give some perspective over what their disappointment should be based upon. The problem isn’t that Dortmund are performing especially badly, but that they have failed to make progress despite spending a lot of money in the summer in the attempt to push on and challenge for the title regularly. The disappointment is specifically that Dortmund are still the same team despite attempts to be much better than that.

But what has held Dortmund back in these last four seasons? Having established that errors explain a lot of their problems this season, it makes sense to consider whether this has been an ongoing problem in the last four seasons.

Errors

In order to analyze Dortmund’s errors, I have limited comparisons only to their closest competitors. Figure 5 plots the average number of errors committed per match, using the median top four from the last four seasons: Bayern Munich, RB Leipzig, Dortmund, and Bayer Leverkusen.

While Bayern and Leverkusen committed more mistakes in 2016/17, and Leipzig have committed more mistakes so far this season, Dortmund have generally been a lot more error-prone than the rest of the top four. Dortmund’s rivals commit a lot fewer errors in 3 out of 4 seasons, and in Leipzig’s case, they have made only marginally more mistakes than Dortmund this season.

Figure 6 plots the errors committed by Dortmund and the median number of errors committed by their Bundesliga competitors in the last four seasons. In this case, I have included all teams that have finished (or are currently) in the top four. This included a total of 7 teams: RB Leipzig, Bayer Leverkusen, Bayern Munich, Hoffenheim, Schalke, Borussia Monchengladbach, and Borussia Dortmund.

In 2016/17, Dortmund’s competitors committed a greater number of errors, but this number has been consistently falling in every season since, while Dortmund’s errors have remained relatively constant. It’s unclear to me whether this is the result of any active changes made by the other top teams in the Bundesliga, but it strikes me as remarkable that the error rate has been constantly falling among the rest of the league’s best, while it has stayed the same for Dortmund.

Table 1 presents the median errors per game committed by all seven teams in the last four seasons.

Dortmund commit more errors than everyone else, including those that have finished in the bottom half of the league in at least one season. This seems extremely telling.

Away Form

I also considered the possibility that Dortmund’s performances are considerably worse away from home. This has, at times, felt self-evident, as Dortmund are capable of playing at their best at Westfalenstadion, but they have fallen short against opponents they should be beating when playing away from home.

Figure 7 splits the expected and observed points by home and away games, and compares performances in the last four seasons. As demonstrated by the two plots, the expected points totals in home games has a much steeper curve than the away games. This is because FiveThirtyEight’s win probabilities account for the home advantage, so any differences in the observed and expected points is not simply the result of home games favoring the home team.

Dortmund spent the first two seasons underperforming away from home, but improved last season, and in total the cumulative observed points are about the same as the expected points. However, Dortmund tend to perform significantly better than expected when playing at home.

So while there doesn’t appear to be any evidence of underperformance away from home, Dortmund do seem to benefit significantly from playing at the Westfalenstadion.

Testing the Relationship between Errors, Home Advantage, and Over/Underperformance

Having identified that Dortmund commit a lot of errors, and that there are observable differences between performances at home and away games, I ran regressions to test the significance of these effects on Dortmund’s over/underperformance.

I computed a mixed regression model, testing the relationship between over/underperformance and three fixed-effects: Dortmund’s xG, home/away games, and Dortmund’s errors. I also included the opponent win probability as a random-effect variable. Mixed models are particularly valuable when measuring effects on the same units of analysis over time or when measuring effects on clusters of correlated units. In this case, this is particularly useful as the likelihood of over and underperformance differs based on the opponent Dortmund face, which is not of interest in this analysis (I’m not trying to prove that better teams win more, because that is self-evident), so a mixed model allows for clustered observations in the data. Including the opponent’s win probability helps reduce the impact that a team’s overall quality has on the propensity for Dortmund underperforming.

The model results show that both xG and errors are statistically significant, and have substantively large, roughly equivalent, but opposite effects on underperformance. While both of these variables are statistically significant, the binary variable representing whether Dortmund played at home or away is not. Playing away from home has a negative effect on over/underperformance, but there is enough variance in performances at home and away that it did not reach significance.

Figure 8 plots the regression results at home and away games, depending on the number of errors committed.

Figure 8 demonstrates that the number of errors has a significant impact on Dortmund’s performance, while playing away from home produces a moderate but non-significant reduction in performance.

This suggests that errors help explain the games where Dortmund underperform their expected points. Cleaning up their error rate so that it falls in line with their competitors would have a significant impact on their points total, and would likely make them serious title challengers.

Conclusion

So the evidence suggests that errors are not only having an impact under Favre, but have impacted Dortmund’s results in the last four seasons. Not only that, but they appear to explain underperformance in games by Dortmund in recent seasons too. Given this, it is remarkable that they have been able to finish in the top four in each of these seasons. If they were able to reduce the errors, this would surely contribute to significant improvements in results, and would significantly close the gap between Dortmund and the top of the table.

In the final article in this series, I will explore some of the best individual performances so far in the Hinrunde, and compare Dortmund’s best players to the their rivals around the Bundesliga.