Autumn is not disappearing but has been moved on the calendar

Summer lasts longer. November, December, and January have more mild days than before

You’ve probably heard or chatted recently about the topic “the Autumn is disappearing”. People talk about that here in Barcelona. After a mild October, suddenly a cold snap brought the winter for the first time. The feeling is that this is more and more common as years go by, but, is really the Fall threatened with extinction? Data rather suggest that it’s been pushed and it’s leaving the October.

I’ve done a brief study with the daily temperature data series from Barcelona Airport, aiming to figure out if there is any clear trend about the number of Autumn days decreasing. The exercise is easy to understand: I’ve gotten an average temperature of October and November, taking the period 1971-2000 as a reference, and after that created an interval of what I decide is an Autumn day. To do so, I widened three degrees Celsius above and below the October/November average temperature. What we get is that every day with an average temperature roughly between 12 and 18 degrees Celsius is what we are gonna say from now on this article that is an Autumn day.

Then, using some SQL queries I’ve counted how many September, October, November, December and January days fitted into this criteria from 1951 to nowadays. The result is that if we sum how many days had these mild temperatures from September to January year by year, there is no clear trend. They haven’t increased or decreased. By average 47 days fitted into this criteria, some years had much more, some years much less, but there’s no significant trend.

So where this perception comes from? There are no clear changes among years, but there are changes in how these Autumn days get distributed among months. The main change is that October has been stormed by the summer. During the second half of the XX century, often about 15 to 20 days of October fitted into our Autumn criteria, but a downward trend from 2000 is crystal clear. During the first decade of the XXI century were already 10 days a year, and from 2011 the average lowered even below 8. 2018 is a clear example of that, just two days of October could be considered Autumn days using our criteria, 26 were warmer, and the other 3 colder.

It’s also less common that the Autumn feeling arrives early during September. On the other hand, November, December and especially January have more Autumn days than before. More than disappearing, the fall has been pushed, at least in Barcelona. -November, December, and January make up for the lack of Autumn days in October.

The cold is vanishing

Probably the lack of Autumn feeling comes from the fact that the summer lasts such a long time that often gets followed by the first cold days, as happened this year, but this doesn’t mean cold settles.

In fact, if something is disappearing from Autumn it’s the cold. I’ve also counted what happens with the days that doesn’t fit on our Autumn category during that five months, so to know which were colder o warmer than this. During many decades on this period, the cold days has been more than the warm days, but the tendency fairly flipped during the last 20 years. Now it’s normal to have more warm than cold days among September 1st and January 31st. Summer it’s getting longer and Autumn it’s moving to winter.


The article was published in the newspaper ARA on November 3, 2018.

Five last minutes are a mine of points for Real Madrid

FC Barcelona would have won two more championships considering the games to end at minute 85

If you’re a Barça fan this should sound very familiar to you: it’s Sunday in the afternoon, you were at the movies. As you leave the theater and switch on the cell phone you get some messages about Real Madrid’s game. Real isn’t winning and there are just 20 minutes left. Don’t trust, but step into a bar to watch the rest of the game though. Let’s go, maybe there’s some good news on the way. 75 minutes and still a tie game, 80 minutes, 85 minutes, almost… but in the end Real Madrid scores, and you go home upset because of a game you were not supposed to have watched. Among the most common Barça supporters mantra’s, there is the one which says that Real Madrid scores last-gasp winners very often. What truth is there in this complaint?

I’ve analyzed every single game of Spanish Liga since 1960 to shed some light on this intuition. The exercise lies in comparing the result of the games on three different moments with the final result. In other words, see which team was winning -or if the game was tied-, after 75 minutes, after 80 and after 85, and compare those results with the final one. Doing this I’ve been able to develop a won/lost points balance during the last minutes, coming from each of this three different moments. For example, let’s suppose that a team is trailing till minute 87 of the game, though the final result is a draw. This would mean a +1 on this balance. On the other side, the team who was winning would get -2 points.

The results confirm the belief of Barça supporters: Real Madrid is by far the team who wins more points during the last minutes of the game. Since 1960, Madrid has made it to win 67 points during the last 5 minutes of the game, far behind it there is Valencia with 31, and Betis and Bilbao with 29. The result also says another important thing: the belief not only comes from the fact that Madrid is good on scoring last-gasp winners but also on that Barça performs very bad on last minutes of the game. Barça owns a balance of just +3 points, and has jumped into the positive figures just because earned +6 points on last season. In a mock league taking care only of the points earned or lost during the last 5 minutes of the game, Barça would be sat on the bottom half of it. Considering just the main teams in the league, only Real Sociedad would perform worst: the Basques owns a -9 points balance, considering the changes of the result in the last 5 minutes of the game.

If we set the boundary at the minute 80 of the game, or at the minute 75, Barça improves. On a mock league of the last 15 minutes of the game, Barça would already be second, having less than half of Madrid’s points though. There is one thing that attracts attention, the more we get close to the end of the game, the more increases the difference between the two teams. Figures of the 75-minute balance show that Madrid wins the double of points than Barça, considering 80-minute boundary, wins four times more, and considering an 85-minute limit Madrid takes 20 times more points.

More points, but no more championships

There’s another important question to ask: how it would change the league list of winners if we get rid of the last minutes of the game? The most stunning answer is that Barça would have won two more championships considering the games to last 5 fewer minutes. Altogether, 8 championships would have changed hands, 70/71, 76/77 and 83/84 Ligas would be on Barça’s museum, none of them would come from Real Madrid though. However, Madrid would have won one of Barça’s Ligas: the first of the back-to-back that were decided on the finish line at Tenerife (season 1991/92). Madrid wouldn’t have less Ligas on its museum according to that figure though, on the contrary, there would be one more. Among the most surprising Ligas that would change hands, there is season 2002/03, in which the team dressed in white resisted against the surprising Real Sociedad of Kovacevic, Nihat, Karpin and a youngster called Xabi Alonso. Real Madrid finished the championship two points over the Basque team (78 to 76), without the last 5 minutes of game Real Sociedad would have taken that Liga with an 8 points margin.

The Ligas that would have changed of owner according to 80-minute balance are exactly the same, but staring at the last 15 minutes balance, Barça gets even more harmed. FC Barcelona would have won four more championships according to last 15 minutes balance, this time some of them coming from Real Madrid. In particular, one of the ‘la Quinta del Buitre’ Ligas (season 1988/89), the first of Cruyff as Barça’s coach. Real Madrid made 62 points on that season, Barça fell short by 5 (57 points). Not considering the last 15 minutes of each game Barça would have made the same number of points, Real Madrid would have got stuck at 56 though. Also Liga’s 2006/07 title would pass from Madrid to Barcelona. That one is especially remembered for the goal scored by RCD Espanyol’s player Raúl Tamudo at Camp Nou in the next-to-last match of the season, which combined with a goal almost at the same time by Van Nistelrooy at La Romareda Stadium left the championship almost on Real Madrid’s hands. The so called ‘Liga del tamudazo’ would be on Barça’s museum with no discussion (5 point margin) if we don’t consider the last 15 minutes of each game. Real Madrid won 8 points during the last 15 minutes of the game on that season.

Change of roles during the 90’s

Almost always during the last 55 years, Real Madrid has been a winning team considering just the last minutes of the game, though temporal sequence comparing Barça and Madrid shows a clear period when the roles changed. During some years at the 90’s Real Madrid lost more points per season than the worst Barça seasons. A kind of hole that was quickly left behind.

Barça’s bad run on the final stage of the games got broken on the last season (2016/17), in which made 6 points on last five minutes balance. Since 1960 just on 1996/97 Liga (second after the change from 2 to 3 points per win) Barça had made so many points in the last minutes of the game. Last season FC Barcelona won on the last gasp at Mestalla, at Vicente Calderón and so did on a home game against Leganés. Moreover, took a draw scoring last-gasp goals at Betis’s and Villarreal’s fields. Even there is one more game that Barça won on injury time which is not considered on this balance: Messi scored the third goal (2-3) at Santiago Bernabéu after minute 90, though those points aren’t considered because at minute 85 the game was on Barça’s side too (1-2). On the contrary, Madrid made two points on first leg game at Camp Nou, scoring the goal that tied the game at minute 90 (1-1).

Article published in newspaper ARA on june 6th, 2017

Why it’s best to bet for underdogs

The commission that bookmakers make you pay for betting it’s quite different depending on the chances of win

An easy search on the net is enough to find thousands of sources making reference to how bookmakers do to earn money, regardless of the outcomes in sports events. They do that in many different ways, but the most basic is simple to understand: they charge a kind of commission that it’s already included on the odds they offer you to win.

To understand this, the first you have to know is that you can turn the odds that the bookmakers offer into a probability. For example, if we have a decimal format odds, just have to divide 1 by the figure, and multiply per 100. So, if bookmakers offer 1.7 euros for each euro we bet on the win of FC Barcelona, somehow we are accepting that the chances of a Barcelona win are 1/1.7×100, so 58.8%. Let’s find some real example, December 19th, 11:30 PM, for the Philadelphia 76ers against Brooklyn Nets game, Bwin offers 1.85 for the 76ers win, and 1.97 for the Nets victory. According to what I said, that means 54% probability of 76ers to win, and 50.7% probability of a Nets victory. Or Philadelphia wins or Brooklyn do so, so the add of this two figures should be 100%, and the fact is that it is 104.7%. The bookmakers overestimate on purpose the odds of victory of each team, to make sure that whoever wins, there is money left for them to earn, a kind of commission. We can repeat as much as we want this figure, and almost everytime we’d get a plus 100% result. Up to this point, nothing much new. This is not the only practice that ensures money for the bookmakers, that wouldn’t be enough, but in this we’ll focus on that.

If you waste some time doing the same exercise for many different games, you’ll figure out that more or less the commission charged is quite the same in most of the matches. That could drive you to think that whoever you bet for, you’re gonna be charged more or less the same. Is it true? By now we’ve figured out how the bookmakers charge us for betting on a specific game but, are we able to know which is the commission for betting on a specific team?

illustration_NBA_MLB_idoia_vallverdu_fora_webTo figure it out I’ve made a frequency distribution with the odds offered by the bookmakers. The analysis is based on each game on the Major League Baseball between 2005 and 2012, also 2016. That makes a sum of more than 22.000 games. Chose the baseball for two basic reasons: a huge number of games on a simple season, no chance of, what makes the exercise easier to do and understand. The source of the data between 2005 and 2012 is website and got 2016 data from Oddsportal. In both cases, the figures represent an average of the odds offered by the main bookmakers just before the game starts.

The exercise lies in passing over every simple bet, making groups of them, putting together the ones which represent a probability between 2% and 5%, another group with the ones between 5% and 8%, and so on. Once this is done, I’ve checked for every group how many of the  were at the end winning ones, and how many were not. If there is an enough amount of bets in a specific group and the estimation of probability is correct, the number of winning bets should be in between the interval. The figure below shows the difference between the estimation and the reality. So, if for the group of bets between 47% and 50% the real number of bets won is 46.5%, on the figure you would see -2%, which is the difference between 46.5% and the center of this interval: 48.5%. The line is a 3-value moving average.


As expected, on every group the number of winning bets is smaller than estimated for the bookmakers market, what confirms that de bookmakers overestimate the real chance of win. Besides, there is something of what I was after in this figure: we don’t pay the same commission betting for favorites than doing it for underdogs. The difference reaches a point in which betting for a team with a chance of win between 65% and 68% on the Major League Baseball it’s been charging by average with a 2,7% commission, whereas betting for teams with chance between 35% and 38% you’d have played for free, even you’d been paid 0.5%. Even if I wouldn’t know who the hell is Clayton Kershaw, betting blindfolded in every team within between this interval would have won some money.

About the reasons, nothing on this figure allows to take conclusions, but I do have a theory. This interval between 35% and 38% is the maybe psychologically the worst for betting. I mean, betting for favorite always seems safer, and in any case, if you’re are willing to bet for and underdog, make sense that you look for one which gives you enough money to worth the risk. Makes sense that betting for this kind of is unattractive, and so the bookmakers reduce here the commission to stimulate betting for this kind of teams. Anyway, it seems we have an answer for the question if, in a long-term thinking, is it better to bet on favorites or underdogs.

Let’s go beyond, is this pattern unique to baseball? I made the exercise also for the NBA, another league that accomplish our basic specifications, though has the half number of games per season. The figure shows such a similar shape, again we have a ‘peak’, between 35% and 40% the commission goes almost non-existent, from then tends to rise.


There’s an important difference between the two sports, while on baseball the frequency distribution is narrow, in the NBA there are many more games with an obvious favorite. One of the reasons for this is the importance of a starting pitcher in baseball. The player on this role a critical part on the final score of the match, and to avoid fatigue and injuries 4 or 5 players rotate on this position. So the best pitcher in the team usually plays only one of every five games, make easier for worst teams in the league to beat the best ones. That’s one of the reasons of a narrow frequency distribution on baseball.

The field factor and the referee’s influence

The referees award almost the same fouls to home teams as to away ones, but away players are sent off more easily

Finished the previous post telling about one of my reference books here in this blog: Scorecasting: The hidden influences behind how sports are played and games are won. One of the studies that the book mentions was made by two Spanish economists that in 2005 set out to see how peer pressure affects human decisions. Luis Garicano and Ignacio Palacios-Huerta counted the extra minutes added by referees in the Spanish league, taking into account the result of the score in the 90th minute.

In most cases, the referees added about three minutes on average, but when the difference was only by one goal there was a clear deviation: if the home team was winning, the average went down to about two minutes. On the contrary, if the away team had an advantage of one goal, the average of extra minutes added went up to four.

Illustration: Idoia Vallverdú
Illustration: Idoia Vallverdú

To learn a bit more about this, we have analysed how many fouls per game have been awarded to home teams and away teams in the last ten years, and also how many yellow and red cards have been shown.

The result is that in the overall of the five leagues 2.6% more fouls are awarded to away teams than to home teams, but in some countries home teams are the ones that are awarded more fouls. The difference, however, soars according to the gravity of the sanction: away teams see 20% more yellow cards than home teams, and up to almost 42% more red cards. It is true that while the referees blow for more than 10,000 fouls per season, they only show 100 red cards, making it easier in this regard for the differences in percentage to be higher. But looking at the figures, it is not nonsense to think that the referees favour more home teams the more difficult the decision they have to take is.

The field factor and the referee's influence

Less fouls, same cards

From the statistics of these ten seasons other curious facts can be drawn: the number of fouls awarded has dropped self-evidently in every league, and specifically in the Spanish league it has dropped from almost 15,000 in the 2005-2006 season to less than 11,000 per season for the last three years. Overall in the five major leagues in ten years, referees have gone from blowing for 35 fouls per game to about 28. The evolution is very similar in four of the five competitions, but in English football is quite different, and the data clearly confirms one of the classic stereotypes of the European football: English football is clearly where the referees give more freedom to play. But also in this aspect there is a trend towards standardization since ten years ago in England referees awarded about 13 fouls less per game than in other leagues, and in the last two season the difference has stood at around 7.

Average fouls signalled per game

This clear and continuous decline in the number of fouls awarded per game has not been accompanied by a decline in the number of yellow cards: in general, about 4 per game are shown, a figure that has hardly changed in the last 10 years. The Spanish league is where more cards are shown (5.2 per game) and in England where less (3.2). What we can see is a slight drop in the number of players sent off, but the trend is nowhere near as remarkable as the number of fouls. On average in the five competitions a total of 0.25 red cards per game are shown, that is there is a player sent off in every 4 games. It is also in this aspect that Spain leads in the card tally, since a player is sent off by the referees every 263 minutes on average, whereas in England a player leaves the pitch every 518 minutes.

Playing at home is no longer the advantage it used to be

A study of every match of the 5 most important European football leagues since 1970 up to nowadays. Overall, at the end of the 70’s teams retained almost 70% of home points, whereas in the last seasons this figure has come down even below 60%

To start this blog site I get back an article I published on the newspaper ARA on august 2015. A study about the scores of the 5 main European football leagues, based on a database I made up using datasets from I’ve split it into two halves. Here comes the first one:

Everyone knows that playing at home is a way to start the match with an advantage, but to what extent playing a game at your home stadium favours a team? To be able to appreciate the importance of the field factor we have analysed the results of all the matches of the five major European football leagues from 1970 to the present, and the overall answer is that in all these years, home teams have won 64.5% of the points and away teams, 35.5%. That is, home teams have managed to secure almost 2 out of every 3 points they have obtained. There are differences between countries that are never higher than 5 percentage points. France is where teams retain more points at home (67%), whereas in England is where they get less (62%). One of the first questions hanging in the air is whether or not the fact that in English matches visiting supporters usually occupy a significant part of the stadium can be related to this difference.

The decline of the home teams advantage in football

If we chop up the figures by season, we find something we were expecting: the field advantage today is no longer what it used to be in the previous decades, and it is important to note that it has been going down over the last seasons. Overall, at the end of the 70’s teams retained almost 70% of home points, whereas in the last seasons this figure has come down even below 60%. Another thing that is obvious by looking at the first chart that accompanies this piece is that over the years the lines of the different countries have been getting closer, that is during the last seasons the differences between leagues have narrowed.

These figures alone don’t provide any evidence, but probably most readers of this article will share the hypothesis that the gradual pacification of the stadiums over the years seems to have been a key factor in explaining this sustained trend. It is easy to think that the lower the level of violence in a stadium is, the more relaxed away players play and the less pressure referees feel when blowing for fouls, and that all of these help to balance the scale between the teams. On this basis, it is interesting to look at the chart for the figure of the final of the European Cup in 1985 between Liverpool and Juventus, played at the Heysel stadium in Brussels. The fights between fans before the match ended up with 39 supporters (most of them Italians) killed and around 600 injured. For the significance of the match, this is probably the biggest tragedy in a European football stadium due to fights between supporters. Interestingly, during the years after the Heysel disaster it is in the English football league where more teams fail to win at home, but in general the downward trend accentuates over the next decade.

Illustration: Idoia Vallverdú

There is one last aspect that is striking in the chart: the country and the period in which winning away is more complicated is in Spain during the transition. There is a clear peak around the year of general Franco’s death, and that has to make one consider whether or not a period of social upheaval and instability like that one could have had an indirect influence on the fact that home victories were easier.

So far we have been talking about violence, but if we asked ourselves why is it easier for teams to win at home, probably most of us would reply that it is due to a mix of the effects of the public on the players and referees, the difference in size or the grass that may exist between one stadium and another, or even perhaps we would add the weather. 4 years ago, the journalist Jon Wertheim and the economist Tobias Moskowitz resolved themselves to clarify precisely the reasons behind the field factor in their book Scorecasting: The hidden influences behind how sports are played and games are won. From the statistical analysis of different data sets in various sports they came to the conclusion that the responsibility falls primarily on the referees. According to them, nor weather, nor traveling, nor fields affect the results, and even the effect of the supporters on the away team is not significant. Actually, that’s what the next post will talk about.

What if we take Barça and Madrid of the league

NOTE: A reader comment when the article was published suggested that the drop on the home field advantage in the Spanish football could be related to the increasing gap between the two main teams (Barça and Madrid) and the rest. That was easy to check, filtering all the games on which Barça and Madrid are involved, and so getting a league without them. The result is a very similar graph, without Barça nor Madrid, the home field advantage would have also declined. On the comments I was also asked about the change from 2 to 3 points per win, I indeed took this into consideration, as well as paid attention to the difference in the year it was applied.