In our previous article, we introduced three ways of testing your skill in sports betting:

• Betting with a small bankroll (or no bankroll, logging ‘fake’ bets in a spreadsheet)
• Backtesting
• Closing Line Value

In this article, I will explain other ways of testing your skill and designing a betting strategy by following one of these three methods:

• Measuring the p-value of your backtest
• Forward testing
• Blind betting

Measure The p-value Of Your Backtest

A backtest is a way of testing your method with past data. You will need a sports events database with at least:

• Date, so that your events can be ordered in time
• Players or teams
• Results of the events
• Odds

Regarding the odds, which odds do we need? Opening odds? Closing odds? 1h before the match? The maximum/minimum value between opening and closing? From how many bookmakers? A simple answer: the more, the better (in Spain we say, the more sugar, the sweeter).

Just be sure that the odds are accurate, and that they would have existed for enough time to place your bets.

There are several free sources for all of this data, like betexplorer, oddsportal, football-data and tennis-data, flashscore… You can build your database from them.

In order to perform your backtest, you can establish a set of rules that must be fulfilled in order to bet.

We can do this very fast with Betaminic and its Betamin Builder tool. For example:

• A team that hasn’t won any of the last 3 games in a row when playing at home
• The away team has not scored >2 goals in its last 3 away matches
• Matches between 2016 and today (26/11/2020)
• If both conditions are fulfilled, then bet for the home team, no matter the odds

We can check the “backtest” of this method with the results and odds of the database, for the period of the last five years (2016 to 2020). This strategy would have obtained a profit of 55 units at average odds of 2.95 with a yield of 7.66%. But these specific rules have produced only 719 bets (of 73009 matches between 2016 and 2020, a 1%). You can check the strategy here (Home revenge):

I changed the number of consecutive matches and number of goals several times before obtaining a profitable strategy, so could the final system have been a result of “data-mining” or “overfitting”? These factors first need to be taken into account to conclude if the positive results may have been achieved by simple luck or variance.

So, let’s analyse whether this system will obtain long term winnings? We can evaluate the p-value with this p-value calculator, that is, the probability of obtaining such yield just by luck.

Not bad! 7.427%. It is a good result for a strategy built in a few minutes. But be aware that we have not calculated the probability of a home win, nor have we estimated the minimum profitable odds, and we have also not taken into account the odds, to decide if we bet or not. The rules are very simple, maybe too simple.

A reasonable value required to consider that we have not been lucky is to obtain a p-value lower than 1% (@12xpert would say even lower than 0.1%). So what can we do to improve our strategy?: We need a higher yield!

We can add an additional rule to obtain a new strategy (Home come back):

• The home team has not conceded more than 1 goals in at least one of the last 3 (not won) matches.

Yes! The yield has improved to 15.90%, with almost the same profit, 44.7 units. But the number of bets has been reduced to 300 in the last 5 years… Consequently, the p-value is again not small enough:

regardless, this doesn’t definitively confirm whether these strategies are profitable or not, we will verify the system using the following form of testing: the forward-test.

The Forward-Test

A forward-test is an evaluation of a method or strategy, without taking into account any future data we wouldn’t have known at the moment of setting the rules of the method.

You may think: Hey, we have done the backtest with past data, we didn’t use future data. But yes, we did. We have set the rules to decide if we bet or not using odds and results data we shouldn’t have known. We have used “future” data (odds and results) to calibrate the parameters of our method.

But now, after you have calibrated your model with past data, we have to maintain the system and avoid changing anything in the future, and evaluate your method with future data.

We can also simulate this in Betaminic. We can build a strategy with past data from a calibration time period, and perform a theoretical forward-test with the following year, for example.

I have created a new simple strategy (Away team surprise):

• Home team has won 2 games in a row
• Home team has not scored more than 2 goals in both matches
• If that happens, we bet AH Away+0.5 (X2)
• Calibration period from 2014 to 2019.

From a total of 84835 matches, in the backtest period between 2014 and 2018/2019, we would have bet 453 matches, with a profit of 12.42 units and a yield of 2.74%. The p-value is 28.1%. What will happen in the next few years? Have we been lucky in the calibration period? We can establish the same parameters and evaluate the strategy again, performing our forward-test with the strategy (Away team surprise Forward-Test):

We have won again! In fact, I didn’t expect to win in the forward-test, but perhaps we have found a “gold mine”. I think I have been very lucky! Let’s see in the future…

The stats of the total period (Away team surprise total) are 591 bets at average odds of 1.98, with a profit of 17.01 units, and a yield of 2.88%. That results in a slightly lower p-value of 23.97%. I still wouldn’t risk real money with this strategy…

Blind Betting

This method requires extreme discipline. Blind betting is not betting with a blindfold. Blind betting is trying to calculate the probability of a match result without knowing anything about the bookmaker odds. Some expert tipsters, like @SquashPicks, use this method.

In order to practice this blind betting method, you need to:

1. Select a match and a market (Home/Away, Over/Under, etc). Avoid seeing the odds of the bookmakers!
2. Try to establish the probabilities of all the possible outcomes. Remember that the sum of all the probabilities should result in 1!
3. Calculate the estimated odds as the inverse of estimated probabilities.
4. Estimate the error by recording if you are “very sure” or “not sure” about the probabilities you have established. You can use several grades of uncertainty.
5. Depending on the error, increase the odds by multiplying with a “safety factor” that depends on the estimated error. For example 1.1 for “very sure” and 1.25 for “not sure”. These are your “minimum profitable odds”.
6. Record all the values previous to consult any bookmaker odds.
7. Check your bookmakers and save the values of the odds. If possible, save also the closing odds. If you want to compare your estimated probabilities, with the probabilities estimated by the bookmaker, remember that you need to calculate the inverse of the bookmakers’ odds and then remove the margin.
8. If the odds are higher than the minimum profitable odds, then you should record a bet. You can use different strategies for the bet size, as we explained here.
9. Wait for the results and evaluate your method.

By doing so, you are training your estimation of probabilities and error. If you track your bets and finally obtain a profit, perhaps you have found a new value betting method! I recommend you calculate the p-value of your bets as well, then start betting if this p-value is small enough for you.

This blind betting method can be also adjusted with the same kind of backtest by changing your “safety factors” or the bet size (you should never change your estimated probabilities after the match has been played, as you probably know the result or the odds, and could be affected by the “anchor bias”). However, the more you increase your safety factors, the fewer bets you will have placed. The yield will likely grow, but the p-value will often still be higher.

Conclusion

We have explained three advanced ways of testing your strategies or methods. All of them require time to “think slow” and it is very difficult to find a profitable method with a very low p-value backtest that also performs well in the forward test; or a blind betting method that obtains good results, even after adjusting the safety factors; or a betting strategy that systematically beats the Closing Line Value.

However, I recommend that you combine all three methods of testing. Use a backtest generated by blind betting, adjust the safety factors, calculate its p-value, and then evaluate the forward-test.

If you are using just your mind and intuition to estimate your probabilities, it’s going to be very hard to perform more than one backtest: “If I could go back in time one year, what probabilities would I have predicted for the next day’s match?” As you probably already know the results or the odds, or you have predicted these probabilities before, you will be influenced by them. This is one of the disadvantages of being a tipster instead of using an algorithm, as it was explained in the Tipsters vs AI Machine Learning article.

The only way I know to do many backtests is by using an algorithm or a statistical method like the examples above, but that requires a lot of data and also programming, statistics and math skills. In the end, if you want to win, you have to invest huge effort and time (and perhaps also you will need money to buy a new powerful computer!).

Let us know how you test your betting systems, which method do you use? If you have any other testing methods you use let us know in the comments below!