Lookahead analysis¶
This page explains how to validate your strategy in terms of look ahead bias.
Checking look ahead bias is the bane of any strategy since it is sometimes very easy to introduce backtest bias - but very hard to detect.
Backtesting initializes all timestamps at once and calculates all indicators in the beginning. This means that if your indicators or entry/exit signals could look into future candles and falsify your backtest.
Lookahead-analysis requires historic data to be available. To learn how to get data for the pairs and exchange you're interested in, head over to the Data Downloading section of the documentation.
This command is built upon backtesting since it internally chains backtests and pokes at the strategy to provoke it to show look ahead bias. This is done by not looking at the strategy itself - but at the results it returned. The results are things like changed indicator-values and moved entries/exits compared to the full backtest.
You can use commands of Backtesting. It also supports the lookahead-analysis of freqai strategies.
--cacheis forced to "none".--max-open-tradesis forced to be at least equal to the number of pairs.--dry-run-walletis forced to be basically infinite (1 billion).--stake-amountis forced to be a static 10000 (10k).--enable-protectionsis forced to be off.
Those are set to avoid users accidentally generating false positives.
Lookahead-analysis command reference¶
usage: freqtrade lookahead-analysis [-h] [-v] [--no-color] [--logfile FILE]
[-V] [-c PATH] [-d PATH] [--userdir PATH]
[-s NAME] [--strategy-path PATH]
[--recursive-strategy-search]
[--freqaimodel NAME]
[--freqaimodel-path PATH] [-i TIMEFRAME]
[--timerange TIMERANGE]
[--data-format-ohlcv {json,jsongz,feather,parquet}]
[--max-open-trades INT]
[--stake-amount STAKE_AMOUNT]
[--fee FLOAT] [-p PAIRS [PAIRS ...]]
[--enable-protections]
[--dry-run-wallet DRY_RUN_WALLET]
[--timeframe-detail TIMEFRAME_DETAIL]
[--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]]
[--export {none,trades,signals}]
[--export-filename PATH]
[--freqai-backtest-live-models]
[--minimum-trade-amount INT]
[--targeted-trade-amount INT]
[--lookahead-analysis-exportfilename LOOKAHEAD_ANALYSIS_EXPORTFILENAME]
options:
-h, --help show this help message and exit
-i TIMEFRAME, --timeframe TIMEFRAME
Specify timeframe (`1m`, `5m`, `30m`, `1h`, `1d`).
--timerange TIMERANGE
Specify what timerange of data to use.
--data-format-ohlcv {json,jsongz,feather,parquet}
Storage format for downloaded candle (OHLCV) data.
(default: `feather`).
--max-open-trades INT
Override the value of the `max_open_trades`
configuration setting.
--stake-amount STAKE_AMOUNT
Override the value of the `stake_amount` configuration
setting.
--fee FLOAT Specify fee ratio. Will be applied twice (on trade
entry and exit).
-p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
Limit command to these pairs. Pairs are space-
separated.
--enable-protections, --enableprotections
Enable protections for backtesting.Will slow
backtesting down by a considerable amount, but will
include configured protections
--dry-run-wallet DRY_RUN_WALLET, --starting-balance DRY_RUN_WALLET
Starting balance, used for backtesting / hyperopt and
dry-runs.
--timeframe-detail TIMEFRAME_DETAIL
Specify detail timeframe for backtesting (`1m`, `5m`,
`30m`, `1h`, `1d`).
--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]
Provide a space-separated list of strategies to
backtest. Please note that timeframe needs to be set
either in config or via command line. When using this
together with `--export trades`, the strategy-name is
injected into the filename (so `backtest-data.json`
becomes `backtest-data-SampleStrategy.json`
--export {none,trades,signals}
Export backtest results (default: trades).
--export-filename PATH, --backtest-filename PATH
Use this filename for backtest results.Requires
`--export` to be set as well. Example: `--export-filen
ame=user_data/backtest_results/backtest_today.json`
--freqai-backtest-live-models
Run backtest with ready models.
--minimum-trade-amount INT
Minimum trade amount for lookahead-analysis
--targeted-trade-amount INT
Targeted trade amount for lookahead analysis
--lookahead-analysis-exportfilename LOOKAHEAD_ANALYSIS_EXPORTFILENAME
Use this csv-filename to store lookahead-analysis-
results
Common arguments:
-v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
--no-color Disable colorization of hyperopt results. May be
useful if you are redirecting output to a file.
--logfile FILE, --log-file FILE
Log to the file specified. Special values are:
'syslog', 'journald'. See the documentation for more
details.
-V, --version show program's version number and exit
-c PATH, --config PATH
Specify configuration file (default:
`userdir/config.json` or `config.json` whichever
exists). Multiple --config options may be used. Can be
set to `-` to read config from stdin.
-d PATH, --datadir PATH, --data-dir PATH
Path to directory with historical backtesting data.
--userdir PATH, --user-data-dir PATH
Path to userdata directory.
Strategy arguments:
-s NAME, --strategy NAME
Specify strategy class name which will be used by the
bot.
--strategy-path PATH Specify additional strategy lookup path.
--recursive-strategy-search
Recursively search for a strategy in the strategies
folder.
--freqaimodel NAME Specify a custom freqaimodels.
--freqaimodel-path PATH
Specify additional lookup path for freqaimodels.
The above Output was reduced to options lookahead-analysis adds on top of regular backtesting commands.
Summary¶
Checks a given strategy for look ahead bias via lookahead-analysis Look ahead bias means that the backtest uses data from future candles thereby not making it viable beyond backtesting and producing false hopes for the one backtesting.
Introduction¶
Many strategies - without the programmer knowing - have fallen prey to look ahead bias.
Any backtest will populate the full dataframe including all time stamps at the beginning. If the programmer is not careful or oblivious how things work internally (which sometimes can be really hard to find out) then it will just look into the future making the strategy amazing but not realistic.
This command is made to try to verify the validity in the form of the aforementioned look ahead bias.
How does the command work?¶
It will start with a backtest of all pairs to generate a baseline for indicators and entries/exits.
After the backtest ran, it will look if the minimum-trade-amount is met
and if not cancel the lookahead-analysis for this strategy.
After setting the baseline it will then do additional runs for every entry and exit separately. When a verification-backtest is done, it will compare the indicators as the signal (either entry or exit) and report the bias. After all signals have been verified or falsified a result-table will be generated for the user to see.
Caveats¶
lookahead-analysiscan only verify / falsify the trades it calculated and verified. If the strategy has many different signals / signal types, it's up to you to select appropriate parameters to ensure that all signals have triggered at least once. Not triggered signals will not have been verified. This could lead to a false-negative (the strategy will then be reported as non-biased).lookahead-analysishas access to everything that backtesting has too. Please don't provoke any configs like enabling position stacking. If you decide to do so, then make doubly sure that you won't ever run out ofmax_open_tradesamount and neither leftover money in your wallet.- In the results table, the
biased_indicatorscolumn will falsely flag FreqAI target indicators defined inset_freqai_targets()as biased. These are not biased and can safely be ignored.