describes how to use freqtrade, an open source trading software written in Python. We will use freqtrade to create, optimize, and run cryptocurrency trading policies using pandas.
Part 1 (this article)
- Basic functions of Freqtrade and encryption market terminology-we will learn how freqtrade works, how to use command-line tools to download historical market data, and how to create new profiles and policies. How to define policies using Python and pandas-We will define a simple moving average strategy to trade between ETH and BTC in an attempt to maximize the number of bitcoins we hold. Retroactively test the strategy based on historical data to determine the performance of our strategy-we will learn how to generate a complete report and how to visualize the graph of the robot simulation transaction.
Basic knowledge of Freqtrade
Freqtrade is a cryptocurrency algorithm trading software written in Python. It allows you to:
- Make a strategy: use Python and pandas easily. We will create a simple policy in this article, and you can view freqtrade's sample policy repository. Sample policy repository for freqtrade two。 Download market data: quickly download historical price data for the cryptocurrency of your choice.
- Backtest: Test your strategy against historical data. Backtesting is a key step in seeing if your strategy has a chance of making money in the real world. It does not guarantee actual performance because market conditions are more complex than the downloaded data.
- Optimization: use hyperopt to find the parameters that best suit your strategy.
- Select currency pairs to trade: your selection can be static or dynamic based on a simple filter, such as whether the transaction volume is greater than a certain amount.
- Trial run: use a simulation wallet to test the strategy based on real-time data.
- Real-time operation: deploy the strategy using real funds through the API of the cryptocurrency exchange. This is the last step after you know what you are doing and are willing to lose all your money.
- Use Telegram Robots: control and monitor your policies through Telegram.
- Analyze and visualize transaction history: utilize transaction data saved on file or through SQL databases.
Install freqtrade
OK, let's start with the installation. Docker-the easiest way to get started on all platforms is Docker, which is the fastest way to get started on all platforms and is recommended by Windows. You need to install Docker and docker-compose first, and then make sure that Docker is running by starting Docker Desktop. You can now set up freqtrade by issuing the following command in the desired directory:
mkdir ft_userdata
cd ft_userdata/
# Download the docker-compose file from the repository
curl https://raw.githubusercontent.com/freqtrade/freqtrade/stable/docker-compose.yml -o docker-compose.yml
# Pull the freqtrade image
docker-compose pull
# Create user directory structure
docker-compose run --rm freqtrade create-userdir --userdir user_data
You should now have the following directory structure:
ft_userdata
└───user_data
├───backtest_results
├───data
├───hyperopts
├───hyperopt_results
├───logs
├───notebooks
├───plot
└───strategies
How do I install freqtrade without Docker? Direct installation for direct installation on your system (Linux, MacOS, Windows) that do not require Docker, I recommend that you review the documentation directly. As far as I'm concerned, with Ubuntu 20.04, it's simple.
git clone https://github.com/freqtrade/freqtrade.git
cd freqtrade
git checkout stable
./setup.sh -i
This step takes some time to complete and requires input to generate the initial configuration. Don't worry too much about this, because we can change it later, but as a rule of thumb, say "yes" to everything. Note: the installation creates a virtualenv (.env), and we should activate it before doing anything with freqtrade.
source ./.env/bin/activate
We can use the freqtrade command. To verify that the installation was successful, run
freqtrade --help
Important: if you install freqtrade directly, you do not need to
docker-compose run --rm
Prefix the command as in the rest of this article. To verify that freqtrade is properly installed and available for use, issue the help command:
docker-compose run --rm freqtrade --help
You should see the following output
usage: freqtrade [-h] [-V]
{trade,create-userdir,new-config,new-hyperopt,new-strategy,download-data,convert-data,convert-trade-data,list-data,backtesting,edge,hyperopt,hyperopt-list,hyperopt-show,list-exchanges,list-hyperopts,list-markets,list-pairs,list-strategies,list-timeframes,show-trades,test-pairlist,instal l-ui,plot-dataframe,plot-profit} ...
The output of the help command shows all possible freqtrade commands. In this series, we will explore the most important commands and how to use them. The first is the backtracking test.
In this article, we want to create a simple strategy and test it back based on historical data. The backtest simulates the transactions expected by the strategy based on the historical data test strategy. Although this does not guarantee performance in the real world, it is a good indication of the winning / losing strategy. Freqtrade uses the following steps to test back the policy:
docker-compose run --rm freqtrade download-data -p ETH/BTC -t 1d --timerange 20200101-20201231 --exchange binance
The arguments to this command tell freqtrade the following: -p ETH/BTC- downloads data of ETH-BTC currency pairs -t 1d-download data with a time range of 1 day -- timerange 20200101-20201231 downloads data from January 1, 2020 to December 31, 2020. -- exchange binance- downloads data from the currency Exchange. Here, you can use any exchange supported by freqtrade. This command generates files:
ft_userdata
└───user_data
└───data
└───binance
ETH_BTC-1d.json
It contains several open, low and closed volume (OHLCV) data artifacts, as follows:
时间 打开 高的 低的 关闭 体积
1622851200000 0.073013 0.075 0.0726 0.073986 143662.752
1622937600000 0.073989 0.075961 0.073626 0.075737 120623.423
The meanings of the columns are as follows: time: Unix timestamp in milliseconds Opening price: the opening price at the beginning of a candlestick. The highest price reached during a candlestick. Lowest price: the lowest price reached during the candlestick. Closing price: the closing price at the end of a candlestick Transaction volume: the number of assets purchased or sold, shown in the base currency (in our case, ETH). BTC is the quoted currency.
We have the data needed to test back the policy, but we need to create a configuration file, which will allow us to easily control multiple parameters of the policy. To create a new configuration, we run the following command:
docker-compose run --rm freqtrade new-config --config user_data/learndatasci-config.json
Before continuing, we need to open the configuration file and set pair_whitelist = ["ETH/BTC"], which will create the currency pair we are interested in. And then we're ready to go. You don't need to worry about anything else for now, but you should make sure you understand what the other configuration options mean, so be sure to visit the documentation.
Here, we will define a simple moving average policy, similar to the one in the "Python for Finance" series. If you haven't read it, be sure to check out. You can find more details in Investopedia. Know the strategy. I'll tell you a little secret. The deal is actually very simple, you only need to do two things: Buy cheaply Sell at a high price Is it that easy? Any problems? The tricky part is finding actual lows and highs. The ideas behind the moving average strategy are as follows: You have two lines. -slow moving average (SMA): the long-term average of past values, representing the overall trend. -Fast moving average (FMA): a short-term average of past values, representing current trends. For our strategy, we will use the following indicators: buy when the FMA crosses the top of the SMA, indicating an upward trend. Selling when the FMA crosses below the SMA indicates a downward trend. This is called the moving average crossover strategy. Defining a custom policy for freqtrade allows us to use pandas to transform the moving average crossover policy in freqtrade. First, we need to create a new strategy file to preserve the logic behind buy/sell signals.
docker-compose run --rm freqtrade new-strategy -s SimpleMA_strategy --template minus
We can now find the newly created file in the policy folder:
ft_userdata
└───user_data
...
└───strategies
SimpleMA_strategy.py
SimpleMAstrategy.py contains an automatically generated class, SimpleMAstrategy, and several functions that we need to update. To define our simple policy, we need to update the following three functions: Populateindicators () Populatebuytrend () Populateselltrend () Let's discuss these contents one by one. 1.populateindicators () Here, we calculate the indicators that our strategy needs to generate buy / sell signals. Fast moving average (blue line) and slow moving average (yellow line) indicators drawn for ETH/BTC according to our strategy, we calculate the fast moving average fastMA (using the last 5 candlesticks) and the slow moving average slowma (using the first 50 candlesticks):
class SimpleMA_strategy(IStrategy):
...
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
fast_period = 5
slow_period = 50
dataframe['fast_MA'] = ta.SMA(dataframe, timeperiod=fast_period)
dataframe['slow_MA'] = ta.SMA(dataframe, timeperiod=slow_period)
return dataframe
Notice that we pass the data frame as a parameter, manipulate it, and return it. All of our functions process frames of data in this way. If you are interested in seeing metrics other than simple moving averages, check out the ta-lib documentation. Note Function definitions in this class use type hints to define parameter and return value types. 2.populatebuytrend () This function populates our buy signal, which is triggered when the fastMA cross line is higher than that in our strategy slowMA. We can do this by updating populatebuytrend () to include the following logic:
class SimpleMA_strategy(IStrategy):
...
def populate_indicators(...):
...
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe.loc[
(
qtpylib.crossed_above(dataframe['fast_MA'], dataframe['slow_MA'])
),
'buy'] = 1
return dataframe
With qtpylib, we can easily find the intersection. In essence, when our purchase condition () is triggered, the above code buy populates the column crossedabove with 1. 3.populateselltrend () In a manner similar to the previous function, this function populates our sell signal. According to our strategy, this is slowMA when the fast_MA crossover line is below.
class SimpleMA_strategy(IStrategy):
...
def populate_indicators(...):
...
def populate_buy_trend(...):
...
def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe.loc[
(
qtpylib.crossed_below(dataframe['fast_MA'], dataframe['slow_MA'])
),
'sell'
] = 1
return dataframe
The crossedbelow we use now is-- vice versa-- when our sell condition is triggered, it populates the column with 1crossedabove.
class SimpleMA_strategy(IStrategy):
...
# the following statement disables selling due to ROI
minimal_roi = {
"0": 100
}
# the following statement disables selling due to stop-loss
stoploss = -1.0
def populate_indicators(...):
...
def populate_buy_trend(...):
...
def populate_sell_trend(...):
...
Now that we have filled in the strategy, we can test its performance on past data. After the backtest strategy has defined our simple strategy, we now want to evaluate it using the historical data of the retest, which allows us to trade in the past and see how they perform. Backtesting is not a perfect representation of how well our strategy performs because other factors affect real-time market returns, such as slippage. To perform backtesting using freqtrade, we can run the following command using the class and function we just created:
docker-compose run --rm freqtrade backtesting
--strategy SimpleMA_strategy -i 1d -p ETH/BTC
--timerange 20200101-20201231
--starting-balance 1
-c ./user_data/learndatasci-config.json
The parameters in the above command are as follows: -p ETH/BTC- trades ETH/BTC currency pairs and changes our BTC to ETH -- timerange 20200101-20201231-2020, from the first day 01max 01max 2020 to the last day 121max 2020. -c. / user_data/config-learndatasci.json We are using the configuration defined earlier in this article. -- starting-balance 1 Let's start with the balance of 1 BTC.
Result for strategy SimpleMA_strategy
========================================================== BACKTESTING REPORT ==========================================================
| Pair | Buys | Avg Profit % | Cum Profit % | Tot Profit BTC | Tot Profit % | Avg Duration | Win Draw Loss Win% |
|---------+--------+----------------+----------------+------------------+----------------+-------------------+-------------------------|
| ETH/BTC | 6 | 8.47 | 50.81 | 0.05086506 | 5.09 | 34 days, 12:00:00 | 3 0 3 50.0 |
| TOTAL | 6 | 8.47 | 50.81 | 0.05086506 | 5.09 | 34 days, 12:00:00 | 3 0 3 50.0 |
==================================================== SELL REASON STATS =====================================================
| Sell Reason | Sells | Win Draws Loss Win% | Avg Profit % | Cum Profit % | Tot Profit BTC | Tot Profit % |
|---------------+---------+--------------------------+----------------+----------------+------------------+----------------|
| sell_signal | 6 | 3 0 3 50.0 | 8.47 | 50.81 | 0.0508651 | 50.81 |
===================================================== LEFT OPEN TRADES REPORT ======================================================
| Pair | Buys | Avg Profit % | Cum Profit % | Tot Profit BTC | Tot Profit % | Avg Duration | Win Draw Loss Win% |
|--------+--------+----------------+----------------+------------------+----------------+----------------+-------------------------|
| TOTAL | 0 | 0.00 | 0.00 | 0.00000000 | 0.00 | 0:00 | 0 0 0 0 |
=============== SUMMARY METRICS ===============
| Metric | Value |
|-----------------------+---------------------|
| Backtesting from | 2020-01-01 00:00:00 |
| Backtesting to | 2020-12-31 00:00:00 |
| Max open trades | 1 |
| | |
| Total trades | 6 |
| Starting balance | 1.00000000 BTC |
| Final balance | 1.05086506 BTC |
| Absolute profit | 0.05086506 BTC |
| Total profit % | 5.09% |
| Trades per day | 0.02 |
| Avg. stake amount | 0.10000000 BTC |
| Total trade volume | 0.60000000 BTC |
| | |
| Best Pair | ETH/BTC 50.81% |
| Worst Pair | ETH/BTC 50.81% |
| Best trade | ETH/BTC 36.59% |
| Worst trade | ETH/BTC -10.57% |
| Best day | 0.03662727 BTC |
| Worst day | -0.01058107 BTC |
| Days win/draw/lose | 3 / 277 / 3 |
| Avg. Duration Winners | 51 days, 16:00:00 |
| Avg. Duration Loser | 17 days, 8:00:00 |
| Zero Duration Trades | 0.00% (0) |
| Rejected Buy signals | 0 |
| | |
| Min balance | 1.02441263 BTC |
| Max balance | 1.06732320 BTC |
| Drawdown | 16.44% |
| Drawdown | 0.01645814 BTC |
| Drawdown high | 0.06732320 BTC |
| Drawdown low | 0.05086506 BTC |
| Drawdown Start | 2020-09-10 00:00:00 |
| Drawdown End | 2020-12-19 00:00:00 |
| Market change | 24.93% |
===============================================
We get a complete report that contains the results of all transactions over the specified period. Freqtrade divides the report into four parts:
We can see that only six transactions have taken place. These transactions generated 5.09 per cent profits, starting at 1 BTC and ending at 1.05086506 BTC. Given the risks involved, the result is not impressive. However, this strategy is very simple and there is a lot of room for improvement: Compared to buying and holding only ETH, that is, converting our entire BTC stack to ETH at the beginning of the test period, we will get 24.93% (market change indicator), but this is not what we can usually expect. We have much less exposure, betting only 10 per cent of each deal, not all of it. It is important to test our strategy under different conditions-not only when the market is growing, but also when the market is shrinking. Trading more currencies for us only considers Ethernet Fong, which is one of the hundreds of currencies we can trade. This restriction allows only one transaction at a time, which is obviously not ideal. With a more advanced strategy, we can say that we use one of the simplest strategies, which only uses a simple moving average as an indicator. Increased complexity does not necessarily mean better performance, but we can test a large number of combinations of metrics against each other to find the best strategy. At this point, we haven't tried to optimize any hyperparameters, such as moving average periods, investment returns, and stops. In a smaller period of time we only think about daily candlesticks, which is one reason why robots find only about 0.02 transactions a day, which is much less than human traders. Robots can make more money by trading more frequently and looking at more detailed candlesticks.
To use freqtrade's drawing command, we need to change the docker-compose.yml file. The only thing we need to do is comment out one line and uncomment the other. Refer to the following file excerpt for an example:
---
version: '3'
services:
freqtrade:
# image: freqtradeorg/freqtrade:stable <--- comment this line
# image: freqtradeorg/freqtrade:develop
# Use plotting image
image: freqtradeorg/freqtrade:develop_plot <--- uncomment this line
...
This tells docker-compose to pull the freqtrade Docker image that contains the correct drawing library. We can now use this command to visualize our data, metrics, and buy / sell signal freqtrade plot-dataframe.
docker-compose run --rm freqtrade plot-dataframe -s SimpleMA_strategy --config ./user_data/learndatasci-config.json -i 1d --timerange 20200101-20201231 --indicators1 fast_MA slow_MA
The-- indicators1 option defines the metrics we want to plot, namely fastMA and SlowMA. These must be defined within the policy specified with the option. By default, this creates an available drawing html file in the drawing directory:. / userdata/plot/freqtrade-plot-ETHBTC-1d.html
ft_userdata
└───user_data
...
└───plot
freqtrade-plot-ETH_BTC-1d.html
Hover over the drawing to see how the robot actually performs what we want it to do, as defined by our simple moving average strategy: FastMA buys slowMA when crossing below. FastMA sells slowMA when crossed at the top. To see what else you can do with plot-dataframe, run docker-compose run-- rm freqtrade plot-dataframe-h or visit the documentation.