Foreword

A few days ago, it was found that the profit and loss curve output of the FMZ strategy backtest result was relatively simple, so I thought about whether to obtain the income result data and then process it myself to get a more detailed capital curve evaluation report and display it graphically. When I started to write out the ideas, I found that it was not so easy, so I wonder if anyone has the same ideas and has already made the corresponding tools? So I searched the Internet and found that there are indeed such tools. I looked at several projects on GitHub and finally chose pyfolio.

What is pyfolio?

pyfolio is a Python library for financial portfolio performance and risk analysis developed by "quantinc". It works well with "Zipline" open source backtest library. "quantinc" also provides comprehensive management services for professionals, including Zipline, Alphalens, Pyfolio, FactSet data, etc.

The core of pyfolio is the so-called "so-called tear sheet", which is composed of a variety of independent graphs that provide a comprehensive picture of the performance of the trading algorithm.

GitHub address: https://github.com/quantopian/pyfolio

Learn to use pyfolio

Due to the fact that there are few online learning materials for this tool, it takes a long time for me to use it easily.

PyFolio API reference:

https://www.quantopian.com/docs/api-reference/pyfolio-api-reference#pyfolio-api-reference

Here is a more detailed introduction to pyfolio's API. The platform can be used for backtesting of US stocks. The backtesting results can be directly displayed through pyfolio. I only learned it roughly. It seems that other functions are quite powerful.

Install pyfolio

The installation of pyfolio is relatively simple, just follow the instructions on GitHub.

FMZ backtest results displayed by pyfolio

Well, the introduction is here, and began to enter the topic. First, get the backtest capital curve data on FMZ platform.

Click the button next to the full screen in the above figure in the floating profit and loss chart of the backtest result, and then select ‘Download CSV’.
The format of the obtained CSV data is as follows (the file name can be changed according to your needs):

If you want to have a comparative benchmark for the analysis results, you also need to prepare a K-line daily data of the trading target. if there is no K-line data, only the income data can also be analyzed, but there will be several more indicators for the results of benchmark data analysis, such as: Alpha, Beta, etc. The following content are written in accordance with the baseline K-line data.

We can obtain K-line data directly from the platform through the FMZ research environment:

# Use the API provided by the FMZ research environment to obtain K-line data which equal to the revenue data
dfh = get_bars('bitfinex.btc_usd', '1d', start=str(startd), end=str(endd))

After the data is prepared, we can start the coding. We need to process the acquired data to make it conform to the data structure required by pyfolio, and then call the create_returns_tear_sheet interface of pyfolio to calculate and output the result. We mainly need to pass in returns, benchmark_rets=None and live_start_date=None three parameters.

The return parameter is required income data; benchmark_rets is the benchmark data, it is not necessary; live_start_datelive_start_date is not necessary.

The meaning of this parameter is: when did your returns start from the real market? For example, our a bunch of returns above, assuming that we are starting real market after 2019-12-01, and the previous are in the simulation market or the result of a backtest, then we can set it like this: live_start_date = '2019-12-01'.

By setting the parameters, we can theoretically analyze whether our strategy has been overfitted. If the difference between the inside and outside of the sample is large, then there is a high probability that this is overfitting.

We can implement this analysis function in the FMZ research environment, or we can implement it locally. The following takes the implementation in the FMZ research environment as an example:

https://www.fmz.com/upload/asset/1379deaa35b22ee37de23.ipynb?name=%E5%88%A9%E7%94%A8pyfolio%E5%B7%A5%E5%85%B7%E8%AF%84%E4%BB%B7%E5%9B%9E%E6%B5%8B%E8%B5%84%E9%87%91%E6%9B%B2%E7%BA%BF(%E5%8E%9F%E5%88%9B).ipynb

# First, create a new "csv to py code.py" python file locally and copy the following code to generate the py code containing the CSV file of the fund curve downloaded from FMZ. Running the newly created py file locally will generate "chart_hex.py" file.

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import binascii

# The file name can be customized as needed, this example uses the default file name
filename = 'chart.csv'
with open(filename, 'rb') as f:
    content = f.read()
# csv to py
wFile = open(filename.split('.')[0] + '_hex.py', "w")
wFile.write("hexstr = bytearray.fromhex('" +
            bytes.decode(binascii.hexlify(content))
            + "').decode()\nwFile = open('" + filename + "', 'w')\nwFile.write(hexstr)\nwFile.close()")
wFile.close()

# Open the "chart_hex.py" file generated above, copy all the contents and replace the following code blocks, and then run the following code blocks one by one to get the chart.csv file

hexstr = bytearray.fromhex('efbbbf224461746554696d65222c22e6b5aee58aa8e79b88e4ba8f222c22e4ba8be4bbb6220a22323031392d31302d33312030303a30303a3030222c300a22323031392d31312d30312030303a30303a3030222c300a22323031392d31312d30322030303a30303a3030222c2d302e3032383434353837303635373338383930350a22323031392d31312d30332030303a30303a3030222c302e3030373431393439393432333839363936390a22323031392d31312d30342030303a30303a3030222c2d302e30323234373732373731373434313231370a22323031392d31312d30352030303a30303a3030222c2d302e30323033393930383333363836353735390a22323031392d31312d30362030303a30303a3030222c2d302e3034393935353039333230393332303435360a22323031392d31312d30372030303a30303a3030222c2d302e303434333232333634383035363033370a22323031392d31312d30382030303a30303a3030222c2d302e3032353631313934393330353935313637360a22323031392d31312d30392030303a30303a3030222c302e3032363331303433393432313739303536360a22323031392d31312d31302030303a30303a3030222c302e3033303232303332383333303436333137350a22323031392d31312d31312030303a30303a3030222c302e3033313230373133363936363633313133330a22323031392d31312d31322030303a30303a3030222c2d302e3031383533323831363136363038333135350a22323031392d31312d31332030303a30303a3030222c2d302e30313736393032353136363738333732320a22323031392d31312d31342030303a30303a3030222c2d302e3032323339313034373338373637393338360a22323031392d31312d31352030303a30303a3030222c2d302e3030383433363137313736363631333438370a22323031392d31312d31362030303a30303a3030222c302e3031373430363536343033313836383133330a22323031392d31312d31372030303a30303a3030222c302e303232393131353234343739303732330a22323031392d31312d31382030303a30303a3030222c302e3033323032363631303538383035373131340a22323031392d31312d31392030303a30303a3030222c302e303138393230323836383338373438380a22323031392d31312d32302030303a30303a3030222c302e30363632363938393337393232363738390a22323031392d31312d32312030303a30303a3030222c302e3036303835343430303337353130313033370a22323031392d31312d32322030303a30303a3030222c302e31343432363035363831333031303231330a22323031392d31312d32332030303a30303a3030222c302e32343239343037303935353332323336370a22323031392d31312d32342030303a30303a3030222c302e32313133303432303033353237373934310a22323031392d31312d32352030303a30303a3030222c302e323735363433303736313138343937380a22323031392d31312d32362030303a30303a3030222c302e323532343832323739343237363235360a22323031392d31312d32372030303a30303a3030222c302e32343931313136313839303039383437370a22323031392d31312d32382030303a30303a3030222c302e31313038373135373939323036393134310a22323031392d31312d32392030303a30303a3030222c302e313633343530313533373233393139390a22323031392d31312d33302030303a30303a3030222c302e31393838303132323332343735393737350a22323031392d31322d30312030303a30303a3030222c302e31363633373536393939313635393038350a22323031392d31322d30322030303a30303a3030222c302e32303638323732383333323337393630370a22323031392d31322d30332030303a30303a3030222c302e32303434323831303032303830393033320a22323031392d31322d30342030303a30303a3030222c302e323030353636323836353230383830360a22323031392d31322d30352030303a30303a3030222c302e31323434363439343330303739303635360a22323031392d31322d30362030303a30303a3030222c302e31303032343339383239393236303637332c302e31303032343339383239393236303637330a22323031392d31322d30372030303a30303a3030222c302e31303637313232383937343130373831360a22323031392d31322d30382030303a30303a3030222c302e31323839363336313133333032313036310a22323031392d31322d30392030303a30303a3030222c302e313337393030323234303239323136320a22323031392d31322d31302030303a30303a3030222c302e31313432333735383637323436303130350a22323031392d31322d31312030303a30303a3030222c302e31323638353037323134353130343038320a22323031392d31322d31322030303a30303a3030222c302e31343139333631313738343432333234330a22323031392d31322d31332030303a30303a3030222c302e31333838333632383537383138383536370a22323031392d31322d31342030303a30303a3030222c302e313136323031343031393435393734350a22323031392d31322d31352030303a30303a3030222c302e31363135333931303631363930313932330a22323031392d31322d31362030303a30303a3030222c302e31343937383138343836363238323231380a22323031392d31322d31372030303a30303a3030222c302e31353734393833333435363438393438320a22323031392d31322d31382030303a30303a3030222c302e32343234393031303233333139323635380a22323031392d31322d31392030303a30303a3030222c302e32313830363838353631363039303035350a22323031392d31322d32302030303a30303a3030222c302e323938383636303034333936303139340a22323031392d31322d32312030303a30303a3030222c302e33303135333036303934383834370a22323031392d31322d32322030303a30303a3030222c302e323938363835393334383634363038370a22323031392d31322d32332030303a30303a3030222c302e333039333035323733383735393130310a22323031392d31322d32342030303a30303a3030222c302e333834363231343935353136383931320a22323031392d31322d32352030303a30303a3030222c302e33343532373534363233383138313130360a22323031392d31322d32362030303a30303a3030222c302e33363235323332383833363737313035330a22323031392d31322d32372030303a30303a3030222c302e33343937363331393933333834333133360a22323031392d31322d32382030303a30303a3030222c302e33303732393733373234353434373938360a22323031392d31322d32392030303a30303a3030222c302e33323238383132323432363135363530370a22323031392d31322d33302030303a30303a3030222c302e33343134363537343239333438363535330a22323031392d31322d33312030303a30303a3030222c302e333435323733393139363237303738320a22323032302d30312d30312030303a30303a3030222c302e33353730313633323035353433343337340a22323032302d30312d30322030303a30303a3030222c302e33343937353937393034363236373934370a22323032302d30312d30332030303a30303a3030222c302e33373032333633333138303534353335370a22323032302d30312d30342030303a30303a3030222c302e33383636373137373837343037313635370a22323032302d30312d30352030303a30303a3030222c302e33383834373536373836393031343634330a22323032302d30312d30362030303a30303a3030222c302e34313331323236353139383433373731340a22323032302d30312d30372030303a30303a3030222c302e34323335323332383237303436333733350a22323032302d30312d30382030303a30303a3030222c302e34363837333531323838353035333330330a22323032302d30312d30392030303a30303a3030222c302e353436373135313832363033383332380a22323032302d30312d31302030303a30303a3030222c302e353530373037323136333937383830310a22323032302d30312d31312030303a30303a3030222c302e35353531373436393236393938310a22323032302d30312d31322030303a30303a3030222c302e353632323130363337343737323731330a22323032302d30312d31332030303a30303a3030222c302e353734373831373030393536383631370a22323032302d30312d31342030303a30303a3030222c302e353632383330303731353536353831350a22323032302d30312d31352030303a30303a3030222c302e363538323839383038313031393136380a22323032302d30312d31362030303a30303a3030222c302e363732323034393830303331333936370a22323032302d30312d31372030303a30303a3030222c302e363537313832383237323238323335380a22323032302d30312d31382030303a30303a3030222c302e363734393831383838383639373536330a22323032302d30312d31392030303a30303a3030222c302e363739373632303637393239383131330a22323032302d30312d32302030303a30303a3030222c302e363334313332373332393636313231370a22323032302d30312d32312030303a30303a3030222c302e363237353837313436323430323734370a22323032302d30312d32322030303a30303a3030222c302e363331313336373230353334393834370a22323032302d30312d32332030303a30303a3030222c302e3630313936323331393931343334360a22323032302d30312d32342030303a30303a3030222c302e363036343239313935383633313431360a22323032302d30312d32352030303a30303a3030222c302e35383130363933393531373337390a22323032302d30312d32362030303a30303a3030222c302e363133313034353130383436353937380a22323032302d30312d32372030303a30303a3030222c302e3632393938323638373737383035350a22323032302d30312d32382030303a30303a3030222c302e363831333134363734333130313533350a22323032302d30312d32392030303a30303a3030222c302e373134303533393533383834313233350a22323032302d30312d33302030303a30303a3030222c302e373433383032353331363031313135360a22323032302d30312d33312030303a30303a3030222c302e373535393639303935383539313330370a22323032302d30322d30312030303a30303a3030222c302e373533383030313630323737353438310a22323032302d30322d30322030303a30303a3030222c302e373534343434333437323732343132350a22323032302d30322d30332030303a30303a3030222c302e373435373138393532343434373738330a22323032302d30322d30342030303a30303a3030222c302e3738373636303035313130343530340a22323032302d30322d30352030303a30303a3030222c302e373935393939343930353732393834360a22323032302d30322d30362030303a30303a3030222c302e373935323037323039363636373034390a22323032302d30322d30372030303a30303a3030222c302e3832393234363232343838363336350a22323032302d30322d30382030303a30303a3030222c302e383239393034373635353939363035350a22323032302d30322d30392030303a30303a3030222c302e383338363639323137313033313436350a22323032302d30322d31302030303a30303a3030222c302e38353830313634373631380a22323032302d30322d31312030303a30303a3030222c302e383130323530393437393936313938330a22323032302d30322d31322030303a30303a3030222c302e383433323631313436333636313030320a22323032302d30322d31332030303a30303a3030222c302e383535383536353834363731333632320a22323032302d30322d31342030303a30303a3030222c302e383337323730363631383738303935360a22323032302d30322d31352030303a30303a3030222c302e383333353332343038383538303234330a22323032302d30322d31362030303a30303a3030222c302e383636383832343034353334343633320a22323032302d30322d31372030303a30303a3030222c302e383836363634323232323038333831310a22323032302d30322d31382030303a30303a3030222c302e393032363430303937303731373033390a22323032302d30322d31392030303a30303a3030222c302e383832373838333631373939333438380a22323032302d30322d32302030303a30303a3030222c302e383530303035363732363738333734320a22323032302d30322d32312030303a30303a3030222c302e3737383436363530373530313739360a22323032302d30322d32322030303a30303a3030222c302e373737383734393835393335313437350a22323032302d30322d32332030303a30303a3030222c302e373731333834393530303532383132330a22323032302d30322d32342030303a30303a3030222c302e373937383030363936353434323134340a22323032302d30322d32352030303a30303a3030222c302e373736383231373934313333363939370a22323032302d30322d32362030303a30303a3030222c302e373938353333313136353336313831310a22323032302d30322d32372030303a30303a3030222c302e383530343335363139343238353239390a22323032302d30322d32382030303a30303a3030222c302e383734333333393138383334393638310a22323032302d30322d32392030303a30303a3030222c302e3838383336363333393338343837380a22323032302d30332d30312030303a30303a3030222c302e383933393737393637343631333438380a22323032302d30332d30322030303a30303a3030222c302e3931323431323035313530303336362c302e3931323431323035313530303336360a22323032302d30332d30332030303a30303a3030222c302e383733353632323939353238363532330a22323032302d30332d30342030303a30303a3030222c302e383532353336353235333030343039310a22323032302d30332d30352030303a30303a3030222c302e383633323633313830363733313335350a22323032302d30332d30362030303a30303a3030222c302e383734303237343632353730373730350a22323032302d30332d30372030303a30303a3030222c302e383634323439323631363431353135360a22323032302d30332d30382030303a30303a3030222c302e38373630353132313331363135333031').decode()
wFile = open('chart.csv', 'w')
wFile.write(hexstr)
wFile.close()

!ls -la

cat chart.csv

# Install pyfolio library in research environment

!pip3 install --user pyfolio

import pandas as pd
import sys
sys.path.append('/home/quant/.local/lib/python3.6/site-packages')
import pyfolio as pf
import matplotlib.pyplot as plt
%matplotlib inline
import warnings
warnings.filterwarnings('ignore')
from fmz import * # import all FMZ functions

# Read fund curve data, FMZ platform download, cumulative income data
df=pd.read_csv(filepath_or_buffer='chart.csv')
# Convert to date format
df['Date'] = pd.to_datetime(df['DateTime'],format='%Y-%m-%d %H:%M:%S')
# Get start and end time
startd = df.at[0,'Date']
endd = df.at[df.shape[0]-1,'Date']

# Read the target asset daily K-line data, and use it as the benchmark income data
# Use the API provided by the FMZ research environment to obtain K-line data equal to the revenue data
dfh = get_bars('bitfinex.btc_usd', '1d', start=str(startd), end=str(endd))
dfh=dfh[['close']]
# Calculate the daily rise and fall based on the closing price of k-line data
dfh['close_shift'] = dfh['close'].shift(1)
dfh = dfh.fillna(method='bfill') #  Look down for the nearest non-null value, fill the exact position with this value, full name "backward fill"
dfh['changeval']=dfh['close']-dfh['close_shift']
dfh['change']=dfh['changeval']/dfh['close_shift']
# Frequency changes keep 6 decimal places
dfh = dfh.round({'change': 6})

# Revenue data processing, the FMZ platform obtains the cumulative revenue, and converts it to the daily revenue change rate
df['return_shift'] = df['Floating Profit and Loss'].shift(1)
df['dayly']=df['Floating P&L']-df['return_shift']
chushizichan = 3 #  Initial asset value in FMZ backtest
df['returns'] = df['dayly']/(df['return_shift']+chushizichan)
df=df[['Date','Floating Profit and Loss','return_shift','dayly','returns']]
df = df.fillna(value=0.0)
df = df.round({'dayly': 3}) # retain three decimal places
df = df.round({'returns': 6})

# Convert pd.DataFrame to pd.Series required for pyfolio earnings
df['Date'] = pd.to_datetime(df['Date'])
df=df[['Date','returns']]
df.set_index('Date', inplace=True)
# Processed revenue data
returns = df['returns'].tz_localize('UTC')

# Convert pd.DataFrame to pd.Series required for pyfolio benchmark returns
dfh=dfh[['change']]
dfh = pd.Series(dfh['change'].values, index=dfh.index)
# Processed benchmark data
benchmark_rets = dfh

# The point in time when real-time trading begins after the strategy's backtest period.
live_start_date = '2020-02-01'

# Call pyfolio's API to calculate and output the fund curve analysis result graph
# "returns" Parameters are required, the remaining parameters can not be entered
pf.create_returns_tear_sheet(returns,benchmark_rets=benchmark_rets,live_start_date=live_start_date)

The output analysis result:

Interpretation of results

There are a lot of output data, we need to calm down and learn what these indicators mean. Let me introduce a few of them. After we find the introduction to the relevant indicators and understand the meaning of the indicators, we can interpret our trading strategy status.

• Annual return

Annualized rate of return is calculated by converting the current rate of return (daily rate of return, weekly rate of return, monthly rate of return, etc.) into annual rate of return. It is a theoretical rate of return, not a rate of return that has actually been achieved. The annualized rate of return needs to be distinguished from the annual rate of return. The annual rate of return refers to the rate of return for one year of strategy execution and is the actual rate of return.

• Cumulative returns

The easiest concept to understand is the return on strategy, which is the rate of change in total assets from the beginning to the end of the strategy.
Annual Volatility
The annualized volatility rate is used to measure the volatility risk of the investment target.

• Sharpe ratio

Describes the excess return that the strategy can obtain under the total unit risk.

• Max Drawdown

Describing the biggest loss of the strategy. The maximum drawdown is usually the smaller, the better.

• Omega ratio

Another risk-reward performance indicator. Its biggest advantage over Sharpe ratio is-by construction-it considers all statistical moments, while Sharpe ratio only considers the first two moments.

• Sortino ratio

Describes the excess return that the strategy can obtain under the unit's downside risk.

• Daily Value-at-Risk

Daily Value at Risk-Another very popular risk indicator. In this case, it means that in 95% of cases, the position (portfolio) is kept for another day, and the loss will not exceed 1.8%.

Reference: https://towardsdatascience.com/the-easiest-way-to-evaluate-the-performance-of-trading-strategies-in-python-4959fd798bb3

• Tail ratio

Select the 95th and 5th quantiles for the distribution of daily return, and then divide to obtain the absolute value. The essential meaning is how many times the return earned is greater than the loss.

• Stability

This is called stability. In fact, it is very simple, that is, how much the time increment explains the cumulative net value, that is, the r-squared of the regression. This is a bit abstract, let's explain briefly.

Reference: https://blog.csdn.net/qtlyx/article/details/88724236

Small suggestions

It is hoped that FMZ can increase the evaluation function of the rich capital curve, and increase the storage function of historical backtest results, so that it can display the backtest results more conveniently and professionally, and help you create better strategies.

From: Evaluation of backtest capital curve using "pyfolio" tool (fmz.com)