Quant Basics 9: Plotting A Response Surface
In the last section we looked at bootstrapping by random sampling one of our best strategy PnL curves in order to determine how stable and reliable the returns are. In this section we will look at the response surface of the returns, that is the PnL with respect to the underlying parameters. In our case we have only two parameters (our moving average look-back periods), so we can conveniently plot a surface in 3 dimensions. Mathematically, more parameters will simply create a multi-dimensional hyper surface, it’s not that more complex but it can be very hard to visualise. We can deal with this scenario only numerically. Response surfaces give us interesting information about our strategy, when the surface is quite smooth, our strategy is generally “well behaved”, however, steep peaks and troughs indicate that small deviations from our tested behaviour will cause large and unexpected changes in our PnL. This is behaviour that we generally try to avoid. In other words, for our parameter sets we look for “plateaus” in the response surface rather than peaks. It is easy to inspect that visually in 3D, in higher dimensions, we need to find a parameter set with a small enough gradient.
∇f=(∂f/∂x)i + (∂f/∂y)j + (∂f/∂z)k
Where f is our response surface and i, j, k are the unit vectors in the x, y, z direction. Numerically speaking, ∂f/∂x around parameter set Pn is nothing but (PnLPn+1 – PnLPn-1)/2 if our parameters are single-spaced integer values as in the case of our moving averages.
Plotting surfaces needs a little bit of vector magic. The code below shows how to plot a surface with arbitrary point and a regular grid superimposed. The meshgrid() function produces that regular grid around our parameters of interest and the griddata() subsequently interpolates between our PnLs to determine the PnLs that would match the points on the regular grid we’ve just produced. Finally, we plot the point and the wireframe in 3D using mplot3d.
def plot_response_surface(pnls,params,tickers,start,end,backend='file'): from mpl_toolkits.mplot3d import Axes3D p = prices(tickers,start,end,backend=backend) best_params = get_best_parameters(params,pnls,50) x =  y =  z =  for par in best_params: x.append(par) y.append(par) z.append(calc_pnl(calc_signals(tickers,p,min(par),max(par)),p)[-1]) n_points = 50 X,Y = np.meshgrid(np.linspace(min(x),max(x),n_points),np.linspace(min(y),max(y),n_points)) Z = scipy.interpolate.griddata(np.array([x,y]).T,np.array(z),(X,Y),method='cubic') fig = plt.figure() ax = fig.add_subplot(111, projection='3d') ax.plot(x,y,z,'ro') ax.plot_wireframe(X,Y,Z) plt.xlabel('x');plt.ylabel('y') plot_response_surface(pnls1,params,tickers,start,end) plt.show()
This seems a bit complicated but in practise, you can just simply use the function above whenever you want to plot such a surface. Our 3D surface produced from our parameters (x and y) and our PnLs (z) looks like this:
With mplot3d you can rotate the plot to inspect your result more closely. In our case we can see that our surface does not have any excessive peaks or troughs, it is quite well-behaved.
So far, we have focused mainly on PnL but it would be nice to know how our strategy performs in terms of Sharpe ratio and drawdown. In the next section we will look at this more closely.
The code base for this section can be found on Github.