Your advice: what's a reasonable way to test RANDOM EXITS?

[sluggo]

We've seen the Tharp-Basso system that makes trade entries at random, then uses a trailing stop to exit (link). But what about the reverse: a system that makes trade exits at random, but lets us use our favorite type of entry. To my knowledge, nobody has ever discussed such a thing.

It got me wondering: What is the most reasonable way to "test random exits"? What plan of testing is the fairest? On my own I came up with a few ideas, but would love to hear YOUR ideas for how to "test random exits".

I thought of these:

1. Wait for your entry signal to make a trade entry. Then go ahead and enter the market according to that entry signal. Remain in the trade for "D" days after entry, then exit at the market price at the end of the D'th day. "D" is a number chosen at random, uniformly distributed between D=1 days and D=100 days (or whichever boundaries you like).
2. Same as #1 except the random numbers D follow a probability distribution of trade holding times that resembles a real-world trading system's holding time distribution. For example, Figure 1 below is taken from straight from the Blox backtest of a futures trading system; it is column U of "Trade Log.csv".
3. You restrict yourself to entry signals taken from pure-reversal systems that are always in the market, either long or short but never out. The random number generator is used to generate random DATES (rather than random trade holding-times). For each exit date, you look back and see what was the most recent entry signal. Then your "trade" consists of that most recent entry signal, plus the randomly chosen exit date.

I am not so fond of #3 because it seems too restrictive to forbid any kind of entry signals. After all we are trying to answer the question, "How profitable is a system that uses a good entry but exits at random?" and it seems unwise to forbid any kinds of good entries.

Would love to hear your thoughts.

+SLUGGO+

[Jens Albrecht]

I would like to see added:
Take entry, then given an exit chance x% roll each day wether to leave trade. Would be interesting stepping through x%.

[Roscoe]

Hi Sluggo. I have a form of #1 as part of my standard testing code but it is very simple - it exits at market after D bars but D is just a static value. I never really found a lot of use for it really, it was based on a suggestion from Chuck LeBeau as a means for testing entries (try 5, 10, 20 bars) but my observation was that the interaction between entry and exit was far more relevant.

[Forum Mgmnt]

I've tested using both #1 and a close cousin to #2, namely an exit that was a random number of days between some minimum and maximum. For example, randomly exiting between 20 and 150 days after entry.

I have not tested using a distribution as you outlined in #2 but this is certainly an interesting idea. My only fear with using the distribution is that you skew your results toward the type of system you pick, i.e. long-term trendfollowing, medium-term, super-long-term, medium-term counter-trend, etc.

- Forum Mgmnt

[sluggo]

Preliminary results on

• Portfolio of 10 US futures markets. One of them was "ES", the eMini S&P 500 futures.
• Test ran for 10 years, from 3/19/1997 to 3/21/2007
• $100 per contract, round trip, for commissions+slippage
• Textbook entry signal
• Random Exits of some kind

It's fun, I encourage Blox owners to experiment.

TENTATIVE CONCLUSION: Textbook entry signals are non-negligible contributors towards system overall profits. Even with random exits, good entry signals can make money.

[td80]

Excellent work sluggo, very interesting results.

I am experimenting with some volatility concepts myself with the same sort of focus on exit efficiency.

If you get bored try plotting an indicator with:

(20 bar high - 20 bar low) / (50 bar high - 50 bar low)

Notice what happens on a variety of timeframes on a variety of instruments when the percentage drops below 40%. It tends to be either a reversal or an acceleration of the trend right around that area and the instrument normally goes ballistic and the ratio goes over 80% .... I'm experimenting with it trying some things, trying to see if it's possible to develop a system around it or perhaps entries/exits. If it dips below 40%, I consider that a trigger, and then until it goes over 80%, I consider the market to be "springy". Then until another dip below 40, market will probably do average price acceleration....

Thanks again for your insightful posts Sluggo,

-Dave

[nickmar]

Here's a system inspired by this discussion:

System 1 Rules:

Code: Select all

1 - Pick a random number X between 20 and 200
2 - Wait X trading days
3 - Pick another random number Y between 20 and 200
4 - Flip a coin. If it is heads, enter a Long position and hold for Y trading days. If it is tails, enter a Short position and hold for Y trading days
5 - Repeat from step 1

System 2 Rules:

Code: Select all

1 - Pick a random number X between 20 and 200
2 - Wait X trading days
3 - Pick another random number Y between 20 and 200
4 - If the current close is greater than the Y day moving average, enter a Long position and hold for Y trading days. If the current close is less than the Y day moving average, enter a Short position and hold for Y trading days.
5 - Repeat from step 1

• Portfolio Breakdown:
  
  Currencies - 6
  Energies - 6
  Grains - 7
  Long_Rates - 7
  Meats - 4
  Metals - 9
  Short_Rates - 5
  Softs - 8
  Stock_Indices - 9
  
  Total Markets - 61

- Position-sizes were volatility-adjusted
- Slippage of 15% and roll slippage of 10%
- Test ran from 1990-03-31 to 2007-02-28

Each system was executed 2000 times - here's an interesting visual:

EDIT: System 1 is represented by the bottom graph and system 2 is represented by the top graph.

[sluggo]

nickmar, that's outstanding! It seems that (random entries + random exits) are significantly worse than (moving average entries + random exits). Conversely, moving average entries are better than random! This contradicts the teachings of a few gurus.

[svquant]

Just to throw out another idea on the random exit method would be to have a random selection of exit methodologies per trade. In the fixed N-day or distribution based N-day exit we are really testing a fixed time exit criterion. Yes we vary the days but not the method. I'm suggesting to look at picking based on a random variable which exit to use after your entry point, eg N-day min/max, MA-xover... Of course if the exit gives a signal immediately this would be like a 1-day fixed exit. A way of thinking about this is that it is similar to the N-day drawn from a distribution but we are generating the distribution in a more random unbiased way vs using a current trading system which will show a bias as mentioned above. Best to have a good selection of exit methodologies/criterion that include trend, counter trend, profit target...

Have not coded this - but it is food for thought. Thanks to those who have coded and shown some results!

Marc

[Forum Mgmnt]

nickmar's tests above reminded me very much of the research I did for the chapter "Lies, Damn Lies, and Backtests" in Way of the Turtle. Since this is a topic that I had already promised to discuss in detail in a blog entry on my blog site at www.wayoftheturtle.com. I finished that blog entry tonight so those of you who don't have the book yet can see what I was talking about.

In short, my tests showed very similar positive skew to the random exit system if I considered only random long entries when the 25-day moving average was higher than the 350 day moving average, an vice versa for shorts. In my case I randomly exited between 20 and 120 days after entry.

[sluggo]

svquant / Marc, thanks very much for the suggestions! I certainly agree, what has been discussed so far is only one particular "method" of exit: a time based exit. The amount of time you wait before exiting each particular trade is random, but the fact that it is a time-exit and nothing but, remains.

Your suggestions seem to be along the lines of "Let's create a grab bag of numerous realistic exits, and then for each trade, reach into the bag and pull out an exit." But is this what we really want? We seek to isolate the effects of entry signals by pairing them with "Random" exits; but to only use "realistic" exits may be to reduce the randomness, perhaps?

Another thought: No matter what procedure is used to choose an exit, eventually the exit order is filled at some instant of time. The two instants of time: Entry Fill Time, and Exit Fill Time, uniquely identify this trade. Since the Entry Fill Time is chosen by the entry signal, the purpose of the exit signal is to choose the Exit Fill Time. Why not make this Exit Fill Time be a random number? Does that not meet the objective of creating a "Random Exit?"

sluggo

[sluggo]

Test results placed in a new thread: viewtopic.php?t=3779

[svquant]

Sluggo:

Yes in the end all trades are just entry and exit dates... oh but the effort we put into those dates

My comments were really in terms of #2 and c.f.' comment about the distribution. Of course another way would be to just look at the distributions from a few different systems/exits and use the N-day drawn from this mixture of distributions. But in the end will it really tell you anything more about your entries then unif-random in a range of days? Perhaps not!

Reminds me of academic papers that always do a bootstrap procedure vs just a plain old t-test since many financial time series or trading system time series violate the normality assumption, but in the end the t-test vs bootstrap does not give different answers. Better to use a more computationally difficult non-parametric test due to the time series properties? Yes, the theory tells us so. Matter in practice - not from what I can tell.

[GENX]

Flip out the old stethiscope at day of entry and document the following:

ATR @ Entry Signal X Calendar Day
ATR X March 26 (ATR*26)/30days = Stethiscope number (S#)

Running Mean of ATR for last 30 days= 30!ATR

S#/30!ATR and smooth by say 15 days= Exit monitor

When Exit Monitor displaces by 1STD exit trade.


Of course I have not tested this, and I can't tell you pedigree info, it was truly just a random jittle of thoughts itself. So accept it for what it is, something to consider.