The Art and Science of Trading:

Course Workbook
Detailed Examples & Further Reading

Hunter Hudson Press, New York, New York MMXVII

Copyright ©2017 by Adam Grimes. All rights reserved.

Published by Hunter Hudson Press, New York, NY.
No part of this publication may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, electronic, mechanical,
photocopying, recording, scanning, or otherwise, except as permitted under
Section 107 or 108 of the 1976 United States Copyright Act, without either
written permission.
Requests for permission should be addressed to the author at
[email protected].
Limit of Liability/Disclaimer of Warranty: While the publisher and author
have used their best efforts in preparing this book, they make no
representations or warranties with respect to the accuracy or completeness of
the contents of this book and specifically disclaim any implied warranties of
merchantability or fitness for a particular purpose. No warranty may be
created or extended by sales representatives or written sales materials. The
advice and strategies contained herein may not be suitable for your situation.
You should consult with a professional where appropriate. Neither the
publisher nor author shall be liable for any loss of profit or any other
commercial damages, including but not limited to special, incidental,
consequential, or other damages.

ISBN-13: 978-1-948101-01-1
ISBN-10: 1-948101-01-7
Printed in the United States of America
10 9 8 7 6 5 4 3 2 1
To my readers:
Your support for my work has touched me.
Your dedication and perseverance have inspired me.
Your questions have challenged me, and I’ve learned so much from you.
I am honored to be a part of your journey.
Thank you
“It’s what you learn after you know it all that counts.”
 – John Wooden
Wooden
 Forward (and how to use this book)
This is not a typical trading book. If you’re going to use it most effectively,
you need to know a few things about it.

What Is In This Book?

This book is a companion to the trading course, The Art and Science of
Trading, available free of charge at MarketLife.com, and also to my first
book, The Art and Science of Technical Analysis: Market Structure, Price
Action, and Trading Strategies (2012). The content in this book fall in four
broad categories:

Exercises and “homework” for the trading course, available at

MarketLife.com
Collected blogs and other short pieces of writing relevant to
each section and topic.
More extensive studies in “whitepaper” format that provide a
statistical foundation for this style of trading.
A reading plan for The Art and Science of Technical Analysis.

A few thoughts on each of these:

Exercises for the Trading Course

One of the things that makes learning to trade difficult is that there has
never been a solid curriculum to develop the skills of technical trading. This
book includes exercises that have been shown to be effective over years of
teaching and coaching traders: chartreading exercises that develop skills from
reading inside individual bars to understanding large-scale moves of markets.
The ways in which markets trend, and those trends come to an end, are
examined in detail, as are other aspects of price action.
There are also some little gems hidden throughout this work: There’s a full
history of the Dow Jones Industrial Average in chart format, with
performance summaries for each decade, and important geopolitical events
marked on the charts. You literally hold in your hands the history of stock
trading in North America. There are also charts that explore important and
volatile situations in the market, as well as some charts that were specifically
chosen to show conflicted and confusing technical patterns.
Effective trading requires much more than looking at chart patterns, and
there’s material here to help you craft an enduring and effective trading
program: exercises aimed at developing a comprehensive trading and business
plan, managing various aspects of trader psychology, keeping effective
records, and doing statistical analysis and your own market research. Traders
at any stage of development should find something of value here, and, for
many traders, this information will lay a solid foundation for a lifetime of
successful trading.
Developing traders often struggle with an insufficient perspective on
market history. Time spent studying these historical examples, deeply, is time
well spent. Do not focus too heavily on page count! Some of the most
important work in this book occupies only one or two pages for each exercise.

Collected Blogs and Other Shorts

Over the past decade, I’ve blogged and written
writt en regularly in various formats.
This book collects selected blog posts that will give insight on the topics
covered. I have made an effort to retain the casual, informal tone of this
format; these posts have been only lightly edited to fit into this book, and
most posts feature the original graphics which, in some cases, were slightly
lower production quality than what you might usually find in a book.
I admit to having some concerns about effectively republishing material I
have already shared in another format. As this book started to take shape,
those concerns faded because I saw how well the posts addressed the
students’ learning along the way. Also, one of the strengths of a blog is also
its weakness—it’s a living thing. Readers focus on current topics, and blog
posts from years past are sometimes overlooked. Writing this book was a
good opportunity to collect some of those older, historical blog posts and to
connect them to overarching concepts.

Whitepapers
The whitepapers in Part II of the book have never been published in their
entirety, though some of the information found its way into various
presentations and blog posts I have done over the years. They give some good
examples of ways in which we can apply quantitative techniques to market
data. Hopefully, I’ve communicated some of the nuance involved with this
work, and stressed the need for humility—we never have firm, final answers
to most of our questions, and there’s always another way to consider the
problems involved. The last chapter in this section provides some solid
examples of quantitative tendencies that support the style of trading in my
first book and the online course.
There are several ways you can use the material in this book most
effectively,
effectively, depending on your experience and objectives.

How to Use this Book

Any book can be read cover to cover, and that might be a good way to
familiarize yourself with the contents. After that first read through, there are
several other ways you may best use this material:

As the Workbook to the Course

This is how this book started: as a collection of pdf documents and charts
that were designed to extend the work in the course. If you are working
through the online course, simply use the material in this book as your
homework, working through each module consecutively.
consecutively.
If, for whatever reason, you do not have the course material, you can still
do many of the homework exercises. Every effort has been made to make the
explanations and descriptions as useful as possible without going into
unnecessary and redundant detail. Some of these exercises may stand on their
own better than others, but they will really shine when you work through
them, as intended, with the online course.
The exercises and studies presented here are the result of many years of
practical trading, teaching, thinking about markets, and feedback from readers
and traders I have worked with. There are no “filler” or “throwaway”
exercises—everything
exercises—everything is important.

As a Study Guide for My First Book

You may also read my first book, The Art and Science of Technical
Analysis (Wiley, 2012), with this book in hand, following the guided reading
plan for that book. People who have read the book effectively generally read
it more than once, take
t ake notes, and create exercises to help them understand the
concepts. The online course was originally intended to be a companion to the
book, to help people work through it in a structured fashion, and to make sure
readers were getting the most out of it.
If you work through that book following the topically-ordered reading plan
here, also take a look at the associated exercises; some of those exercises will
appeal to you and will offer good opportunities to deepen your understanding
of the material.

As a Stand-alone Reference
The whitepapers in Part II can be read by themselves, in the order in which
they are presented. They will bring some challenges to some of the tools
traditionally used by many technical traders. It is necessary to reiterate a
point, here: the objective of these papers is not to disprove anything. In fact, it
is not the nature of scientific inquiry to think in those terms. Rather, we are
seeking evidence that these tools, which are purported to be very powerful,
offer a statistical edge in the market.
These tools, in my studies, do not show an edge, but there could be many
reasons for this. Perhaps the tests are flawed, perhaps the data was flawed,
perhaps the methodology missed something important, or perhaps the tools do
not have an edge. Regardless, these papers will give you some perspective on
the problems of technical trading, and may suggest some new directions for
your own investigations and research.
I hope you find this material interesting, useful, and fun. I have enjoyed
writing it for you, and I wish you all possible success in your trading
endeavors!
Adam Grimes
October 2017
New York, New York
 Course Catalog
This is a list of modules and units from the online course, available at no
charge from MarketLife.com.

I. Chartreading 101
Introduction
Basic Principles
Basic Chart Setup
Charts: Going Deeper
Reading Price Charts

II. Chartreading, Going Deeper

Pivots and Swings
Trends
Support and Resistance
Trading Ranges
The Problem of Randomness
Random Walks

III. Market Structure & Price Action

Trend and Range
Trend to Range and Back Again
Tools for Trends
Ends of Trends
The Two Forces
Market Cycles

IV. The Pullback

The Pullback
Expected Value
 Where’d Your Charts Go?
Quantitative Techniques I
Record Keeping
Manual backtesting

V. The Anti
The Anti
Quantitative Techniques II
What Works
Cognitive Biases
Trading System Design
Support & Resistance Project

VI. The Failure Test

The Failure Test
Basic Trading Stats
Papertrading
History of Traditional Technical Analysis
Classical Chart Patterns
Research Journal

VII. The Breakout

The Breakout
Trade Management
Your Trading Plan
Multiple Timeframe Analysis I & II
Basic Market Knowledge

VIII. Pattern Failures

Pattern Failures I, II, & III
 Position Sizing I & II
Risk

IX. Practical Trading Psychology

Two Mistakes
Learning High Level Skills
Meditation and Mindfulness
Routine and Process
Discipline
Understanding Yourself
 Reading Plan for The Art and Science of Technical
Analysis
Page numbers refer to The Art and Science of Technical Analysis by Adam
Grimes, Wiley (2012)

I. Charts, chart setup, becoming a trader

1-8 (having an edge)
9-12 (basic chart setup)
22-30 (reading inside bars, charting by hand)
375-385 (becoming a trader)
399-408 (trading primer)
375-384 (becoming a trader)

II. Pivots and swings, support and resistance, basic patterns of 
trend and range
13-18 (two forces intro, pivots)
19-21 (basic swing patterns)
49-64 (trends)
97-120 (ranges)
78-84 & 93-96 (trend analysis)

III. More detail on trends and ranges, interfaces, the market

cycle
85-92 (trendlines)
121-148 (between trends and ranges)
189-212 (indicators and tools for confirmation)
31-48 (market cycles and the four trades)

IV. Pullbacks and journaling

65-77 (pullback intro)
154-169 (pullback detail)
 291-315 (pullback examples)
385 - 388 (journaling)

V. The Anti and cognitive biases

170-173 (the anti)
327-336 (anti examples)
353-359 (cognitive biases)
409-424 (deeper look at MACD and MA)

VI. Failure test and basic trading stats

149-153 (failure test)
314-326 (failure test examples)
254-262 (basic trading stats)
389-398 (trading stats)

VII. Breakouts, multiple timeframes, trade management

174-188 (breakouts)
337-345 (breakout examples)
213-230 (multiple timeframe analysis)
231-253 (trade management)

VIII. Risk
263-290 (risk)

IX. Trader psychology

346-374 (trader’s mind)
 Acknowledgments
No creative work springs forth fully formed from a vacuum. It is the
interactions with other thinkers that drive creativity, and this book, perhaps
more than most, owes a lot to its readers. Over the years, your questions— 
whether they be simple, profound, or unanswerable, and your interactions
with me—whether you were supportive, challenging, or downright angry— 
have driven me forward. People often wonder at the amount of free content I
create, but I must honestly say I have probably benefited from this as much or
more than anyone else.
Some other people contributed to this work in a very focused and direct
way. Hannah Guerrero, our discussions, years ago, about teaching a complete
beginner to trade provided the seed from which all of this course material
grew. To Maria Tadros, thank you for unlimited proofreads on a very tight
time schedule—in the end, you were a veritable ATM machine of ideas and
valuable perspectives. Tom Hansbury, Peter Lawless, and Stewart Button,
thank you for your critical thoughts and careful edits. Jose Palau helped me
bridge the gap from experience to theory and back again; thank you!
To my wife, Betsy, thank you for being supportive, looking at multiple
drafts of cover graphics that differed by a few millimeters, humoring me
while we agonized over the virtues of color #0947D5 vs. #0945D4 (I
exaggerate only slightly), and generally letting me disappear for days while
writing this book!
To each of my readers, I owe a debt and many words of gratitude: thank
you. You have helped create this work you now hold in your hands; I could
not have done it without you.
 Chapter 1

Module 1–Chartreading 101

T
his module focuses on some important foundational concepts that are
often overlooked. We begin with an investigation of what it means to
have trading edge and why it matters. Our goal in all of this work is
to focus on practical application, but to also supply enough theory to support
the work and to make sure that the trader understands the “whys” as much as
the “hows”.
This module also includes a solid look into price charts. Too often, traders
begin their work without truly understanding what the chart represents. Chart
display choices are made based on vague visual appeal, similarity to
something seen elsewhere, or a recommendation from a friend (who may or
may not know what he is doing!) Thinking deeply about the chart also leads
us to our next area of focus: chart stories.
I came up with the term “chart story” when I was working with beginning
traders. When we think in these terms, we imagine that every aspect of every
bar is important, and we try to understand the part every tiny detail plays in
the developing story of the market. (We must acknowledge right away that
this line of thinking is misleading because it does not respect the random
variation in the market. Its value is only as a training tool to help build solid
habits in chartreading.) This is one way to look at and to think about price
charts, and it lays a solid foundation for developing market feel down the
road.
The supplementary readings for this section also cover some thoughts on
the process of learning to trade and why it can be so difficult. Simply put, we
do not learn to trade at all—rather, we become traders. And that journey,
richly rewarding as it may be, is long, challenging, and fraught with danger.
The trader who understands this from the first steps is much better equipped
to succeed.

Section 1: Chart Setup

You should begin to set up your charts, or, if you are an experienced trader,
to rethink your existing chart setup. While there’s no right or wrong, it’s
probably a good idea to move toward simplicity and a focus on the chart itself
(rather than on indicators.)
A common question is what my chart settings are. There’s no need to
duplicate exactly, but the charts in this section use:
• Modified Keltner Channels set 2.25 average true ranges around a 20
period exponential moving average.
• A modified MACD that uses simple instead of exponential moving
averages, and the settings of 3-10-16 (with no histogram) for inputs.
For the developing trader, it’s probably a good idea to use the same chart
setup for all markets and timeframes. Of course, there may be reasons you
want to use different indicators or setups on, say, monthly vs. 5 minute charts,
but the purpose of these exercises is to train your eye to see the data in a
consistent way.
Experiment and play with the options your charting package presents.
Sometimes different color settings can be more pleasing to your eye, and it’s
also worth taking some time to look at the choices between candles and bars,
spending more time on whichever you are less comfortable with.

Section 2: Chart Stories

This is an exercise that is designed to do a few specific things:

To force you to slow down and to look at the details of the

charts
To get you to start thinking about the forces that might be
behind the price formations
To start thinking about emotional context in extreme situations
To show you a handful of important historic moments in
financial markets. (Not every chart in this sector is designed to
that end; some are simply illustrations of interesting patterns.)
To begin to awaken some sense of intuition and inductive
learning.

For the purpose of this exercise, assume that every bar has a story; your job
is to tell that story. Rather than worrying about being right or wrong, focus on
the thought process and inductive nature of this analysis. There really are no
wrong answers here, and you may even find value in doing these exercises
more than once. Finding interesting examples on your own would be another
way to extend the analysis.
If the chart has text, answer the question or do the specific analysis on the
chart. If there is no text, then write a separate explanation for each labeled bar
 —in all cases, make sure that each bar designated with a label receives your
attention and a text explanation.
Adequate explanations will usually be 2-6 sentences long and will focus on
concepts such as:

The position of the open and close within the day’s range
The position of the open and close relative to each other
The range of the bar relative to previous bars
Consider each bar both alone and in relation to previous bars
Any “surprises” (This is a deliberately large category.)
Action around any obvious support and resistance levels. (This
is not an exercise in support and resistance, so do not focus on
this aspect.)
It may be useful to think in terms of large groups of buyers and
sellers driving the market, and the battle between those groups.
 Section 3: Trendlines
The charts in this section are slightly more compressed and are presented
without commentary. Draw trendlines, following the rules from the module
(and pp. 84–86 of The Art and Science of Technical Analysis). A trendline
should:

Capture the swing low before the high of the trend in an uptrend
(and the reverse is true for a downtrend.)
Be attached as far back into the trend as possible, but capturing
the beginning of the trend may not be possible.
Not cut any prices between the two attachment points. A
trendline may cut prices after the attachment point (i.e., the
trendline was broken.)

Note the transitions into ranges and then the breaks into trending action,
and do whatever analysis you feel appropriate. Many charts will include
several trends, and you may find different definitions depending on the
timeframes you consider. As long as the trendlines are drawn according to the
rule set, they are valid.
These charts were chosen to be a mix of relatively straightforward and
more complex patterns.
 Section 4: Trendline Research Project
A word of warning: done well, this is a big project. It will take some time,
but that time will be well rewarded!
The point of the study is to understand what happens as trendlines are
drawn in evolving market data. Ideally, you would use a software program
that would allow you go one bar at a time (to replicate, as much as possible,
the experience of having the chart form in real time.) As the market develops
on your screen, decide when and where to draw trendlines. (Review Module
3, Unit 4 from the course for guidelines on drawing trendlines.)
Once the trendlines are drawn, notice what happens when the market
touches them. Does the touch of the trendline hold? Does it indicate the end
of the trend? What happens if you combine it with bands and/or swing
analysis?
Start to think about how you might trade these structures.
You need to keep some type of records. Screencaps of your charts would be
one possibility, but it would also be a good idea to somehow score the
interaction with the trendlines.
This project is deliberately broad, but should encourage you to spend at
least several days investigating action around trendlines. Do not trust what a
book tells you—ask the market itself!
If you are not able to generate bar by bar charts, you may work in the
middle of a chart but try, as much as possible, to imagine the chart is being
revealed one bar at a time. You will not replicate the feeling of hidden
information, but you will draw consistent trendlines on correct pivots this
way. If you simply start drawing trendlines on charts, you will likely make
many mistakes on attachment points and pivots based on what trend
information was available at the time.
Remember: draw the trendline, and then see what happens “to the right” of
the correctly drawn trendline when the market engages the trendline.

Section 5: Charting by Hand

This is still a valuable exercise. In the first module, we discussed the
several ways you can do this exercise. The details are not important; it is far
more important that you simply do it.
Do not spend too much time on this exercise. A few minutes every day (but
every day!) directed to drawing your charts, or a few minutes each hour if you
are an intraday trader, will reward you with deep understanding of price
action. This is one tool to stay in the flow and to build your intuitive
understanding of market action.

Section 6: Readings
From The Art and Science of Technical Analysis: Market Structure, Price
ction, and Trading Strategies by Adam Grimes, Wiley, 2012:
85-92 (trendlines)
121-148 (between trends and ranges)
189-212 (indicators and tools for confirmation)
31-48 (market cycles and the four trades)
These readings will lead you deeper into the intricacies of the transitions
between trends and ranges, and will give you further examples of correctly
and incorrectly drawn trendlines.

On Trends

The Trend Is Your Friend, Except at the End…

Let’s talk about endings today. If we can better understand the patterns that
often occur when trends end, we are better equipped to manage risk, place
stops, and, in general, to trade those trends. Crude oil provides a good
example of a dangerous ending in a specific kind of trend.
Recall the “slide along the bands” trend. This is essentially a low-volatility
trend, and we can identify it when price goes to one band or channel—it
doesn’t really matter whether you use Bollinger bands, Keltner channels, or
some other similar tool—and stays there while the market “slides” in the
direction of the trend. This type of trend doesn’t have a lot of the normal
pullbacks and retracements that most trends do, and it can be incredibly
powerful. Look at this chart of crude oil futures:
 It’s easy to miss a trend like this because the market may look very dull and
boring, but this type of trend can go much further than anyone thinks possible.
Identifying this pattern early allowed us to catch the recent downtrend, even
though we thought, in early summer, that crude oil might be putting in a long-
term bottom. The grinding trend pattern can roll over many obstacles that set
up against the trend.
This trend also provides a good example of what happens when everyone is
on one side of the trade. In some ways, this pattern quantifies the classic one-
sided trade; trading activity and volume dry up as volatility (measured by
standard deviation of returns) craters. These trends are a powder keg, and they
tend to end in one way: with sharp, volatile counter-trend pops.
Now, call this a “short covering rally” or whatever you will, but the point is
we knew, weeks in advance, that this trend was not likely to end politely. We
knew that even a minor bounce against the trend was likely to lead to a multi-
day pop.
So, how do you use this information? Well, the art of stop placement (and,
for discretionary traders, there is maybe more art in this aspect of trading than
in any other) depends on understanding expectations. There are times we need
to be far from the market—too many traders probably try to use stops that are
too tight. However, there are also times that our stops should be close; if we
know that a minor move against the trend is likely to lead to a nasty move
against the trend, why not put our stops close enough that we are among the
first out?
This is the lesson: learn to identify these “slide along the bands” trends,
which are rare but powerful. Try to get on board, but don’t plan to do so via
retracements, because those retracements can be painful. If you are in a trend
like this, first, congrats, but, second, play defense. Use tight stops, perhaps
even tightening them after each bar. This is not a time to endure much pain
against the trend—this is a time to lock in profits.
One of the principles of trading is to identify those points when the crowd
might be right, to stand with the crowd while they are right, and to try to be
among the first out when the crowd turns. It’s a nice plan, but usually hard to
execute. If you can identify this (rather rare) type of trend and use the correct
stops with perfect discipline, you’ve found trader psychology encoded in the
patterns of the market, and this often points the way to winning trades and
substantial profits.

Slip and Slide (Along the Bands)

Effective trading patterns are usually simple. The more complicated an idea
is, the more likely it is the result of trying to fit random data (i.e., the market)
into a preconceived mold. I think many of the patterns and ratios of traditional
technical analysis are the result of very misguided Procrustean “analytical
techniques”—this is one reason they look so good in the past, but don’t have
that much utility at the hard right edge of the chart.
First of all, let’s take a quick look at a daily chart of crude oil. Notice how
crude oil is pressing into the bottom Keltner channel without any real
retracements:

This pattern can be dangerous, and understanding it can save you a lot of
grief. I probably need a better name for it, but I’ve taken to just calling it
“slide along the bands”, because that’s what the market does. In bullet points,
here are the main concepts:

Markets normally move in alternating periods of with-trend

movement (up or down), interspersed with pullback against the
trend. The normal way markets move is in a zig-zag, which is
why trading pullbacks can be such an effective trading strategy.
Sometimes this pattern gets short-circuited, and the market does
not pull back. There are many ways this can happen, but the
basic idea is that the dominant group pushes the market in the
trend direction at the same time volatility dries up.
This creates an interesting situation in which the market
“drips”, “bleeds”, or “slides” higher or lower (depending on the
trend direction).
If we use properly calibrated bands, we can see this pattern
when the market just presses into one band and sits there while
it continues to trend.

This pattern is the proverbial double-edged sword. On one hand, the market
can go much further than we might expect. When you are fortunate to be
positioned on the right side of such a move, the best thing to do is to focus on
trading discipline: maintain a correct stop and tighten that stop every 2-3 bars
as the market makes new extremes. Don’t look at P&L, and don’t over think.
Let the market tell you when to get out by hitting your stop.
However, this pattern does bring some unusual risks. When it ends, it often
ends in a volatile spike against the trend. We absolutely must respect our
stops, and we cannot be upset if we are stopped out in noise. When this
pattern ends, the market is probably going to become very emotional. The
market can be emotional; you, as a trader, cannot.
So many times, in trading, our entire job description boils down to one
simple directive: don’t do anything stupid. Don’t make mistakes.
Understanding this simple pattern can help you avoid many mistakes and
navigate this difficult trading environment.

On Developing a Style

Simplify, Simplify, Simplify…

 You may know that I studied cooking formally—it was something I’d
always wanted to do, and I arrived at a spot in life where I could pursue a
culinary degree and apprenticed in the kitchen of one of the top French chefs
in America (who was a disciple of Paul Bocuse, but that is a story for another
day.) I remember my early days as a trainee chef; I cooked dinner parties for
friends often and I had a box that I carried with me that had over forty
different herbs and spices. I also had a set of maybe 15 knives that I took with
me—a knife for every possible purpose. Today, fifteen years or so later, how
has my cooking changed?
First of all, it’s better. I would put many of the things I make up against
similar dishes made by any restaurant kitchen anywhere, and keep in mind I
live in a city with great restaurants. More to the point, I have simplified,
simplified, simplified everything to bare essentials. Rather than forty-five
different herbs and spices, now it’s usually salt and pepper, parsley, garlic and
fresh thyme. I almost always have one of two knives in my hand, and I might
pick up the paring knife for small jobs. My long study of Japanese cuisine
means that when I cook something, I try to express the essence of the thing.
For instance, for steak, hot cast iron, salt, pepper and a little butter when it’s
finished will do the trick. Less can be so much more.
It has been the same with my trading. I’ve been around the block a few
times. I have used multiple indicators with hundreds of lines of code each.
I’ve had complicated systems with rules that fill two pages for bar counts,
indicator crossings, and action in related markets. I’ve traded every possible
timeframe from scalping to building baskets and portfolios for
monthly/quarterly timeframes and beyond. I’ve built multifactor econometric
models to forecast rates and asset class returns, looked at cross correlations
between higher moments of return distributions, and screened for trades
across a universe of thousands of assets. Much of this journey was a
legitimate part of my learning, but I have realized something over the years— 
I have truly become a minimalist.
Gone are all the complicated indicators and rules, and, in their place, is a
simple system with rules I can explain in five minutes, and two indicators that
are non-essential and only used in a supporting role. I do not look at a huge
number of supporting (complicating) factors; I try to limit my inputs to only
those that I know are significant. The end result is that I can make a trade
decision in a matter of minutes or less, and trade and risk management rules
are clearly defined. What’s more, I find I can trade this system with no
emotional involvement, so total, objective control is within grasp. The more I
have simplified and removed the unessential, the better my results have
become.
You probably don’t need many or most of the things you use in your
trading. Your results might even be a lot better if you just focused on the
things you really do need.
“Hack away at the unessential”, as Bruce Lee said. What is unessential in
your trading?
What can you simply?

Traders get an edge by thinking in categories

Richard Wyckoff was one of the founding fathers of technical analysis, and
one of his useful concepts was to divide the market into an idealized cycle of
accumulation, mark-up, distribution, and markdown. Though it can be
difficult to apply this model in actual trading, it has many important lessons
and can shape the way we think about market action. From a practical
perspective, it lays the foundation for a simple categorization of technical
trades into four trading categories. There are two trend trades: trend
continuation and trend termination, and two support and resistance trades:
holding and failing.
Though this may seem like an arbitrary classification system, it is not.
Every technical trade imaginable falls into one of these categories. Trades
from certain categories are more appropriate at certain points in the market
structure, so it is worthwhile to carefully consider your trades in this context.
The first question to consider is: are all of your trade setups in one
category? If   so, this may not be a bad thing—a successful trading
methodology must fit the trader’s personality—but many traders will find the
best results when they have at least two counterbalancing setups. There is an
old saying: “If the only tool you have is a hammer, every problem you
encounter will look like a nail.” There is certainly room for the specialist who
does one trade and does it very well, but there’s also value in a broader
approach.
Some market environments favor certain kinds of plays over others. If you
trade within each of these trading categories, then you need to ask yourself:
Are you applying the right kind of plays to the right market environments? If
you are a specialist who focuses on only one setup or pattern (this is not a
criticism if you are successful this way), then you need to realize that only a
few specific market environments favor your play and your job is to wait for
those environments. You can redefine your job description to include not 
trading most of the time! Wait on the sidelines, and wait for the environments
in which you can excel. Clarify your setups. Categorize them into trading
categories, and then simplify, simplify, simplify.
Let’s look briefly at each of the four broad trading categories:

Trend Continuation
Trend continuation plays are not simply trend plays or with-trend plays.
The name implies that we find a market with a trend, whether a nascent trend
or an already well-established trend, and then we seek to put on plays in the
direction of that trend. Perhaps the most common trend continuation play is to
use the pullbacks in a trend to position for further trend legs. It is also
possible to structure breakout trades that would be with-trend plays, and there
is at least one other category of trend continuation plays—those trades that try
to get involved in the very early structure of a new trend, before the trend has
emerged with certainty.
There is a problem, though: It is important to have both the risk and the
expectation of the trade defined before entry; this is an absolute requirement
of any specific trade setup, but it can be difficult with trend continuation
trades. The key to defining risk is to define the points at which the trend trade
is conclusively wrong, at which the trend is violated. Sometimes it is not
possible to define points at which the trend will be violated that are close
enough to the entry point to still offer attractive reward/risk characteristics.
On the upside, the best examples of these trades break into multileg trends
that continue much further than anyone expected, but the most reliable profits
are taken consistently at or just beyond the previous highs.

Trend Termination
More than any other category, precise terminology is important here. If we
were less careful, we might apply a label like “trend reversal” to most of the
trades in this category, but this is a mistake. That label fails to precisely define
the trader’s expectations. If you think you are trading trend reversal trades,
then you expect that a winning trade should roll over into a trend in the
opposite direction. This is a true trend reversal, and these spots offer
exceptional reward/risk profiles and near-perfect trade location. How many
traders would like to sell the high tick or buy the very low at a reversal?
However, true trend reversals are exceedingly rare, and it is much more
common to sell somewhere near the high and to then see the market simply
stop trending. Be clear on this: This is a win for a trend termination trade— 
the trend stopped. Anything else is a bonus; it is important to adjust your
expectations accordingly.
Trend termination trades are countertrend (counter to the existing trend)
trades, and trade management is an important issue. Most really dramatic
trading losses, the kind that blow traders out of the water (and that don’t
involve options) come from traders fading trends and adding to those
positions as the trend continues to move against them. If this is one of the
situations where the trend turns into a manic, parabolic blow-off, there is a
real possibility for a career-ending loss on a single trade. For swing traders,
there will sometimes be dramatic gaps against positions held countertrend
overnight, so this needs to be considered in the risk management and position
sizing scheme. Perhaps more than any other category of trade, iron discipline
is required to trade these with any degree of consistency.

Support or Resistance Holding

There is some overlap between these trading categories, and it is possible to
apply trades from these categories in more than one spot in the market
structure. We might expect that most support/resistance trades will take place
in accumulation or distribution areas while the market chops sideways, but a
trader trading with-trend trades could initiate those trades by buying support
in the trend. Are these trend continuation trades or support holding trades?
The answer is both, so traders must build a well-thought-out classification
system that reflects their approach to the market. Your trading patterns and
rules are the tools through which you structure price action and market
structure, and they must make sense to you. Take the time to define them
clearly.
Many trading books will show you examples of well-defined trading
ranges, where you could buy and risk a very small amount as the market
bounces off the magic price at the bottom of the range. These trades do exist,
but they are a small subset of support holding trades. Support, even when it
holds, usually does not hold cleanly. The dropouts below support actually
contribute to the strength of that support, as buyers are shaken out of their
positions and are forced to reposition when it becomes obvious that the drop
was a fake-out.
For the shorter-term trader trading these patterns, there are some important
issues to consider. If you know that support levels are not clean, how will you
trade around them? Will you sell your position when the level drops, book
many small losses, and reestablish when it holds again? Will you simply
position small in the range, plan to buy more if it drops, and accept that you
will occasionally take very large losses on your maximum size when the
market does drop? Every decision is a tradeoff, and you must understand the
consequences of these decisions.

Support or Resistance Breaking or Failing

Support/resistance breaking trades are the classic breakout or breakout from
channel trades and, ideally, would be located at the end of accumulation or
distribution phases. In fact, these trades define the end of accumulation or
distribution, as the support or resistance fails and the market breaks into a
trend phase. Another place for support/resistance breaking trades is in trends,
but many of these are lower time frame breakout entries into the trading time
frame trending pattern. Many traders, especially daytraders, find themselves
drawn to these patterns because of the many examples where they work
dramatically well. Many trading books show example after example of
dramatic breakouts, but there is one small problem with breakout trades— 
most breakouts fail.
In addition, the actual breakout areas tend to be high-volatility and low-
liquidity areas, which can increase the risk in these trades. They occur at very
visible chart points, and so they are often very crowded trades. The presence
of unusual volume and volatility can create opportunities, but it also creates
dangers. Execution skills probably matter more here than in any other
category of trade, as slippage and thin markets can significantly erode a
trader’s edge. These trades can offer outstanding reward/risk profiles, but,
especially in short-term trades, it is important to remember that realized losses
can sometimes be many multiples of the intended risk, significantly
complicating the position sizing problem. This is not a fatal flaw, but it must
be considered in your risk management scheme.
Depending on the time frame and intended holding period for the trade, it
may be possible to find that there are patterns that precede and set up the best
examples of these trades. The best resistance breaking trades will be driven by
large-scale buying imbalances, and these imbalances usually show, for
instance, as the market holds higher lows into the resistance level before the
actual breakout. Breakouts driven by small traders who are simply trying to
scalp small profits in the increased volatility are less reliable and are usually
not set up by these larger-scale patterns. In the very best examples of these
trades, buyers who are trapped out of the market by the suddenness of the
breakout will be compelled to buy into the market over coming days or
weeks, and this buying pressure will provide favorable tailwinds for the trade.
Traders specializing in breakout trades usually spend a lot of time studying
the patterns that set up the best trades, and maintain a watch list of potential
candidates for trades at any time. Executing unplanned breakout trades in a
reactive mode is unlikely to be a formula for long-term success.

On Market Rhythm

Toward a Simple Model of Price Behavior

I am a quantitative discretionary trader: I am a discretionary trader, but
everything I do is subject to quantitative and statistical verification. Over the
years, I have accumulated a lot of statistical evidence for what works and
what does not work in the market, and the results may be surprising. In
conversations with other traders, I’m sometimes accused of being overly
negative, as it’s hard to find real quantitative support for many of the
traditional tools of technical analysis (such as Fibonacci ratios, most
applications of moving averages, etc.) Many traders find this message
challenging, but most people only seek confirmation of their beliefs—a
common and dangerous cognitive bias.
I want to share a model of market behavior that I’ve found very useful.
Though this is a theoretical model, it works. It is supported by rigorous
statistical research, and, even more importantly, it has proven itself in actual
trading for many years. The point of this work is not to disparage anything
anyone does; the point is to save your time and money.
The concept is simple. Imagine, for a moment, that there are two forces in
the market:  Mean reversion, the tendency for large moves to be reversed in
part or completely, and  Momentum, the tendency for large moves to lead to
further moves in the same direction. When the forces are in balance, and they
usually are, markets will move more or less randomly. Price will move up and
down in ways that look a lot like a random walk. Future prices and price
direction will be unpredictable, and there is no technical reason for having a
position. We refer to this as a market in equilibrium, and these are the types of
markets we try to actively avoid.
However, there are other points where one force predominates. In trends,
momentum-fueled, with-trend thrusts lead to further moves in the same
direction, though eventually mean reversion overtakes them and the market
rolls over into a pullback. (These concepts are timeframe dependent.) At other
times, mean reversion will predominate and large moves can be faded. The
question of technical trading now becomes this: “is it possible to identify
patterns that show, in advance, when one force is likely to be stronger than the
other?” If so, then we have a reason for taking a position, putting risk in the
market, and the possibility of harvesting trading profits from markets that are
otherwise random and unpredictable. Fortunately, the answer to that question
is yes.
That’s all there is to the model; it is so simple that it’s easy to overlook the
importance and usefulness of such a simple framework. These forces shape
everything from traditional chart patterns to long-term trends to ultra-short-
term HFT behavior. There are many nuances here that may not be appreciated
in traditional technical analysis—for instance, the balance of momentum and
mean reversion are different in different asset classes and different
timeframes. This is why technical tools cannot be simply applied “just as
well” to “any market and any timeframe”, but, rather, why some adaptation
and experience is necessary to translate concepts and tools across different
applications. We’ll dig into all of this in more depth soon, but, for now, begin
to think toward a simpler model of price behavior that is shaped by these two
primordial market forces.

Hey, that’s different!

 It pays to think deeply about markets, how they move, risk, opportunity,
and how it all plays out in the grand scheme of probability. We can and should
spend a lot of time crafting our trading plans, understanding our risk
tolerance, and monitoring our adherence to those plans. But, in the heat of the
moment, trading does not have to be complicated. Though there may be
tremendous quantitative work supporting a method, often the simplest tools
work best. One of my favorite patterns is simply knowing when something
has changed in the market.

I took the screenshot above in the middle of the day (10/6/14) when I
noticed that many of the grains were putting in large standard deviation up
days, which is another way to say that they were making large moves relative
to their own volatility. Here is also a case where the right tools can be helpful;
would you have seen that this was a significant day just by at the chart?
Maybe, but the panel below the chart quickly shows the significance of this
move. On a volatility-adjusted basis, this was the largest upward move in
nearly a year.
Now, this is only the first stage of analysis, but it is an important one. Over
the years, when I have worked with, coached, and trained traders, I used to
okingly call this “hey, that’s different!” In reality, it is not a joke. Noticing
that the dominant market pattern is shifting can be an important piece of
information.
The point of this is the concept, rather than the specific example here. Find
an obvious break in the existing market pattern, and then pay attention to
what happens afterward. So, what can be different? Here are some examples:
 Largest volatility-adjusted move over a certain time period.
Obvious move that breaks a chart pattern.
Counter-to-expected breakout, but, again, it must be obvious.
Sudden, sharp reversal like a single day that reverses the
previous week’s movement.
Quiet market goes into an extended period of volatility, or vice
versa.

This is just a starting point; you can make a much longer list of things that
indicate market dynamics might be shifting. One key point: though this is a
simple concept and is simple to use, it must be based on things that really
work. If it is based on technical ideas that have no foundation in market
reality then you are only analyzing insignificant noise. Understand how
markets really move, how they usually move, and then—look for something
that breaks the pattern. Look for something that jumps out and say, “hey,
that’s different!”

Staying in Step: Finding Rhythm of the Market

Rhythm is a fundamental aspect of the human experience: our bodies pulse
with the rhythm of blood and breath. We experience the rhythms of day and
night, and the longer cycles of the seasons. Rhythm is fundamental to music,
whether it’s the relentless pounding bass of a rock song, the syncopated stabs
of a Jazz guitar, or the nearly baffling asymmetry of Messiaen. Visual rhythm
ties together much of architecture, design and visual art. There are natural
rhythms in our mood and energy level—rhythm pervades everything we
touch or experience, in some way. And, yes, the rhythm of the market is ever
present and undeniable.
Philosophically, the understanding of rhythm is a critical division between
Eastern and Western thought, with most Western understandings favoring a
linear perception of time and experience—history, of individuals, nations, and
the world, is seen as a journey along a one-way timeline. In the East, a more
cyclical perspective prevails, an understanding that events lead to other events
in an ongoing cycle of creation and destruction, and that the rhythmic
interplay of forces creates much of the human experience. The Western
approach might talk about the march of history, while the Eastern perspective
might say, “what is has been before and will be again.”
 In financial markets, the cyclical approach is usually the right one. This has
all happened before, and what has been shall be again, in some form.

The market rhythm

Financial markets move in cycles and there is a rhythm in that movement.
Some of this is well-documented: we can measure cycles in prices and returns
with tools like Fast Fourier Transforms or Kalman filters, we can measure
cycles in volatility with other tools. We also can measure the rhythm between
trending and ranging activity with different tools. There are also many more
arcane cycles based on time and angles, and there are cycles in instruments
that are bounded and in the relative performance of different markets.
Trading cycles is not as easy as we might expect; cycles shift and abort
without warning. To stay in step requires frequent adjustment, and the trader
often finds himself out of rhythm with the perceived cycle. Traders who
discover cycles often think that they are the answer to many trading problems,
but actual application is elusive. An understanding of cycles can be useful,
but trading them in a pure form can be very difficult. For most traders,
understanding the cycles of the market can be a useful “first filter” for
knowing when and how to apply specific trading techniques to the market.

The trader’s rhythm

Markets have cycles, but so do traders, and there are some lessons here that
we can begin to apply right away. First, understand that there is some natural
flexibility in all cycles. Do not expect perfect regularity. This applies as much
to your trading results and performance as to market action itself—expect that
periods of good performance will be followed by lackluster performance. Just
know that this is part of the “game”: you’re never as good as it seems during
the good periods and never as bad as you feel when things are hard. Your
overall performance (and the right way to think about your performance) lies
somewhere in the middle.
Also, realize that your activity in the market will be governed by the
market’s action and the variation in that action. There will be times when
active trading is required, and times when the right thing to do is to do
nothing. Sometimes, you may go through long periods of time in which your
only job is moving stops and managing open positions, and you may see these
open positions stopped out one by one. This is all ok, and all normal.
Staying in step
In my experience, a lot of the bad things that happen to traders happen
when we try to apply the wrong tools for current market conditions—the tools
may well be great tools, but a great tool at the wrong time is the wrong tool!
(Examples: applying a mean reversion system to fade a strong trend, a
daytrader forcing trades when markets are dead, etc.) So, the first line of
defense is intellectual: know that markets are cyclical and know that your
performance will also have some cycles and rhythm.
The next piece of the puzzle comes from good record keeping and analysis
of your trading results. Your intuition about your performance may well be
valid, but it’s a lot better if it’s supported by some data. Simply keeping a
running total of, say, your last 20 trades’ win ratio (assign a 1 if the trade is a
win and a 0 if it’s a loss (decide what to do with breakeven trades too), and
keep a moving average of the last X trades) can give some good insights into
performance. Of course, monitoring P&L will get you into roughly the same
place, but win ratio is often a good early warning sign.
Last, have good rules that help you decide what to do. The right answer, of
course, is the standard and not-immediately-helpful “it depends”, but it does
depend: it depends on what markets, timeframes, and style of trading you do.
At one extreme, a long-term trend follower may well ignore this information,
knowing that she is simply going to be out of step with the market for most of
the time, and also know that it doesn’t matter because she will be profitable
over a long enough period. On the other extreme, a daytrader might pull the
plug on a day if he has 5 or 6 losses in a row, because he knows that is
unusual for his style of trading and probably reflects a market where
conditions are not favoring his play. It’s hard to know what these rules should
be until you’ve traded a while, but developing these “meta rules” that govern
your behavior and trading activity is an important task for the developing and
professional trader alike.

Too Much of a Good Thing

When markets trend, they do so in alternating waves of with-trend strength
interspersed with countertrend pullbacks. Analysis of these swings can give
many insights into the strength and integrity of a trend, and this can be done
both quantitatively and qualitatively. Generally speaking, the with-trend legs
need to be stronger than the countertrend pullbacks, and the stronger the with
trend legs are, the better.
However, excessive  strength can indicate a climax extreme that can mark
the end of a trend, or at least can cap that trend for quite some time. This
problem—distinguishing “good” with-trend strength from “overheated”
overextensions is one of the core problems of technical analysis. Many
solutions have been proposed; all of them work at times and fail at others (one
of the recurring problems is that these tools would often take a trader out of a
trend too early), but this is another case where simple can be better. Though
nothing works all the time, a simple pattern can give important insight into
the future direction of prices.
Let’s take a look at an example in sugar futures. Many of the Softs (sugar,
cotton, orange juice, coffee, cocoa, etc.) can be thin and can have some
surprises; inexperienced traders should be cautious, but sugar tends to be a bit
more tractable than some of the other Softs. The chart below shows sugar was
in a two month uptrend. Don’t get too caught up in how you define the trend
 —the point is simple: Sugar was going up.

This was a reasonably strong trend, possibly threatening to reverse a multi-

year downtrend. The next day, Sugar had a sharp spike higher.
 There are several signs of a possible buying climax here. First, the range of
this bar is many times the average range of recent bars. (It’s not necessary to
quantify this, but, in general, a bar that has a range that is three or more times
the average range of recent bars should catch your attention.) It also comes to
a new trend extreme, and it is obvious that the pattern of the trend has been
broken. It’s often enough to notice that something has changed, that
something is different, and don’t assume that violations of the trend only
matter if they are against the trend. (In other words, many traders would tend
to watch for downward spikes as a warning that an uptrend is weakening.
This is not wrong, but upward spikes can also be significant.) Long shadows
on candles are often indicative of exhaustion. In this context, it might be ok to
think that the large upward shadow (“wick”) on the last candle shows where
many traders made mistakes.
So, what do we do now? If you’re long, you need to play defense. It is
appropriate to tighten stops dramatically on long positions, perhaps even
working them under the last 1-3 days’ lows. If you’re looking to short, it
might make sense to aggressively pursue short entries following a pattern like
this. At the very least, do not enter long positions immediately following a
spike like this. Be careful of buying pullbacks or breakouts, and be very
reluctant to buy weakness. A buying climax often indicates that the proverbial
“last willing buyer” has bought, and the market will often collapse into the
vacuum on the other side.
 In this case, we can see that the buying climax marked the trend high, at
least for several weeks. Generally speaking, if we get a strong enough selloff
following a buying climax, the next bounce is often a good, high-probability
short setup. It’s also possible that the market will pause for a while, absorb the
overextension, and then head higher. It’s possible that a buying climax could
lead to further spikes in the same direction; sugar could have traded to 23.00
on the next day following the “climax” (though the long shadow made that
considerably less likely). Anything can happen. Nothing works all the time,
and the best we can do is to quantify the probabilities and understand how the
balance of buyers and sellers might be shifting in a market at any time, but
this pattern, being aware of climax points, can offer great insight for traders in
all markets and all timeframes.
 Chapter 4

Module 4–The Pullback

This module looks at the pullback: a simple, but powerful, trading pattern
that uses the normal fluctuations in trend strength to set up and to manage
trades. Though the concept of trading pullbacks is probably familiar to most
traders, many traders overlook the power and utility of this pattern.
The second major area of focus is on applying quantitative techniques to
market data. On one hand, we are asking a simple question: “does it work?”
We quickly find that getting the answer to that not-so-simple question is
fraught with complications and difficulties. After a look at calculating
expected value, we take a long detour into investigating the patterns in the
relationship and geometry of swings, specifically looking at applications of
Fibonacci ratios to market data.
This module concludes with a section on journaling and doing manual (bar
by bar) backtesting of trading ideas. You will do a research project centering
around the pullback concept. The goal of this project is both to understand the
edge in the pattern and to familiarize yourself with the process of backtesting
a trading idea. This will lay a solid foundation for both discretionary trading
and deeper quantitative work.

Section 1: Record Keeping

Your work for this module might appear to be less than for previous
modules—certainly, the page count is lower. But that is an illusion! In this
module, the emphasis and focus shifts to you doing the work, and this begins
with record keeping.

Journal
This is the perfect time—today, right now, immediately—to start keeping a
ournal.
Though you can consider the issues of format and exactly what you want to
put into the journal later, this practice will be most effective if it is a routine
done consistently. Essentially, you want to make journaling a habit—a very
good, constructive habit that will ultimately play a big part in your success.
 For the beginning trader, it’s sometimes confusing to know what to put your
in journal. If you aren’t sure what to write, write that. Write about your
feelings about journaling, your feelings about building habits. Write about
things in your life and world you want to change. Write about your
experiences trading. Write about your future trading, and what you think of
the work you’re doing in this course. Write about kittens. It doesn’t matter!
Just write a little bit, each day, and let this work evolve as you go along.

P&L Sheet
You also do need a P&L sheet that allows you to record at least the
following datapoints for each trade:

Date In
Price In
Price Out
Initial Stop

You will use this in your research project this week.

Section 2: Pullback Backtest

First of all, it needs to be said that this will not be a proper, rigorous
statistical backtest. Rather, it is a process designed to do a few things:

Train your eye to see the pullback pattern in the market

Get some idea what edge might (or might not) be in the specific
way you see the pattern
Point you toward some improvements in your perception
And to get you used to doing some work on historical data

Before you can really do this work, you need access to historical charts and
some record keeping system; pencil and paper will work, but electronic
formats are much better. You then need to define the pullback pattern. This
can be difficult, because there is admittedly (and deliberately) some element
of discretion. Do not be discouraged. The way in which you see these patterns
will evolve and change, but it is the exposure to market data that will let you
evolve. This is truly a case where the only way you learn is by doing.
So, define the pullback pattern. What, specifically, will get you interested in
looking for a pullback? How will you define a strong enough or sharp enough
move to tell you that a pullback might set up? How will you monitor the
shape of the pullback as it develops? Where will you actually get into trades?
Where will you place that initial stop?
Take some time to answer those questions, and come up with a rule sheet
for pullbacks. (Write it down.) Then, go through some market data bar by bar,
recording key stats for each “trade”, and see how it works and how it feels;
your subjective sense or feeling is valuable. At the first stages, this exercise is
as much about you as it is about the market.
Like anything else, this process becomes easier the more you do it. To have
a valid test, you need a significant number of trades, but just get started with
the exercise this week.

Section 3: Charting by Hand

Continue your work on charting by hand throughout this Module. We have
already considered several ways you can do this exercise; choose one that
works for you. The details are not important; it is far more important that you
do it.
This should also not be a tremendous time suck—this is not a unique form
of torture I have devised. A few minutes every day (but every day!) directed
to drawing your charts, or a few minutes each hour if you are an intraday
trader, will reward you with deep understanding of price action.

Section 4: Readings
From The Art and Science of Technical Analysis: Market Structure, Price
ction, and Trading Strategies by Adam Grimes, Wiley, 2012:
65-77 (pullback intro)
154-169 (pullback detail)
291-315 (pullback examples)
385 - 388 (journalling)
The readings this week will help us to move from the purely theoretical,
high-level perspectives on markets to looking at applied trading patterns.
Seeing many examples of the pullback, and considering how to manage trades
that set up based on this pattern, will give the trader good ideas for continuing
to explore this aspect of market behavior.

On Journaling

Your Best Trading Book Is Your Own

There are a few practices that consistently separate winning traders from
the losers. True, there is an incredible diversity of winning strategies,
approaches, and personalities. However, we when look at and talk to
consistently profitable traders, we start to see a few common threads. There
are some things that winners do that losing traders often ignore. Keeping a
trading journal, of some kind, is one of those practices.
A reader from by blog asked the following question, which I have
paraphrased:
What should go into a daily trading journal?
should it just narrate the day’s activities like “price opened slightly
higher than yesterday’s close and then it closed lower forming a
down trend bar”?
should it include my [the trader’s] thought process which went 
through during the day while seeing the price and/or also after
seeing the closing price?
how to see the last bar in the context of previous bars? Should I look 
at range of the bars or a doji bar in context of earlier bars?
 Does it really help to evaluate each bar on daily timeframe (except 
the sigma spike types bar) if you are swing trading? Or I am just 
analyzing the noise but if bars seen together will make sense?
Of course as you have mentioned in many of your blogs/videos/book 
the market structure/context plays an important part. But while
noting down in my trading journal, I am not sure it is really helping
me to improve my decision process.
So, the answer, I think, is that you need to keep some kind of trading
ournal, but I don’t think you need to keep a specific kind of journal. In other
words, you probably need to keep a journal, but there’s no one, right way. Let
me try to simplify this a little bit, and hit some bullet points:
 The most important thing about your journal is that you do
it! All the planning and best intentions in the world are worth
nothing if you don’t follow through. Consider this carefully
when you plan your journaling—it has to be realistic. If your
 journal involves filling out three forms and writing a page of
text twice a day, you’re probably not going to do that. On the
other hand, scribbling three lines in your note book at the end of
the week (“It was a good week. Made a ton of money. Bought
pizza.”) probably doesn’t really provide a lot of value. If there’s
a key to successful journaling, it’s finding that balance between
ease of use and complexity.
Just to expand on that point, it’s the routine that matters.
Keeping your journal, whether you do it daily, weekly, or
monthly (depending on your timeframe) is a matter of
discipline. In trading, being disciplined means that you are
always  disciplined. This means that you must follow through
and do your journal! Get into the routine, and do it without fail.
Another note from my own experience: the less you want to do
your journal on any period, the more you need to. Never
underestimate your mind’s ability to avoid tasks that really need
to be done.
I often suggest that traders keep two journals: a market
research journal and a personal, behavioral, journal. These can
be combined, but I’ve found it useful to understand that I’m
shining the spotlight of my focused attention alternately on the
market and on myself. This helps to separate out what is market
behavior from errors that I may make as a trader. To me, this is
an important part of understanding who you are as a trader.
Think about the format of the journal. I’m convinced that paper
and pencil engage a different part of the brain, and writing
things by hand helps us to learn differently. Most of us don’t do
a lot of writing by hand today, so sitting down with paper,
feeling the rhythm of your body as you write, pacing your
thoughts to match that rhythm, and focusing your attention on
the smooth glide of ink onto paper—knowing that your abstract
thoughts are taking form in the outside world—that’s powerful
magic. However, it’s not so easy to go back through five years
of trading journals to understand some market tendency if those
  journals are a Hemingway-esque pile of Moleskine notebooks
bound with a string in the corner of a closet. Using a database
program or even a searchable word processor document makes
for much easier review. My personal answer is to do a lot of
stuff on paper, and to separate out “research” from journaling as
a separate process. Research is done in Python or Stata or
Excel, but I want to hold my journal in my hands. Different
strokes for different folks, and there’s no right or wrong, here.

These are some points that will get you started on this very important
practice

What Do You Believe?

I want to share an important exercise with you. The journey along the path
to trading mastery can be difficult because it requires conflicting skills—on
one hand, you must rule certain aspects of your behavior with iron discipline
and must also begin managing your thoughts and belief systems. On the other
hand, learning to trade requires patience and gentleness. You can’t force much
of what will happen, as some of the changes and adaptations take time—it’s a
growth process (though we’d like to encourage that growth to be as quick as
possible.) This exercise has the potential to give you the power to make
sweeping changes in many areas of your life, and maybe to help you a few
steps along the path to mastery.
Many authors have pointed out that we don’t trade the market, we trade our
beliefs about the market, or another way to think of it is that we don’t truly
interact with the actual market, we face our beliefs, both enabling and
limiting, about the market. This is true of many things beyond the market.
(For instance, how much miscommunication is happens because people are
not really listening to each other? Because they are paying more attention to
their own beliefs and preconceptions about what that other person is saying,
rather than listening?) Most people careen through life, bouncing from one
experience to the other, without really digging into the beliefs and motivations
that drive them. Socrates put it in black and white: “the unexamined life is not
worth living”, yet I wonder if so many of the distractions of day-to-day life
are designed to help us avoid that examination—to help us avoid the hard
questions.
 Today’s exercise is about those hard questions. I would like you to block
out a full hour’s time. Turn off the phone and computer, no texting, no
television, maybe put on some background music. Sit down with some blank
paper and your favorite beverage of choice, and start to list your beliefs. I
would suggest focusing on three areas: your beliefs about the market, your
beliefs about the process of trading, and your beliefs about yourself. If you
want to go a step further, maybe you can also work on a list of beliefs about
the universe and reality itself, but realize that any of these lists is probably
enough material to write a substantial book. Do the first part of this exercise
with no judgment, simply listing beliefs as fast as you can think of them. List
both empowering and limiting (you may choose to think of them as good and
bad, but those aren’t the best labels) beliefs, and just keep going until no more
come.
After you’ve done this, return to the exercise over the next few days and
add things you’ve missed. You should have several pieces of paper filled with
what is probably a jumbled list, and then the second part of the exercise
begins: organize and clean those lists. Notice the difference here: the first part
of the exercise was stream of consciousness with as little judgment as
possible (brainstorming), and now you switch gears to an editor’s mindset.
There are other ways to think about this division: unstructured / structured,
creative / reductive, intuitive / rational, subconscious / conscious, etc.
You will find some of your beliefs overlap, some can be edited, some can
be discarded, and some probably were in the wrong list. Clean those lists, and
you will end up with a set of beliefs that give you some deep insight into your
heart and mind. Incidentally, many of the most effective trading practices
work to integrate different types of analysis or different ways of “being”, and
this little exercise can help you experiment with balancing those sometimes
conflicting and contradictory approaches.
The process of trading can be very stressful, and I think many people would
find great benefit in working with a mental health professional at different
points along the way. This little exercise, however, encourages you to be your
own therapist, to do your own work. You can next begin the process of
transforming some of those limiting beliefs, or learning how to operate within
the belief structure you have. It’s difficult to just “change what you believe”,
but most people are blissfully unaware of the power of their beliefs to
influence their actions and their results.
 So, once again, I encourage you to find a few quiet moments, shut out the
outside world, turn inward, look deeply with, and open yourself to
understanding and growth. Why not take those first steps today?

On Quantitative Techniques

Quantitative Analysis: What and Why

I think most traders and investors understand that we live in a world in
which markets are becoming ever more competitive. To make money in those
markets, you must have an edge, and you must truly understand your edge.
But the question remains: where and how do you find an edge and how can be
you sure you understand it? I want to share a little bit of my own experience
in this regard, and give you some ideas for doing your own work.
At the beginning, I think it’s important to understand the questions we are
asking. No matter what your trading/investment methodology is, you are
assuming that something is more likely to happen than something else, based
on whatever tools you are using. Think about that a minute, because some
people are confused on this point. Whatever you do to decide when to get in
and out of markets, you are doing so because you think one outcome is more
likely than another. If we don’t have this understanding, then we can easily
fall into a mindset that says, “x must happen”, “x will happen”, or “x has to
happen.” That way of thinking is dangerous; at best, x is a little bit more
likely to happen. So, how do we know that is true?
Quantitative analysis and quantitative tools can seem to be intimidating,
but, at their core, they are very simple—all we’re doing is looking at a bunch
of things that happened in the market, defining some conditions, and seeing if
those conditions have been tied to certain outcomes. We then make a
(hopefully small) leap and assume, if we’ve done our work right, that seeing
these conditions in the future will make certain outcomes more likely. Here
are some concrete examples of statements we can test. Notice that they each
include a condition or set of conditions, and an outcome: Stocks with good
earnings stability are likely to go up. Stocks making 52-week highs are likely
to go up. After a decline in price, if volatility contracts, stocks are more likely
to go down. After making new 20-day highs, a commodity is more likely to
continue to go higher. (By the way, some of those statements might be true,
and some might be false. They are simply examples.)
How do we answer those questions? Well, there are some semi-
sophisticated techniques that you may have encountered in a math class a
long time ago like linear regression or principal component analysis. These
tools do have their place, but we don’t need them for much of work. We can
do some useful analysis with these steps:

Get a bunch of market data together. We need to make some

decisions about what timeframe (daily, weekly, 1 minute?),
what markets, what time span (recent? 5 years? 50 years?) we
want to cover. This stage of getting and managing our data is
harder than you might think because we have some thorny
issues like dividends and splits for stocks and rolls for futures to
consider. In addition, nearly all data sources have some errors,
so we’re going to need to spend (too much of our) time cleaning
and wrangling this data.
Define a set of conditions. These conditions can include price
patterns, other technical factors, tools calculated from price
inputs (e.g., MACD), fundamental factors, changes in
fundamental factors, economic data, macro factors, sentiment
data, etc. The only serious caveat here is that these conditions
need to be defined precisely because we are going to test them
over hundreds or thousands of occurrences.
Look for every time the condition occurred in every market we
are analyzing.
See what happened following every occurrence of the
conditions.
A useful construct is to frame the question so it’s binary: either
one thing happens or another. We can look at magnitude of
effects and variability, but, often, this simple binary “counting”
approach leads to good insights.
Compare what happens after the conditions to all other market
data. What we’re looking for here is some evidence that our
condition set has the power to influence markets.

That’s really it, and it’s not so intimidating: define a condition; test it, and
then look at the results. Whether you’re working with pencil and paper, a
spreadsheet, or working within a programming language, this technique of
asking questions and seeing what the data says will help you understand the
market better and find opportunities for profitable trading.

How Do You Know If You Have a Trading

Edge?
In a recent blog post, I made the statement (borrowed from Jack Schwager),
that you must have an edge to be successful in the marketplace, and, if you
don’t know what your edge is, you don’t have one. This, of course, prompted
the logical question from a few readers: “how do I know if I have an edge?”
Before I answer, I need to start with a disclaimer: There are a lot of ways to
make money in the marketplace. I’ll try to be as inclusive as possible in this
answer, but just realize that everything here will be somewhat biased from my
own perspective as a primarily technical trader who uses hybrid systematic
and discretionary techniques. I’ll point out some of the places where I’m
reasonably sure an edge does not exist, but, undoubtedly, there are things that
I’m missing here, too.
Your edge must be realistic. There are many arcane and silly approaches
to the market. Sadly, it’s not easy to avoid this stuff because it is everywhere.
In fact, some certification programs focus on a lot of the this kind of stuff, so
we have an army of technicians with letters after their names who talk about,
say, the 161.8% Fibonacci extension, or argue over wave counts. Prices in
financial markets are driven by buying and selling decisions people make.
I’ve been down the “mysteries of the Universe/probing the Mind of God”
rabbit hole myself, earlier in my trading career. If your edge depends on woo-
woo and magical thinking, you’re probably in trouble before you begin.
Also, being realistic means you must understand that trading returns are
uncertain; you’ll have rich periods and lean periods. Your trading account is
not going to be an ATM, and your edge may work for a while and then stop
working. It’s not uncommon to see a trader make quite a lot of money, and
then to enter a period where he cannot make money at all as market
conditions change. Even if your edge is solid and stable, it will only be so
within the bounds of probability, and those margins can be pretty fuzzy,
indeed.
You have to commit to the work and commit to continually evolving as a
trader. You must commit to the process.
Your edge must fit your timeframe . Do fundamentals matter? Do short
term movements, say on a 5 minute chart matter? How about relationships
between prices in different markets? The answer, of course, is yes and no,
depending on what kind of trader you are. If you are a short-term trader,
focusing on fundamentals probably doesn’t make sense. If you’re a long term
trader, you need to figure out how to filter out the noise. Your edge must
respect factors that are relevant to your timeframe.
You probably should be able to verify your edge statistically . This is a
good news/bad news situation: you must have some basic knowledge of
probability and statistics to tell if you have an edge. There’s no way (that I
know of) around that, but the good news is that the math is pretty simple.
Even someone who is “mathematically challenged” can acquire the skills
needed in far less than a year’s time, with a little work and focus.
What you’re looking for is an understanding of the concepts behind
statistical significance, but also some common sense, real-world application.
Imagine I shuffle a deck of cards and deal them to you, and you find three red
cards in a row. How surprising is that? What if the next ten cards are also red?
How surprising is this now? Would you begin to suspect something about the
deck? If I assure you the deck is fair and was shuffled (and you know I’m not
lying), what are the odds the next card is black? You should be able to do
math like that quickly and easily, or at least have some solid intuition about
the answers.
As for backtesting, I think there’s a place for it, and it’s a topic we cover in
some depth in this course. There are also limitations, and you need to
understand those, as well. The best way to verify your edge is to clearly
define your rules, test them, and then out of sample or forward test them
before committing real capital and risk. I could list a thousand bad ways to
think you have an edge: relying on the authority of a guru, finding a simple
pattern and not testing it, etc., but most of those ways would fail at this step.
If you can’t codify and test your edge, it’s very hard to understand it. (Note
that this applies to fundamental approaches as well.)
Your edge must fit you.  We’re all built a bit differently. Some of us have
widely varying attitudes toward risk, patience, emotional control, and many
other aspects of our personality. I think there are many ways that most traders
could trade successfully, but there are also many ways that just will not work.
For instance, can you sit through 40% drawdowns that might last 2 years if
you knew, with a high degree of certainty, that you’d make money over the
long haul? If not, then you shouldn’t be a long-term trend follower. Can you
devote every second of every day to focusing on the market? If not, you can’t
daytrade. Are you prepared for a 3-5 year learning curve, and do you have the
capital (mental and financial) to sustain your learning through that time
period? If not, then you probably can’t trade at all.
Good things happen when a trader, who is at the right stage of maturity,
finds a system that fits him like a glove. So many of the struggles and
problems he has faced will resolve themselves, seemingly effortlessly, but, of
course, it’s the years of work to get here that make it all happen.
It’s hard to have the confidence to execute without having a high degree of
certainty that your edge is real—in fact, you shouldn’t   have that confidence!
Confidence in the wrong thing is a sure way to financial ruin in the markets.
Knowing your trading edge, understanding it, is a critical step in trading
successfully.

How Do You Know When You’re Wrong?

How can we have a bias and stay out of trouble? One simple rule will solve
most of your problems: if you have a bias on a market, it must be based on
something you can see. At the risk of repeating myself, let me say that a few
different ways, just to be sure it’s clear.
If you have a bias on a market, check yourself for emotional reactions or
commitments. Particularly, once you’ve made a public statement (even a
simple tweet or post in a trading room, for developing traders), it is only
human to shift your attention to defending that statement. This will get you in
trouble. I think many trading problems can be solved with a small adjustment
to mindset: as soon as you are in a trade, begin to shift your focus to finding
reasons the trade is wrong. This is unnatural, to say the least. In everyday life,
who argues like that? Who makes a point, and then immediately starts looking
for reasons that whatever they just said is wrong? This, however, is a very
helpful perspective for the trader because too many traders ignore evidence
that they are wrong (even their stops being hit, and it can’t get much clearer
than that), and hold on to losing trades long past the point they should’ve
exited. Many trading accounts have been destroyed by this practice, and many
traders have left the field for some other endeavor simply because they
couldn’t exit when they were wrong.
A logical extension of that idea is that a bias is only meaningful if you can
clearly state what would disprove or contradict it. For instance, if I’m
short because of a higher timeframe bias on a market, what should not
happen? Well, the market should not “go up”, but that needs to be better
defined—reasonable levels on that higher timeframe should not be broken,
but I probably can’t use them as stops because then I am trading that higher
timeframe.
We also can use price action on the trading timeframe to get clues that our
bias is right or wrong. The trading timeframe will trend both up and down
within the higher timeframe bias, but we should not see many consolidation
patterns resolve easily to the upside, and we should not see too many large
standard deviation spikes to the upside. If we do, and if this continues for
some time, it’s evidence of underlying bullish conviction, which contradicts
our bearish bias. The exact points where you “pull the plug”, even before your
stop is hit, on the bias might be up for debate, but the key is that you have
conditions that would contradict your bias.
Compare this to common amateur biases: you hate a stock because you hate
the food at a restaurant, you had a bad customer service experience, you think
a competitor’s product is best, you lost money in a stock so you are out for
revenge, someone in a chat room told you it was a great stock, etc. Those
types of biases are less meaningful than a disciplined bias that you can
understand and monitor as the market unfolds.
There is another potential problem with a bias; it can blind you to trades
that set up against your bias. A bias is often nothing more than a higher
timeframe structure. If you are a daytrader, maybe you have a bearish bias
based on the weekly or daily chart. If you are a position trader, maybe your
bias comes from monthly charts, but you do not know your bias is right! It
may well be wrong, and, if it is a “good” bias based on structures that have a
statistical edge, then, if it fails, it may fail dramatically. Wouldn’t it be nice to
be able to participate on the right side of that failure, and maybe even make
some money? You can’t do this if you are emotionally attached to your bias,
and if you will not trade against your bias, but a better mindset is something
like “I hold a bearish bias, but everything I see is screaming long in my face.
Maybe I should get long on my chosen timeframe, manage the trade properly,
and see what happens.” Easy to write, and easy to do once you’ve developed
the right mindset, but it’s usually impossible for the struggling, emotional
trader to do this.
Check your biases. See if they are truly based on something you can see, if
you can clearly state what would contradict those biases, and if you have the
flexibility to trade against them when needed.

What’s Wrong with Fibonacci?

 I see you don’t believe in Fibonacci ratios, but it seems every book 
and website says they are really important. I read your analysis on
 your website and it was convincing, but your stance is so different to
the CTA program and all the other technical people out there. You’ve
shared stories of your development as a trader and I was wondering
Could you maybe talk a little bit more about how you came to the
conclusions you have? Maybe seeing the path you walked will help
me find more confidence to stand apart from the Fibonacci crowd.
Could you tell me what’s wrong with Fibonacci and even more how
 you came to believe this?
This is a tought-provoking question from a student, and one that deserves a
good answer. First of all, I’m assuming if you are reading this you understand
how Fibonacci is applied to trading. At a bare minimum, you should
understand retracement ratios, extension ratios, and time ratios. If you are
fuzzy on that, just google it and familiarize yourself with the common
practice in today’s technical analysis.
As a very short summary, there are two issues here to consider: First, many
people believe that the so-called “Golden Ratio” (a number that begins with
1.618) describes many important relationships in the universe and human art.
Second, people note that this ratio can be derived from the so-called
“Fibonacci sequence” {0, 1, 1, 2, 3, 5, 8, …} and further assume that the
actual numbers have significance themselves—i.e., that the number 13,
because it is a Fibonacci number, might have some special qualities that 12
and 14 do not. These ideas are extended to financial markets, usually in
measuring length and magnitude of a market’s movements in time and price,
or in some relation.
I don’t know when I first encountered Fibonacci numbers. I was interested
and curious about various arcane schools of thought even as a kid, so I’m sure
I had some encounters with the idea before I reached high school. My first
serious investigation came in college as I was working on a degree in music
composition. Part of the process of learning to write music is learning what
people have done before you, and there are various schools of thought about
how to understand and analyze a composer’s work. I was working with a
technique called Schenkerian Analysis that basically takes a piece of music
and reduces it to a few key elements—a way to see the skeleton of the body
or the frame of the building. In reading the works of other analysts, I found
that people said the Golden Ratio was very important in the structure of many
pieces. I probably looked at 100 examples, and then launched into deep
analysis myself.
I’ll spare you the gory details, but I spent many months on this project. At
first, I was excited because I had unlocked some key to the mysteries of the
universe. If I could understand the use of this ratio, then I could improve my
own compositions. I could probably use the idea in the computer-generated
compositions I was working on, and maybe could develop some alternate
tuning systems that would take advantage of different ratios of resonances.
What I discovered early on was that the Golden Ratio did not “work”.
Sometimes important things fell near the ratio, sometimes (quite rarely)
exactly on, but often not very close at all. There were also inexplicable things,
such as very important features coming at some other ratios, while some very
minor detail hit a precise Fibonacci ratio—and “pro-Fibonacci” people would
point out how the ratio worked in this case. I scratched my head; maybe there
was something I didn’t understand, but it seemed like hanging a work of art
on a wall and then marveling over some detail of how the floor tiles hit the
wall—probably simply due to chance, and almost certainly not significant.
I then encountered the “measurement issue”: when I talked to people who
were supposed to be experts, they were extremely evasive. I remember a
conversation with someone who had written an influential book. When I
explained that I could not make the ratio work in a specific Beethoven piano
sonata, he told me that was because I needed to measure ratios in space on the
page; in other words, inches. This, of course, is nonsense. Our perception of
music is ordered in time, not physical space. Physical space on a page is
arbitrary; I found examples of the same sonata in different editions that were
4 or 20 pages long, and things were spaced proportionally very differently in
those different editions.
I discovered something that I would later encounter in trading: people
would use whatever measurement system they needed to make the theory fit
the facts. Once I looked at the problem objectively, I saw that “big things”
tended to happen later in a piece of music rather than earlier, usually after the
middle and somewhere a bit before the end. This, of course, is a dramatic
pattern that makes sense in a book, a play, a film, or pretty much any other
structure—it’s just common sense. But the idea that 61.8% was some magical
ratio in music—that just didn’t hold water.
When I started digging into relationships between ratios and musical notes,
that didn’t really “work” either. So, after many months of intensive work, I
decided that Fibonacci ratios in music were overhyped and simply didn’t
represent reality in any meaningful way. Did it make more sense that
composers (who, in most cases, were writing music for a living because it was
their job) would follow a formula of “big thing happening somewhere after
the middle and near the end”, adapting it to the flow of whatever piece they
were writing, or that someone was staying awake at night trying to hide a
secret code of the universe somewhere in between the notes?
Fast forward a few years to where I was learning to trade. I’d had some
successes and failures, and realized that I was going to have to study very
hard to make this work. I tracked down a bunch of original source material
from legends like Schabacker, Gann, and Elliott, as well as a big selection of
the modern books written on the subject. In the beginning, I quickly forgot the
Fibonacci lessons I learned in music, and was awed by the power of the ratios
in financial markets. I read book after book talking about the different ratios
and how they described moves, where to put stops, where to put targets, when
to predict turns—I saw it worked often. Of course, there were cases where it
did not work, but there was also this strong appeal to mysticism in much of
the writing: these are “sacred ratios” upon which the “foundation of the
universe rests”. How could I ignore such portentous information when I was
entering a trade on a currency chart?
A few things eventually shook my belief in the concept, best told in bullet
points:
You couldn’t be sure which level would work, but some level
always worked after the fact. I began to realize that levels
would be violated in live trades; I had dutifully placed my stop
a few ticks beyond, but then another level clearly held at the
end of the day. There was no way, and no way in the literature,
to predict which level would hold. Once I learned about the idea
of confluence, I realized that we were drawing so many levels
on charts that it might just be luck that they seemed to work.
I began to understand randomness. I had a weird formal
education. My quantitative training in undergrad was sorely
lacking. While I would not recommend this to anyone, it did
leave me with a curious hole to fill: I had to re-think the
problem of randomness from the ground up, as I did not have a
good understanding of things like confidence intervals and
significance tests. From practical trading, I saw that there was a
lot of noise in data, but I wasn’t sure how to tease it out. As I
was getting a better education, I came up with a stop-gap; I
generated many charts of random market data according to
various techniques and spent a lot of time looking at them. If I
had a better formal education I probably would have thought
this was a waste of time, but I experienced so many cognitive
errors as I did this. It’s one thing to know them academically,
but when you see how easily your perception is swayed and
how easily you find patterns in random data, you start to think
deeply. Does it invalidate the idea of patterns in real market
data? Of course not, but it certainly challenges the claims of
“just look at a chart! You can see it works! How can you
question it? Look at these examples…” Armed with that
firsthand experience (and, again that word is critical—it was
experiential, not academic knowledge) I became very critical of
examples and claims.
I did some background work on the people making claims for
the tools. I won’t dwell on this because I don’t think it’s
constructive, but suffice it to say that someone could make a
good career out of debunking Fibonacci experts, just like
Houdini did with mediums in his day. I realized that we have a
tendency to put some aura of greatness around past gurus, who,
in many cases, were part-time traders who had poor access to
data and no analytical tools. Alexander Elder, in his excellent
book tells of interviewing the great W.D. Gann’s grandson, and
that his grandson said there was no fortune and no profits from
his trading in the stock market. When I dug into the current
gurus on the internet, I discovered that many trades were done
at improbable prices (“How do you always get filled on the bid
every time?”), and, years later, one of the big gurus from the
early 2000’s told me that all her trading, scalping NQ futures,
was on a simulator and she never had a live trading account.
Sadly, I had seen hundreds of people try to replicate her
methods, with no success. I could go on and on. I wasn’t trying
to tear down any idols; rather, I was desperately searching for
some evidence that someone was really applying these tools to
make money.
The last straw was adjusted price charts. This might seem odd
to newer traders, but when you look at past price charts, those
prices may or may not represent prices at which the asset
actually traded. Much of the discipline of technical analysis
rests on the idea that people have a memory around specific
prices. While this may or may not be true, there are different
ways that historical charts must be adjusted. With futures, there
are at least three common methods (difference, ratio, or
unadjusted), and the question of when to roll to new contracts.
With stocks, there are issues of dividends, splits, spinoffs, and
other corporate actions that may or may not be accounted for on
the price charts. You still see this today: ask someone showing a
Fibonacci extension on a crude oil chart how their chart is back-
adjusted. How many days open interest or volume to roll? How
does your chart compare to spot prices? The answer I got from
asking many people was either confusion or “it doesn’t matter.”
(I have seen this dismissal over the years from many people
who use levels in various capacities. I remember explaining to a
stock trader who had traded from more than 20 years why SPY
prices were so different today from yesterday—in 20 years of
using “levels” he had never accounted for dividends.) Simple
logic here: if I tell you I have some powerful pills, but it doesn’t
matter which pills you take, how much you take—take 1 or 20,
or when you take them, is it more likely that it magical
 medicine or that it does nothing at all?
The whole thing died, for me, when I realized that it rested on
vague appeals to authority. I knew this all along, but once I had
been around the block a few times, it was even more obvious.
The Emperor had no clothes. No one will ever provide you with
a quantitative proof of Fibonacci levels working. (I’ve made
this challenge many times, and I will renew it here. Show me
something good and I will publish it and admit I’m wrong.
Show me something possibly flawed but still substantial, and
I’ll publish it for discussion. It’s possible my thinking on this
subject is wrong, and I would love to expand my thinking in
another direction. Despite me having said this hundreds of
times, I have yet to receive a single shred of actual work done
on these ratios.) The last apology for ratios I read was a few
weeks ago when someone said that you could just look at charts
and see they worked and a lot of his friends, who were medical
doctors, said Fibonacci ratios were really important in the body
and in art.

So, that’s my journey, and that’s why I place no emphasis at all on

Fibonacci levels. Here’s the real key: you do not need them. That’s the point.
It’s not that I’m trying to tear down anything or simply show you that
something doesn’t work; I’m showing you that this is probably confusing
baggage and noise, and does not add any real power to your analysis. Why
not focus your attention on things that do work?

On Pullbacks

How to Trade Pullbacks

Some of my readers may have questions as to whether it is possible to trade
profitably based on simple patterns, given that so much modern quantitative
trading focuses on volatility, spread relationships between two or more
markets, and other types of trading that move us away from simple,
directional trading. The answer to that is a qualified ‘yes’: patterns are
heuristics, shortcuts, if you will. A trader who can read and analyze patterns
can quickly drill down to the essential elements of market structure.
Opportunity exists in markets when buying and selling pressure is not
Three exercises to further investigate the patterns of support and resistance

Exercise 1: Action around levels

Find support and resistance levels on charts. This instruction is deliberately
broad; you can do this in many ways. One way might be to look for a market
to make important pivot highs and lows, and then to see what happens when
price re-engages those levels. Another possibility is to find an area in which
support or resistance is working, and then zoom in, perhaps on lower
timeframes, and look at the action closely.
Keep some kind of notes and records, but this is also a deliberately
subjective project. It may be enough to characterize the level as working
(holding) or failing (breaking). Many traders find it useful to take screenshots
of the charts being investigated. Over time, this will build a library of patterns
and variations of patterns around support and resistance.

Exercise 2: Random lines on charts

Of course, the concern with the previous exercise is that we may have been
cultivating cognitive bias. Any line drawn on a chart will appear to be
significant. It is impossible to stress this enough, so you need to see it
yourself to fully appreciate this quirk of human perception.
Take a number (at least 20) charts and somehow draw random lines on
those charts. There are many ways to do this, depending how certain you want
to be that the lines are random. In many software packages, it is possible to
mask the price bars, perhaps setting them to the color of the background of
the chart. Another possibility might be to set a chart to a higher timeframe,
draw some random lines on that chart, and then “zoom” in to lower
timeframe.
Lines may be flat or sloping. Some of your lines, if they are truly random,
will not engage prices, but your job is to look carefully every time price
comes close to one of these lines. Imagine the line is significant (or that you
believe it to be) and then investigate action from the same perspective you did
the “real” levels in Exercise 1.
Once you have spent some time with this exercise, you will likely be
suspicious of support and resistance in general, and especially of calculated
levels. (Think of the so-called “pivot levels” such as S1, S2, R1, R2, … Is it
possible that these levels are no better than your random levels? That is the
key question for anyone who would use levels in his work.)

Exercise 3: Statistical Analysis

This exercise is more difficult. Define levels somehow; it might be
interesting to use a mixture of some calculated level (ratios, pivot levels, etc.)
and levels derived from clearly visible chart points.
Move through the chart bar by bar, seeing what happens as price engages
the level.
Make a serious effort to avoid “information leakage” from the future; be
very careful of seeing future price data and to focus on information you would
have had as the chart unfolded in real time.
Make “mock trades” at each level and record returns for each of several
days after. This is a manual approach to the event study methodology. If you
have programming skills, you might be able to code a system that could take a
discretionary entry on a specific bar and then generate statistics such as those
you saw in Module 5 of the course.

Section 4: Charting by Hand

This is still a valuable exercise, and this will be the last module that carries
an explicit reminder to do hand charting. You should be thinking about how
(and if) you wish to incorporate this practice into your ongoing trading,
training, and market analysis. In the first module, we discussed the several
ways you can do this exercise. The details are not important; it is far more
important that you do it.
This should also not be a tremendous time suck—this is not a unique form
of torture I have devised. A few minutes every day (but every day!) directed
to drawing your charts, or a few minutes each hour if you are an intraday
trader, will reward you with deep understanding of price action.
You may wish to leave this exercise, but come back to it whenenever you
feel out of touch with the market, or when you’ve been away from trading for
a period of time. There is something about the practice of writing down prices
and/or charting by hand that has deep power to connect us to the flow of the
market.
 Section 5: Readings
From The Art and Science of Technical Analysis: Market Structure, Price
ction, and Trading Strategies by Adam Grimes, Wiley, 2012:
170-173 (the anti)
327-336 (anti examples)
353-359 (cognitive biases)
409-424 (a deeper look at the MACD and moving averages)

On Doing Statistical Analysis

Some Important Lessons I’ve Learned

I thought I’d share a few thoughts today, kind of a “what I’ve learned in the
past 10 years” post. This list was refined and crystallized by the process of
writing the book, but this list was also a major reason why I wrote it in the
first place. I saw so many people struggling to learn to trade that I wanted to
try to put down the lessons and truths I had found in a concrete format. True,
there is no one way to trade, and many different approaches can be successful
in the market, so long as they are aligned with some fundamental truths.
These are some of those fundamental and undeniable truths, as I have come to
understand them over the course of my trading career:
Most of the time, markets are very close to efficient  (in the academic
sense of the word.) This means that most of the time, price movement is
random and we have no reason, from a technical perspective, to be involved
in those markets.
There are, however, repeatable patterns in prices. This is the good news;
it means we can make money using technical tools to trade.
The biases and statistical edges provided by these patterns are very
small. This is the bad news; it means that it is exceedingly difficult to make
money trading. We must be able to identify those points where markets are
something a little “less than random” and where there might be a statistical
edge present, and then put on trades in very competitive markets.
Technical trading is nothing more than a statistical game.   There are
close parallels to gambling and other games of chance; a technical trader
simply identifies the patterns where an edge might be present, takes the
correct position at the correct time, and manages the risk in the trade. This is a
very simplified summary of the trading process, but it is useful to see things
from this perspective. This is the essence of trading: find the pattern, put on
the trade, manage the risk, and take profits.
It is important to be utterly consistent  in every aspect of our trading.
Many markets have gotten harder (i.e. more efficient, more of the time) over
the past decade and some things that once worked, no longer work. Iron
discipline is a key component of successful trading. If you are not disciplined
every time, every moment of your interaction with the market, you are not a
disciplined trader.
It is possible to trade effectively as a purely systematic trader or as a
discretionary trader, but the more discretion is involved the more the trader
himself is a key part of the trading process. It can be very difficult to sort out
performance issues that are caused by markets, by natural statistical
fluctuations, by the trading system not working, or by the trader himself.
There is still a tremendous bias in many circles toward fundamental
analysis and against technical analysis. The fundamentalists have a facile
argument because it is easy to point to patterns on charts, say they are absurd,
and point out that markets are actually driven by supply, demand and
fundamental factors—the very elements that fundamental analysis deals with
directly. However, many times the element of art involved in fundamental
analysis is overlooked. How much does your valuation change if your
discount rate is off by a percentage point? How dependent is your model on
your assessment of some manager’s CapEx decisions in year four? Do you
really have a good sense of how the company’s competitive position will
evolve with the industry over the next decade? Does everyone else? There’s a
lot more “wiggle room” in fundamentals than most people realize.
One advantage of technical trading is that, done properly, it clearly
identifies supply/demand imbalances from their effect on prices. This is a
form of look-back analysis, but good technical tools force you to deal with the
reality of what is happening right now. There is no equivocation, wishing, or
emotional involvement in solid technical trading. The best risk management
tools are technical, or are based on patterns in prices themselves.
 Most people (and funds) who try to trade will not be successful, and I
believe this is because most of them are simply trying to do things that do not
work. Taking a good, hard look at your tools, methods, and approach can be
scary, but there is no other way to find enduring success in the market.

Bad Stats Lead to Bad Decisions

More and more, we are bombarded with market statistics from every side.
Anyone with a spreadsheet can tell you that such and such week has been up
9 times out of the past 7 years; stats are cheap. The problem is that many of
those statistics are false or misleading and making market decisions based on
those statistics can be hazardous to your financial health.
There have been a lot of statistics floating around about what happens after
a weak first day of the month or quarter, claiming that a down first day gives
a downward bias to the entire quarter (or month). First of all, let’s look at the
stats some “experts” are sharing. Using the S&P 500 cash index back to 1961,
if the first day of the month is up (which happened 370 months), the entire
month was up 64.9% of the time, for an average monthly gain of 1.29%. If
that first day of the month was down (which only happened 287 times), the
month only closed positive 51.2% of the time, and the average return for all
of those months was -0.21%. Here are the results presented in table format:

These are strong numbers that seem to show a tremendous edge in the
marketplace, but let’s dig deeper. We should, first of all, be on guard because
the effect is so strong—when we find something that appears to be a
statistical homerun, we’ve probably made a mistake somewhere. Let’s find
that mistake.
The first step is to ask if the results make sense. I would argue, right out of
the gate, that they do not. Why should such a strong effect exist? Perhaps we
could make a case for some kind of monthly momentum—maybe managers
tend to put money to work at the beginning of the month and that has some
persistence through the entire month. Maybe there is another reasonable
explanation. It is, at least, possible to make an argument, but we are already
suspicious because there is no clear logic driving these results.
However, something interesting happens if we examine day two of the
month; we find the same effect. No, not for day 1 and day 2 being a
cumulative decline for the month (though we can do that test, too), but simply
if day 2 is up, the month tends to be up. Also, day 3… and day 4…. In fact,
no matter which day of the month we examine, we find if that day is up, the
month tends to be up! If it is down, the entire month tends to be down.
So, now we have a problem. Can we make any possible argument to
explain this? How do we feel about the argument of monthly momentum or
managers putting money to work on, say, day 17 of the month? Obviously,
this is now completely illogical, so we must look elsewhere for an
explanation. Maybe we should turn our attention to the way we ran the test.
Maybe there is a problem with the methodology.
As an aside, I once knew a trader who had a system that was based on a
similar idea. He had done enough research to know that his system “worked”,
statistically speaking, no matter which day or days of the month (or quarter,
or week, or even which hour of the intraday period) used as a trigger. Over the
course of about a decade he had traded the system live, and had lost a
significant amount of money—high seven figures on this particular system.
You might ask why he kept going back to it, but the reason was (bad)
statistics. Every way (except the right way) he looked at the system and
twisted the inputs, the results were astoundingly strong, yet the system failed
to produce in actual trading.
This is not an academic exercise; statistical mistakes are not abstract. For
traders, statistics are life and death; statistics are the tool through which we
understand how the market moves. Bad statistics lead to bad decisions, and
bad decisions cost money.
Before we get to the methodological error, here are the “correct” statistics
for the “first day of the month” effect:
 Based on these test results we can say we see no effect—that, whether the
first day of the month (or, in fact, any particular day) is up or down has no
significant effect on the overall direction of the month. There is no tradable
edge here, and, unless our test has missed something ((always a possibility— 
stay humble!)) there is simply nothing here worth thinking about.
There is a simple solution to mistakes like this, and I’ll share it with you
later on. For now, spend some time thinking about where the error might be in
the method, why it matters, and why it might be hard to catch. Here’s a hint:
what is a monthly return?
From here on out, I hope to accomplish two things:

To understand how easily “future information” can contaminate statistical studies, a nd how
even a subtle bias can introduce serious distortions.

To suggest one simple condition—asking yourself if the statistic could possibly have been
executed in the market as a trade—can protect us from all errors like this.

The specific error around the first day of the month effect is a common
mistake. I’ve certainly made it myself in tests and analysis enough times to
know that it is something always worth checking for, and I’ve seen it in stats
people use for many technical factors like moving average crosses,
seasonality, trend indicators, buying/selling at 52 week highs, the January
effect, and many others.

Day of the week effect?

Let’s re-cast the day of the month effect as the “day of the week effect”.
Here are (erroneous) stats that show that the return for each day of the week
(1 = Monday, 2 = Tuesday, etc.) is a strong influence on the weekly return.
For instance, if Wednesday is down, there is a high probability the entire
week is down. For comparison, looking first at all weeks (2,856) in the S&P
500 cash index, going back to 1962, the average weekly return was 0.15%
with a standard deviation of 2.14%, and 56.1% of all weeks were positive.
Here are statistics for the weekly returns, based on whether any Day of the
Week (left column) was up or down for the week. We seem to find a very
strong effect, and, again, that’s our first warning that something is amiss:
Market data is abstract. Think of physical “things” to simplify
problems.
To find the mistake, it helps to think of each week as a physical card. On
one side of the card, there are five boxes, each of which has a positive or
negative number (the daily return). Flip the card over, and you will find a
single number that is the return for the week; though not strictly correct, let’s
ust simplify slightly and say the weekly return is all the daily returns added
together. You can’t add percentage returns (you must compound them), but
that’s a complication we don’t need for this example.
So, put the 2,856 cards in a bag and randomly draw one out. You will find
each of the five boxes is more or less just as likely to have a positive or
negative number (i.e., each day is just about as likely to be up as down). If
you did this a lot, you’d find that the numbers are slightly more likely to be
positive than negative—about 52.7% of the days are up—but you would have
to look at a lot of cards to see that. At first glance, it just looks like we have a
mix of positive and negative days in each small box.
For this exercise, let’s focus on box 1, which is the return for the first
trading day of the week. Imagine pulling a few cards out, looking at that day,
and finding, just like any of the boxes, some are up and some are down. Ok,
now put all the cards back in the bag; things are about to get interesting.
Now, dump all the 2,856 cards out on the floor, and separate them into two
piles based on one factor: if that day 1 is positive put it in the pile on your left,
and if day 1 is negative, put it in the pile on your right. Now, do you see what
has happened here? Your selection process has guaranteed that at least one of
the days will be positive for the week for every card in the left-hand pile, and
there’s the mistake.
If you now turn the cards over and average the weekly number for every
card in each pile. you’ll discover that the pile on your left has a lot more
weeks that were positive, and the pile on your right has most weeks in the red.
When one out of five days is guaranteed to be positive, the week will
overwhelmingly be biased to be positive. While this might not seem that this
error would create a large bias in the test, it does. Market data is random
enough that even a very light “thumb on the scale” is enough to seriously
distort the results.

One test can catch many errors

How do we avoid errors like this? Well, the standard I apply to any test is
simple: Always ask how could you trade the given effect. In this case, we are
looking at weekly returns, which means buying or selling on a Friday and
exiting the following Friday (except in shortened weeks); this is the only way
we can capture the weekly return as traders. However, we don’t know if we
should have bought or sold on Friday until we see what the following Monday
does! This trade would have been impossible to execute, so the statistic is
suspect.
In this case, a better test, and one that would be tradable, would be to define
the week in one of two ways: either from the close of the day being examined
to the next Friday, or maybe doing a rolling week, always 5 trading days out.
In the case of the original test, the month should be defined from the close of
the day being examined to the end of the month, and a test like this will find
no bias. If you’re curious, here is a comparison of the two methods for the
first day of the week effect:

This is a sneaky error, and it’s one I’ve made many times myself. Though
the test will work—thinking about whether the tendency was tradable on the
timeline—it takes some careful thought as mistakes are not always apparent at
first glance. Be ruthless in examining the information you use and be even
more vigilant with your own thinking. Bad statistics lead to biases and poor
decisions.
 On Moving Averages

Death Cross: Omen or Not?

One of the recurring themes in my work is that most of the technical tools
most people use do not show an actual edge in the market—in other words, as
hard is it may be to say, most things most people do simply do not work. In
particular, there are a few technical conditions we hear discussed in the media
frequently; I can think of four offhand: any market is “finding support” at its
X-day moving average, the ominous-sounding “Hindenburg Omen”, and the
“Death Cross” and “Golden Cross”. What all of these have in common is that
they are easy to explain, easy to show visually, and most of them have catchy
names. Unfortunately, they are all meaningless—they have, at best,
questionable statistical significance.
The Russell 2000 is nearing a potential “Death Cross”, as reported
yesterday in the media:
The Russell 2000, which consists of small-cap stocks, is approaching
a key technical level, dubbed the “death cross,” that has market-
watchers wary. Its 200-day moving average is close to breaking
through its 50-day moving average on the upside, which signals a
bear market may be on the horizon.
Laying aside concerns about anticipating technical events, or that it is much
more accurate to think of the 50-day breaking through the 200 to the
downside, we should ask the key question: is the Death Cross really a “key
technical level”? Let’s ask the market itself and see what the data has to say.
Using the Russell 2000 cash index (6613 trading days, going back to
6/24/1988), we find 19 cases of the Death Cross. A reasonable way to test the
Death Cross is to take all the events that have happened, and to see what the
market has done following those events. We need to ask some questions and
make some decisions like: how far after the event should we look? Should we
look at every day or just hit a few spots? What is the correct measure of
performance?
For this quick test, let’s look at what happens to the market at a few spots
following the Death Cross, up to one year out. I will use 5, 20, 60, 90, and
252 trading day windows, which correspond roughly to one week, one month,
one quarter, one half year, and a full calendar year. This is not an exhaustive
test ((for instance, we could have a distortion due to a weird event 20 days
following one of the events, while 19 and 21 days would have shown a very
different picture)), but it’s good enough for a first look. At least, by looking at
a range like this we avoid the error of simplistic analysis that is so common:
people will say things like “following the Death Cross, the Russell 2000 was
up, on average, x% a month later.” When we see a single time period, we
should ask what happens over other time windows and why that particular
one was the only period examined.
There is another important mistake to dodge here. For instance, this is a
completely faulty analysis of a technical event: “Market XYZ was up/down
x%, on average, following the event.” Something important is missing from
that thought process—we should compare the event to the baseline of the
market, so we have to say “Market XYZ was up/down x% over/under its
baseline return following the event.” In other words, we look at what the
market normally does, and compare post-event action (condition) to that
normal (unconditional) market movement. Only if the market was absolutely
flat (i.e., had an average daily return (return, in this case, simply means
percent change) of zero) would it be valid to compare the event to zero. The
Russell does not have a zero baseline, as the following table shows:
 If we look at returns following every trading day in the history of the
Russell 2000, we find it is up, on average, 10.06% a year later. Now, let’s
compare this with returns following the Death Crosses:

When you look at that chart, focus your attention on the bottom line, which
shows what the Russell 2000 did, relative to its unconditional (baseline)
return. For instance, looking at the entire history of the Russell, we find it is
up, on average. 0.2% one week later. Following the Death Cross, it is down,
on average, -1.92%, meaning that it underperformed its baseline return by
2.12%. So, here is the first interesting point: the Death Cross actually does
show a significant sell signal in the Russell 2000 one week later. However,
this effect decays; the key question here is how large is the effect, relative to
the variation for the period? A year out, we are seeing standard deviations
greater than 15.0%, against a very small positive effect of 1.93%. So, what we
can say from this test is that we do find a statistically significant, very short-
term sell signal in the Russell 2000 in the data examined. This sell signal
appears to be strong for a week, and then decays and we see no longer term
significance. Interesting.
Where do we go from here? Well, first, I’d flip the test and look at the so-
called Golden Cross, which is the inverse of the Death Cross, when the 50
period crosses over the 200 day moving average:
 Here, we do not have any significant effects at all; this is not necessarily a
condemnation of the test (perhaps there is a reason the effect would not by
symmetrical), but it certainly calls for further study. What other questions
should we ask? It is entirely possible to find a valid signal just due to chance,
so we’d be wise to repeat this test with other assets and other timeframes. We
also might dig a bit deeper and look at each of the events, though we should
be careful of doing too much work like that because it is easy to “curve fit”
and select what we want to see. Still, actually looking into the data can help to
build a deeper understanding of how the market looks. A summary test is only
that: a very broad, rough, and blunt summary that may miss much significant
detail.
Another question that I find very interesting is “why do people focus so
much attention on mediocre or, in some cases, absolutely meaningless
technical tools?” One reason is probably due to cognitive bias. For instance,
one of the largest one-week selloffs following a Death Cross in the Russell
was in 2008; anyone who identified it then and remembers the strong selloff
is likely to have some emotions associated with that event. Furthermore, the
signals certainly can look convincing on a chart:
 It would be easy to find a few charts like that, and “show” that these crosses
work very well, but this is simply a case of choosing good examples. As I’ve
written before, much of the discipline of traditional technical analysis is
visual, not quantitative, so technical analysts are prone to these types of
errors, even with the best of intentions. The only defense against these errors
is using the tools of statistics to take a proper look at the data and to consider
the effects in the cold, hard light of quantitative analysis. In the case of the
Death Cross, there does not appear to be any reason to focus on this event,
and it appears to have no long-term significance for the market.

Death and Golden Crosses: A Deeper Look

The previous article looked at the (apparently) impending Death Cross in
the Russell 2000 index. In that article and the data examined, I found an
interesting and statistically significant short-term signal for the Death Cross,
but no effect for the Golden Cross. In this post, I will look at the Death and
Golden Crosses on much more data. First, the DJIA going back to the 1920’s,
and then on a large basket of stocks from more recent trading. I’ll also discuss
a bit more about test procedures, and what I have found to be the most
common opportunities to make mistakes that will cost you money.
The sample size for the Russell 2000 test was quite small, with only 19
events to examine. Let’s take a look at an index with much larger history, the
Dow Jones index going back to the mid 1920’s. Here are the statistics for both
Crosses on that index:
 With more data, this test shows a very different picture from what we saw
before. Now, there appears to be no effect at all, though we might note with
some curiosity that the excess returns, for both crosses, seem to be pretty
consistently negative. (Though we cannot tell from these tables, none of these
numbers are statistically significant, meaning that we are quite likely looking
at noise.) This test seems to show no effect whatsoever from the Crosses, and
we could draw the conclusion that we should probably not pay any attention
to them. However, here is our first cautionary note:

Caution: More may not necessarily be better

In examining more data, we have gone back further in time. It is certainly
possible that older data does not relate to current conditions. Something (or
many things) could have changed. In all analysis, we make the assumption
that “the future will look something like the past”, but this is an assumption
that is worth considering carefully. In this case, it seems reasonable to assume
that any effect would be more or less stable, but this may not always be the
case.

Crosses on Individual Stocks

Both of these tests have been done on stock indexes. I thought it would also
be interesting to look at the test on a basket of individual stocks. Here are the
test results (in a different format) for a basket of 100 stocks, using the past 10
years of data. This will be a way to get more events to study, and also to look
at a different asset class. (Individual stocks may or may not behave differently
than stock indexes.)

This table is, perhaps, slightly harder to read, but it gives the same type of
test results in more depth, focusing on a shorter time period (20 trading days
after the event). I would suggest you focus on the second column in each box,
which shows the excess return ( X - X  is the signal mean — baseline mean) and
the p= column, which shows the p-value for the test.
In this case, we see echoes of the “weird” negative return that we saw in the
DJIA test. (This is, more or less, an artifact. The “juice”, i.e., big returns, in
stocks appear to happen at the extremes. Tests that select events more likely
to be in the “middle” of the data (relative to high/low range), as this average
crossing test does, are likely to show a natural element of underperformance.)
Most importantly, there is no clear and strong effect here.

Caution: Sample sizes with individual stocks

I just presented a test on over 125,000 daily bars. This would seem to be a
lot of data, until you realize that stocks are very highly correlated, and many
of these events were driven by the broad market, occurred on or near the same
dates. It is easy to do tests on many individual stocks, but avoid being misled
by the sample sizes; assume that the tests are not as powerful as we might
usually assume, given the apparently (and possibly misleadingly) large
sample sizes.
A look at statistical significance
One of the problems with most tests of technical patterns is that statistical
significance is rarely reported. When a technical analyst is telling you about a
candlestick pattern or some other favorite technical tool, ask a question like
“what is the statistical significance for that pattern? Do you have a  p-value?”
You’ll be met by blank stares because a) that type of testing has not usually
been done in technical analysis and b) traditional technical analysts are not
used to thinking like this.
Nearly all data includes some degree of noise or random fluctuation, and
market data usually has a lot of noise. What we’re trying to do, with any test,
is to peer deeply into that data and maybe tease out some real effect. To be
able to do that, we must be able to sort out what might be “real” or
“significant” from what is random noise. Most traders know if they just go
into the market and buy on five days over the past year randomly, there is
some chance that they will make money on all five trades. The critical
question we need to ask is how sure can we be that these results, no matter
how good or bad they look, are something other than randomness? What if we
ust got lucky?
Traditional tests of significance look at the data, the effect sizes, the amount
of variation in the data, sample sizes, etc., and give us one answer to that
question. (Note that even this answer only works within the bounds of
probability. Stay humble, my friends. We don’t truly know anything for
certain!) This answer is often expressed as a  p-value, which is the probability
of “seeing a result at least as extreme as the one observed simply due to
random chance.” (Formally, the last part of this sentence should read “given
that the null hypothesis is true”, but, for purposes of most market analysis,
that null hypothesis is that the effect is due to randomness.)
Without that assessment, any test of a technical effect is bound to be
misleading. We don’t care that X was up Y% of the time after the event, that
stocks rallied Z% following the super-secret signal, or that buying with this
system produced any number of wins over the past ten years. Those are
examples of how technical results are often presented, but, without an
assessment of statistical significance, they aren’t terribly meaningful.

Surprise: a hidden gem

 Look again at the test table for individual stocks, specifically at the Up and
Down columns, that give the % of stocks that were up or down on those days
following the event, without regard to the magnitude of those moves. (Note
that the baseline was: up, unch, down : 50.6%, 1.02%, 48.38%.) What is
going on here? Without going into a lot of supporting detail, this effect is due
to mean reversion in stocks, which is a powerful force—almost overwhelming
over some timeframes. Individual stocks mean revert strongly. Consider the
case of a Death Cross, and the price movement required to generate that
event. Nearly always, price will be going down, pulling the short-term
average through the longer-term average; furthermore, the day of the event is
almost certain to have a lower close than the day before. Simply buying
stocks on this condition, a close lower than the day before, will result in a
small win in a test, due to the power of mean reversion in stocks. This is also
why quantitative trend following systems have trouble with individual stocks.
Perhaps think about your trading experience with individual stocks. What
happens when you “chase” entries? How many breakouts fail? How many
breakout trades are likely to snap back against you, if you enter on a strong
day in the direction of the breakout? What do the statistics from your trading
say? Though we might not have found an effect with the soundbite darlings,
the Death and Golden Crosses, we have found something important: we’ve
uncovered one of the fundamental principles of price behavior and seen it in
action, and that’s not a bad day’s work.
  A Pullback Variation

Patterns Within Patterns

One of the problems swing traders in stocks face is correlation; the sad
reality is that stocks mostly move together, and trades in stocks mostly win
and lose together. If you buy four names and short four at the same time,
there’s a pretty good chance that one set four is going to make money and one
will lose money. Anything we can do, as technical traders, to loosen the bonds
of correlation in our pattern analysis is useful.
Logically, stocks will become less correlated to the market when something
company-specific is driving changes in the stock price. In the absence of
those company-specific factors, most stocks will drift more or less with the
market. There are a handful of patterns, all tied together by urgency
(expressed in different ways), that can give an edge to a pattern playing out
with less market influence than would otherwise be expected.
One of these is a pattern that I called a “nested pullback”; the pattern occurs
when a consolidation or flag is “in force” and is moving toward a target. That
move makes a pause of a few bars (on your trading timeframe), and this pause
is the nested pullback. It is a way to incorporate multiple timeframe
influences without explicitly pulling up a higher timeframe chart; the higher
timeframe momentum is “in force”, and we are simply looking for a lower
timeframe inflection in that momentum. Schematically and conceptually, this
is the trade:
 Now, consider this idea in the chart of TWTR:

It’s easy to see that the 2nd and 3rd bars from the right of the chart may be
a pause in the breakdown that started in the high teens. Though this is a daily
chart, the price structure is a complex (two-legged) pullback on the weekly
timeframe, so this daily pause is essentially a tiny pullback within the context
of that big weekly structure.
What do you do with this information? Well, as always, the answer depends
on who you are as a trader and how you make trading decisions. Critically,
the action out of this pattern—whether it leads to clean breakdown
(confirmation of the trade) or not (possible contradiction) can give insight into
the character of the move and to the conviction behind the move.
Many traders will be attracted to the idea of using a little pattern like this to
finesse an entry into the weekly trade with tight stops. Conceptually, this is
possible, but it’s more difficult in practice. One of the biggest mistakes
discretionary traders make is using stops that are too tight, and then their
stops simply become targets in a noisy market. If you’re going to do this,
make sure you understand the tradeoffs between tight stops and probabilities.
One last point: this is an example of a pattern that is fairly “easy” for the
human trader to handle, but that is very difficult to quantify. You could write
code to describe a nested pullback, but that code would take a lot of tweaking
and refining before it worked well. (We’d have to define the initial move,
what setups are valid, how much momentum would set up the nested
pullback, the scope and location of the pullback, and we’d have to strike a
balance in all of this between precision and leaving a wide enough range that
we catch all the patterns we want—not an easy task!) This is a pattern that
puts the human skill of pattern recognition to work in a disciplined
framework, and points us toward a type of trading in which discretionary and
quantitative tools can work together.
I’ve written about this pattern before, and even have a section on my blog
dedicated to it. Why do I write about the same patterns over and over?
Because focusing on a defined subset of patterns can lead to great
understanding of complex and complicated markets. Because these patterns
work. Because they are important.
 Chapter 6

Module 6–The Failure Test

This Module introduces a pattern which I have called the failure test. This
is not a pattern I invented; in fact, Wyckoff talked about a similar concept,
using the terms of springs and upthrusts. Victor Sperendeo has called this a
2B entry, and other authors have written about this pattern—it is a simple, and
enduring aspect of market behavior.
The simplest way to understand this pattern is that the market makes an
attempt to break beyond support and resistance, and that attempt fails: it is
both a test of the support and resistance level, and a failure to break beyond
the level. This is a pattern that is often an aggresive countertrend pattern.
Even if a trader does not wish to use such an entry in an outright capacity, it is
useful to watch for these failures as they often mark at least short-term ends
of trends, and sometimes major turning points.
Next, we think about the concept of papertrading. Many writers have said
that papertrading or simulated trading (trading a theoretical account with no
real money on the line) is worthless because it does not replicate the
emotional aspects of trading. These writers correctly point out that the
emotional challenges are where many traders get stuck, but papertrading does
have a purpose. Done properly, it is another kind of statistical investigation of
the trader’s edge. One thing is fairly certain—if a trader cannot make money
in a papertrading account, she will not be able to make money in a real money
account. Papertrading is an important stage of the trader’s development.
This module also includes a look at some basic statistics we can calculate to
understand a trader’s performance, and how that performance may vary
through time. Though this is far from a complete analysis of a trading
program, we now have the tools to understand the basic quality of that trading
edge.
Last, the Module concludes with a deep look at traditional, classical
technical analysis—both its history and examples of chart patterns drawn
from the original source material. We consider both the utility and limitations
of these patterns, and how understanding the market dynamics behind the
patterns can lead to a solid understanding of market behavior.
In the upcoming pages, we also include an in-depth look at nearly a century
of market history and data for the US stock market. The trader should now
have the tools to analyze price patterns with an eye to supply/demand
imbalances, surprises, and probabilities.

Section 1: Failure Test Backtest

Testing patterns should now be second nature, as you’ve spent considerable
time working through pullbacks and the anti pattern. The failure test is a
simple and clear cut pattern, but it is not without its problems. It often sets up
in a countertrend setting that can lead to stunning, dramatic losses when the
pattern fails.
It may require significantly different psychology and trade management
skills, compared to the with-trend pullback examples first exampined. In your
work on this backtest, consider both the objective aspects of the pattern, and
how comfortable you would have been taking the trade. Keep in mind that
time in these trades will typicall be shorter, and many trades will hit profits or
stops within a few bars. It might also be a good idea to examine both longer-
term and intraday charts, whether or not you intend to trade intraday. Price
action, particulary around the current high or low of the session, can reveal
some interesting aspects of market behavior.
As in previous tests, first define the pattern as precisely as possible. Write a
short set of rules, and then turn to the charts and find examples of the
patterms. You may now also record your results and consider those results
using the basic trading stats from Unit 4 of this Module.

Section 2: First Steps in Papertrading

In this module, you will begin your first explorations of papertrading.
Conventional wisdom (correctly) notes that the most significant problems of
trading are probably psychological and emotional, and (also correctly notes)
that papertrading does nothing to replicate the emotions of actual trading—so
many writers and educators, quite wrongly, suggest that papertrading is
worthless.
 The confusion comes from not understanding what we can and should do
with papertrading. Basically, papertrading is a type of forward test of our
edge. Backtesting always looks back at historical data, but papertrading does
the same type of testing in live market data. The trader papertrading will deal
with the unknowable future at every step.
Yes, we need to always remember that a profitable papertrading record is
no guarantee of profits with real money. It is very possible to trade well in a
simulated account and to lose money when the trader “goes live”, but one
thing is pretty certain: if you can’t make money papertrading, you are unlikely
to make money in a real account!
How can we get best value from this exercise? One answer is to take it very
seriously. Anything you can do to create some emotional “charge” or
accountability around these results will help. This is where working in a
community and sharing your results can be valuable. Also, never allow
yourself to “recreate history”; there is a temptation to look at a bad decision
made a few days ago and say, “what if I had just…” Do not do this—even a
simple instance of changing a past decision will completely invalidate your
results.
The module discusses how to effectively do papertrading in detail, but a
few reminders here:

You need a system to carefully track your entries and exits. If

you’ve been doing the suggested backtests, then you already
have a system that can be used, with little or no adaptation, for
your papertrading.
Clearly define the conditions that will get you in and out of
trades. In the next module, we will drill deeper into the creating
your trading plan, but have some type of written plan for your
papertrading. This plan can always be revised, so don’t get stuck
on it—just do it!
Try to do exactly what you will do when you trade real money:
do your homework, have a sheet of potential trades, and then
note the prices where you would have been filled in your
executions. Be pessimistic in your assumptions about slippage
or fills.
Keep doing this consistently for a number of weeks. Initially,
 this exercise is just to get you in the flow of doing the work. In
coming modules, we will spend much more time creating your
plan and moving toward actual trading.

Section 3: Classical Charting Research

In this unit, we have seen many examples of classical charting techniques
and patterns. This is a good opportunity to investigate the statistical edges
behind some of those patterns, but there are some important complications
and limitations to be aware of.
First, these patterns are undeniably subjective. Though it may be possible to
define some of these patterns algorithmically, the usual application of these
patterns comes from being identified by the trader’s eye. Before beginning a
research project, you should have a clear understanding of the pattern or
patterns you wish to investigate: what do they look like and how will you
identify what is an example of the pattern and what is not?
Second, trading these patterns involves some nuance. Few classic charting
advocates would claim that every occurrence of a head and shoulders pattern
will lead to a change of trend. Rather, there are usually other qualifications
and complications and triggering price movements that will indicate if a
pattern is “in play” or not. These are other issues you should consider before
investing time and energy in a research project.
Once you have addressed these issues, write a clear set of rules that
explains how you will identify these patterns. Then, move through charts bar
by bar and identify the patters as they emerge on the chart. As before, be very
careful of any future information contaminating your results; even a glance
ahead at the chart will invalidate your test.
Keep theoretical papertrading records of trades around these patterns. You
may also consider working this research into some type of event study
structure, whether completely by hand or with an automated component for
pattern evaluation. The objective of this research project is to gain some
insight into the edge behind these patterns, and perhaps to find patterns that
might be useful in your own trading plan in the future.

Section 4: Dow Jones Industrial Average: Full

 History
The charts in this section are weekly charts of the full history of the Dow
Jones Industrial Average. They are presented here for your use and analysis.
In most cases, the charts cover roughly a decade from the “zero years” the
same year ten years later. In a few cases, you will find charts that snapshot the
middle of the decade—for the 1929 crash and the 1987 crash. This was done
to allow you to better connect subsequent price action to these distortions in
the market.
A few points to considere:
First, apply the basic tools of trends and ranges you have learned through
the course. Think about both descriptive (ability to define what is happening
right now) and predictive (ability to predict what will happen in the near
future) power of the tools. Too often, traders make naïve assumptions that do
not carry through to actual trading. Doing deep investigation of these
historical charts is a good first line of defense—understand what the tools can
do, and what they cannot do.
Second, major historical events have been marked on the charts. These fall
into two categories: market specific events, such as the 1987 crash, are noted,
and also major geo-political events. You should spend some time considering
the market’s reaction to geopolitical events. Though many of these defined
culture and thought for decades after the event, market response might not be
what you expect. No claim is made that we marked all or even the most
relevant historical events, but looking at these events should give you a good
starting point.
Third, spend some time contemplating these charts. It’s easy to glance at a
chart in a few seconds, but you hold most of the significant history of the US
stock market in your hands. If the events and market movements spark an
interest to dig deeper into some of the events marked, this will only deepen
your understanding of the market environment and historical context. This is
time well spent.
Last, consider the investment returns in the windows below each of the
decade-long charts. Though the beginning and ending points are arbitrary, you
will begin to gain some respect for the nature of long-term returns in stocks,
and for the long-period cycles in these returns.
 This chart shows the beginning of the DJIA. These early charts are
particularly important because this was the environment within which Charles
Dow formulated his early Wall Street Journal editorials, and from which most
of the principles of classical technical analysis were derived.
 1. It is not just that risk is hard to understand— some some risks may be
impossible to understand. Your risk management plan must leave room for
impossible to
the unknown and the unknowable. This is why advice from great traders
always includes a reminder to “stay humble” or to avoid hubris. There’s a lot
you don’t know in the market, but there’s also a lot you cannot possibly know.
know.
And, in markets, what you don’t know certainly can hurt
can hurt you.
2. Think about the extremes.  Understand the most extreme events that
have happened in your market, then look backward and out: look at related
markets and go back in history. What is most extreme thing that has ever
happened, in the entire recorded history of markets, in markets that might be
like yours? Once you understand this, realize that more extreme events lie in
the future. Ask the questions: What would happen if you had a position on?
How bad could it be? Then, assume that your answers vastly understate the
risks.
3. Think about the “middles” . What are the common risks you will face
in this market? What happens a few times a year that could be unsettling?
How can you prepare for and protect yourself against these events? Many
traders only focus on the extreme risks, but a lot of trading accounts die sad
deaths from a thousand cuts. Mundane risks add up, and mundane risks can
take you out of the game permanently.
4. Your trading strategy is a risk.   One of the biggest risks most
developing traders face is that they are doing something that simply doesn’t
work. How well do you know your strategy and its characteristics, and how
sure are you of those numbers? The unexamined life may not be worth living
 —Socrates was probably right—but the unexamined trading system is
certainly not worth trading!
5. You are the biggest risk.  Yes, that’s right,  you.
 you. All of your talk of
discipline, preparation, planning, all of the hours of screentime, all of the
chats with trader friends—all of that isn’t worth much if you don’t follow
through and do the right thing. If you aren’t disciplined, every moment of
every trading day, you are not a disciplined trader. The market environment is
harder than you can imagine, and it will challenge you, at times, more than
you imagined possible. Spend a lot of time thinking about the most critical
part of your trading system: you, yourself.
6. Plan for risks outside the market.  Everyone, from the institutional
scale to the individual trader, will have outside influences challenge their
market activities. Institutionally, regulatory changes and developments in
market structure can dramatically change the playing field. Your investors
will make mistakes—becoming fearful and exuberant at exactly the wrong
times. If you’re an individual investor, you will face outside financial stresses,
personal issues, health issues, etc. All of these things will have an effect on
your trading that is hard to capture in the numbers, but prudent planning will
allow you to navigate these challenges.

What’s Luck Got to Do With It?

Let’s cut straight to the punchline: the answer is, “it turns out, quite a lot.”
Here’s a look at probability and luck, and the part they play in our investing
results.
People selling investment tools or advice are, in some sense, selling an
illusion of certainty. We can use whatever words we like—consistency,
security, risk management—but what it comes down to is that humans have a
very natural desire (and need) for safety and security. We want to know what
the right actions are, and that those actions will lead to the desired results. We
think we are prepared, and that we can handle some variability and surprises
the market throws our way.
Think about it like this: let’s say we want to cook a steak. We might know
that sometimes the piece of meat will be tougher, more flavorful, or have a
little better fat marbling—so we know that the steak we cook tonight will not
be identical to the one we cooked last week. But what if we cut the meat, get
the grill to just the right temperature, season the meat perfectly, start to sear it,
and then our house gets hit by a small asteroid that wipes out our entire town?
That’s a degree of variability we, generally, are not expecting. Though the
example becomes a little silly (and the steak overcooked), surprises like this
happen in financial markets far more often than we’d like to admit.
Randomness is a problem for traders and investors. It’s a problem
objectively because it makes our results highly variable (even if we’re doing
exactly the right thing), and it’s a problem subjectively because we have very
poor intuition about randomness. Looking at any market, most of the time,
most of what you see is random fluctuation—meaningless noise. We don’t
deal well with meaningless noise; our brains are fantastic pattern recognition
machines; they find patterns with ease, even when no patterns exist. Many
pages have been written about cognitive biases in investing, but one of the
most serious is that we don’t understand randomness intuitively very well.
Consider these questions:

How many people do you need to get together in a room to have

a 50% chance that two of them share a birthday?
If you flip a perfectly fair coin a million times, is there a good
chance or a bad chance you will have 500,000 heads at the end
of that run?
If I flip that coin only 30 times, what is the chance I have 5
heads in a row, given that it’s a fair coin? You don’t have to
give me a number, just tell me is there a “pretty good chance”
or “almost no chance”.
How likely is it for someone to win a lottery twice?

You probably get the point that, unless you’ve studied probability and
statistics, your first answers to those questions are probably dramatically
wrong.
Because we don’t naturally understand randomness and variation, few
traders are prepared for the natural degree of randomness in their results., and,
yes, let’s use that dirty word: some traders get lucky and some get very
unlucky. It’s easy to show models where traders have radically different
results trading the same system with no errors (on different time periods).
Even with no mistakes, no emotions, no analytical prowess, no tweaking of
the system, results diverge, dramatically,
dramatically, due to luck.
So, that’s great, right? Some people get lucky in the market and some don’t.
If that’s all I have to say, maybe we should just go to Vegas and put it all on
black for the first roll, but hold on, there’s
there’s a better way.

Managing your luck

Now for the good news: you are not powerless against luck. People say
many different things about luck, like “luck is what happens when
opportunity meets preparation.” Well, there’s certainly some truth to that (and
value, from a motivational perspective), but that is not exactly how we are
using the word here. Luck, in this case, means random variation that you
cannot control. Luck can be good or bad, but it is, by definition, unpredictable
and uncontrollable. Even if we can’t control it, there are some things we can
do to better understand and to manage the impact it has on our trading results:
You absolutely must have an edge.  In the short run, you can get lucky and
make money doing something that has no edge, but expected value will catch
up with you. Don’t gloss over this point, because it might just be the single
most important thing we can say about trading—you have to have an edge.
You must be consistent. You must trade with discipline. Nearly everyone
who writes anything about trading says these things, but the why is important:
you must be consistent because the market is so random. You cannot change
your approach based on short-term results because those short-term results are
confounded by the level of noise in the market. In other words, you can lose
doing the right thing and make money doing the wrong thing. Too many
traders make adjustments based on evaluating a handful of trades, and this is
likely a serious (fatal) error.
error. Markets are random; you don’t have to be.
Luck matters. There’s no denying that, but so does skill and so does edge.
In fact, the more skillful you are as a trader, paradoxically, the more luck
matters. You can be successful without luck, but the wildly successful traders
(who are outliers) always have some significant component of luck. If the
overall level of investment skill in the market is rising (far from a certain
conclusion, in my opinion), then performance will converge and luck will
play a bigger part for the top performers.
If you understand the part luck plays in your results, you will realize that
emotional reactions to your results are largely inappropriate. Too many
traders ride the emotional roller coaster from euphoria to depression based on
their short-term results, and this really doesn’t make sense because you’re
letting luck (random fluctuation) jerk your emotions around.
Some people take this message, that luck, good or bad, is important and
unavoidable, as bad news. It isn’t bad news at all. You
You are still responsible for
developing your trading method and your skill as an investor. Without those
things, you won’t make it. It’s very unlikely that you can do everything right
over a long period of time and not perform adequately well. However, many
people focus on outlier-type stories as their motivation for getting into the
market: people who took a few thousand dollars to hundreds of millions,
people who make incredible returns in a short period of time, etc. If you
understand the part luck plays then you will understand that this cannot be a
goal or motivation for getting into the markets, because those outlier returns
were people who got lucky. Maybe you will too (by definition, you probably
won’t), but your focus must be on building a disciplined, core trading
methodology rather than trying to get lucky.
If this topic interests you, let me recommend a fantastic book. In The
Success Equation: Untangling Skill and Luck in Business, Sports, and
 Investing, 
 Investing,  Michael Mauboussin takes a deep look at measuring the
contribution of skill and luck to success, and draws some fascinating parallels
from sports. This is one of the most important books I’ve read in a long time,
and I’d highly recommend it to anyone who manages money or trades in the
markets.
 On Trading Styles

Discretionary and Consistent? How?

I’ve been asked a few variations of the same interesting question recently.
Here are a few examples:
The one thing I keep struggling with is finding an edge I can use in a
disciplined and consistent way. How do you define an edge? To me
an edge can’t be discretionary; an edge can only be defined by a set 
of rules. Otherwise it can’t
can’t be applied in a disciplined and consistent 
way, am I right?
Or, as another reader asked in the comments to one of my blog posts:
 As an avid reader
reader of your posts
posts and your book it strikes me how
often you mention the importance of discipline and consistency.
consistency. Does
this mean that your trading is more rule based than discretionary?
Or are some parts… more rule based and other parts… more
discretionary?
 In my opinion, consistency is only possible if you have rules, but on
the other hand it is also important to be adaptive and flexible and
this might sometimes challenge consistency.
consistency.
So, the question really comes down to something like “how can we be
discretionary and consistent at the same time?” Let’s think, first, about what
“discretionary” means.
I think many people would categorize trading approaches across a spectrum
from discretionary to systematic. A purely systematic trader has a set of rules
and simply always applies those rules; a computer could do it, probably better
than a human. On the other extreme, imagine a purely discretionary trader
who simply buys or sells whatever he wants based on his opinion or mood at
the time with no consistency (1 share or 1,000 shares; holding 1 minute or 10
years; etc.) The problem in this (flawed) classification system comes from a
misunderstanding of what it means to be a discretionary trader.
To my way of thinking, discretionary trading simply means that we allow
some degree of human intuition into the trading process. This does not mean
that trading becomes unstructured and undisciplined. On the contrary, the
application of discretion can be highly structured. Here are some ways this
might be done:

First, specify where discretion is allowed. In choosing entries

only? Sizing? Trade management? Stop location? Getting out
early? Doubling up? All of the above? Depending on your
trading methodology, some of these choices may make more
sense than others.
Have a rule that specifies what you will do if intuition and other
rules are strongly in conflict. This requires some thought
because it may be different at different points in the process.
For instance, if your intuition is to skip an entry specified by
your system, perhaps that would not be allowed, but if you are
inclined to exit a trade early,
early, that might be ok.

Note the words used here: “specify”, “have a rule”, etc. This means that
you have rules, in advance, that say what you will and will not do in the
market.
That last point is the key to consistency. We do not have to remove all
human input from the trading process. In fact, we can’t. For instance, a purely
systematic trader follows a set of rules, but where did those rules come from?
In most cases, from some type of statistical or scientifically-informed
(hopefully) research process, but, even here, things are not absolute. Research
involves decisions, and those decisions are made, at least on some level, by
humans. Why some markets and not others? Why certain parameters? Why
was this particular approach to the data explored in the first place? Why did
we choose to look at certain data sets and exclude others?
Even more important, someone (or something) is probably monitoring our
systematic guy. If market conditions change in some way we don’t
understand, his rigid set of rules may no longer work, and, at some point, he
will get be pulled out of the game. Following that event, there’s a good
chance that humans (again, using some intuition and discretion) may look at
his rule set, refine it in the context of recent data, and might send him in to
play again. Even in this most rigid systematic approach, human discretion and
intuition dance in the margins. The key is that discretion can be structured and
disciplined, just as any other input to the trading process.
I will agree that exact adherence to bullet point rules is easier than
incorporating discretion into your trading. It takes a lot of experience to use
discretionary inputs in a way that is not unduly influenced by emotion. (Note:
removing all emotion from trading should not be a goal.) Being consistent and
being disciplined simply means we follow a set of rules with consistency, and
those rules certainly can include an element of discretion.

On Reversals

The Problem with Reversals

One of the curious elements of human psychology is the tendency to fade
moves or to try to pick turning points in trends. When I started trading, this
was how I started out: looking for patterns that showed a trend was ending
and trying to take a counter-trend position. Over the years, I’ve seen many
traders focus on fading moves—a few do so successfully, but most struggle.
We must be aware of this bias, know that it colors our thinking, and work
very hard to see market action with objectivity and clarity.
Calling tops and bottoms is seductive. If you have a public platform, all you
have to do is get it right once; it’s amazing to see people consistently calling
for reversals, being wrong time after time, and yet to see other people still
following that advice. One reason this works is that we look so smart and feel
so right when we do call those tops and bottoms that it’s easy to forget the
accumulated losses from the failed trades would be very large indeed.
It’s also difficult because this is a very inconsistent game. For instance,
many stock traders at prop firms specialize in fading large moves. They will
wait for a news item to hit and for a stock to make a huge move relative to its
normal trading history. (Think about a $50 stock that normally has an ATR of
$1 making a $5 move in less than a minute. Those are the kinds of moves we
are talking about.) These traders will take a position against that move and are
prepared to add to it if the stock keeps moving against them. Down, buy
more. Down, buy more. Down more, keep buying. These traders will often
have long strings of winning weeks (or even years), but then may give back
years of profits in a single trade, and may even be taken out of the game
completely. I’ve seen it happen more than once.
There are psychological reasons behind many traders’ focus on fading, but I
think this is really driven by a deep, fundamental misunderstanding of how
markets work. Ask yourself: when a market makes a large move, a very large
move out of all proportion to its recent history, what is more likely to happen?
Is that move more likely to lead to another move in the same direction, or is it
more likely to reverse? Though our inclination is to think the market has gone
“too far” (and, perhaps, to be sorry we missed the move so we look for a
reversal), it’s far more likely for a market to make another large move in the
same direction after it has made a large move. (Mandelbrot makes this point
in The Misbehavior of Markets 
Markets   and explains why it is so.) If we do not
understand that, then we are doomed to be forever biased to reversal.
Are you automatically inclined to look for reversals? Do you see trending
charts, and immediately try to find where you can jump on board against the
move? If so, you have a bias. It may even be a bias that can be harnessed
constructively. I am not advocating blind adherence to “trend following” as a
methodology, and there are tools that can be used to find trend termination
trades. There’s room for many different styles of trading, but we need to
understand how we think, how we see, and work to manage our biases with
ruthless efficiency.
efficiency. Fail to do so, and we will
wil l fail as traders.

Don’t Be This Guy

Take a look at this picture which I took a few years ago, on a Friday
afternoon, on a New York / New Jersey ferry. After a long and stressful work
week (it was 2008), the gentleman in the photo was more than a little
inebriated (i.e., could barely stand up), probably the victim of an early happy
hour. Now, you should also know that these ferries are fast, and the winds on
the river are strong—the wind is often strong enough to blow glasses off your
face. This poor soul had urgent business that was unable to wait for the trip
across the river, so he walked to the front of the ferry, unzipped, and relieved
himself over the bow—directly into what was probably a 35 knot headwind.
Though this happened a while ago, the lesson and the aftermath made a
lasting impression (probably more so on the people who did not see it coming
and did not step out of the spray). Though few of us might commit the
Technicolor version of this error, financial commentators do it all the time, in
other ways.
I have spent some time doing a lot of reading—everything from social
media, “big” media, gurus and pundits, and paid research. It is always
interesting to see the commonalities across the group (a less kind assessment
might be “groupthink”), but one error crops up repeatedly: attempts to catch
or call a trend turn with no justification. This error can be hazardous to your
financial health, so let me share a few thoughts.

Why we are always looking for the turn

I think there are good reasons why traders are always looking for the end of
the trend. Many of us who do this are competitive and contrary in the
extreme. I joke with people that I could have a conversation like this: Me:
“Look at the pretty blue sky.” You: “Yes, that really is a pretty color of blue.”
Me (now concerned because I agree with someone else): “Well… is it really
blue? Isn’t it more blue green? And we know it’s essentially an optical
illusion anyway…”
This tendency is natural and pretty common among traders. On one hand,
it’s a very good thing—you will do your own work, be naturally distrustful of
outside opinions and cynical about information, and will work to think
critically about everything. But it’s also a weakness because it makes us
naturally inclined to see any market movement and think that the crowd is
wrong. The crowd is not always wrong; often, they are right and they are right
for a very long time.
I think this is a simple reason why so many of us are always looking for the
turn—many traders are simply wired to be contrary and to think in a contrary
way. We are different, and we want to stand apart from the crowd. For many
of us, this is a part of our personality and we must learn to manage it, and to
understand that it is the lens that can distort everything we see.

Trading lessons and psychology

 Beyond this element of personality, there are also some trading and market
related reasons why we are always looking for a turn. There’s a misguided
idea that we have to catch the turn to make money. Decades of trend
following returns (for example., the Turtles) have proven that you don’t have
to catch the turn; it’s enough to take a chunk out of the middle. There’s also a
natural inclination to be angry and distrustful of a move we missed—if we see
a long, extended, multi-month trend in which we are not participating, it’s
natural to be scornful of those who did participate and to look for reasons the
trend might be ending.
Many classical chart patterns are taught and used out of context. Any trend
will always show multiple “head and shoulders” patterns, and inexperienced
chartists will not hesitate to point these out. The problem with poorly defined
chart patterns (out of context) is that you can see anything you wish to see in
a chart—it’s always possible to justify being long, short, or flat a market, so
it’s always possible to find evidence to support whatever you want to do, at
least in the absence of clearly defined trading rules and objectives.
Another problem is that many traders use tools that are supposed to
somehow measure extremes. Overbought/oversold indicators, sentiment
indicators, ratios, bands—the problem is that these all measure the same
thing, in a different way. If I get an oversold signal from sentiment, RSI, and
some Fibonacci extension, I do not have three signals; I only have one
because the tools are so tightly correlated. This is important to understand—if
we don’t understand this (the correlation of inputs into a trading decision),
then we will have false confidence in our calls, and performance will suffer.
Better to know you don’t know, than to think you know more than t han you do.

Commentators and asymmetrical payoffs

If a trader places a trade, she makes money if the trade is profitable and
loses money if it is not. This is simple, logical, and just. However, for a
commentator (blog writer, research provider, tv personality, guru, etc.), the
payoffs are very different—the public remembers the times we are right, and
very quickly forgets the times we are wrong. The fact there even are
permabears (people who have been bearish stocks for decades) who are called
to be on TV and in the paper when the market goes down is proof of this fact.
It’s possible to run a newsletter or blog business for years making outrageous
claims that never come true such as “end of the financial world”, “the coming
crash”, “how to protect your assets from the coming seizures”, “where to bury
your gold”, etc.
The crazier and more outlandish the forecast, the better: if someone says the
S&P is going down 500 points tomorrow and he’s wrong, no one will long
remember because it was a dumb call. If, however the S&P should, for some
reason, go down 500 points, that person is, instantly and forever, the expert
who “called the crash”. In fact, if that forecast doesn’t come true but there’s
some mild decline in the next few months, creative PR can still tie the
forecast in.
Why does this matter? You can read blogs and listen to commentators, but
read with skepticism. Realize that the person writing has a reason for calling
ends of trends and turns. Your trading account, however, has a different
standard: if you lose more on your losing trades than you make on the sum of
your winners, that’s going to be a problem, in the long run.

Finding ends of trends

One way I have found to avoid the situation where I’m going against the
trend is to require some clear signal from the market that the trend might have
ended. There are specific patterns that can help: (exhaustion, climax, three
pushes, failure tests, price rejection), and then seeing the change of character
(new momentum in the other direction) to set up a pullback in the possibly-
new trend is key.
In the absence of that sequence: 1) something to break the trend and 2) new
counter-trend momentum and change of character, the best bet is to not try to
fade the trend and to wait for clear signals. And that guy back at the top of
this post? Yeah, don’t be that guy.
 Chapter 9

Module 9–Practical Trading Psychology

Psychology is not an afterthought. In the first version of the Course, each
Module had a component focused on psychology, so the topic was taught
throughout the course. One of the most significant changes in the 2017
revision was the most of the psychology-focused units were moved into the
last Module.
There are two reasons for this change: progress in the psychological tasks
of trading are more individual and each trader will progress at her own pace.
One of the most consistent pieces of feedback was that students would find
themselves working many units in the past on psychology while they
continued to forge ahead with the market pattern work.
Second, the ending of this Course really is the beginning. Putting the
psychology and inner work here highlights the importance of this material,
and should help the developing trader and experienced trader alike to better
integrate this material into their
t heir continued learning.
The readings, carefully selected from my blog posts over the years, are
more extensive for this Module and should be read carefully as they include
some hints and directions that are not covered elsewhere.

Section 1: Introspection
Introspection
Introspection is not easy. We spend much of our lives lying to ourselves;
some of this is even constructive, but with can easily either over or
underemphasize weaknesses or strengths. Looking at other people, we easily
identify people who think too much of their abilities, or can probably also
think of someone who we think should have far more confidence. Having an
accurate and balanced assessment of our strengths, weaknesses, and potential
is not normal, but that is your goal in doing this work!
First, we can define the scope of the project a little more clearly. We want
to understand our:

Skills and abilities, natural, developed, and as-yet undeveloped

 Core beliefs and principles
Motivations
Needs
Fears
Strengths and weaknesses
Directions for growth
Anything else that catches your attention as you work through
the project

There is no right or wrong way to do this, but here are some ideas that work
for other people. Feel free to adapt and expand them for your own practice
and exploration.
Sit down with a blank piece of paper and write stream of consciousness
lists of your strength and weaknesses. Spend time doing this, and come back
to it every day for a week. At the end of the week, edit and categorize the list.
Reflect on the list, and let it grow over another week or two. Make sure that
you bring your focused attention back to this list several times each day, even
if for only a few minutes.
Alternatively, make five lists (personal attributes, values, emotional
characteristics, habits, needs and desires) and work in filling
filli ng out those lists.
Though this is introspection, you may find value in talking to people who
know you well and getting feedback from others. It can be very helpful to
work with people from different settings (friends, coworkers, family
members), but use their perspectives to spark your own work and self-
reflection.
This exercise is an important part of knowing yourself. Once you have a
clearer picture of your motivations, strengths, and weaknesses, you will be
better equipped to figure out how to shape yourself into the person you want
to be.
Be honest—not brutal, not cruel—be fair and honest with yourself. The
goal of this exercise is to see clearly and accurately.
accurately.

Section 2: Understanding Beliefs

Do not begin this project until you have completed both the Belief
Inventory in the previous module, and the introspection in this module. Once
those two projects are done (and they will take considerable time to do well),
we can turn our attention to understanding our beliefs and motivations better,
and to transforming some of the limiting beliefs into beliefs that are useful
and empowering.
To do this exercise, take your work from both the introspection and belief
inventory, and focus on the beliefs. Separate them into categories based on
two criteria. First, is the limiting-empowering spectrum. For instance, a trader
might hold the belief that “money is dirty”, and it’s easy to see how that belief
could limit all of his operations in the market. An empowering example of the
same belief might be “I live in a universe of abundance.”
The second category is “how strongly you believe this.” Rank your beliefs
on a scale from “deeply held, almost certain to be true” to “very uncertain,
and essentially a guess.” You should also mark beliefs that you believe
(deliberate word choice, there) are very important and not very important to
you. Come up with a system that makes sense to you.
You will need to trust your intuition on this project, and may well need to
revisit it several times. It has only taken me a few paragraphs to explain a
profoundly important exercise. If you do this well, you will have a much
better understanding of who you are, why you are, and perhaps even have
some insights for how you might grow into the person you want to be.

Section 3: Transforming Beliefs

Now we begin the fun stuff! Here is where you transform the beliefs that
have held you back, that have separated you from your potential. Before you
start, consider that this is work that might benefit from an outside perspective.
A coach can be particularly effective as you do this work, but so can a friend,
a therapist, or a counsellor. Sometimes it’s much easier to see someone else’s
limitations and problems than it is to understand our own. The work you have
done to this point has laid a strong and solid foundation for transforming your
beliefs. You can make great progress on your own (perhaps adapting your
ournal or self-talk for the project), but don’t be afraid to avail yourself of
some outside help. Even a few sessions with someone who is used to doing
this work can be life-changing.
You have arrived at your beliefs as the total of all your learning, thinking,
and life experiences. Many of these beliefs have been reinforced time and
time again, and these beliefs form an important part of the fabric of your
reality. However, some of these beliefs are not true. Some of them are
unimportant, but some of them are harmful. If you want to change yourself— 
if you want to change the results you get from the Universe—one of the best
ways to do so is to change your beliefs.
I should say up front that the common approaches of positive affirmations
(e.g., writing down a new belief and saying it repeatedly, perhaps with strong
emotion, every day, maybe while looking in the mirror, etc.) are not usually
very effective. We hold our beliefs, for the most part, because they are
anchored in our perception of reality. If we simply try to change a belief by
force, our minds will rebel. (Take a silly example of trying to transform the
belief “I cannot walk through walls” with positive affirmation. That belief is
going to be strongly reinforced, no matter how you frame your affirmation!)
There is no one, right way to do this work. (Indeed, for some people the
positive affirmation approach may be very effective.) A framework that I have
found useful is this:
Identify where you want to be or what you want to do, and then think about
why you are not there.
Identify beliefs that have gotten in your way. These are limiting beliefs.
Think about these limiting beliefs. Where do they come from? Why do you
believe these things? What anchors, things you see or experience, reinforce
this belief and lead you to think it is true? What emotions and life experiences
are associated with this belief?
Think about what this belief is doing. Every piece of your mental
framework serves some purpose. If it’s not a useful purpose, it’s at least
intended to be useful—many things that become problems could be
constructive in another context. Think about why you believe this, and what it
is doing for you. What are you getting from this belief?
Spend a little bit of time thinking about what you might be missing? Is it
possible you are missing information or misinterpreting experiences that you
have always taken to support this belief? You know there are people in the
world who don’t believe the same things you do. Can you imagine what it
would be like to see through someone’s eyes who did not have this belief?
How would that person interpret the data you think supports the belief?
 Working with intention—with responsibility, care, and love for yourself— 
you will likely see some paths to transform your beliefs. Simply working
through this approach will cause you to see the belief from another
perspective, to be able to question its foundations. You can choose, in many
cases, to replace the belief with something that is more constructive and more
empowering. Perhaps here there is a place for some positive-affirmation-type
reminders that can help you see the world through new eyes.
This is only the very surface of this work. If it interests you, you can easily
find additional sources and materials that will carry you deeper. Change your
beliefs, and you change your world. Change your beliefs, and you change
yourself.

Section 4: Readings
From The Art and Science of Technical Analysis: Market Structure, Price
ction, and Trading Strategies by Adam Grimes, Wiley, 2012:
346-374 (trader’s mind)

On Psychology

Are Beliefs All That Matter? Thoughts on

Trading Psychology
Here’s an interesting question on a controversial topic—trading psychology.
A reader sent me a note asking:
 Hi. I hope you can help me with a question. I am a developing trader
and am having a friendly argument with some trading friends. One
 person says that trading psychology is all there is. She says that we
don’t trade the markets we trade our beliefs about the markets and
all we need to do is act with our beliefs and we will make money.
 Another person says psychology is just an excuse to sell courses and
books and that anyone who has a system can follow it. Can you give
us a final answer on this?
I know this is a good question because there’s no way I can answer it
without making someone, somewhere very angry. People have passionate
feelings on this topic, and, no, I don’t think I can give you a final answer.
What I can  do is to share some thoughts based on my experiences—my
personal experience as a trader, working with other traders, and conversations
with other traders, both developing and professional. Also, it should be said
upfront that I’m not a trained mental health professional and anything I say
here is just the observation of a layman in the field.
My personal belief is that trading psychology is often overemphasized at
the expense of actually having an edge in the market. I bet I know the books
your friend who says “we just trade our beliefs about the market” is reading,
and those books also suggest that it’s very easy to find a system that works— 
all you have to do is have a trend following and a range system and just know
when to switch back and forth between the two. In my experience, there are
some serious and deep problems with this line of thought.
First the statement “you only trade your beliefs about the market” is a non-
falsifiable statement. I could give a list of reasons why I think this statement
is not true, but then someone who believes it could say “well, that’s your
belief…” This is a statement that has almost religious overtones. Because it
holds out such promise, it inspires strong emotion and almost fanatical
defense from people who believe it is true. Also consider the statement: “it’s
all psychology. Trading psychology is all there is.” Plenty of books and
people say this, but since there is nothing anyone could do to disprove the
statement, no evidence that would make believers stop believing, a
reasonable, logical, course of action is to consider the statement is
meaningless. (If you’re not familiar with this idea: a statement must be
potentially falsifiable, work it into your thinking. It will change the way you
see the universe.)
Methodologically, there’s another perspective: it’s a tautology to say that
successful traders trade their beliefs about the market. Of course they do, and
those beliefs are well-aligned with the way markets work; that’s why these
traders are successful. It’s quite another thing to tell a developing trader you
ust have to believe something, and then trade in alignment with your beliefs
because there are plenty of market beliefs that are wrong. Successful traders
have beliefs that explain how the market really moves. I think we should
teach beginning traders how to understand how the market works rather than
focusing too much, early on, on psychology.
Now, there’s another side to this coin: there are plenty of examples of
 traders who know what to do and can’t do it. There are traders who have a
solid system and just can’t execute. There are investors who do good
research, have a good plan, and then panic when the market goes down 5%
and sell everything. It’s not enough to just know what to do; some of the most
serious challenges of trading relate to actually doing what we know is right.
There are significant overlaps between human peak performance in other
endeavors (athletics, chess, music, etc.) and trading. I personally spent some
time working with a therapist when I saw that I needed better skills and
strategies to deal with the pressures of trading. Knowing what to do isn’t
enough; it’s the doing that matters. And, to actually do, we need the
psychological strength and resilience to deal with the stresses of the markets
 —psychology is indeed important.
I apologize for the long-winded answer to your question, but there is no
definitive answer: the answer is different for traders and investors in different
stages of the journey. I do think—and I know that this is a controversial
stance—many, if not most, psychological questions can be addressed through
trading methodology. If we don’t have an edge then no psychological work
will stop the bleeding from our accounts. Maybe we’ll lose money and feel
better about it, maybe we will be happier, but we will still lose money. We
must have an edge.
It seems counterintuitive to say this: but many problems that are assumed to
be trading psychology problems can be fixed by correct trading, with an edge,
in alignment with the realities of the market.

Six Keys to Managing Trading Psychology

We live in a “quick fix” world in which everyone is looking for “hacks” to
solve and shortcut problems. The internet is filled with “you’re doing it
wrong” posts that purport to show you how to do everything from tying your
shoes to drinking orange juice correctly. For traders struggling with
psychological weaknesses—and, to be clear, this is all of us!  All traders are
vulnerable to making mistakes driven by psychological issues—there are no
shortcuts to solving psychological issues. It’s hard work, but if you don’t do it
you’re going to lose a lot of money in the market. It is work that must be
done. Here are six of the most important tools I’ve found to help you manage
your psychological issues in the marketplace.
closed off from another level of experience. There’s a way to achieve a
radically different perspective, and to bring different strengths of your
analytical tools to every situation.
This different way of being and experiencing might be called “being in the
moment.” (Be careful with the word “mindfulness”—it’s often applied to this
type of work, but it really means something else. It’s a specific set of
meditation techniques, not a way to sit at your desk and answer emails.) For
traders in financial markets, being in the moment can be a powerful
perspective: you can see with clarity. You are free from emotional baggage
and past mistakes. You are free to be your most powerful and successful self.
Achieving this state takes two things: first, you have to want to do it. You
must have a clear intent to be more in the moment. Second, it requires
attention and a gentle patience to bring your attention back to the work when
it wanders. In some ways, this is a framework for extending meditation to
your everyday life, and here are some ways to do it.

Work with intent. Too often, we are robots. I find that I focus

much better if I declare an intent—for the day, for a specific
task, or for an entire venture. Simply working with intent will
focus the power of your will, and you’ll get things done that
might seem impossible.
Use a list.  Use two lists! One of the best pieces of time
management advice is to start each day by writing a list of the
three to five things you must do that day. When something goes
on that list, you get it done, no matter what. Putting something
on the list declares a clear intent for the day. (Nice when things
are coherent, isn’t it?) You also might have a longer-term list of
projects and things you are working on that will require days,
weeks, or months to accomplish, but start with the daily list. If
you do nothing else, do this.
Create ritual and structure. There is great power in the little
rituals of everyday life. Do it when you sit at your desk. Do it
when you leave the house and when you return. Do it when you
eat—there’s power in conscious eating—take a few moments to
extend gratitude for the fish, animal, or plant that is no longer
what it was so that you can eat. This might sound touchy-feely,
but it’s also an invitation to really understand where we fall in
 the dance that is life.
Be careful of “I am ____” language.   I am afraid. I am
nervous. I am angry. No… you are not . You are feeling 
feeling  afraid,
nervous, or angry, but  you are not these emotions.
emotions. Particularly
for traders, the flood of emotions can be overwhelming. Simply
being aware of the self-talk and natural tendency to think “I am
 ____”, and then to gently contradict that—this is one way to
achieve powerful clarity and perspective.
Spend a few moments paying attention to your breathing. Do
this right now. Read this paragraph, then close your eyes and
bring all of your attention to your breathing… in… and out…
for the next three breaths you take. That’s it—very simple and it
will only take maybe half a minute to do that. Don’t alter your
breathing pattern and don’t try to think anything; just focus on
your breathing. Do this a few times a day, and you’ll begin to
interrupt the “normality” of our everyday experience.

We could go on and on with these ideas, but these will get you started. The
key concept is that we want to shake things up just enough that we can
achieve a new perspective. Simple things work. Simple things are powerful.

You Are What You Read: Try an Information

Diet
A developing trader said something to me a few years ago that has stuck in
my mind: he said that he has to be careful what websites he reads because
negativity can be dangerous; when you live in a constant state of fear, it
colors everything you see. Be careful: you are what you eat (and read and
think about!)
I think it goes even deeper. Negativity and fear change our perception
deeply. When we absorb certain kinds of information, or information placed
in emotional contexts, we create blind spots for ourselves. We are not able to
see and think rationally if we allow too much emotion, and too much
emotional contamination from others, into our thought process. Emotional
context changes how we think and how we can think.
I’d challenge you to take a moment and think about the information you
consume on a daily basis. First, major media is not perfectly unbiased.
Anyone who works in media wants to engage an audience. We (I’m including
myself here because, among other things, I’m a blogger) all know that
emotion generates clicks, shares, and reads and negative emotion resonates
much more strongly than positive. So, news stories naturally come wrapped in
an emotional wrapper with a provocative and emotional headline, and our
brains naturally accept the emotion with the facts. And what about blogs and
newsletters and sources that are intended to be biased? Some of the most
popular sites on the internet feature an incredible dose of emotion—fear and
hate will provoke and engage an audience and build a core audience very
well.
So, take a few moments today and do an information inventory. Ask
yourself what sites and sources you regularly read and how much emotion is
baked into them. (I’m not going to call out any specific sites or authors, but I
think you will be able to read between the lines.) For traders, there are a few
specific things to watch for:

Do you frequent discussion forums that are filled with

frustrated, failed traders and investors? Some of these forums
also include a healthy dose of constructive work by traders who
are either committed to figuring out the game or helping others,
but be sure you can separate out the wheat from the chaff. I’ve
been involved in these forums since they were mailing lists, and
I can say the degree of anger that comes from failed traders is
astounding. They are certain the system is rigged, and just
throw abuse at anyone and everyone in the forum like a monkey
throwing poop at the zoo. You can hang out in these forums, but
watch out for the flying poop.
Speaking of flying poop, there’s been a lot of focus lately on
fake news, especially politically-driven news on Facebook and
other social media feeds. It takes less than a minute to do some
basic fact checking, so please do this before you share and get
upset about something you read.
Do you read newsletter writers who constantly predict the end
of the financial world, World War 3, or who focus on
conspiracy theories? If so, you’re probably hurting yourself.
There are writers who have built multi-million-dollar industries
 playing on the emotions of the public. These newsletter writers
have been constantly predicting THE END for decades, and
they’ve been wrong. They will continue to be wrong, but the
harm they have done to their readers is almost beyond measure.
If you read anyone like this, consider carefully what your world
would be like without this influence.
Do you frequent blogs that repost
r epost or aggregate content like this?
Is there a constant flood of posts predicting crashes, talking
about why the system is rigged, or drawing great significance
from hidden datapoints? Is everything blamed on HFTs? Again,
this stuff generates clicks and shares, but it does not help you as
a trader and investor.

Once you’ve taken a serious look at the information you consume, then
you’re in a position to figure out what do to about it. I suggest trying an
information diet: First, cut out virtually everything you read. If you are a news
unkie, this may feel uncomfortable. You may feel blind or cut off, but give it
a chance. Avoid clicking or reading for a week, and see how your emotional
state changes.
Then you can start adding things back, a little at a time, and monitoring
your emotional state while you do so. If you find yourself sucked into hours
of clicking and sharing emotional and negative articles, first of all
congratulate the blog owner on building an engaging blog, but then step away
and rethink. You can absorb information, but you need to be careful about the
emotion that goes with it.

On Dealing with Loss

Managing your Fear of Loss

People talk about “learning how to lose” and “losing gracefully”. Truth be
told, that’s a concept I’ve always had trouble understanding. Though I know
there is some wisdom in the idea of losing gracefully, I am very competitive
in most things and a terrible loser. I hate to lose.
But trading is different. In trading, I am a great loser, and, in fact, I have
actively cultivated the skill of learning to lose. Early on, I struggled with the
same emotions any new trader faces; when I put on a trade my palms would
sweat, my heart would race, and I would find myself taking a bathroom break
every five minutes. It soon became obvious that being emotional to that
degree was overwhelming whatever rational analysis I was capable of
making, and that I simply was not going to be successful until I learned to
manage those emotions to some degree.
The breakthrough, for me, came when I understood something very
important about risk and loss. (And this is important. It might even be the
single most important thing I know about trading psychology.)
psychology.) There are risks
are risks
in trading you should be concerned about. Yes, but the risk of losing on your
next trade is not one of them. In fact, that risk, of any one trade being a loss,
is not actually a risk at all!
Think about the things that probably should concern you as a trader: if you
are trading too large (overleveraged), then you should be terrified. In fact, you
should be so scared you can’t do anything else until you reduce your position
size and are trading with an appropriate degree of risk. (Fear can be good.)
Maybe you should be afraid about tail risk, or about the impact of some
unforeseeable geopolitical risk. Perhaps you should be concerned about
breaking discipline and doing something dumb. You should definitely be
concerned that maybe you’re trading without an edge and your approach
doesn’t actually have the potential to pull profits out of the market, and a lot
of your work should be aimed at proving this fear unfounded.
In all of this, I’d argue that that last trade you just put on, whether it is in
Apple, Facebook, the euro, gold, or the crush spread—whether that trade
wins, loses, or breaks even is not something you should even be thinking
about. In fact, the best mental state is simply to not care about the outcome of
any one trade.
Why is this? Well, your trading results are the sum of your wins and losses,
and you absolutely are going
are going to have losses. Trying to avoid those losses is
very dangerous; you must accept them and welcome them. The grocery store
owner with a thriving business (i.e., his business is working—just like a trader
with an edge) does not agonize over buying inventory, or that some
percentage of his lettuce will spoil before it can be sold. This is simply a cost
of doing business, and he could have no profits without lettuce—some of the
lettuce he sells, and some of it spoils. As a trader,  your losses are your cost o
doing business and you cannot have profits without those losses . You must
learn to think over the next 20 trades, not the next single trade. You must
avoid attaching too much importance to any outcome.
When I realized this, trading changed for me. I knew it was different the
day I put on a position, asked myself what I thought would happen, and found
the answer “I really don’t care.” That’s the key: you have to be able to not
care. (This does not mean you are irresponsible. You still must do the hard
work of trading, be perfectly disciplined, and manage all of the other risks of
trading. About those things, you must care deeply.) I think this is also
something that goes beyond intellectual understanding. I can point you toward
the realization. You can read the words I write here and understand them, but
you have to accept the lesson into a deeper part of your self, and that takes
some time. If you truly understand loss, you can accept that loss, make it a
part of who you are, and use it for your success.

A Little Mental Trick…

I want to share a quick thought with you—a little trick in thinking that
made a big difference in my trading a few years ago. One of the problems
with traders is that we can be stubborn. This happens to everyone, and no one
is ever immune to it. It takes a lot of confidence to pull the trigger, and
sometimes much analysis and hard work has gone into justifying the trade.
What do we do when contradictory information emerges? Well, sometimes
the shock of seeing that the trade is wrong can cause traders to freeze, with
disastrous consequences, or sometimes a trader is just unable to process the
conflicting information.
I developed a little trick that might seem trivial, but it is very important.
Simply put, anytime to you put a trade on, assume that the trade is going to
be a loser.
loser. No matter how much analysis, how many supporting factors, or
how perfect the pattern is, assume that the trade will lose money. This creates
a profound shift in your focus because, rather possibly discounting
contradictory evidence (which can sometimes be as simple as “I just bought
and now it’s going down…”), you will be open to and will readily accept that
contradicting information.
Of course you will, because you assumed 
assumed  the trade was wrong to begin
with. When you find confirmation that the trade is right, it is almost a pleasant
surprise. Shift your thinking into this mode, and you will be much less likely
to overstay your welcome in suboptimal setups that are not working out— 
you’ll be far more likely to do the right thing, which is usually to pull the plug
on the trade and look for a better opportunity.

Wins Live in the Land of Loss

I’m sitting here at my desk looking out the window. On my desk is a
(perfectly normal and fair) coin that I had been flipping a while. Question:
can you tell me how many heads I flipped?
Now, think about that question a few moments. If you had to guess, what
would you guess? Maybe I’ve been sitting here watching Netflix for a couple
of hours flipping a coin. Maybe I flipped it while I was on the phone for just a
few minutes. Maybe I just flipped it once or twice. You probably could figure
out some reasonable estimate of how long a (mostly sane) person would sit
flipping a coin, add into it an estimate of how many times you could flip each
minute, and come up with some kind of answer. You can probably guess that
there’s some practical upper limit to how many times I flipped it, and the
probability distribution of how long I’ve been flipping probably has a lot of
weight near the origin and not much out in the tail. If this was an interview
question for a consulting firm, you could come up with an answer!
Now, here’s another question: how sure are you of that answer? How
confident can you be that you’re within 10% of the right number of coin
flips? This is the question that really matters. Ok… hold on to that thought.
Let me give you one more piece of information. If I tell you I flipped 67
tails, does that change your answer to how many heads I flipped? It should— 
now you know if 5, or 65, or 65,000 is a better guess. How confident are you
about your guess now? Much more confident, yes?
Here’s the point of this post, and it’s not a math lesson: heads and tails go
together. You can’t have one without the other. The same process that
generates the tails generates the heads. You might even say that heads and
tails “live together” or that “tails live in the land of heads”.
This is such a simple observation that it’s easy to miss, but I want you to
think of this in two ways. First, this lesson generalizes to life. I think you can
often tell how successful someone is by asking how many times they have
failed. Most big, successful businesses (and individuals) have a string of
dramatic failures behind them. In some sense, we learn from those failures,
move forward, and eventually get “lucky”.
But there’s another way to think about this—life and business are really
about the odds. Chance and luck do play a part, but you must put yourself in
those situations where chance and luck can work in your favor. The more
times you’re at bat, the better the chance you’re going to hear that crack and
everyone can just stand and watch the ball sail over the fence. One of the
most inspiring quotes I’ve ever heard—and maybe it’s as much for its gritty
realism as anything else, comes from Mark Cuban:
 It doesn’t
doesn’t matter how many times you fail. It doesn’t
doesn’t matter how
many times you almost get it right. No one is going to know or care
about your failures, and neither should you… All that matters in
business is that you get it right once. Then everyone can tell you how
lucky you are.
If you’re a trader, think about your losses in this context. How do you feel
about those losses? Do you feel shame? Fear that your next trade might be a
loser? Do you do things designed to minimize the size and chance of your
losses? (Be aware that those two things work against each other—fearful
traders often generate very small, but very certain losses. This is one way to
die as a trader.)
Or do you welcome your
welcome your losing trades?
Do you know that all of your winning trades live in the land of loss? Do
you realize, wholeheartedly, that your losses and wins are two sides of that
same coin? Do you know your losses are essential to your success? The more
winning traders you have in the course of your career, the more losing trades
you will have—winning lives in the land of losing, just like the heads and
tails on that coin I flipped.
When we understand this—and if we are sure that we are trading with an
edge—then trading becomes easy. Fear of loss simply melts away. To me, this
is a simple, but profound truth. It also just happens that this simple thing is
one of the keys of approaching trading with a professional mindset, and
becoming fully comfortable with probability, risk, and the nature of a trading
edge—all essential to your eventual success as a trader or investor.

Four Steps to Mastering Your Fear of Trading

 Every trader faces fear, sooner or later. In a perfect world, we’d be fearful
at exactly the right times and use that fear to manage our risk, but we all know
it doesn’t work like this—markets naturally seem to provoke the “wrong”
emotions at the wrong turns, and following those emotions blindly can get us
in a lot of trouble. For some traders, fear can be nearly debilitating, and this is
as true for purely systematic/quantitative traders as it is for discretionary
daytraders. Here, I’ll share a few ideas that have helped me, and many traders
I know and have worked with, to manage the emotions of trading.
Acknowledge the fear and allow yourself to experience it. There’s a
natural tendency to avoid our fear; we aren’t that much different from the
child who knows monsters lurk in the darkness under the bed, but pulls the
covers over his head to avoid looking. While this is a very effective strategy
for children dealing with monsters at bedtime, it is not so effective in most
other walks of life. The first step is to allow yourself to experience your fear,
and one of the ways to do that is simply to say, “I am feeling fear right now.”
Whatever works for you, just invite that emotion of fear to be fully present in
your experience—this also has the effect of grounding you in the “now” and
bringing you into a mindful awareness of the present moment. It just so
happens that this present moment is tinged with fear, but that is perfectly fine.
However you do it, first,
fi rst, allow yourself to fully experience the fear.
Ask if the fear is there for a reason.  Fear often serves a good purpose.
The Darwin Awards
Awards (would I date myself too much if I said I remembered the
early days of this Usenet phenomenon, when the cost of connecting was a
long-distance phone call for my Apple II modem?) are full of humans who
probably should have had more fear at critical points in their existence.
Though we usually are not going to reason fear
reason fear away—it is counterproductive
to try to rationalize most emotions away—there’s a good chance that your
fear is telling you something. Are you afraid to put on 25 long positions and
no shorts in stocks, risking 2% on each trade? Are you afraid to put on 5 large
currency pairs in the same direction against the USD? Do you have some
issues with all that naked short option premium you just sold? Good. Listen to
your fear; it’s telling you not to be stupid. On a deeper level, do you have a
nagging fear because you might be trading without an edge, are
undercapitalized, or are simply doing something that doesn’t work? Your fear
might be doing something useful, and it’s worth thinking about that before we
try to casually dismiss that fear.
 Inflate the fear to ridiculous extremes.  This might seem
counterproductive, but asking “what’s the worst that can happen?” can be a
useful step. In some cases the worst that can happen is unacceptable, and you
might need to take some steps to manage that scenario. In many more cases,
we see that the fear is unjustified, and that the possible consequences are
much smaller than the fear we are feeling. Allowing your fear to expand to
extremes, without being afraid of experiencing it, can often make a cartoon
mockery of that fear, and it will naturally release. You might even find
yourself laughing at the fearful state you were in just a few moments before;
fear has a very difficult time surviving where there’s laughter. If nothing else,
this step will help you to understand your fear on a deeper level.
Work on releasing the fear.  There are many ways to do this. In several
podcasts I talked about the Sedona Method, which is a simple, targeted
method to release unhealthy emotions. It’s so simple that you might overlook
it, but it can be powerfully effective. Give it a chance, or maybe simply take a
few deep breaths and allow the fear to dissipate. There’s no right or wrong
here—managing emotions is a personal journey and we all find what works
well for each of us.
Those few,
few, simple points can reshape your experience of fear in trading. Let
me remind you that we are not going to eliminate emotions in trading, nor
would that be a productive outcome. As humans, every decision we make
includes some mix of rational analysis and emotion. Many traders find fear is
an essential emotion in trading, but there are different kinds of fear: if you are
“white knuckling it” through trades, or experiencing a high “pucker factor”
every time you look at a chart, then you won’t be able to trade effectively— 
the emotions of trading will drown everything else out. Everything in its
place, in its time, and in the correct proportion. These simple ideas will help
you get there.

The Power of Vulnerability

“I’m looking for a portfolio that will make about 20% a year but I 
don’t want any down years.”
“Can you give me a hedging strategy with options that will protect 
me from all downside without limiting my upside and at zero cost?”
“I’m working on my trading plan and I plan to just eliminate my
 losing trades.”
Believe it or not, I didn’t make those quotes up—they are exact quotes from
questions or conversations I’ve had with either professional managers or
developing traders. They are all tied together by one thing: fear of loss.
We spend much of our lives taking reasonable precautions. We look both
ways before crossing the street. We smell fish before we cook it. We buy
insurance, hopefully after carefully considering the cost and risks. In trading,
we find ways to limit or to define our loss and we avoid taking stupid risks. It
is good that we do these things, and good that we are aware of the risks.
Healthy respect for risk is one thing, but when it becomes fear, problems
arise. Fear leads to tension and conflict; we don’t see information correctly
and clearly, and we will make mistakes. Cognitive biases and emotional
reactions seize control of our behavior, and, in the worst cases, traders are
completely unable to act and end up quitting in desperation. Attempts to
counteract this, by focusing on risk control and aspects of the trading plan,
can often work against us as they simply highlight the probability of losses.
Too many traders get stuck in this spiral.
What’s the solution? One powerful solution is to accept vulnerability. If we
know, with certainty, that losses will happen—and they will; losses are a part
of every trading methodology—then we are not afraid of those losses and
certainly are not surprised when they happen. For “scarier” things, like tail
risk, we must first accept and emotionally open ourselves to the reality of
those risks before we can turn rational analysis to understanding the tradeoffs
between magnitude and probability of these risks.
Life, existence itself, is fragile. There are no guarantees in anything—by
opening ourselves to this reality, we can understand our trading and
interaction with financial markets as part of that web, and can better
understand our place in the universe.
Something else happens here too: risk and opportunity truly do go together.
To answer all of those questions at the top of this post, we could eliminate the
risk or cost. Just get out of the market completely! But we cannot have any
upside if we eliminate all the risk. There’s always going to be a cost or a
chance of a cost or the chance of a loss with any valid approach. If we fully
accept this, it will shape our understanding of markets and may even point out
some hidden opportunities we might’ve missed if we were only focused on
the possible loss.

On Discipline

A Letter to Myself: Pattern Interrupts

A reader’s question:
 Hi. I know that most of what you talk about isn’
isn’t really
really about 
daytrading, but I am a daytrader and I have a question. Overall, I’m
not making money.
money. I’ve found that my problem is that I have a few
big losing days each month. I set a daily loss limit and am supposed
to stop when I reach the limit but I find myself getting angry and
trading bigger the closer I get to it. Yesterday I lost almost three
times my limit. How do I stop myself from hurting my trading
account? Thank you, I know I’m not the only person who does this to
myself.
Well, no—you are not the only person who does this. In fact, it’s very
common for traders on all timeframes to do some kind of “revenge trading”.
Getting this aspect of trading under control is crucial; if you don’t do it, you
aren’t going to succeed in this business. Whatever profits you make will
wil l leave
faster than you make them. Sadly, it’s not so easy as just saying “follow your
rules” and leaving it at that. It’s vitally important to avoid emotional, revenge
trading, and to avoid “going on tilt”.
People have different psychological makeups and respond to this kind of
stress differently.
differently. Some people do well being down and behind and fighting to
get back—there are even successful traders who do their best work like this,
and some of them do manage to constructively harness an impressive degree
of emotion. For others, cultivating a detached perspective is important
because any strong emotional content sends them over the edge. You must
work hard to understand yourself, your triggers, and how to deal with them.
This is a critical part of the “know yourself” aspect of trading.
For me, there were two parts to the solution: 1) realizing how stupid this is
and 2) creating pattern interrupts. Both of those were important; the first is
intellectual, but that cannot adequately and fully address the problem.
Understanding, reason, analysis—these are all great, but we are dealing with
an emotional and behavioral problem and reason has limited power. I spent a
lot of time in reflection after some of the times I went on tilt and became an
irrational trader. I finally realized that this was not how I wanted to live my
life and I couldn’t trade well if I continued to be stupid. (The image of a video
game character getting stuck in a loop and running into a brick wall was
powerful imagery for me—seeing him keep hitting it, then getting bloody,
then making a bloody splat on the wall, but his legs won’t stop—they are out
of his control and keep picking him up and running him into the wall again
and again and again, even after he starts bleeding, even after his head falls off,
even after he turns into a big, gross, bloody pulp… you get the idea. This was
the equivalent of what I was doing to myself as a trader about once a month…
and it didn’t feel good!)
Bad things happen to us often because we make them or let them happen;
we go through chains of behavior that lead to predictably bad results. After
the fact, we are driven to ask, “why would you do that to yourself? Wasn’t it
obvious?” Well, it is obvious when we look at it rationally, but in the heat of
the battle we are not rational. Our brains are flooded with flight or fight
chemicals, and we are not able to make good decisions. What you need is
some really simple tool to break that usual pattern of behavior and change it.
This is where pattern interrupts come in.
I had a friend who stopped a serious smoking habit with the classic “snap
the rubber band on the wrist” method—cliché, but it is a pattern interrupt and
it works. Consider situations outside of trading: Do you typically have a fight
with your spouse? Can you identify what leads up to it? Then do something
different: when ____ happens, go for a walk, get a glass of water, randomly
say “I love you”… do something different. How about minor road rage?
Instead of getting angry at the “idiot” who someone got in front of you, what
would happen if you made yourself take a deep breath and smile? Do you
think you might forget to get angry?
As a daytrader, my answer was to write myself a letter. The letter was to be
read when I had reached a specific loss limit, either in numbers of consecutive
losses or in actual dollars lost. Today, I realize that some small fraction of
those days I might have been down just due to bad luck (anyone can have 4
losing traders in a row), but I also know that, far more often, I was in that spot
because I was doing something stupid. I attach the letter below so you can
adapt it to your own situation. Here are the points I found useful:
 Write simply. Write to yourself as if you were a small child. Use simple
sentences and words. Be tough on yourself. Maybe not Gunny Sgt. Hartman
Full Metal Jacket tough,
t ough, but be tough. You’re
You’re reading this to yourself because
you are doing something pretty dumb and destructive and let’s just say things
as they are. Don’t sugar coat it. Include specific behavioral alternatives. Tell
yourself what to do. In my case, it was get up and go for a walk. Above all,
stop trading. Include some positive reinforcement. Remind yourself that you
don’t actually suck at trading, even though you might suck today.
So, here is my “open letter to myself: today, a loser, but tomorrow, probably
not:”
 If you are reading
reading this, it is because you are
are within ______ of your
loss limit today. Let’s slow down and consider how we got here.
There are three ways this usually happens. If you got here because
 you lost track of your risk
risk on a trade or a number
number of trades, you are
are
 just an idiot and I can’t
can’t help you. It is also possible that you just 
just 
aren’t seeing the market clearly today, for whatever reason. Think 
hard about that. If either of those things are true, maybe you should
 pack it in and just go home. Tomorr
Tomorrowow is a new day.
day.
 However,
 However, it is more likely that
that today’s
today’s market is not the kind of 
market that plays to your strengths. You have a specific skill set, and
within this realm you are probably as good as anyone out there. You
understand the ebb and flow of a trending
trending market, and how to press
press
trades in those markets to maximum advantage. You usually don’t get 
shaken out by noise—you can hold trades in this kind of market 
better than almost anyone else… and, more often than not, you catch
the turn at the end so you know when to get out. You do need to
remember that maybe 1 in 5 trading days really rewards this style of 
trading. Chances are, today is not one of those days or you wouldn’t 
be reading this right now.
What do you do now? Well, first of all take a deep breath and go for
a short walk. Now. You are not going to make a brilliant trade in the
next 15 seconds. You do not have to be here. In fact, your brain
chemistry is compromised so you aren’t quite in your right mind— 
 you cannot make good
good decisions right now.
now. There
There is a really
really good
chance the next trade you make will be your last one for the day,
day, and
then you can sit there and think about what an idiot you are for the
rest of the day. Not pleasant or productive, so go for that walk right 
now.
When you come back to the screen, the right course of action is to
chip away at this loss. Think carefully about your next trading
decision. You are going to be tempted to put on a lot of risk to try to
make a big dent in your loss, but you know that almost never works.
On a day you’re really in sync with the market, you can recover this
loss in a single trade in a few minutes, but this clearly is not that day
or you wouldn’t be reading this right now. Make a few ticks on small
size and then use that cushion to take larger risk. Make $10 at first…
$5 is even ok. Green is good. Hit singles, not home runs.
The Discipline Rule
You have to be disciplined to trade successfully; we all know that, and it’s
one of the few pieces of trading advice that might apply universally. No
matter what your trading style, market, timeframe, risk profile—whatever you
do, you have to do it consistently and with discipline. It’s easy to overlook
something we hear this often, but the reason this advice is so often repeated is
that it really is that important. However, there is one little detail that makes all
the difference—if you are going to be disciplined, you must be disciplined all
the time.
So, you make your trading rules, and you refine those rules. You monitor
how they work in real time, and you make adjustments as necessary. Your job
is now “simply” to follow those trading rules—do what the rules say always.
Again, I’d be willing to bet that every one of you reading this knows what I
ust said is important, but can you truly say that you are disciplined every
moment, in every interaction with the market?
Sometimes you will hear traders say they are “usually disciplined” or they
“almost always” follow their trading rules. Wrong. You are either disciplined
or you are not—it’s a yes or no thing. If you are disciplined 999 times out of
1,000, but you break discipline that one time, you are not a disciplined trader.
There is no halfway, usually, mostly, or almost always—either you are or you
aren’t.
This way of thinking is a good reminder because it forces us to be mindful
of the damage we can so easily do to ourselves. All it takes is one slip—one
trade held too far past the stop, one crazy addition to a losing trade, or one
impulsive entry—to undo weeks of disciplined work. Not only can the
financial impact be extreme (it’s not that hard to lose a month’s profits in five
minutes), but the emotional and behavioral impact can be devastating. Once
you’ve opened the door and done something silly, you are much more
vulnerable to future mistakes. And, if you make money doing something
stupid… well… that’s almost the worst possible thing that can happen to any
trader because you will do it again, and eventually the bill will come due.
You must be disciplined, and you must be disciplined every day, every
minute, every moment, and every interaction with the market. If you are not
always disciplined, you do not have the right to call yourself a disciplined
trader. Another simple rule, but one that can make all the difference in the
world.

Five Steps to Trading Discipline (And Why

You Should Care)
You have to be a disciplined trader. This is one of the nuggets of trading
wisdom we hear constantly, and for good reason—it’s true. Without
discipline, we will make many mistakes in the highly competitive market
environment. But it’s not as simple as saying to yourself “be disciplined” and
watching all your trading problems melt away. Discipline, for many traders, is
a battle to be fought over many years and a goal that is only achieved with
some maturity as a trader. Here are five things you can start doing, today and
right now, that will move you a little further along the path to being a
disciplined market participant.
1. Understand, intellectually, the reasons why discipline is important.
Many of the battles you will fight with discipline are emotional and
behavioral, so you cannot simply reason them away, but you are unlikely to
be successful unless you understand why it matters so much.
2. Have a plan.  This is another perennial nugget of trading wisdom that
holds great truth. By this point in the course, you should have spent a lot of
time constructing a full and detailed trading plan. Discipline, then, is “simply”
following that plan. Without the plan, it’s pretty hard to evaluate discipline.
3. Evaluate your performance regularly.  In this context, I don’t mean
whether you made money or not; I mean one simple thing: did you follow
your plan? That’s the only question that matters. If you aren’t disciplined
every moment in the market, you aren’t disciplined. Every second, every day,
every tick, every trade, every exit, every interaction with the market—you
must be disciplined. If you aren’t a disciplined trader all the time, then don’t
say you are a disciplined trader. Evaluate your ability to stick the plan
regularly, and be aware of the times you’ve fallen short. (Hint: keep a journal.
Write this stuff down!)
4. Become aware of your “tells” and behavioral triggers . We all have
things that set us off. Maybe it happens when someone cuts us off at an exit
driving. Maybe it’s the way someone looks at us or uses a certain tone of
models still use this assumption of normality as one of their core
assumptions.) However, the presence of fat tails or any other kind of
nonnormal distribution is no evidence at all against either the EMH or the
RWH.

Baseline Drift
When we look at long price histories of some assets, we observe a baseline
drift. For instance, if you were to randomly buy a large basket of stocks and
hold them over a significant time, you would likely make money: over any
extended holding period, stocks tend to go up. It would be easy to assume that
this kind of slight bias might be evidence of nonrandom price action, but it is
not. Mathematically, we could express the probability of a step up or down in
our original random walk equation like this:
 Prob up = Prob down = 0.50

But there is no reason that the probabilities in that equation have to be

equal. Making a simple change—allowing the probability of a step up to be
slightly larger than the probability of a step down—would generate a random
walk with a positive drift component.
The presence of a baseline drift in stock prices is well established, and it is
the reason behind the prevalence of index funds, which seek to capture this
drift for their investors. One of the goalposts for traders seeking to outperform
markets is that they must beat this baseline return without incurring extra risk
(volatility) compared to the baseline. It is trivial to create a system that
captures the baseline return with baseline volatility (buy and hold), and also
simple to increase both the baseline capture and the risk through leverage. It
is also not that difficult to create an active trading system that will outperform
the baseline, albeit with increased volatility and risk. In this case, investors
receive no benefit over what would be achieved through a simple leveraged
portfolio, and in many cases are worse off because they are assuming risks
that may not be well-defined or completely understood. The challenge is to
achieve superior risk-adjusted returns, which is not a simple task. Far from
disproving either the EMH or the RWH, this baseline drift is embraced by
both. This is the “above-average” that Malkiel was referring to when he said
that efficient markets “do not allow investors to earn above-average returns
without accepting above-average risks.”
Examples of Outperformance
Critics of the EMH often point to examples of funds or traders who have
outperformed the market as evidence against the theory. The academic world,
of course, is also aware that a group of traders consistently outperforming the
market would be a serious challenge to the theory, so these cases of claimed
outperformance have been carefully examined in the literature. The EMH
does not say that no one can make money trading, as is commonly assumed.
First of all, survivorship bias and the prevalence of counterintuitive runs in
random data will result in a handful of traders showing exceptional
performance purely due to chance. For instance, imagine a room with 10,000
people who will all flip a fair coin. If the coin comes up heads, the flipper is
declared a winner and can stay in the room; tails, the flipper leaves the game.
After the first flip there will be about 5,000 flippers left in the room since
roughly 50 percent will win this game of chance. If we continue to 10 flips,
there will be, give or take, 10 remaining flippers of the original 10,000, all of
whom have displayed remarkable coin flipping acumen.
Assume, for a minute, that you are presented with the coin flipping record
of one of these remaining star flippers without being told the selection process
that produced them. The probability of someone being able to flip 10 heads in
a row is less than one in a thousand, so we could reasonably assume that
someone able to do this had a remarkable skill. However, when we do
consider the selection process, that these 10 expert flippers are the remnants
of the 10,000 who began the game, we may regard the one-in-a-thousand
odds a little differently. Taken out of context, these flippers look like experts,
but they are obviously the result of random chance when the process is
considered.
We could structure a simple test to see if these flippers have actual skill. All
we would need to do is to monitor their performance going forward; if they
do not have any skill above random chance (which should be our initial
assumption), then roughly half of them should leave the game on the next flip,
and so on. If we see this group has remarkable performance, say 75 percent of
them flip heads on every flip from the time we identified the group, then we
might begin to consider that they have a real edge in this game. If they
continue to exit the game at a rate of about 50 percent per flip, then we realize
that their past flipping records, however impressive, are simply the result of a
random process.
 There are, of course, examples of traders and funds that outperform the
market, but the academic argument is that they may well be the result of a
selection process not unlike the coin flipping game. In fact, mutual funds
sometimes cultivate this process by launching a number of products with
various strategies in diverse asset classes, selecting the winners after a period
of time and then marketing those products to the public. Going forward,
however, these funds do not, as a group, show outperformance. Market
history is littered with stories of winners with extended periods of
outperformance and seemingly inexplicable track records that eventually
revert to the mean. (Value Line, as well as Bill Miller and the Legg Mason
Value Trust are two well-known examples of track records that have faltered a
bit in recent years.)
There are several possible explanations here. One that should not be
dismissed out of hand is that perhaps EMH advocates are correct in their
claims. Perhaps consistent risk-adjusted outperformance is not possible in the
long run. For traders trying to make profits in the market, this would be
disheartening, to say the least, but there are other possibilities to consider.
Market efficiency may be an evolving process, and it may be possible to
identify profit opportunities that are eroded over a period of time. (Whether a
trader can successfully identify new inefficiencies in the future is another
question entirely.) It is also possible that there are traders with impressive
track records who simply have no economic motivation to offer those records
for analysis. At any rate, we must realize that a simple example of fund or
trader outperformance (even on a risk-adjusted basis) does not present a
credible challenge to the EMH.

Inefficiencies Due to Market Microstructure or Transaction

Costs
Traders doing initial research into price patterns will observe that there
appear to be glaring inefficiencies in illiquid instruments such as inactive
futures contracts, less frequently traded currency rates, and microcap stocks.
A word of caution is needed here, because price records in these instruments
may not present realistically exploitable opportunities. Price changes in
illiquid instruments can often be nothing more than bid-ask bounce, where
prices trade back and forth between wide bid-ask spreads. Exploiting these
opportunities would require being able to reliably buy on the bid and sell on
the offer—a most unrealistic assumption.
 Another issue, especially when comparing relationships between two
markets, is that quoted prices almost always consist of the last actual trade.
Imagine two markets that are very closely related, one of which trades
actively but the other rarely trades. Examining the price records of these
markets, a trader would find long periods when prices seemed to diverge, but
this is only because current trades in the active instrument are being compared
to a single stale print in the inactive one. If the trader attempts to actually
execute on this apparent divergence, he would probably find that there is no
divergence at all because the bid-ask spread of the illiquid instrument tracks
the liquid instrument very closely. The bid-ask spread represents true value, as
any trade will execute there first, certainly not against a stale price from many
minutes or hours ago. This is not just a hypothetical example—the same issue
occurs every day in options, in back months of futures, and in stocks that
trade multiple share classes or that are listed in different international markets.
It is also important to consider whether inefficiencies are economically
significant. Volume in some illiquid instruments may be very light, for
instance, with only a few hundred shares a day trading in very thin stocks.
Even if actual inefficiencies were found after accounting for market structure
issues, it might not be possible to exploit them on any economically
meaningful scale.
There have also been cases throughout market history of one group of
participants gaining a structural or transaction costs–based advantage over the
rest of the marketplace for a period of time. The New York Stock Exchange
specialist system, floor traders in most open-outcry markets, and the SOES
bandits of the dot-com boom are obvious examples. These traders’ profits are
the result of structural imbalances in the market, and are not evidence against
efficient markets. In addition, markets have tended to evolve and erase these
inefficiencies over time.

Credible Challenges to the Theory

Before proceeding, it is important to clarify one point. There are many
people on both sides of the argument for and against the EMH; it is a very
polarizing subject, and some people in academia and in some areas of the
finance industry have built careers out of defending the theory. Our goal in
this section is not to disprove the EMH, but to understand what types of
information could potentially present a serious problem for believers in the
EMH. If markets are, in fact, less than efficient, it is possible that some of
these legitimate challenges could point the way to potential profits.

Assumptions Are Flawed

In classic EMH, the assumption is that market participants receive new
information, process it, and immediately act in a rational manner. Their
immediate, but well-considered, actions cause prices to instantly adjust to the
correct level justified by the new piece of information. Models are important
because they can help us understand reality, but it also important to consider
the limitations of those models. A model must, by definition, simplify
elements of reality. No one thinks that a model is a perfect representation of
reality, but no one is quite sure how far models can diverge from reality and
still be useful. Model building, even in the case of simple models, is a blend
of both art and science. (Think of the old joke about a group of scientists
stranded on a desert island with a case of canned food and no way to open it.
After the physicist, the chemist, and the biologist tried unsuccessfully to open
the cans, they asked the economist for help. He replied, “This one is easy. You
see, first we must assume a can opener exists. …”)
Some of the assumptions of the EMH fall well short of reality. First of all,
there is great debate about whether people act rationally. There is a growing
body of evidence that is confirming what traders have probably known since
the beginning of time—people make emotional decisions, sometimes they
make mistakes, and they do not always act in their own best interests.
Furthermore, information is not received and processed by all market
participants simultaneously, and sometimes errors are made in analysis. In the
case of breaking news, traders make immediate and emotional decisions that
can have a dramatic impact on prices. Particularly intraday, when news breaks
in a stock, the stocks of companies with similar ticker symbols (e.g., AAP and
AAPL, PCBC and PCBK) will sometimes have large moves as well. It does
not matter that the two companies are completely unrelated or are in
completely different industries. This is a clear example of traders reading the
news and simply making a mistake by executing in the wrong symbol. If
markets were truly informationally efficient, if investors always made rational
decisions, this should not happen.

Overreactions and Underreactions to News

When new information comes into a market, most EMH models assume
that the information is processed and the stock immediately moves to the
correct level justified by the news. This process would look something like
the example in Figure 10.2.

 Figure 10.2 Idealized EMH-Style Reaction to News

It rarely works like this. Prices are much more likely to overreact or
underreact to news, or even to move initially in the wrong direction entirely.
These are sometimes periods of great uncertainty, and the situation is often
exacerbated by wide spreads and low liquidity. Markets work, but the process
of price discovery is sometimes messy, noisy, and even dangerous. Market
makers and liquidity-providing traders step back, spreads widen, and large
orders have a disproportionately large impact on price. In reality, it is
exceedingly unusual for a market to move cleanly and instantly to the new
price level; the process is much more likely to look something like Figure
10.3.

 Figure 10.3 A More Realistic Reaction to News

In addition, not all market participants receive all news items at the same
time, and some certainly do their homework on a different schedule. Imagine
process. Furthermore, many of these funds have policies that prohibit them
from reacting immediately to news. Even if they wanted to execute, as a
group they may own most of the outstanding shares of the company and
positions like this cannot be unwound in seconds—they must be cut up into
smaller pieces and sold over a period of days to weeks in a manner that will
not have an extremely adverse impact on prices.
Researchers are quick to point out (quite correctly) that, while markets may
not move immediately and cleanly to new price levels, overreactions are
about as common as underreactions. This, they claim, prevents traders from
making easy, reliable profits off reactions to news events. Even if markets do
fluctuate, overreact, and underreact to new information, if these fluctuations
are random and unpredictable, market efficiency is maintained. (Note that
here is a case where the RWH could be violated and the EMH still hold.)
Based on my experience, I would say that they are correct in pointing out that
there are no easy profits to be made in this environment. Of course, it is also
possible that there are opportunities to profit from these initial mispricings of
news events, but that the academic tests have failed to capture some of the
factors that traders consider in their decisions.
This is reality. It is messy, dirty, and ugly, and it is very different from the
idealized world presented in some forms of the EMH. Some of these traders
are making emotional reactions, and many of them are actually making
trading decisions based off the price movements themselves rather than a
rational analysis of the news. In the idealized world, if $40 were the correct
price for the stock based on its future earning potential (and it would be,
because market prices always accurately reflect all available information in
this idealized world) and the news item now justified a drop to $30, this
adjustment to $30 would happen instantly with no intervening prices. But this
is simply not the way the world works. There would be tremendous volatility
around the announcement, and the impact of large funds adjusting their
positions (some of whom will be selling and some will be buying because
they will find the stock trading at a discount to the fundamental value justified
by their models) will be felt for days or weeks afterward.

Inside Information
Strong form EMH says that all information, even secret inside information,
is priced, so it is not possible to make a profit trading off that information.
Obviously, in the case of the aforementioned drug stock, a group of people
knew the drug approval would be denied before the news was released to the
public. Those people had a chance to initiate short positions at $40 and could
have taken profits as the news hit the wires. For strong form EMH to be valid
there would have to be some mystical adjustment mechanism that moves
price to the correct level the instant the FDA committee votes to deny the
drug approval, before the information hits the newswires, even before the
company knows the news, and even before the news leaves the room. (We
should be suspicious of market explanations and models that require a deus ex
machina. Buying and selling move prices—only those and nothing else.) It is
also worth considering that regulatory agencies, at least in the United States,
spend an enormous amount of time, energy, and resources prosecuting traders
who trade on inside information. Why would this be so if there were no
money to be made on that information? Obviously, there are some serious
logical problems here, so strong form EMH has few adherents today, even in
academia.
Long-Term Mispricings: Booms, Bubbles, and Busts
Some forms of the EMH could allow for short-term mispricings due to
market microstructure issues, but there is also evidence that markets may stay
mispriced for extended periods of time. In an efficient market, such
mispricings would represent a profit opportunity for rational investors, and
the pool of educated, rational arbitrage money should be able to quickly
overcome any irrationality. This does not seem to be the case. Market history
has many examples of markets that reached irrational bubble valuations and
then deflated with astonishing speed. The financial crisis of 2007, crude oil at
$150 in 2008, Nasdaq stocks in the dot-com bubble, the collapse of Long-
Term Capital Management in 1998, the Asian financial crisis of 1997, and the
Hunt Brothers’ impact on the silver market in 1980 are recent examples, but
the South Sea Bubble of 1720 and the Dutch Tulip Mania of 1634 suggest that
this has been going on for a long time. Figure 10.4 shows a chart that could be
the chart of a high-flying tech stock from the dot-com bubble of 2000, but it
actually is the price history of shares of the South Sea Company from January
1719 to mid-1721. In fact, we even have evidence from ancient Babylon that
there were commodity bubbles in antiquity—it seems likely that, far from
being an aberration, bubbles and busts are natural features of markets. There
is nothing new under the sun. In these bubbles, valuations typically expand to
hundreds or thousands of percentages of fair, justifiable valuations over a
period of months or years. When the bubble pops, as it always does, there is
the inevitable bust as the market quickly recalibrates. It is difficult to
reconcile this behavior with the idea that investors always make rational
decisions. At the very least, it seems like something else is going on here.
  Figure
 Figure 10.4 Stock Price of the South
South Sea Company

Cost of Information
Semistrong form EMH states that no one can earn above-average profits
based on analysis of any publicly available information. Therefore, there is no
value in doing any fundamental analysis or, indeed, any analysis at all,
because all information is already reflected in the price. Again, we run into a
logical absurdity in the limit. If the theoretical construct were true, then no
one would do any analysis, so how would the information be reflected in
price? In fact, there would be no motivation for anyone to ever do any
analysis of any information, and, eventually, no reason to ever trade. Markets
would collapse and cease to exist (Grossman and Stiglitz 1980). However, in
the real world, market-related information is quite expensive. Institutions,
traders, and analysts pay a lot for information, and they pay even more to get
it quickly.
quickly. Why would they do that, and continue to do so decade after decade,
if this information were worthless?

Autocorrelation
Autocorrelation in Returns
In a random walk, the probability of the next step being up is usually very
close to 50 percent (plus or minus any drift component), and that probability
does not change based on anything that has happened in the past. In other
words, if the probability of a step up is 50 percent, after three steps up the
probability of the fourth being up is still 50 percent, exactly as it would be for
flips of a fair coin. Under the assumptions of random walks, price has no
memory and each step is completely independent of previous steps.
Autocorrelation is the statistical term that basically deals with trends in data.
If a series of price changes displays high autocorrelation, a step up is more
likely to follow a previous step up. Though this is a simplification that is not
always true, prices tend to trend in an autocorrelated price series.
Many academic studies have found no autocorrelation in returns, but the
evidence is mixed, with other studies finding strong autocorrelations. The
question seems to depend heavily on the sample being examined, as different
time frames or time periods of the same market may exhibit radically different
qualities. It is worth mentioning that this is consistent with what I have come
to expect as an experienced trader in a wide range of markets and time
frames. One of the major themes of this book is that all market action exists in
one of two broad states: range expansion or mean reversion, or, to use terms
that might be more familiar, trends and trading ranges. In a trending
environment, over any given time period, the next price change is more likely
to be upward if the preceding price change was also upward, or downward if
the preceding change was downward. In a trading range environment,
negative autocorrelation is more likely—the following price change is more
likely to reverse the previous price change. If these two environments are
aggregated, the end result is that the autocorrelations essentially
counterbalance each other, and the overall sample shows near-zero
autocorrelation.
I would suggest that one of the potential problems with the academic
studies regarding autocorrelation is that they begin with the assumption that
prices move randomly, so perhaps not enough effort has been made to find
patterns that could differentiate, a priori, between these discrete regimes. Note
that some writers supporting technical analysis have made the suggestion that
trending and trading range environments should be analyzed separately. This
is a mental exercise on par with dividing a set of random numbers into those
greater and less than zero, and then being surprised that the former set has a
higher mean than the latter! For the analysis to be useful, it must be possible
to define a set of conditions that will have some predictive value for
autocorrelations over a finite time horizon. This is actually one of the core
tasks of competent discretionary trading—to identify the most likely
emerging volatility environment. One of the major focuses of this book is to
identify conditions and factors that will help a discretionary trader make just
such a distinction.
Volatility Clustering
Real market prices show at least one very serious departure from random
walk conditions. A random walk has no memory of what has happened in the
past, and future steps are completely independent of past steps. However, we
observe something very different in the actual data—large price changes are
much more likely to be followed by more large changes, and small changes
are more likely to follow small changes. For practical purposes, what is
probably happening is that markets respond to new information with large
price movements, and these high-volatility environments tend to last for a
while after the initial shock. This is referred to in the literature as the
persistence of volatility shocks and gives rise to the phenomenon of volatility
clustering. Figure 10.5 shows the absolute value of the standard deviations of
daily changes for several years of daily returns in the S&P 500 Cash index.
Note that the circled areas, which highlight clusters of high-volatility days,
are not dispersed through the data set randomly—they tend to cluster in
specific spots and time periods.

 Figure
 Figure 10.5 Absolute Values
Values of Standard
Standard Deviation of Returns, S&P 500
 Index, Mid-2008 through
through 2010 (Values
(Values < 2.0 Filtered
Filtered from
from This Chart)
What we see here is autocorrelation of volatility. Price changes themselves
may still be random and unpredictable, but we can make some predictions
about the magnitude (absolute value) of the next price change based on recent
changes. Though this type of price action is a severe violation of random walk
models (which, by definition, have no memory of previous steps), do not
assume that it is an opportunity for easy profits. We have seen practical
implications of an autocorrelated volatility environment elsewhere in this
book—for instance, in the tendency for large directional moves to follow
other large price movement—but it is worth mentioning here that there are
academic models that capture this element of market behavior quite well.
Autoregressive conditional heteroskedasticity (ARCH), generalized ARCH
(GARCH), and exponential GARCH (EGARCH) are time series models that
allow us to deal with the issue of varying levels of volatility across different
time periods. A simple random walk model has no memory of the past, but
ARCH-family models are aware of recent volatility conditions. Though not
strictly correct, a good way to think of these models is that they model price
paths that are a combination of a random walk with another component added
in. This other component is a series of error components (also called
residuals) that are themselves randomly generated, but with a process that sets
the volatility of the residuals based on recent history. The assumption is that
information comes to the market in a random fashion with unpredictable
timing, and that these information shocks decay with time. The effect is not
unlike throwing a large stone in a pond and watching the waves slowly decay
in size. If this topic interests you, Campell, Lo, and MacKinlay (1996) and
Tsay (2005) are standard references.

Behavioral Perspectives
Strictly speaking, it is not possible to disprove either the EMH or the RWH,
but sufficient empirical evidence has accumulated to challenge some of the
core beliefs. Strong form EMH, in particular, exists today more as a vestigial
intellectual curiosity than as a serious theory. In addition, the price
adjustments under other forms of the EMH cannot occur through some
magical, unexplained mechanism; they must be the result of buying and
selling decisions made by traders. There will necessarily be temporary and
intermediate mispricings as prices move from the preevent to the postevent
levels, and there may be some degree of emotion (irrationality) along the way.
New theories have evolved in an effort to better explain the conditions we
observe in the real world, and many of those focus on investors’ behavior.

Markets Are Mostly, but Not Always, Efficient

Many of the potential issues with and challenges to the EMH essentially go
away if we allow for some form of intermittent efficiency. Perhaps markets
are informationally efficient most of the time, interspersed with periods where
prices diverge from fundamental values. This model sacrifices one of the key
advantages of the EMH—consistency (“Markets are efficient and market
prices are the correct prices. Always.”)—but the trade-off could be a model
that more closely fits the real world. Temporary mispricings such as bubbles
and crashes could be embraced by a theory that allows intermittent efficiency,
efficiency,
but work would need to be done to understand the mechanism by which
markets enter and leave the state of efficiency. It seems likely that some
understanding of the participants’ emotional reactions and behavioral issues
would be necessary to understand that mechanism, and that the actions of
rational traders would ultimately be the driving force behind a return to
efficiency and equilibrium. It is also worth considering that, if these models
are correct, there could be profits for traders who can identify and act on
inefficiencies. These traders could well be the process through which markets
become efficient, and they could be compensated for their efforts and risk
through superior risk-adjusted profits.

Reflexivity
Reflexivity is a term coined by George Soros to describe the process where
asset prices have an impact on fundamentals and fair valuations. Most
theories suggest that prices are a reflection of actual value, but Soros (1994)
suggests that the causative link runs both ways. In
I n his own words:
The generally accepted theory is that financial markets tend towards
equilibrium, and on the whole, discount the future correctly. I 
operate using a different theory, according to which financial
markets cannot possibly discount the future correctly because they
do not merely discount the future; they help to shape it [emphasis
mine]. In certain circumstances, financial markets can affect the so-
called fundamentals which they are supposed to reflect. When that 
happens, markets enter into a state of dynamic disequilibrium and
behave quite differently from what would be considered normal by
the theory of efficient markets. Such boom/bust sequences do not 
arise very often, but when they do, they can be very disruptive,
exactly because they affect the fundamentals of the economy.
Essentially, Soros’s argument is that the traditional causative model in
which asset prices correctly reflect fundamentals is flawed because it assumes
a one-way link between fundamentals and prices. There are times when
markets become so irrational and so mispriced that market participants’ biases
biases
and impressions actually change the fundamentals that are supposed to be
driving valuations.
The academic world has taken little note of this theory, but it does seem to
describe many elements that we observe in boom-and-bust cycles. Perhaps the
clearest example of this concept would be to imagine a wildly changing
currency rate. Exchange rates are supposed to reflect fundamentals, but it is
not terribly difficult to imagine an unbalanced situation where the rates
actually become so extreme that they have an effect on the fundamentals of
each country’s economy. Similarly, how hard is it to imagine an extreme price
for a commodity affecting fundamentals of production, transportation, or
marketing for that commodity? Do farmers
f armers not make planting decisions based
on grain prices? This certainly is not the normal mode in which markets
operate, but these types of extremes occur several times each decade. Even if
we do not wish to wholeheartedly embrace the theory of reflexivity, it is
worthwhile to spend some time considering the concept that prices may
influence fundamentals, that the observer may affect the outcome of the
experiment. Perhaps far more complex relationships exist in financial markets
than classic economic models suggest.

Market Efficiency Is an Evolving Process

Victor Niederhoffer (1998) offers the intriguing idea that markets may be
viewed as ecosystems, with each participant playing some role in the food
chain. In nature, there is an ongoing arms race between predators and prey:
ecosystems rest on a large base of prey animals, which evolve new forms of
deception and camouflage over time. In response, predators also become
more efficient over time. Both predators and prey depend on each other in a
complex network of relationships, and neither side can truly win the arms
race. If predators win the battle, then they eventually die because there are no
more prey animals. If prey animals evolve to the point where none of them
are eaten, predators first die from starvation, but prey animals quickly follow
as they overpopulate and consume all available food. The process of
evolution is not an option. Ecosystems are locked into these processes, but
they must, by definition, end in stalemate.
What happens if we replace the ecosystem with the market, prey animals
with small individual traders, and predators with market makers and HFT
algos? There is also an arms race here, and many parallels exist between the
deceptive practices animals evolve in nature and the behavior of market
participants. Technology
Technology evolves, the marketplace itself evolves, but there has
always been a hierarchy in which some traders are predators and some are
prey.
Under these assumptions, market efficiency is not a static state, but an
evolving process itself. There are many examples from market history of large
groups of traders making outsized profits. These profits persist for a period of
time, but then something changes in the marketplace and they are unable to
adapt. Two glaring examples are floor traders on commodity exchanges and
stock daytraders in the latter part of the 1990s. Floor traders had access to
order flow and the ability to execute orders for their own accounts much
faster than the public could. A floor trader could literally trade 20 times in the
amount of time it took a member of the public to place an order and get
confirmation of the fill. In addition, floor traders’ transaction costs were
extremely low, enabling them to profit from very small moves in the market.
Eventually, the major commodity exchanges moved away from the floor
structure to an electronic market, and the vast majority of these traders were
unable to make profits under the new paradigm.
In the case of some stock traders, there were specific inefficiencies that also
allowed them access to order flow in the form of visible, large orders in the
early electronic trading systems. This group also had the ability to execute at
much higher speeds and for much lower costs than the general public, and, as
a group, these traders were wildly profitable for several years. Over time,
markets evolved. New execution technology for large orders made it
impossible for this group of day traders to consistently identify large orders
and to trade in front of those orders. At the same time, the general public
began to access the market electronically through low-cost direct-access
brokers, also eroding the day traders’ informational
informational and cost advantages. Last,
a new predator evolved in this ecosystem as high-speed electronic trading
systems were developed to target the order flow generated by this group of
day traders. The predators had become prey.
In both of these examples, there was a group of traders who were able to
pull consistent, outsized profits out of the market for a number of years (or
decades). Their advantage was based on access to information, execution
speed, and lower transaction costs—taken together these things are the very
definition of a market inefficiency. In the end, markets changed and evolved,
and these traders, as a group, were no longer able to profit.

New Directions: The Adaptive Markets Hypothesis

One of the most exciting ideas from academic research in the past decade is
the adaptive markets hypothesis (AMH), from Andrew Lo (2005) at MIT. As
he says,
[T]his new frame-work is based on some well-known principles of 
evolutionary biology—competition, mutation, reprodu
reproduction,
ction, and
natural selection—and I argue that the impact of these
t hese forces on
financial institutions and market participants determines the
efficiency of markets and the waxing and waning of investment 
 products,
 products, businesses, industries
industries and, ultimately,
ultimately, institutional and
individual fortunes. In this paradigm, the EMH may be viewed as the
“frictionless” ideal that would exist if there
there were no capital market 
imperfections such as transactions costs, taxes, institutional
rigidities, and limits to the cognitive and reasoning
reasoning abilities of 
market participants. However, in the presence of such real-world
imperfections, the laws of natural selection, or, more appropriately,
“survival of the richest,” determine the evolution of markets and
institutions… . Although the AMH is still primarily a qualitative and
descriptive framework, it yields some surprisingly concrete
concrete insights.
The EMH and the AMH begin with the same assumption: that individuals
always intend to act in their own self-interest, after careful consideration of
the probabilities, payoffs, and marginal utilities attached to all possible
scenarios. However, where EMH postulates consistently rational actions,
AMH acknowledges that traders and investors make mistakes and learn from
those mistakes. Over a period of time, specific groups of market participants
grow and evolve, while other groups are unable to adapt and eventually
perish. The market environment also evolves as these groups wax and wane in in
influence, and this process of evolution drives market dynamics. Under the
EMH, market prices reflect all available information, but the AMH points out
that prices reflect all available information and the specific ecology of the
market environment at that time.
The efficient markets hypothesis poses some serious questions that traders
must consider. Even if the theory is not entirely correct, traders need to
acknowledge that markets are usually informationally efficient. New
directions in academic thinking about markets, including concepts of evolving
efficiency, reflexivity, and Lo’s adaptive markets hypothesis, suggest that the
academic world is beginning to acknowledge that there may be legitimate
opportunities for traders to profit. Markets usually trade with a high degree of
randomness, and price action usually approximates some form of a random
walk quite well. I suggest that the trader’s job is to find those very few times
when markets are something less than random, when there is an imbalance of
buying and selling pressure, and when there is some degree of inefficiency.
This is the first task of trading and is the point of all of our quantitative
thinking about markets.

Summary
Much modern academic research suggests that markets are efficient,
meaning that new information is immediately reflected in asset prices and that
price movements are almost completely random. If this were true, then it
would not be possible to make consistent risk-adjusted returns in excess of the
baseline drift in any market. However, many studies and much empirical
evidence exists that challenge the EMH, and new directions in academic
thinking leave open the possibility that some skilled traders may be able to
profit in some markets. Trading is not easy, for markets are very close to
being efficient and price movements have a large random component.
However, with skilled analysis and perfect discipline, traders can limit their
involvement to those rare points where an imbalance in the market presents a
profitable trading opportunity. The final two chapters of this book look at
some tools for that analysis and lay out some specific market patterns that
offer verifiable edges to short-term traders.
 Chapter 11

Tools for Quantitative Analysis of

Market Data
 If someone separated the artart of counting and measuring and weighing
weighing
from all the other arts, what was left of each of the others would be,
so to speak, insignificant.
-Plato

M
any years ago I was struggling with trying to adapt to a new
market and a new time frame. I had opened a brokerage account
with a friend of mine who was a former floor trader on the
Chicago Board of Trade (CBOT), and we spent many hours on the phone
discussing philosophy, life, and markets. Doug once said to me, “You know
what your problem is? The way you’re trying to trade … markets don’t move
like that.” I said yes, I could see how that could be a problem, and then I
asked the critical question: “How do markets move?” After a few seconds’
reflection, he replied, “I guess I don’t know, but not like that”—and that was
that. I returned to this question often over the next decade—in many ways it
shaped my entire thought process and approach to trading. Everything became
part of the quest to answer the all-important question: how do markets really
move?
Many old-school traders have a deep-seated distrust of quantitative
techniques, probably feeling that numbers and analysis can never replace
hard-won intuition. There is certainly some truth to that, but one of the major
themes of this book is how traders can use a rigorous statistical framework to
support the growth of real market intuition. Quantitative techniques allow us
to peer deeply into the data and to see relationships and factors that might
otherwise escape our notice. These are powerful techniques that can give us
profound insight into the inner workings of markets. However, in the wrong
hands or with the wrong application, they may do more harm than good.
Statistical techniques can give misleading answers, and sometimes can create
apparent relationships where none exist. I will try to point out some of the
limitations and pitfalls of these tools as we go along, but do not accept
anything at face value.

Some Market Math

It may be difficult to get excited about a chapter about market math, but
these are the tools of the trade. If you want to compete, you must have the
right skills and the right tools; for traders, these core skills involve a deep
understanding of probabilities and market structure. Though some traders do
develop this sense through the school of hard knocks, a much more direct
path is through quantifying and understanding the relevant probabilities. It is
impossible to do this without the right tools.
Many trading books have attempted an encyclopedic presentation of
mathematical techniques, but I see little value in this. It is not a good use of
your time to read a paragraph on harmonic averages in this section, never to
see the term again. In fact, this may do more harm than good, as most
mathematical techniques will give misleading answers if they are incorrectly
applied. I have attempted a fairly in-depth examination of a few simple
techniques that most traders will find to be immediately useful. If you are
interested, please explore these further through some of the resources in the
bibliography. For some readers, much of what follows may be review, but
even these readers may see some of these concepts in a slightly different light.

The Need to Standardize

It’s pop-quiz time. Here are two numbers, both of which are changes in the
prices of two assets. Which is bigger: 675 or 0.01603? As you probably
guessed, it’s a trick question, and the best answer is: “What do you mean by
bigger?” There are tens of thousands of assets trading around the world, at
price levels ranging from fractions of pennies to many millions of dollars, so
it is not possible to compare changes across different assets if we consider
only the nominal amount of the
t he change. Changes must at least be standardized
for price levels; one common solution is to use %ages to adjust for the
starting price difference between each asset.
When we do research on price patterns, the first task is usually to convert
the raw prices into a return series, which may be a simple %age return
calculated according to this formula:
 In academic research, it is more common to use logarithmic returns, which
are also equivalent to continuously compounded returns.

For our purposes, we can treat the two more or less interchangeably. In
most of our work, the returns we deal with are very small; %age and log
returns are very close for small values, but become significantly different as
the sizes of the returns increase. For instance, a 1 % simple return = 0.995 %
log return, but a 10 % simple return = 40.5 % continuously compounded
return. Academic work tends to favor log returns because they have some
mathematical qualities that make them a bit easier to work with in many
contexts, but, for the sake of clarity and familiarity, I favor simple %ages in
this book. It is also worth mentioning that %ages are not additive. In other
words, a $10 loss followed by a $10 gain is a net zero change, but a 10 % loss
followed by a 10 % gain is still a net loss. (However, logarithmic returns are
additive, which is one reason why researchers prefer to use them over simple
%age returns.)

Standardizing for Volatility

Using %ages is obviously preferable to using raw changes, but even %
returns (from this point forward, simply “returns”) may not tell the whole
story. Some assets tend to move more, on average, than others. For instance, it
is possible to have two currency rates, one of which moves an average of 0.5
% a day, while the other moves 2.0 % on an average day. Imagine they both
move 1.5 % in the same trading session. For the first currency, this is a very
large move, three times its average daily change. For the second, this is
actually a small move, slightly under the average. This method of average
returns is not a commonly used method of measuring volatility because there
are better tools, but this is a simple concept that shows the basic ideas behind
many of the other measures. Note that if you are going to use average %
returns as a volatility measure, you must calculate the average of the absolute
values of returns so that positive and negative values do not cancel each other
out.

verage True Range

 One of the most common ways traders measure volatility is in terms of the
average range or  Average True Range  (ATR) of a trading session, bar, or
candle on a chart. Range is a simple calculation: subtract the low of the bar
(or candle) from the high to find the total range of prices covered in the
trading session. The true range is the same as range if the previous bar’s close
is within the range of the current bar. However, suppose there is a gap
between the previous close and the high or low of the current bar; if the
previous close is higher than the current bar’s high or lower than the current
bar’s low, that gap is added to the range calculation—true range is simply the
range plus any gap from the previous close. The logic behind this is that even
though the space shows as a gap on a chart, an investor holding over that
period would have been exposed to the price change; the market did actually
trade through those prices. Either of these values may be averaged to get
average range or ATR for the asset. The choice of average length is, to some
extent, a personal choice and depends on the goals of the analysis, but for
most purposes, values between 20 and 90 are probably most useful.
To standardize for volatility, we could express each day’s change as a %age
of the ATR. For instance, if a stock has a 1.0 % change and normally trades
with an ATR of 2.0 %, we can say that the day’s change was a 0.5 ATR%
move. We could create a series of ATR% measures for each day and average
them (more properly, average their absolute values) to create an average
ATR% measure. However, there is a logical inconsistency because we are
comparing close-to-close changes to a measure based primarily on the range
of the session, which is derived from the high and the low. In some sense, this
may be comparing apples to oranges.

 Historical Volatility
 Historical volatility (which may also be called either statistical or realized
volatility) is a good alternative for most assets, and has the added advantage
that it is a measure that may be useful for options traders. Historical volatility
(Hvol) is an important calculation. For daily returns:

where p = price, t = this time period, t – 1 = previous time period, and the
standard deviation is calculated over a specific window of time.
Annualization factor is the square root of the ratio of the time period being
measured to a year. The equation above was specifically for daily data and
there are 252 trading days in a year, so the correct annualization factor is the
square root of 252. For weekly and month data, the annualization factors are
the square roots of 52 and 12, respectively.
For instance, using a 20-period standard deviation will give a 20-bar Hvol.
Conceptually, historical volatility is an annualized one standard deviation
move for the asset based on the current volatility. If an asset is trading with a
25 % historical volatility, we could expect to see prices within +/–25 % of
today’s price one year from now, if returns follow a zero-mean normal
distribution (which they do not.)

Standard Deviation Spike Tool

The standardized measure I use most commonly in my day-to-day work is
measuring each day’s return as a standard deviation of the past 20 days’
returns. I call this a standard deviation spike, and use it both visually on
charts and in quantitative screens. Here is the four-step procedure for creating
this measure:
1. Calculate returns for the price series.
2. Calculate the 20-day standard deviation of the returns.
3. Spike = Today’s return ÷ Yesterday’s BaseVariation.
Keep in mind that this tool does have the term standard deviation  in its
name, but, because it compares today’s move to a very small window,
extremely large standard deviation moves are not uncommon. If you have a
background in statistics, you will find that your intuitions about standard
deviations do not apply here, and the purpose of the tool is not to use standard
deviations in any traditional concept. Even large, stable stocks will have a few
five or six standard deviation moves by this measure in a single year, which
would be essentially impossible if these were true standard deviations. The
value is in being able to standardize price changes for easy comparison across
different assets.
The way I use this tool is mainly to quantify surprise moves. Anything over
about 2.5 or 3.0 would stand out visually as a large move on a price chart.
After spending many years experimenting with many other volatility-adjusted
measures and algorithms, this is the one that I have settled on in my day-to-
day work. Note also that implied volatility usually tends to track 20-day
historical volatility fairly well in most options markets. Therefore, a move
that surprises this indicator will also usually surprise the options market
unless there has been a ramp-up of implied volatility before an anticipated
event. Options traders may find some utility in this tool as part of their daily
analytical process as well.

Some Useful Statistical Measures

The market works in the language of probability and chance; though we
deal with single outcomes, they are really meaningful only across large
sample sizes. An entire branch of mathematics has been developed to deal
with many of these problems, and the language of statistics contains powerful
tools to summarize data and to understand hidden relationships. Here are a
few that you will find useful.

Probability Distributions
Information on virtually any subject can be collected and quantified in a
numerical format, but one of the major challenges is how to present that
information in a meaningful and easy-to-comprehend format. There is always
an unavoidable trade-off: any summarization will lose details that may be
important, but it becomes easier to gain a sense of the data set as a whole. The
challenge is to strike a balance between preserving an appropriate level of
detail while creating a useful summary. Imagine, for instance, collecting the
ages of every person in a large city. If we simply printed the list out and
loaded it onto a tractor trailer, it would be difficult to say anything much more
meaningful than “That’s a lot of numbers you have there.” It is the job of
descriptive statistics to say things about groups of numbers that give us some
more insight. To do this successfully, we must organize and strip the data set
down to its important elements.
One very useful tool is the histogram chart. To create a histogram, we take
the raw data and sort it into categories (bins) that are evenly spaced
throughout the data set. The more bins we use, the finer the resolution, but the
choice of how many bins to use usually depends on what we are trying to
illustrate. Figure 11.1 shows histograms for the daily % changes of LULU, a
volatile momentum stock, and SPY, the S&P 500 index. As traders, one of the
key things we are interested in is the number of events near the edges of the
distribution, in the tails, because they represent both exceptional opportunities
and risks. The histogram charts show that that the distribution for LULU has a
much wider spread, with many more days showing large upward and
downward price changes than SPY. To a trader, this suggests that LULU
might be much more volatile, a much crazier stock to trade.

 Figure 11.1 Return Distributions for LULU and SPY, June 1, 2009, to
 December 31, 2010

The Normal Distribution

Most people are familiar, at least in passing, with a special distribution
called the normal distribution  or, less formally, the bell curve. This
distribution is used in many applications for a few good reasons. First, it
really does describe an astonishingly wide range of phenomena in the natural
world, from physics to astronomy to sociology. If we collect data on people’s
heights, speeds of water currents, or income distributions in neighborhoods,
there is a very good chance that the normal distribution will describe that data
well. Second, it has some qualities that make it very easy to use in simulations
and models. This is a double-edged sword because it is so easy to use that we
are tempted to use it in places where it might not apply so well. Last, it is used
in the field of statistics because of something called the central limit  theorem,
which is slightly outside the scope of this book. (If you’re wondering, the
central limit theorem says that the means of random samples from populations
with any distribution will tend to follow the normal distribution, providing the
population has a finite mean and variance. Most of the assumptions of
inferential statistics rest on this concept. If you are interested in digging
deeper, see Snedecor and Cochran’s Statistical Methods [1989].)
Figure 11.2 shows several different normal distribution curves, all with a
mean of zero but with different standard deviations. The standard deviation,
which we will investigate in more detail shortly, simply describes the spread
of the distribution. In the LULU/SPY example, LULU had a much larger
standard deviation than SPY, so the histogram was spread further across the x-
axis. Two final points on the normal distribution before moving on: Do not
focus too much on what the graph of the normal curve looks like, because
many other distributions have essentially the same shape; do not assume that
anything that looks like a bell curve is normally distributed. Last, and this is
very important, normal distributions do an exceptionally poor job of
describing financial data. If we were to rely on assumptions of normality in
trading and market-related situations, we would dramatically underestimate
the risks involved. This, in fact, has been a contributing factor in several
recent financial crises over the past two decades, as models and risk
management frameworks relying on normal distributions failed.

 Figure 11.2 Three Normal Distributions with Different Standard Deviations

Measures of Central Tendency

Looking at a graph can give us some ideas about the characteristics of the
data set, but looking at a graph is not actual analysis. A few simple
calculations can summarize and describe the data set in useful ways. First, we
can look at measures of central tendency, which describe how the data
  Figure 11.4 Running Mean of the First 100 Apples Picked Up
At this point, the line seems to be settling in on a value, and maybe we are
starting to be a little more confident about the average apple. Based on the
graph, we still might guess that the value is actually closer to 4 than to 4.5,
but this is just a result of the random sample process—the sample is not yet
large enough to assure that the sample mean converges on the actual mean.
We have picked up some small apples that have skewed the overall sample,
which can also happen in the real world. (Actually, remember that “apples” is
only a convenient label and that these values do include some negative
numbers, which would not be possible with real apples.) If we pick up many
more, the sample average eventually does converge on the population average
of 4.5 very closely. Figure 11.5 shows what the running total might look like
after a sample of 2,500 apples.

 Figure 11.5 Running Mean After 2,500 Apples Have Been Picked Up
 (Y-Axis Truncated to Show Only Values Near 4.5)
With apples, the problem seems trivial, but in application to market data
there are some thorny issues to consider. One critical question that needs to be
considered first is so simple it is often overlooked: what is the population and
what is the sample? When we have a long data history on an asset (consider
the Dow Jones Industrial Average, which began its history in 1896, or some
commodities for which we have spotty price data going back to the 1400s),
we might assume that that full history represents the population, but I think
this is a mistake. It is probably more correct to assume that the population is
the set of all possible returns, both observed and as yet unobserved, for that
specific market. The population is everything that has happened, everything
that will happen, and also everything that could happen—a daunting concept.
All market history—in fact, all market history that will ever be in the future— 
is only a sample of that much larger population. The question, for risk
managers and traders alike, is: what does that unobservable population look
like?
In the simple apple problem, we assumed the weights of apples would
follow the normal bell curve distribution, but the real world is not always so
polite. There are other possible distributions, and some of them contain nasty
surprises. For instance, there are families of distributions that have such
strange characteristics that the distribution actually has no mean value.
Though this might seem counterintuitive and you might ask the question
“How can there be no average?” consider the admittedly silly case earlier that
included the 3,000,000-year-old mummy. How useful was the mean in
describing that data set? Extend that concept to consider what would happen
if there were a large number of ages that could be infinitely large or small in
the set? The mean would move constantly in response to these very large and
small values, and would be an essentially useless concept.
The Cauchy family of distributions is a set of probability distributions that
have such extreme outliers that the mean for the distribution is undefined, and
the variance is infinite. If this is the way the financial world works, if these
types of distributions really describe the population of all possible price
changes, then, as one of my colleagues who is a risk manager so eloquently
put it, “we’re all screwed in the long run.” If the apples were actually Cauchy-
distributed (obviously not a possibility in the physical world of apples, but
play along for a minute), then the running mean of a sample of 100 apples
might look like Figure 11.6.

 Figure 11.6 Running Mean for 100 Cauchy-Distributed Random Numbers

It is difficult to make a good guess about the average based on this graph,
but more data usually results in a better estimate. Usually, the more data we
collect, the more certain we are of the outcome, and the more likely our
values will converge on the theoretical targets. Alas, in this case, more data
actually adds to the confusion. Based on Figure 11.6, it would have been
reasonable to assume that something strange was going on, but, if we had to
guess, somewhere in the middle of graph, maybe around 2.0, might have been
a reasonable guess for the mean. Let’s see what happens if we decide to
collect 10,000 Cauchy-distributed numbers in an effort to increase our
confidence in the estimate (see Figure 11.7).

 Figure 11.7 Running Mean for 10,000 Random Numbers Drawn from a
Cauchy Distribution
 Ouch—maybe we should have stopped at 100. As the sample gets larger,
we pick up more very large events from the tails of the distribution, and it
starts to become obvious that we have no idea what the actual, underlying
average might be. (Remember, there actually is no mean for this distribution.)
Here, at a sample size of 10,000, it looks like the average will simply never
settle down—it is always in danger of being influenced by another very large
outlier at any point in the future. As a final word on this subject, Cauchy
distributions have undefined means, but the median is defined. In this case,
the median of the distribution was 4.5—Figure 11.8 shows what would have
happened had we tried to find the median instead of the mean. Now maybe
the reason we look at both means and medians in market data is a little
clearer.

Three Statistical Tools

The previous discussion may have been a bit abstract, but it is important to
think deeply about fundamental concepts. Here is some good news: some of
the best tools for market analysis are the most simple. It is tempting to use
complex techniques, but it is easy to get lost in statistical intricacies and to
lose sight of what is really important. In addition, many complex statistical
tools bring their own set of potential complications and issues to the table. A
good example is data mining, which is perfectly capable of finding
nonexistent patterns in random data. It is fair to say that nearly all of your
market analysis can probably be done with basic arithmetic and with concepts
no more complicated than measures of central tendency and dispersion. This
section introduces three simple tools that should be part of every trader’s tool
kit: bin analysis, linear regression, and Monte Carlo modeling.
  Figure 11.8 Running Median for 10,000 Cauchy-Distributed Random
 Numbers

Statistical Bin Analysis

Statistical bin analysis is by far the simplest and most useful of the three
techniques in this section. If we can clearly define a condition we are
interested in testing, we can divide the data set into groups that contain the
condition, called the signal group, and those that do not, called the control
group. We can then calculate simple statistics for these groups and compare
them. One very simple test would be to see if the signal group has a higher
mean and median return compared to the control group, but we could also
consider other attributes of the two groups. For instance, some traders believe
that there are reliable tendencies for some days of the week to be stronger or
weaker than others in the stock market. Table 11.2 shows one way to
investigate that idea by dividing the returns for the Dow Jones Industrial
Average in 2010 by weekdays. The table shows both the mean return for each
weekday as well as the %age of days that close higher than the previous day.

Table 11.2 Day-of-Week Stats for the DJIA, 2010

Weekday % Close Up Mean Return

Monday 59.57% 0.22%

Tuesday 53.85% (0.05%)

Wednesday 61.54% 0.18%

Thursday 54.90% (0.00%)

Friday 54.00% (0.14%)

All 56.75% 0.04%

Each weekday’s statistics are significant only in comparison to the larger

group. The mean daily return was very small, but 56.75 % of all days closed
higher than the previous day. Practically, if we were to randomly sample a
group of days from this year, we would probably find that about 56.75 % of
them closed higher than the previous day, and the mean return for the days in
our sample would be very close to zero. Of course, we might get unlucky in
the sample and find that it has very large or small values, but, if we took a
large enough sample or many small samples, the values would probably (very
probably) converge on these numbers. The table shows that Wednesday and
Monday both have a higher %age of days that closed up than did the baseline.
In addition, the mean return for both of these days is quite a bit higher than
the baseline 0.04 %. Based on this sample, it appears that Monday and
Wednesday are strong days for the stock market, and Friday appears to be
prone to sell-offs.
Are we done? Can the book just end here with the message that you should
buy on Friday, sell on Monday, and then buy on Tuesday’s close and sell on
Wednesday’s close? Well, one thing to remember is that, no matter how
accurate our math is, it describes only the data set we examine. In this case,
we are looking at only one year of market data, which could possibly not be
enough; perhaps some things change year to year and we need to look at more
data. Before executing any idea in the market, it is important to see if it has
been stable over time and to think about whether there is a reason it should
persist in the future. Table 11.3 examines the same day-of-week tendencies
for the year 2009.

Table 11.3 Day-of-Week Stats for the DJIA, 2009

three different lines with different slopes. The variable b  in the equation
moves the whole line up and down on the  y-axis. Formally, it defines the
point at which the line will intersect the y-axis where x = 0, because the value
of the line at that point will be only b. (Any number multiplied by x when x =
0 is 0.) We can safely ignore b for this discussion, and Figure 11.9 sets b = 0
for each of the lines. Your intuition needs to be clear on one point: the slope
of the line is steeper with higher values for m; it slopes upward for positive
values and downward for negative values.

 Figure 11.9 Three Lines with Different Slopes

One more piece of information can make this simple, idealized equation
much more useful. In the real world, relationships do not fall along perfect,
simple lines. The real world is messy—noise and measurement error
obfuscate the real underlying relationships, sometimes even hiding them
completely. Look at the equation for a line one more time, but with one added
variable:
 = mx + b + ε
ε ~ i.i.d. N (0, σ)
The new variable is the Greek letter epsilon, which is commonly used to
describe error measurements in time series and other processes. The second
line of the equation (which can be ignored if the notation is unfamiliar) says
that ε is a random variable whose values are drawn from (formally, are
“independent and identically distributed [i.i.d.] according to”) the normal
distribution with a mean of zero and a standard deviation of sigma. If we
graph this, it will produce a line with jitter, as the points will be randomly
distributed above and below the actual line because a different, random ε is
added to each data point. The magnitude of the jitter, or how far above and
below the line the points are scattered, will be determined by the value of the
standard deviation chosen for σ. Bigger values will result in more spread, as
the distribution for the error component has more extreme values (see Figure
11.2 for a reminder.) Every time we draw this line it will be different, because
ε is a random variable that takes on different values; this is a big step if you
are used to thinking of equations only in a deterministic way. With the
addition of this one term, the simple equation for a line now becomes a
random process; this one step is actually a big leap forward because we are
now dealing with uncertainty and stochastic (random) processes.
Figure 11.10 shows two sets of points calculated from this equation. The
slope for both sets is the same, m = 1, but the standard deviation of the error
term is different. The solid dots were plotted with a standard deviation of 0.5,
and they all lie very close to the true, underlying line; the empty circles were
plotted with a standard deviation of 4.0, and they scatter much farther from
the line. More variability hides the underlying line, which is the same in both
cases—this type of variability is common in real market data, and can
complicate any analysis.
  Figure 11.10 Two Lines with Different Standard Deviations for the Error
Terms

 Regression
With this background, we now have the knowledge needed to understand
regression. Here is an example of a question that might be explored through
regression: Barrick Gold Corporation (NYSE: ABX) is a company that
explores for, mines, produces, and sells gold. A trader might be interested in
knowing if, and how much, the price of physical gold influences the price of
this stock. Upon further reflection, the trader might also be interested in
knowing what, if any, influence the U.S. Dollar Index and the overall stock
market (using the S&P 500 index again as a proxy for the entire market) have
on ABX. We collect weekly prices from January 2, 2009, to December 31,
2010, and, just like in the earlier example, create a return series for each asset.
It is always a good idea to start any analysis by examining summary statistics
for each series. (See Table 11.4.)

Table 11.4 Summary Statistics for Weekly Returns, 2009–2010

Ticker N= Mean StDev Min Max

ABX 104 0.4% 5.8% (20.0%) 16.0%

SPX 104 0.3% 3.0% (7.3%) 10.2%

Gold 104 0.4% 2.3% (5.4%) 6.2%

USD 104 0.0% 1.3% (4.2%) 3.1%

At a glance, we can see that ABX, the S&P 500 (SPX), and Gold all have
nearly the same mean return. ABX is considerably more volatile, having at
least one instance where it lost 20 % of its value in a single week. Any data
series with this much variation, measured by a comparison of the standard
deviation to the mean return, has a lot of noise. It is important to notice this,
because this noise may hinder the usefulness of any analysis.
A good next step would be to create scatterplots of each of these inputs
against ABX, or perhaps a matrix of all possible scatterplots as in Figure
11.11. The question to ask is which, if any, of these relationships looks like it
might be hiding a straight line inside it; which lines suggest a linear
relationship? There are several potentially interesting relationships in this
table: the Gold/ABX box actually appears to be a very clean fit to a straight
line, but the ABX/SPX box also suggests some slight hint of an upward-
sloping line. Though it is difficult to say with certainty, the USD boxes seem
to suggest slightly downward-sloping lines, while the SPX/Gold box appears
to be a cloud with no clear relationship. Based on this initial analysis, it seems
likely that we will find the strongest relationships between ABX and Gold
and between ABX and the S&P. We also should check the ABX and U.S.
Dollar relationship, though there does not seem to be as clear an influence
there.
  Figure 11.11 Scatterplot Matrix (Weekly Returns, 2009–2010)
Regression actually works by taking a scatterplot and drawing a best-fit line
through it. You do not need to worry about the details of the mathematical
process; no one does this by hand, because it could take weeks to months to
do a single large regression that a computer could do in a fraction of a second.
Conceptually, think of it like this: a line is drawn on the graph through the
middle of the cloud of points, and then the distance from each point to the line
is measured. (Remember the ε’s that we generated in Figure 11.11? This is the
reverse of that process: we draw a line through preexisting points and then
measure the ε’s (often called the errors).) These measurements are squared, by
essentially the same logic that leads us to square the errors in the standard
deviation formula, and then the sum of all the squared errors is calculated.
Another line is drawn on the chart, and the measuring and squaring processes
are repeated. (This is not precisely correct. Some types of regression are done
by a trial-and-error process, but the particular type described here has a
closed-form solution that does not require an iterative process.) The line that
minimizes the sum of the squared errors is kept as the best fit, which is why
this method is also called a least-squares model. Figure 11.12 shows this best-
fit line on a scatterplot of ABX versus Gold.
  Figure 11.12 Best-Fit Line on Scatterplot of ABX (Y-Axis) and Gold
We now look at a simplified regression output, focusing on the three most
important elements. The first is the slope of the regression line (m), which
explains the strength and direction of the influence. If this number is greater
than zero, then the dependent variable increases with increasing values of the
independent variable. If it is negative, the reverse is true. Second, the
regression also reports a p-value for this slope, which is important. We should
approach any analysis expecting to find no relationship; in the case of a best-
fit line, a line that shows no relationship would be flat because the dependent
variable on the y-axis would neither increase nor decrease as we move along
the values of the independent variable on the x-axis. Though we have a slope
for the regression line, there is usually also a lot of random variation around
it, and the apparent slope could simply be due to random chance. The p-value
quantifies that chance, essentially saying what the probability of seeing this
slope would be if there were actually no relationship between the two
variables.
The third important measure is R2  (or  R-squared), which is a measure of
how much of the variation in the dependent variable is explained by the
independent variable. Another way to think about R 2 is that it measures how
well the line fits the scatterplot of points, or how well the regression analysis
fits the data. In financial data, it is common to see R 2 values well below 0.20
(20 %), but even a model that explains only a small part of the variation could
elucidate an important relationship. A simple linear regression assumes that
the independent variable is the only factor, other than random noise, that
influences the values of the independent variable—this is an unrealistic
lets us see if the tendencies hold for different markets, different timeframes,
and whether they have been stable over time. Proper selection of the test
universe is the first step in this process.
From a quantitative perspective, one of the main questions about markets is
whether asset returns are stationary. This is a slight oversimplification, but,
mathematically, a stationary process is a stochastic (random) process whose
probability distribution does not change over time. In market data, if returns
were stationary, we would not see a shift in such things as measures of central
tendency or dispersion when the same market is examined over different time
periods. There is a significant debate in academia over whether returns are
stationary, but, from a trader’s perspective, they do not seem to be. For
instance, consider the stock of a company that in its early days was a minor
player but later rose to be the industry leader, or a commodity that might at
one time have been primarily produced and consumed domestically, but for
which a large global marketplace developed as domestic supplies were
depleted. As these large, structural changes occur we know that the assets
will, in fact, trade very differently. (Note that there are also processes that are
cyclostationary, meaning that they appear to be stationary except for having a
fairly predictable cyclic or seasonal component. If something is predictable, it
can be backed out of the data, and the cyclostationary data can be transformed
to a stationary set. Examples of real market data that have a strong seasonal
component might be retail sales and volatility of natural gas prices.)
Non-stationarity of asset returns is a challenge to the enduring validity of
any technical methodology; any trading edge should be examined over a long
enough time period to assess its stability. Our results will vary greatly
depending on the time period examined and whether it was a rip-roaring bull
market, a flat period with no volatility, or a bear market. Traders doing
analysis must consider this factor. Too many tests are presented to the public
that were run on the last year or two years of data. These tests may be
valuable if the future looks like these recent years, but that is a tenuous
assumption on which to build a trading program.
Assets and asset classes are also correlated to varying degrees. In times of
financial stress, it is not uncommon to see everything trading in the same
direction, and to see, for instance, coffee, cocoa, crude oil, stocks, and even
gold futures being crushed on the same day. One of the assumptions of many
standard statistical methods is that events are independent, but if we examine
a set of 1,000 stocks, we may find that more than 800 of them have large
moves in the same direction on the same day. Of course, this violates any
reasonable definition of independence, so we need to be aware that large tests
on many different assets may be considerably less powerful (in terms of the
information they give us) than a comparable test in another field. In addition
to challenging assumptions of independence, tight correlations can effectively
reduce the same size for many tests. For instance the equities sample for these
tests includes over 1,380,000 trading days, but, because of correlations, these
are not  1,380,000 independent events.
For the tests in this book, I have attempted to address some of these issues
in three ways: by using a diverse sample of assets and asset classes, by
focusing on a reasonably large historical period that includes several volatility
regimes, and through a methodology that adjusts for the baseline drift
inherent in each data set. In terms of asset class selection, I have taken a
basket of 600 individual equities, randomly sampled from three large market
capitalization tranches, 16 different futures markets, and 6 major world
currencies trading against the U.S. dollar. All tests were also run on a set of
stochastic (random) price models for calibration: a random walk i.i.d. ~
normal (the notiation “i.i.d. ~” means “independent and identically distributed
according to the … distribution”), a random walk i.i.d. ~ actual returns from
the Dow Jones Industrial Average of 1980 to 2010, and a GARCH model.
All tests in this chapter are run on the time period from 1/1/2001 to
12/31/2010. This period was chosen partially out of convenience, and
partially because of the assumption that the most recent data are likely to be
more relevant going forward. (There could be less value in examining results
from, say, 1940 to 1950.) In addition, this period includes, for equities, a
period of protracted low volatility in the first part of the decade, the dramatic
and volatile bear market associated with the 2007 to 2008 financial crisis, and
the sharp recovery bull market of 2009 to 2010. One could, quite correctly,
make the argument that this period does not include a large-scale secular bull
market, which is likely to be the focus of many equities traders, but it is
possible to replicate these tests on other time periods to check for consistency.
It is also worth considering that these 10 years include some dramatic
transitions for nearly all of these assets, many of which underwent a change
from open-outcry to electronic markets. For others, such as U.S. equities, the
electronic marketplaces matured and evolved as they underwent significant
regulatory and structural changes.

Equities
For the equities universe, the entire set of domestic equities (N = 7,895)
was ranked by market capitalization as of 1/1/2011, and break points were set
at the 500th, 1,500th, and 2,500th members to approximate the divisions into
large-cap, mid-cap, and small-cap universes. Note that though there are nearly
8,000 tickers listed, only a few thousand of them trade actively; many of the
very small market capitalization stocks are extremely illiquid, sometimes
going days or weeks without so much as a single trade. Exchange-traded fund
(ETF) products and stocks with very short trading histories, and then 200
stocks from each market cap group were randomly chosen as the sample for
that market cap. Table 12.1 presents industry statistics by each group.

Table 12.1 Industry Statistics for Equities Sample

Large-Cap Mid-Cap Small-Cap

Basic Materials 7 15 15

Capital Goods 13 18 7

Consumer Cyclical 6 8 8

Consumer Noncyclical 15 9 7

Energy 15 19 8

Financial 33 31 44

Health Care 21 9 14

Services 50 41 46

Technology 17 21 28
Transportation 5 6 7

Utilities 17 23 16

Totals 199* 200 200

*One company was designated as a conglomerate and is missing from this

tabulation.

In general, smaller market capitalization stocks tend to trade less actively

and are less in the public focus. As a consequence, smaller-cap stocks are also
usually less liquid and often trade more erratically than their large-cap
cousins. In the sample used for this research, we see clear evidence of this as
measures of dispersion are considerably larger for the mid- and small-cap
tranches. One other common characteristic of smaller stocks is that they tend
to outperform their large-cap cousins with higher average returns, which is a
well-known anomaly in academia called the size effect. The explanation
commonly offered for this is that smaller companies carry more risk than
larger companies, so investors demand a higher return to hold these stocks.
Though not a comprehensive study of the size effect, it is interesting to note
that we see no evidence of the effect in our samples. Table 12.2 presents
simple summary statistics for the equities test universe. Note that many of
these are very small numbers, so they are presented here in basis points rather
than percentages. (For most people, a number like 2 is much more intuitive
than 0.0002.)
This sample, no doubt, suffers from survivorship bias. Many of the
companies in the large-cap sample grew into that market cap; the companies
did well and their stocks appreciated in value. Similarly, many of the small
caps may have been mid caps at one time, but the companies faltered and the
stocks declined. This is a likely explanation for why we do not see the classic
size effect in this sample: because the market cap slices were taken at the end
of the trading period. Had we taken a sample by market cap in early 2009 and
then followed the returns going forward, we almost certainly would have seen
dramatic outperformance in small caps over large caps.
Table 12.2 Summary Statistics for Equities Universe, Daily Returns

All StocksLarge-CapMid-CapSmall-Cap

Total Trading Days 1,381,685 484,002 454,649 443,034

Mean 5.6 bp 5.2 bp 5.9 bp 5.8 bp

Median 2.3 bp 2.5 bp 3.4 bp 0.8 bp

Standard Deviation 248.7 bp 230.8 bp 246.2 bp 270.9 bp

Interquartile Range (IQR)227.2 bp 215.3 bp 223.5 bp 244.0 bp

Coefficient of Variation 44.4 44.8 41.4 47.1

Up% 50.07% 50.21% 50.43% 49.56%

Unchanged% 1.42% 1.15% 1.46% 1.66%

Down% 48.51% 48.64% 48.11% 48.78%

For our tests in this book, this is not a fatal flaw, but it is something that
needs to be considered carefully for many other kinds of tests. If we were
really interested in different trading characteristics by market caps, one way to
compensate for survivorship bias would be to create a dynamic universe. For
instance, it could be resampled, say, every six months and the membership
based on current market cap rankings at each resampling point. On 1/1/2006
you could segregate the stock universe into market caps based on 1/1/2006
rankings, take your samples, and do your analyses over the next six months.
You would then need a list of market caps as they stood on 7/1/2006, take
your sample, and repeat. This is a laborious process, but it may be the only
way to correct for this bias in some kinds of studies. Be clear that these
market cap tranches were done on a look-back basis for the purpose of the
studies in this book; the goal of these tests was to understand the behavior of
equities as a whole rather than focusing too much attention on market cap
division, so this was not a significant limitation for these tests.
Another factor to consider is that the test universe was created from stocks
trading as of 12/31/2010, so companies like Enron, Bear Stearns, and
Lehmann Brothers are not included, nor are stocks that did exceptionally well
and were taken over, sometimes at significant premiums to their market
values. This introduces an element of survivorship bias that could distort or
alter our results. Particularly for traders developing rule-based trading
systems, the impact of these events can be very significant, even to the point
of making unprofitable systems appear to be very good in backtests. For
rough tests of price tendencies like the ones on this chapter, this factor may be
less significant, but the potential impact of survivorship bias cannot be
quickly and casually dismissed
There are several ways in which this research could be extended. For one
thing a number of companies in the sample are American depositary receipts
(ADRs), which are foreign companies that also list on U.S. exchanges. For
most of these companies, U.S. trading hours are not the primary trading
session and there may be significant differences between the way these
companies trade and domestically domiciled companies. It also may make
sense for interested traders to drill down a bit deeper into segregating stocks
by sectors, because biotech companies, for instance, trade very differently
from utilities or technology companies. There may be some important
quantitative differences that these tests have not captured, because they did
not segregate the equity groups by these or other factors.
Last, many traders access markets through a variety of indexes and ETF
products. We should not assume that the behavior of a large number of stocks
in aggregate will mirror the behavior of an index; traders who choose to focus
on index products should probably conduct similar research on the indexes
themselves. However, research done exclusively on indexes is subject to
several deficiencies: limited data history, changing index composition, and
potential microstructure issues such as nonsynchronous trading are a few of
the most common. Even traders who choose to focus on these products will
find their work significantly enhanced by a deeper understanding of the
behavior of the individual components that make up the index.

Futures
 Chapter 14

Moving Averages
Statistics are used much like a drunk uses a lamppost; for support, not 
illumination.
-Vin Scully
Traders use moving averages for many purposes; we have seen a few ideas
for specific applications earlier in this book, but it is worthwhile to now take a
deep look at price action around moving averages. It is always surprising to
me how many of the same concepts are repeated by traders, in the literature,
and in the media without ever being tested: moving averages are support and
resistance, moving averages define the trend, moving average crosses predict
future price movements, and so forth. In devising quantitative tests of moving
averages, we immediately run a problem: many traders use them in ways that
are not amenable to testing. They may be part of a more complex
methodology incorporation many discretionary factors, but there may also be
some sloppy thinking involved in many cases. For instance, traders will argue
that moving averages sometimes work as support and resistance, or that a
moving average crossover is sometimes a good definition of trend. It is
always possible to find an example of something working, and traders often
tend to focus undue attention on a few outstanding examples. However, the
real question is whether it is repeatable and reliable over a large sample size,
and whether the tool adds predictive value to the trading process.
If you flip a coin whenever you have a trading decision you will find that
the coin sometimes does seem to help you. It is also likely that, at the end of
many trades, you would be able to remember a few dramatic examples where
the coin made you a lot of money; perhaps it got you out of a losing trade
early on, or kept you in a winner for the big trend. In fact, you might conclude
that the coin doesn’t work all the time, but nothing in trading works all the
time. Perhaps you realize that it does help in certain cases so you are going to
keep it and just use it sometimes. In fact, you might say, as many traders do of
their indicators, that you “have to know when to look at it.” Obviously, I am
making a bit of a joke here, but the point is important. You have to be sure
that the tools you are using are better than a coin flip, that they are better than
random chance.

Moving Averages as Support or Resistance

A moving average is an average of a window of prices that is moved
forward and recalculated for every bar. For instance, a standard 50-period
moving average averages the last 50 closing prices, including the current bar’s
close. Since today’s close, or the close of whatever time period is being used,
cannot be known in advance, the current price is used until the closing price
of the bar is established. This brings up an important point about moving
averages: a standard moving average, calculated for the current bar, will
change with every tick as the bar unfolds. For longer-term moving averages,
today’s price might be one of several hundred prices, so it would take a very
large change today to significantly move the average. For shorter-term
averages—some traders use averages as short as three to five bars for some
purposes—the last price may have a dramatic impact. This effect creates a
serious problem when examining moving averages on historical price charts,
because the moving average you see on the chart did not exist in real time.

Moving Moving Averages?

Some writers (for instance, Kirkpatrick 2006) claim that moving averages
function as support and resistance levels. This claim may be based, at least in
part, on a casual inspection of historical charts, but it is important to
remember that some of the most dramatic examples of moving average tests
never actually occurred. Points where price appeared to have touched and
moved away from an average may not have existed in real time because the
moving averages actually move while the bar is being formed. Figure 14.1
shows an example of a near-perfect, to-the-tick test of a 20-period moving
average on a daily chart of the S&P 500 index (NYSE: SPY). Based on this
chart, we could assume that price traded down to the average, touched it, and
was immediately pushed away as the market exploded to close at the highs of
the day. This chart suggests that a trader who buys at the moving averages
could have practically bought the very low of the day, as that is where the
moving average intersects the price bar. Furthermore, this was at a structurally
significant point in the market; catching this trade off the moving average
would have given the trader a shot at catching a 13% advance over next few
months.
  Figure 14.1 Near-Perfect Example of a Moving Average Appearing to Act 
as Support 
However, it was not possible to execute this trade at the average. Figure
14.2 shows the daily bar as it unfolded, at five separate points during the
trading day, with the actual value of the moving average highlighted to the
right of each small price chart. On the open, the moving average was well
below the low of the day, and, later, as the market traded down to put in the
low for the session (which was 108.94), the average was still well below the
low of the day. As price rallied in the afternoon, the average also moved up,
but it was still below the low of the day. It is important to remember that
every time the current price changed, it pulled the moving average with it; try
to see these five charts as snapshots in time. Finally, as the market exploded
to close near the day’s high, the average was pulled up into the low of the day,
but this low had already been set many hours before. A trader executing on
this signal would have had to buy the close of the day, not the moving average
test at the low. This is not an example of a moving average acting as support
 —this is an illusion. The trade simply never happened. Any backtesting or
system development based on price touching a moving average, including
some of the tests in this chapter, suffers from this effect.
  Figure 14.2 The Moving Average Is Pulled Up to Touch the Low of the Day
as Price Rallies into the Close

Moving Average Variations

To deal with this issue, some traders offset the moving average a number of
bars. For instance, you may use yesterday’s moving average value, calculated
on the close of that day, as today’s average, or even the average of 5 to 10
bars ago. This has the benefit of being defined before the current bar begins,
and, depending on the specific application of the moving average, may or
may not have an impact on its ultimate utility. In addition, there are several
variations of moving averages, including averages that weigh recent prices
more than past prices, and averages that do not drop data points but rather
decay them exponentially. Chapter 8 examines the behavior of simple moving
averages (SMAs) and exponential moving averages (EMAs) in considerable
detail from a perspective that might be useful to a discretionary trader. In this
section, we focus on the hard facts of price behavior around simple moving
averages.

The Tests
For the tests that follow, I make no attempt to test complete trading systems
or complex methods. Instead, I am simply looking to see whether there is
discernible, nonrandom price movement around moving averages. Does it
make sense to make buying or selling decisions based on the relationship of
price to a moving average? Do moving averages provide support? Are some
moving averages more useful (special?) than others? Can moving averages
define the trend of a market? Consider these to be fundamental tests of the
building blocks and basic principles. There could well be modifying factors,
not addressed in these simple tests, that do show an edge when combined with
moving averages, or there could be ways to trade them profitably that are
outside the scope of these tests. For now, let’s start at the beginning and
consider what happens when price touches a simple moving average.

Moving Average Touch and Hold on Close

There are only two possible outcomes as price engages a moving average:
either the moving average level can hold on the close—meaning that, if price
traded down to the moving average, the bar closes above the level—or it can
fail as the bar closes under the average. Let’s consider the first case, in which
price touches the moving average but the moving average holds on close.
Figure 14.3 shows visual examples of valid tests of a 20-period EMA. We
will use the Pythia framework for testing purposes, which takes a criteria set,
generates a buy or sell signal if a bar fulfills the condition, and then generates
a composite return for each day after the signal. Note that I am not suggesting
this or any signal could be traded based on such simple criteria; this is just a
robust way to do a pure test of tendencies around a potential trading signal.
Also, it is very possible that I have the buy and sell signals flipped—in other
words, what I think is a buy may actually be a sell. This is not a problem
within this testing framework, as we would simply see that buy signals had an
edge for negative excess returns and sell signals showed positive excess
returns.
Criteria for Moving Average Touch and Hold Test
AVG = Average(close, N) //Set average length here.
IF all of the following are true:
Low of previous bar > AVG on previous bar.
Low of current bar ≤ AVG //Moving average touched on this bar.
Close of current bar ≥ AVG //Market is unable to close below
moving average.
THEN buy this bar on close.
Conditions are symmetrical for sell signal.
  Figure 14.3 Examples of Valid Moving Average Touch and Hold on Close
Many traders watch the 50-day moving average, with the justification that it
provides support or resistance because “hedge funds watch it” or that many
other traders like to buy around the 50-day moving average. Of course, there’s
an important assumption here—do people really watch and use this level? If
this were true, it should be possible to find nonrandom price tendencies
around this average. Table 14.1 shows summary statistics for successful tests
and holds of a 50-period simple moving average. Take a minute to familiarize
yourself with the table format, because you will be seeing it many times
throughout the rest of this chapter.

The rows in the table are for days following the signal event.
Days one to five are presented, and the table then switches to a
weekly format. This allows assessment of short-term tendencies
while still maintaining a longer-term perspective.
Excess mean returns (labeled µ sig – µ b), may be the most useful
summary statistic for these tests. To generate this number, the
mean return for the baseline of the sample universe (e.g., the
drift component of stock returns) is subtracted from the mean of
all signal events.
Diff median is the excess median, or the baseline median return
subtracted from the signal median return.
Because these may be very small numbers, they are displayed in
basis points (one basis point equals one-hundredth of a percent:
0.0001 = 0.01% = 1 bp) rather than percentages.
Asterisks indicate statistical significance at the 0.05 and 0.01
levels.
The percentage of days that close higher than the entry price is
does not matter since we do not intend to trade these ideas systematically.
They are useful only as a hint or a guidepost to suggest where trading
opportunities might lie, and to reveal some otherwise hidden elements of
market structure. You could also challenge these results on the basis that they
might not be representative of the whole market. I would disagree because a
large sample of stocks from all market caps and industries is represented,
along with most major futures and currencies, but it is a valid criticism to
consider. Last, this is not a standard and accepted test for random walk, even
though the results strongly suggest a nonrandom process is at work in this
sample.

Moving Average Crossovers

Moving average crossovers occur when a short-term moving average,
which tracks price more closely, crosses over or under a longer-term moving
average. These events are commonly regarded as good trading signals or as
indicators of trend because they are supposed to identify inflection points
where markets have changed direction. In their purest form, these systems are
traded on an always-in (the market) basis, meaning that the system goes long
on a buy signal and immediately reverses to a short position on a sell signal,
without any intervening flat period. The idea is that always being in the
market, whether long or short, will allow the trader to be positioned to take
advantage of a large trend when it emerges. Figure 14.9 shows an example of
an excellent trade that would have been captured, in its entirety, by a moving
average crossover system. This brings up an important point about moving
average crossovers: they will always capture the majority of a large trend. To
be sure, there is some lag depending on the length of the moving averages
chosen, so they will never sell the exact high or buy the exact low of a move,
but, in backtests, they will always take a big chunk out of a large trend move.
  Figure
 Figure 14.9 Trading
Trading a 10-/50-Day Moving
Moving Average
Average Crossover
Crossover System
in a Trending Environment 
However, this is only part of the story. Moving average crossover systems
always work well in trending environments, but, just as surely, they will get
destroyed in ranging markets. If there is no clear trend, the market will chop
back and forth, forcing the moving average system to buy at the highs and sell
at the lows. These whipsaws usually end up giving back all of the profit made
in the trending environment, and sometimes even more. Contrary to what
some authors and books suggest, it is not economically feasible to trade basic
moving average systems on a stand-alone basis, because whipsaws and
trading frictions make them breakeven at best. Figure 14.10 shows an
example of a moving average applied in a range-bound market, where it finds
itself repeatedly buying at the high and selling at the lows. Significant losses
would have accrued in this period.
  Figure
 Figure 14.10 Trading
Trading a 10-/50-Day Moving Average Crossover
Crossover System
System in a
 Range-Bound Environme
Environment 
nt 

cademic Research
There is a significant body of academic research investigating the
profitability of trading rules based on moving average crossovers. Brock,
Lakonishok, and LeBaron’s landmark paper, “Simple Technical Trading Rules
and the Stochastic Properties of Stock Returns” (1992) is noteworthy because
it was one of the first papers to show evidence that technical trading rules
could produce statistically significant profits when applied to stock market
averages. They used a set of rules based on moving average crossovers and
channel breakouts on the Dow Jones Industrial Average from its first record
day in 1897 to the last trading day of 1986. The results, as they say in the
paper, are striking: they find statistically significant profits on both the long
and short side for every moving average combination they examined.
For traders familiar with moving average studies in modern trading
applications, the choices of moving averages that Brock et al. chose may be
surprising: for the short average 1-, 2-, or 5-period, and 50-, 150-, or 200-
period for the longer. A 1-period moving average is not actually a moving
average at all—it is simply the price of the asset, so many of their moving
average crossovers were actually tests of price crossing a moving average.
Most authors and system developers tend to use averages much closer in
length, like 10/50, 50/200. The original 1992 paper is fairly accessible to the
lay reader and should be required reading for every trader who would trade
based on technical signals. However,
However, it might also be instructive to investigate
their results in terms that traders will more readily understand.
The most profitable signals in their study were not the stop and reverse
versions, but rules that entered on a moving average cross and exited 10 days
later. In addition, some of their tests added a band around the moving
averages and did not take signals within that band, in an attempt to reduce
noise and whipsaw signals. For the sake of simplicity,
simplicity, let’s look at their 1-/50-
period moving average crossovers with no filter channel. Figure 14.11 shows
the equity curve calculated for every day from 1920 to 1986, assuming the
trader invested $100,000 on each trading signal, or sold short an equivalent
dollar amount for the short signals.

 Figure
 Figure 14.11
14.11 Daily Equity Curve for 1/50 Moving Average Crossover
Crossover on
 DJIA, 1920–1986 ($100,000
($100,000 Invested on Each
Each Signal)
These results do appear to be remarkable at first glance: a steadily
ascending equity curve that weathered the 1929 crash, both World Wars, and
several recessions with no significant drawdowns. In addition, this system
was stable for most of a century, while the economy, the sociopolitical
landscape, and the markets themselves underwent a number of dramatic
transformations. This stability is exactly what systematic traders are looking
for. Table 14.8 shows a few summary stats for this system, assuming no
transaction costs or financing expenses.

Table 14.8 Summary Stats for 1/50 Moving Average Crossover on DJIA,
1920–1986 (Total
(Total Net Profit Assumes $100,000 Invested on Each Signal)

N= 911

Total Net Profit $743,794

% Profitable 28.0%

Mean Trade 84 bp

Mean Winning Trade 589 bp

Mean Losing Trade –112 bp

These numbers are not bad, though traders not used to seeing long-term
trend-following systems might wish for a higher win percentage. It is not
uncommon for these types of systems to have win rates well under 40 percent,
and, as long as the winners are substantially larger than the losers, such a
system can be net profitable. In this case, we have to wonder if the average
trade size is large enough to be profitable after accounting for trading
frictions. It is important to remember that this is a backtest on the cash Dow
Jones Industrial Average (DJIA), which is the average of a basket of 30
stocks. Today, investors can access this market through a variety of derivative
products, but, for most of the history of this backtest, it would have been
necessary to have purchased each of the stocks in the average individually
and to have rebalanced the basket as needed to match changes in the average.
In addition, financing costs, the impact of dividends, and tax factors also need
to be considered with a system like this. Considering financing costs alone, if
an investor were able to earn 4% risk-free for 66 years (the actual rate would
vary, but this is probably in the ballpark. See Dimson, Marsh, and Staunton
2002), the initial $100,000 investment would have grown to over $1.3 million
over the same time period. Also, the actual price of the DJIA itself increased
2,403% over this period—a simple buy-and-hold strategy would have
returned over $2.3 million, albeit with impressive volatility and drawdowns
along the way.
Stronger condemnation, though, comes from an out-of-sample test. Since
the published results end in 1986, this is an ideal situation to walk forward
from 1/1/1987 to 12/31/2010, which effectively shows the results investors
might have achieved had they started trading this strategy after the end of the
test period. (This is not a true out-of-sample test, as it is likely that Brock et
al. examined some of the later history in their tests, even if the results were
not published.) Figure 14.12 and Table 14.9 show the results for this time
period, which are disappointing to say the least. It is also worth noting that a
simple buy-and-hold strategy would have returned over $550,000 over the
same time period.

 Figure
 Figure 14.12 Daily Equity Curve for
for 1/50 Moving Average
Average Crossover
Crossover on
 DJIA, 1987–2010, Quasi
Quasi Out of Sample ($100,000 Invested
Invested on Each Signal)

Table 14.9 Summary Stats for 1/50 Moving Average Crossover on DJIA,
1987–2010, Quasi Out of Sample (Total Net Profit Assumes $100,000
Invested on Each Signal)

N= 449

Total Net Profit $19,830

% Profitable 20.9%

Mean Trade (Basis Points) 7 bp

Mean Winning Trade 477 bp

Mean Losing Trade –117 bp

What is happening here? The first question we should ask is: is it possible
that this is simply normal variation for this system? Just by looking at the
equity curves, this seems unlikely because we are not able to identify any
other 15-year period when the curve is flat and volatile, but this is not an
actual test. Comparing the in- and out-of-sample returns, the Kolmogorov-
Smirnov test, which is a nonparametric test for whether or not two samples
were likely to have come from the same distribution, gives a p-value of 0.007.
Based on this result, we can say that we find sufficient evidence to reject the
idea that the out-of-sample test is drawn from a similar distribution as the in-
sample test; this suggests that something has changed. It might be unwise to
trade this system in the future after such a significant shift in the return
distribution. This is not to say that anything was wrong with the research or
the system design; market history is littered with specific trading ideas and
systems that eventually stopped working. This is especially common after
working ideas have been published, either in academic research or in
literature written for practitioners.

Optimization
Though this is not a book on quantitative system design, a brief discussion
of the value and perils of optimization is in order. (System optimization is
also sometimes wrongly referred to as “curve fitting”.) In a nutshell,
optimization is simply taking trading systems and changing inputs until you
find a set of conditions that would have performed well on historical data. In
the case of the moving average crossover system just discussed, using a 65-
period short moving average and a 170-period long moving average would
have made the system profitable over the 1986 to 2010 window, producing a
profit of $169,209 in over just 35 trades, with an impressive 40% win ratio.
These numbers were found through an exhaustive search of many possible
combinations of moving averages, but had we also searched for the inverse of
this system (allowing shorting when the slow moving average crossed over
the long and vice versa), we could have found combinations that made well
over a million dollars during the same time period. The danger of
overoptimization is that the best historical values will rarely be the best values
in a walk-forward test or in actual trading. In the worst case, it is possible to
create an optimized system that looks incredible on historical data, but will be
completely worthless in actual trading.
Overoptimization can be accidental or deliberate. In the case of system
vendors selling trading systems to the public, optimized systems can produce
very impressive track records. Some systems are built by system designers
who lack the education and experience to avoid this trap; they may truly
believe they are producing something of value and are surprised when the
actual trading results do not match their backtests. More often, overoptimized
systems are created in a nefarious attempt to extract money from the public.
Designers can build and optimize a trading system in a single weekend that
shows dramatic results on a handful of markets. If they can sell a few copies
of the system for $2,000 to $3,000, that’s not a bad return for a few days’
work.
In these examples, the method and effect of optimization is obvious, but it
can also be more subtle and much harder to detect. It is even possible to
overoptimize a simple market study yourself, without realizing you are doing
so. The term overoptimization implies that there might be an appropriate
degree of optimization—which is correct. Some optimization is necessary and
is actually a vital component of the learning process. For instance, if you
wanted to do a research project on gap openings what would be the
appropriate gap size to study? You might first start with 400 percent, but
would quickly discover that there may not be a single market gap that size in
your entire test universe. If you next look at 0.25% gaps, you will discover
that these are so common as to be meaningless; eventually you might settle on
gap openings between 5 and 20%. Each step does bring some dangers because
you are potentially overspecifying the question, but this is also how you learn
about the market’s movements.
movements. Here are some guidelines that will help you to
guard against overoptimization:

Understand the process. If this is your own work, think

critically about every step in the process. If you are only seeing
the product of someone else’s work, you cannot trust the results
unless you also understand their process. Knowing and trusting
the person who created the results is a good first step, but
remember, anyone can make mistakes. In general, do not pay
money for systems designed by unknown third parties.
Good systems and good research questions usually have few
conditions. A set of conditions like “Buy when RSI is below X
 and price is above Y” is preferable to “Buy when RSI is below
X, MACD is above Y, the A- and B-period moving averages are
above the C-period and rising, while the C-period is declining,
and only execute entries on a Monday, Thursday, or Friday,
avoiding the last three days of the month.” Though the first
criteria set could well be the result of overoptimization, the
second is almost certain to be.
Guard against small sample sizes. It is not possible to draw
good conclusions from most tests with a small number of
events, and someone presenting you a study based on six
occurrences has probably pared that down from a much larger
starting universe. Large sample sizes can also be misleading,
but very small ones are a huge red flag.
Be suspicious of incredible results. In trading, if something
looks too good to be true, it almost certainly is not true. There
are no systems that crush the market over a long period of time.
There are no systems that return, unlevered, 100% a year on
capital. There are no systems with an extremely high win rate
and extremely high reward/risk ratio. These things simply do
not exist, because the markets are too competitive and efficient,
but historical results like this can be created through
optimization.

This is a deep subject and we have only scratched the surface. For the
automated system designer, this is an important area of study, and ideas like
systems that reoptimize themselves on a walk-forward basis, optimizing for
outcomes other than maximum net profit or considering the results of
optimization tests as multidimensional surfaces, are an important part of that
study. Most discretionary traders will only use quantitative studies as a
departure point for developing trading ideas, so it probably is sufficient to
have an idea about the most serious dangers and risks of optimization. When
in doubt, disregard the results of any study that you feel may be the result of
overoptimization or overspecification. Better to simply know you do not
know than to be misled by spurious information.

Testing Moving Average Crosses

Frankly, it is well established that moving average crosses cannot be traded
as stand-alone systems, so there is little point in presenting study after study
that confirm this. There is, however, another application that is very common.
There are many books and trading methodologies that suggest that some kind
of moving average–derived trend indicator can be a useful tool. The idea
usually presented is that, in an uptrend, the upswings will be larger than the
downswings, so traders should use a tool to identify the uptrend and then
trade only with that trend. Because these ideas are so common in the trading
literature, it is worth our time to investigate them here.
First, think about what you would need to see from a trend indicator to
make it useful. Though there could be different answers to this question, I
suggest that they all are probably some variation of this: long trades should
work better when the indicator shows an uptrend, and the downtrend
condition should produce a more favorable environment for short trades. A
simple way to test this would be to identify the trend indicator and then
categorize all days according to whether the indicator labels them uptrend,
downtrend, or neutral. (Note that if you are testing this mathematically, you
need to assign the current bar’s return to the previous condition. For instance,
imagine a situation where a large up day turns the trend indicator to an
uptrend. If you include that day in the uptrend designation, you will assume
that you were holding a long position from the previous day’s close, which is
possible only if you knew what was going to happen the next day in advance.
This is a small, but critical, adjustment.) Once we have categorized the days
according to trend condition, we can measure the mean return and volatility
for each group. If the trend indicator provides useful information, we should
be able to see some difference between the two groups. Ideally, the uptrend
days would have a higher mean return and perhaps a higher probability of
closing up than in the downtrend days.
Consider a very simple trend indicator: the slope of a moving average.
Immediately, we face the ubiquitous moving average question: “What length
of moving average?” By changing the length of the moving average, we can
usually make a trend flip to either up or down on almost any bar, so there is
an arbitrary element to this definition. The 50-period average is commonly
used in this capacity, so we will limit our testing to this one choice. Another
issue to consider is that, though a trader can easily identify the slope of a
moving average visually, doing so in a structured, quantitative manner is a
little bit more difficult. In this case, we draw a linear regression line through
the last five data points, equally weighted, of the average itself, and use the
slope of that linear regression line as the trend indicator.
Table 14.10 shows the results of a test of a 50-period moving average slope
trend indicator; it shows excess returns (in this case, excess return is the raw
return for the signal group minus the raw return for all bars) for the up and
down trend conditions relative to all days in this test. (Note that “All”
excludes days categorized as neutral when the slope of the average was flat.)
In addition, we have calculated two measures of volatility: the standard
deviation of raw (not excess) returns, and the mean of the 20-day historical
volatility readings for each set. Last, the percentage of days that close up is
calculated for each category.

Table 14.10 Trend Indicator: Slope of 50-Period Moving Average, Categorical

Returns

Equi
Equitie
tiess Futu
Future
ress Fore
Forex
x Rand
Random
om Total
otal

Down

N= 580,302 18,818 6,581 22,205 627,906

Mean Excess Return (bp) 177.2 (94.1) 15.5 (85.2) 157.7

StDev Raw Returns (bp) 302.8 152.4 75.6 119.9 293.2

Mean HisVol 39.9 20.4 10.5 18.1 38.2

% Close Up 50.2% 50.1% 50.5% 51.0% 50.2%

Up

N= 793,221 21,827 9,223 26,283 850,554

Mean Excess Return (bp) (129.6) 81.1 (11.1) 72.0 (116.4)

StDev Raw Returns (bp) 218.9 150.2 65.2 114.1 213.8

Mean HisVol 30.2 20.5 9.

9 .8 17.2 29.3

% Close Up 50.0% 51.1% 51.5% 51.7% 50.1%

All

N= 1,373,523 40,645 15,804 48,488 1,478,460

Mean Raw Return 2.3 1.8 1.5 1.2 2.3

StDev Raw Returns 257.7 151.2 69.7 116.8 250.6

Mean HisVol 34.3 20.5 10.1 17.6 33.1

% Close Up 50.1% 50.6% 51.1% 51.4% 50.2%

The results are not impressive for this trend indicator. Considering the
random column first to better understand the baseline, we do see a negative
excess return for the downtrend and a positive return for the uptrend
condition, with a slightly higher chance of close up (51.7% of days close up
in uptrend condition versus 51.4% for all days. (This is not statistically
significant.) Volatility is slightly higher for the downtrend, but roughly in line
across all groups. Turning to equities, we find something surprising: the
downtrend shows a very large, over 3 percent, positive excess return, while
the uptrend shows well over a 1% negative excess return; this is precisely the
opposite of what we should see if the uptrend indicator is valid. In fact, for
equities, this suggests we might be better off taking long trades in the
downtrend condition because we would be aligned with a favorable statistical
tailwind. Futures show a situation that is more like what we would expect,
with a fairly large negative excess return for downtrend, and a large positive
excess return for uptrend. Forex, paradoxically enough, looks more like
equities, but the actual excess returns are very small, and are not statistically
significant.
 How can this be? If you try this experiment yourself, put a 50-period
moving average on a chart, and just eyeball it, you will see that the slope of
the moving average identifies great trend trades. It will catch every extended
trend trade and will keep you in the trade for the whole move—actually, for
the whole move and then some, and there’s the rub. The problem is the lag,
the same problem that any derived indicator faces. Whether based on moving
averages, trend lines, linear regression lines, or extrapolations of existing
data, they can respond to changes in the direction of momentum of prices
only after those changes have happened. A moving average slope indicator
will also get whipsawed frequently when the market is flat and the average is
rapidly flipping up and down. It is possible to introduce a band around the
moving average to filter some of this noise, but this will be at the expense of
making valid signals come even later.
Figure 14.13 illustrates the problem with a 50-period moving average
applied to a daily chart of the U.S. Dollar Index. It would have been slightly
profitable to trade this simple trend indicator on this particular chart, but
notice how much of the move is given up before the indicator flips. The chart
begins with the market in an uptrend (moving average sloping up), and nearly
one-third of the entire chart has to be retraced before the moving average flips
down. Once the market bottoms in November, a substantial rally ensues
before the trend indicator flips up. This lag, coupled with the fact that markets
tend to make sharp reversals from both bottoms and tops, greatly reduces the
utility of this tool as a trend indicator.
indicator.

 Figure
 Figure 14.13 Slope of 50-Period Moving
Moving Average
Average as a Tren
Trend
d Indicator
  Notice how much of each trend
trend move is given up by
by this tool.
Another common idea is to use the position of two or more moving
averages to confirm a trend change. For example, three moving averages of
different lengths could be applied to a chart, and the market could be assumed
to be in an uptrend when the averages are in the correct order, meaning that
the shortest average would be above the medium-length average and both of
those would be above the longer-term moving average, with the reverse
conditions being used for a downtrend. This type of plan allows for
significant stretches of time when the trend is undefined; for instance, when
the medium-length average is above the longer-term average, but the shortest
average is in between the two. This, like all moving average crosses, is
attractive visually because the eye is always drawn to big winners, to the clear
trends that this tool catches. However, like all moving average crosses, the
whipsaws erode all profits in most markets, leaving the tool with no
quantifiable edge. In addition, more moving averages usually introduce more
lag, with no measurable improvement compared to a simple moving average
crossover.
One of the most popular moving average trend indicators today is based on
simple 10-, 20-, and 50-period moving averages. Traders using this tool are
told to take long trades only when it indicates an uptrend and to short only
when it indicates a downtrend. It is reasonable to ask how the market behaves
in both of those conditions. Table 14.11 shows that traders using this tool in
Equities (and it is primarily used by stock traders) will consistently find
themselves on the wrong side of the market, fighting the underlying statistical
tendency. Simply put, stocks are more likely to go down when this tool flags
an uptrend, and up when it flags a downtrend—traders using it as prescribed
are doing exactly the wrong thing. For the other asset classes, the message is
mixed. There is possibly an edge in futures, particularly on the short side, and
forex looks more random than the actual randomly generated test set. At least
in this sample of markets, this test suggests that traders relying on this trend
tool or on tools derived from it are likely to have a difficult time overcoming
these headwinds.

Table 14.11 Triple Moving Average Trend Indicator, Categorical Returns

Equi
Equiti
ties
es Futu
Future
ress Fore
Forex
x Rand
Random
om Total
otal
Down

N= 365,690 12,372 4,204 14,621 396,887

Excess Ret 166.9 (203.0) 43.6 (74.9) 144.5

StDev Raw Returns 319.2 158.8 78.0 119.9 308.6

Mean HisVol 40.8 20.7 10.7 18.5 39.0

% Close Up 50.2% 49.8% 50.5% 50.9% 50.2%

Up

N= 537,896 14,652 6,137 17,515 576,200

Excess Ret (157.9) 80.7 43.9 98.6 (141.5)

StDev Raw Returns 217.6 152.3 66.4 114.6 212.7

Mean HisVol 30.2 20.2 9.6 17.2 29.4

% Close Up 49.9% 51.5% 51.7% 52.0% 50.0%

All

N= 903,586 27,024 10,341 32,136 973,087

Raw Returns 2.0 1.3 1.9 1.4 2.0

StDev Raw Returns 263.5 155.3 71.3 117.0 256.2

Mean HisVol 34.5 20.4 10.1 17.8 33.3

% Close Up 50.0% 50.7% 51.2% 51.5% 50.1%

Equity Volatility in Relation to Moving Averages

So far, these tests of moving averages have not shown good results, but
there is one use of moving averages that long-term investors might want to
consider. Table 14.12 shows mean returns, standard deviation of returns, and
coefficients of variation for two simple systems applied to the daily Dow
Jones Industrial Average from 1/1/1960 to 12/31/2010. The first system is
simply buy and hold, which returns an impressive 1,504% over this time
period, with a daily mean return of 2.2 basis points and a standard deviation
of 101.5 basis points. The second system is long-only, and is in the market as
long as the market is above the 200-day moving average. On the day the Dow
Jones Industrial Average crosses its 200-day average, the system moves fully
to cash on the close of that day and does not reenter the market until the close
of the first day that closes back above the moving average. The moving
average system returns are still a very satisfactory 1,408% by being in the
market approximately two-thirds of the time that buy and hold was invested.
Furthermore, this system achieves its returns with a higher mean return and a
lower daily standard deviation. (For this test, no interest was paid on cash
balance, and no financing costs or trading frictions were assumed.) Buy and
hold has a coefficient of variation of 46, compared to 24.3 for this simple
moving average system—this is a clear example of superior risk-adjusted
performance from a simple technical system. How many money managers
realize that they could beat the market by applying simple technical criteria
like this? How many dollars of investors’ money could have been saved in the
form of management fees in the often futile effort to achieve the goal of
superior returns with lower risk?

Table 14.12 Buy and Hold Compared to Long-Only above 200-Day Moving
Average,
DJIA, 1960–2010

Buy and Hold Long above 200-Day MA

Total ROI 1,504% 1,408%

Mean Return 2.2 3.3

Standard Deviation 101.5 79.8

Coefficient of Variation 46.0 24.3

Days Invested 12,842 8,450

Conclusions
This section has looked at many variations of tests on moving averages. On
one hand, the answers were not crystal clear because there were some
interesting and statistically significant tendencies in some of the tests.
However, the same tendencies are present regardless of the specific period of
average tested, even if the length of the average changes randomly from bar
to bar, and often even without the moving average being present. This
evidence strongly contradicts the claim that any one moving average is
significant or special. Traders depend on moving averages because they are
lines on their charts and they sometimes seem to support prices, but this is a
trick of the eye. If you are depending on moving averages, considering the 50-
, 10-, or 100-day moving average to be support or resistance, you are trading a
concept that has no statistical validity.
This section also taught us some potentially useful things about market
movements. For example, we saw that equities, futures, and forex markets
sometimes show significant differences in the way they trade. It probably
does not make sense to approach them all the same way and to trade them
with the same systems and methods. We also saw evidence that some of the
tendencies that seem to be around moving averages may actually be deeper,
more universal elements of price action. For instance, the tendency for some
asset prices to bounce after trading down to a moving average may simply be
the tendency for prices to bounce after trading down. We also took a brief
look at trend indicators derived from moving averages, and saw that they
suffered from enough lag and mean reversion that they may often put the
trader on the wrong side of the market.
 variations (e.g., Tuesday-to-Tuesday weeks, or months tied to expiration
cycles in options or futures) are even used in some applications. Here is an
important clue: if we see something that “always works” no matter what we
change or tweak, our first assumption should be that there is some error in our
thinking. The other alternative, that the tendency is so strong and the pattern
so powerful, easily descends into magical thinking and becomes a self-
sustaining proof. (A parallel outside of markets might be a panacea medicine
 —a powerful medicine that can cure any ill in any range of unrelated
conditions, but can do no harm, regardless of how much is taken. This type of
medicine does not exist, and any medicine not capable of doing harm most
likely does nothing at all.)
If you can randomly define the parameters and get the same result, isn’t that
suggestive of some random process at work? This should be our first warning
that perhaps our intuition is faulty, and there is something else going on here.
Figure 15.2 shows a random walk path with the with the maximum,
minimum, and open marked. The next tests will replicate this procedure
thousands of time to help build intuition about the location of the open under
a random walk.

 Figure 15.2 A Single Path Through a Random Walk Tree, with the Max,
 Min, and Open Marked

Random Walk Models

Let’s first deal with a more elementary question. Most people would
assume that, if we randomly select a tick from any point in a random walk
model, it should fall anywhere in the range covered by the random walk with
equal probability. We can test this easily through the following procedure: run
multiple paths through a binomial tree, recording the range (Max – Min) for
each path, and the value of a single randomly selected tick. Figure 15.3 shows
the distribution for the location of this randomly selected tick for 500,000
trials. The results are conclusive: a single, randomly selected tick will not fall
anywhere within the session’s range with equal probability in a random walk ,
but instead will cluster much more often around the middle of the range.

 Figure 15.3 Distribution of Randomly Selected Ticks Within the Session’s

 Range for a Random Walk Model
Upon further reflection, it becomes apparent that we would expect a
random walk to spend more time around the middle of its range than at either
extreme, so perhaps this is not surprising. This is also true in real markets.
There are tools that show the distribution of volume inside each individual bar
(see Figure 15.4). MarketProfile is the best known of these, and it shows that
most sessions have clusters of volume and activity at one or more prices; it is
extremely unusual to see activity evenly distributed over the range of an
actual trading session in a real market. If activity clusters in certain places, a
randomly selected tick is more likely to fall in those clusters than anywhere
else.
  Figure 15.4 Distribution of Volume Within Daily Bars
 Note that volume and trading activity are not evenly distributed throughout 
the range.

Position of the Open

At this point, you may be saying, “Wait a minute. You have just shown that
a random tick is more likely to be in the middle of the range than at the
extremes, so doesn’t this strengthen the case for the open being special?” The
problem here is the assumption that the open is just like any other tick.
To build some intuition about this, imagine that you are standing at the
beginning of a binomial tree that will soon begin its random walk forward.
(Perhaps you are standing at the precipice of a ski slope or something equally
dramatic.) You can squint and visualize the most probable future path of the
random walk as a fuzzy cone of probability that starts from your current
position. If you move, the center of the cone, indicating the most probable
terminal point of the random walk, moves with you. This is true because, in
the absence of any drift component, the expected value of a zero-mean
random walk process is the initial point; in other words, the best guess for the
ending point is the starting value. So, at the beginning of the run before the
first step, the probability cone extends downward, centered on your starting
location. It is equally likely that you will end up to the left or the right, but
you most likely will end somewhere around the middle of that cone.
Now, take the very first step of the random walk. Assume it is to the right,
and notice that the cone of probability has shrunk a little bit because you are
movements off the opening range, and that some profitable systems do
incorporate the opening range, but it is still important to cultivate a deep
understanding of price movements and the dynamics driving them.
 Chapter 16

Quantitative Evidence of the Two Forces

U
p to this point, the goal of this part has been twofold: One, to give
fairly in-depth examples of the kind of thinking and quantitative
analysis that can help to separate the wheat from the chaff, and valid
trading ideas from worthless, random ideas. This is not always simple, as
sometimes even properly defining the question is difficult, and results are
rarely black and white. The second goal has been to dispel some myths about
what works in the market. No moving average consistently provides
statistically significant support. No crossing of moving averages or slope of
moving averages provides a statistically significant trend indicator. What
tendencies we do see around moving averages actually occur in giant zones
around the moving average; we have seen no justification whatsoever for
watching any specific moving average value. There is no point in noting that
a market crosses the 100-day or 200-day moving average. No Fibonacci level
provides any significant reference point—a random number is as good as any
Fibonacci level. None of these things are any better than a coin flip!
Though this may be disheartening to some traders, I believe there is an
important message here—it is better to know you don’t know than to continue
to waste your time and energy trading futile concepts. If you have been using
these concepts in your trading, objectively consider your results. If they are
performing well, meaning that you have substantial and consistent profits
over a large sample size, then you have probably incorporated them into a
framework that includes other inputs, and your positive results depend on
many more elements. However, if you are struggling and are not pleased with
your results, maybe it is time to reevaluate the tools you are using. Give up
your preconceptions and your beliefs, and commit to finding what does work
in the market. Struggling traders using futile concepts have a simple choice— 
let go of beliefs and preconceptions that or holding you back, or let go of your
money. Let’s now turn to some ideas that do have validity, that reveal
significant truths about the nature of the market’s movements.

Mean Reversion
  Mean reversion  is a term used in several different contexts to explain the
markets’ tendency to reverse after large movements in one direction. In its
trivial form, traders say that price returns to a moving average. This is true, as
Figure 16.31 shows, but it is not always a useful concept, because there are
two ways to get to the average: price can move to the average or the average
can move to price. Since traders are paid on price movement, the second case
will usually result in trading losses. All of the points marked C in Figure
16.31 would have presented profitable mean-reversion trades, assuming that
traders had some tool to identify the points where price had moved a
significant distance from the moving average. They could simply have faded
these moves by shorting above the average, buying below, and exiting the
market when it came back to the moving average. This is the pattern that most
traders have in mind when they speak of mean reversion, but they forget the
possibilities of the sequence marked A to B. Here, the trader shorting at A
would eventually have exited the trade when price did, in fact, revert to the
average at point B, but the average was so far above the entry price that a
substantial loss would have resulted.
Mean reversion exists in two slightly different contexts: mean reversion
after a single large move (usually one bar on a chart) or mean reversion after a
more extended move (usually multiple bars on a chart). In reality, these are
the same concept on different time frames: what looks like a large multibar
move will usually resolve into a single large bar on a higher time frame. A
single large bar will usually include multibar trends on the lower time frame.
The bar divisions and time frames that traders create are more or less arbitrary
divisions; one of the skills discretionary traders work hard to develop is the
ability to see beyond those divisions to perceive the flow of the market for
what it really is.
Table 16.8 Trading 20-Day Channel Breakouts (Entry on Close, Five Days
between Consecutive Entries)

Equities—Buy Futures—Buy Forex—Buy

Diff. µsig – Diff. µsig – Diff.

Days µsig – µb Med. %Up µb Med. %Up µb Med. %Up

1 (12.3)**(6.0) 47.9%7.2* (0.1) 50.8%0.6 (2.1) 49.2%

2 (24.3)**(10.6) 48.0%8.5 7.2 54.9%4.0 (4.2) 49.7%

3 (30.1)**(11.4) 48.6%16.1* 8.9 55.4%1.8 2.1 53.6%

4 (32.4)**(12.9) 48.9%10.8 8.5 55.7%1.7 (1.2) 53.0%

5 (36.8)**(11.3) 49.6%12.4 4.5 54.9%2.3 9.7 56.4%

10 (54.5)**(8.0) 51.1%15.3 (9.7) 53.7%6.6 12.8 55.6%

15 (64.3)**2.1 52.4%17.7 (20.5) 52.9%15.2 7.1 56.5%

20 (69.5)**12.8 53.5%15.5 (32.6) 53.0%11.6 16.7 58.2%

Equities—Sell Futures—Sell Forex—Sell

Diff. µsig – Diff. µsig – Diff.

Days µsig – µb Med. %Up µb Med. %Up µb Med. %Up

1 7.8** 6.0 51.4%(1.2) 2.4 52.3%(2.8) (0.3) 51.4%

2 12.0** 13.0 52.4%1.5 (5.1) 50.5%(4.5) (4.2) 49.9%

3 16.0** 17.7 52.8%2.2 (7.6) 50.4%(4.7) (7.9) 48.4%

4 18.5** 23.3 53.2%(2.4) (11.9) 50.2%(11.4) (6.6) 50.5%

5 35.4** 33.3 54.0%(3.0) (10.9) 51.0%(7.2) (9.6) 50.1%

10 30.6** 50.5 54.8%(7.7) (23.0) 51.6%(8.4) (9.5) 53.8%

15 35.8** 82.2 56.1%(22.7) (54.1) 49.5%(15.2) (2.5) 54.9%

20 57.3** 106.3 57.2%(25.7) (65.2) 50.1%(26.1) (16.4) 54.9%

You may have wondered what a test of breakout trades is doing in a section
on mean reversion. The reason is simple: failed breakout trades are evidence
of mean reversion. Table 16.8 is another strong affirmation of mean reversion
in equities, showing negative returns for our buy signal and positive returns
for shorts—the signal conditions were exactly wrong for this sample. It
appears that there may be an exploitable opportunity by shorting new 20-day
highs and buying new 20-day lows in stocks, fading the channel breakout.
However, it would be a mistake to assume that you could apply the same
system to futures and forex just because it works in equities. The futures
sample seems to suggest an edge in going with the direction of the breakout.
Positive returns for buys and negative returns for shorts, though probably not
statistically significant (meaning that the trader actually trading these would
see extreme variability in the results), suggest that fading the channel
breakout in futures could be painful. This is also the first consistent signal we
have seen in forex, as all of the returns on the sell side are negative. However,
the small size of these returns, a few basis points at most, is a warning that
this may not be an easily exploitable tendency.
Twenty-day channel breakouts are common, occurring on approximately 4
percent of all trading days; using a longer period for the breakout might result
in more significant levels, so Table 16.9 shows the results of a 100-day
channel breakout. At this point, futures and forex finally start to show
something interesting, and we see that positioning with the direction of the
breakout is clearly the correct trade in these markets. Particularly in forex, the
signal size is small, but means and medians are consistently on the same side
of zero, and the series appears to be flirting with statistical significance. Is this
a stand-alone trading system? Probably not, but it is pretty strong evidence of
an underlying tendency in the market. Note that mean reversion is still alive
and well in equities in this test—this is perhaps the clearest evidence so far of
different behavior between these asset classes.

Table 16.9 Trading 100-Day Channel Breakouts (Entry on Close, Five Days
between Consecutive Entries)

Equities—Buy Futures—Buy Forex—Buy

Diff. Diff. Diff.

Days µsig – µb Med. %Up µsig – µb Med. %Up µsig – µb Med. %Up

1 (12.0)**(6.2) 47.4%10.1* 2.6 52.9%0.5 2.1 51.9%

2 (24.2)**(11.1) 47.3%14.1 17.9 58.2%10.8* 3.0 53.1%

3 (33.3)**(12.6) 47.9%23.7* 18.2 57.7%4.8 9.9 57.0%

4 (39.1)**(18.7) 47.8%25.0* 21.1 59.2%13.6* 18.6 58.5%

5 (46.7)**(17.4) 48.3%21.0 14.7 57.5%14.2 29.0 60.9%

10 (77.1)**(22.7) 49.2%43.2* 22.2 56.1%18.3 19.0 57.6%

15 (87.2)**(18.2) 51.0%59.6** 10.7 56.7%29.7* 18.8 59.4%

20 (98.4)**(15.0) 52.0%73.3 ** 0.3 57.3%31.8 * 45.6 61.5%

Equities—Sell Futures—Sell Forex—Sell

Diff. Diff. Diff.

Days µsig – µb Med. %Up µsig – µb Med. %Up µsig – µb Med. %Up
move was at least as large as the prevailing short-term volatility (in many
cases, it was much larger on the trigger days); and a filter that forced the
market to close strong on the day. This is an example of a test that moves us
much closer to a condition that could be traded in practice. Criteria were
reversed for shorts.

Table 16.13 Volatility Compression Breakout Test

(Ratio of 5-day to 40-day ATR < 0.5. Current day’s range is ≥ 5-day ATR
and current day closes in top 50 percent of day’s range.)

Equities—Buy Futures—Buy

Days µsig – µb Diff. Med. %Up µsig – µb Diff. Med. %Up

1 45.1 ** 22.1 55.2% 53.8 3.3 51.4%

2 74.1 ** 28.5 56.0% 69.6 30.8 56.8%

3 72.1 ** 19.1 54.1% 91.8 * 53.6 62.2%

4 56.1 ** 13.2 52.7% 128.2 ** 78.7 64.9%

5 28.7 * 14.3 52.7% 89.5 * 26.6 73.0%

10 (70.5)** (22.7) 49.7% 28.1 (10.7) 59.5%

15 (108.6)** (18.5) 50.9% (18.0) (43.2) 51.4%

20 (130.0)** (68.5) 48.6% (58.5) (7.2) 56.8%

Equities—Sell Futures—Sell

Days µsig – µb Diff. Med. %Up µsig – µb Diff. Med. %Up

1 (4.3) (0.1) 50.2% (37.7) (16.8) 43.8%

2 (22.5)* (4.5) 49.7% (39.5) (7.5) 50.0%

3 (27.5)* (10.9) 49.3% 2.1 10.7 58.3%

4 (42.8)** (15.2) 48.6% (15.0) 4.9 54.2%

5 (50.8)** (17.2) 49.1% (6.1) 21.4 56.3%

10 (120.8)** (46.0) 48.8% (28.0) (111.3) 37.5%

15 (123.8)** (71.7) 47.8% (105.2) (99.6) 41.7%

20 (76.2)* (57.5) 49.4% (119.2) (122.3) 45.8%

First of all, note that forex is excluded from this test. The reason is that
volatility fluctuates differently in forex, and these conditions produced only
five long trades and two short trades for forex. It would be extremely
misleading to draw conclusions from such a small sample; volatility
compression is alive and well in forex, but this particular way to quantify it
does not work very well. Leaving that aside for now, this test is one of the
most convincing we have seen so far. Means and medians are consistently on
the correct side of zero, which suggests that there is a real, underlying
tendency driving this trade. Two factors complicate this analysis. First,
sample sizes are small across the board, as this setup occurs approximately
once in every 500 trading days. Second, it is by definition a volatile trade,
with a wide dispersion of returns.
Up to this point, every test on equities has shown a clear tendency for mean
reversion—based on those past tests, it seems as though you could actually
trade equities simply by fading large moves. However, the addition of a very
simple filter has completely changed the results, and we have now identified a
subset of those large days that are more likely to continue in the same
direction. Furthermore, this filter condition seems to strengthen the tendency
for continuation in futures as well. Can we use this information to filter out
profitable breakout trades? Could we also use it to increase the probability of
success of mean reversion trades, by not taking them in times of volatility
compression? The answer to both questions is a resounding yes.

Pullbacks
The other condition that can set a market up for a range expansion move is
a pullback after a sharp directional move. What usually happens is the large
move exhausts itself, mean reversion takes over, and part of the move is
reversed while the market reaches an equilibrium point. After a period of
relative rest, the original movement reasserts itself and the market makes
another thrust in the initial direction. (We looked at this structural tendency in
some detail in the section on Fibonacci retracements, and it was one of the
most important trading patterns from earlier sections of this work.) The
concept of impulse, retracement, impulse is valid—it actually is one of the
most important patterns in the market.
We also saw earlier in this chapter that expecting moving averages to
provide support and resistance is not likely to be a path to profitable trading.
However, there is more to the story. A moving average does mark a position
of relative equilibrium and balance, but the key question is “relative to what?”
The answer—relative to the market’s excursions from that particular average
 —implies that some way must be found to standardize those swings and the
distances from the average. Fortunately, both Keltner channels and Bollinger
bands present an ideal way to do this. They adapt to the volatility of the
underlying market, and so, when properly calibrated, they mark significant
extensions in all markets and all time frames. Figure 16.8 shows four short
entries according to the following criteria:

Shorts are allowed after the market closes below the lower
Keltner channel.
The entry trigger is a touch of the 20-period exponential
moving average.
Only one entry is allowed per touch of the channel; once a short
entry has been taken, price must again close outside the lower
channel to set up another potential short.
Rules are symmetrical to the buy side.
Fisher, Mark B. The Logical Trader: Applying a Method to the Madness.
Hoboken, NJ: John Wiley & Sons, 2002.
Grimes, Adam H. The Art and Science of Technical Analysis: Market 
Structure, Price Action, and Trading Strategies. Hoboken, NJ: John Wiley
& Sons, 2012.
Grossman, S., and J. Stiglitz. “On the Impossibility of Informationally
Efficient Markets.” American Economic Review 70 (1980): 393–408.
Harris, Larry. Trading and Exchanges: Market Microstructure for
 Practitioners. New York: Oxford University Press, 2002.
Hintze, Jerry L., and Ray D. Nelson. “Violin Plots: A Box Plot-Density Trace
Synergism.” American Statistician 52, no. 2 (1998): 181–184.
Jung, C. G.  Psychology and Alchemy.  Vol. 12 of Collected Works of C. G.
Jung. Princeton, NJ: Princeton University Press, 1980.
Kelly, J. L., Jr. “A New Interpretation of Information Rate.” Bell System
Technical Journal 35 (1956): 917–926.
Kirkpatrick, Charles D., II. Technical Analysis: The Complete Resource for
 Financial Market Technicians. Upper Saddle River, NJ: FT Press, 2006.
Langer, E. J. “The Illusion of Control.” Journal of Personality and Social
 Psychology 32, no. 2 (1975): 311–328.
Lo, Andrew. “Reconciling Efficient Markets with Behavioral Finance: The
Adaptive Markets Hypothesis.” Journal of Investment Consulting,
forthcoming.
Lo, Andrew W., and A. Craig MacKinlay.  A Non-Random Walk Down Wall
Street. Princeton, NJ: Princeton University Press, 1999.
Lucas, Robert E., Jr. “Asset Prices in an Exchange Economy.”  Econometrica
46, no. 6 (1978): 1429–1445.
Macnamara, Brook N., David Z. Hambrick, and Frederick L. Oswald.
“Deliberate Practice and Performance in Music, Games, Sports, Education,
and Professions: A Meta-Analysis.”  Pyschological Science Vol 25, Issue 8
(2014), pp. 1608–1618.
Malkiel, Burton G. “The Efficient Market Hypothesis and Its Critics.”
Journal of Economic Perspectives 17, no. 1 (2003): 59–82.
Mandelbrot, Benoît, and Richard L. Hudson. The Misbehavior of Markets: A
 Fractal View of Financial Turbulence. New York: Basic Books, 2006.
Mauboussin, Michael J. “Untangling Skill and Luck: How to Think about
Outcomes—Past, Present and Future.” Maboussin on Strategy, Legg Mason
Capital Management, July 2010.
Miles, Jeremy, and Mark Shevlin.  Applying Regression and Correlation: A
Guide for Students and Researchers.  Thousand Oaks, CA: Sage
Publications, 2000.
Niederhoffer, Victor. The Education of a Speculator. New York: John Wiley &
Sons, 1998.
Plummer, Tony.  Forecasting Financial Markets: The Psychology o
Successful Investing, 6th ed. London: Kogan Page, 2010.
Raschke, Linda Bradford, and Laurence A. Conners. Street Smarts: High
 Probability Short-Term Trading Strategies.  Jersey City, NJ: M. Gordon
Publishing Group, 1996.
Schabacker, Richard Wallace. Technical Analysis and Stock Market Profits.
New York: Forbes Publishing Co., 1932 .
Schabacker, Richard Wallace. Stock Market Profits.  New York: Forbes
Publishing Co., 1934.
Schwager, Jack D. Market Wizards: Interviews with Top Traders. New York:
HarperCollins, 1992.
Snedecor, George W., and William G. Cochran. Statistical Methods, 8th ed.
Ames: Iowa State University Press, 1989.
Soros, George. The Alchemy of Finance: Reading the Mind of the Market.
New York: John Wiley & Sons, 1994.
Sperandeo, Victor. Trader Vic: Methods of a Wall Street Master.  New York:
John Wiley & Sons, 1993.
Sperandeo, Victor. Trader Vic II: Principles of Professional Speculation. New
York: John Wiley & Sons, 1998.
Steenbarger, Brett.  Enhancing Trader Performance: Proven Strategies From
the Cutting Edge of Trading Psychology.New York: John Wiley & Sons,
2008.
Taleb, Nassim.  Fooled by Randomness: The Hidden Role of Chance in Life
and in the Markets. New York: Random House, 2008.
Tsay, Ruey S. Analysis of Financial Time Series. Hoboken, NJ: John Wiley &
Sons, 2005.
Vince, Ralph. The Leverage Space Trading Model: Reconciling Portfolio
 Management Strategies and Economic Theory. Hoboken, NJ: John Wiley &
Sons, 2009.
Waitzkin, Josh. The Art of Learning: An Inner Journey to Optimal
 Performance. New York: Free Press, 2008.
Wasserman, Larry.  All of Nonparametric Statistics.  New York: Springer,
2010.
Wilder, J. Welles, Jr.  New Concepts in Technical Trading Systems.
McLeansville, NC: Trend Research, 1978.

About the Author

Adam Grimes has over two decades experience as a trader and system
developer. He’s worked for small firms and big firms, from the farmlands of
the Ohio Valley to the trading floors of the New York Mercantile Exchange.
He is Managing Partner and CIO for Waverly Advisors, LLC, a New York-
based research and advisory firm for which he writes daily market
commentary . He blogs regularly at adamhgrimes.com, and is also a
contributing author for many publications on quantitative finance and trading,
and is much in demand as a speaker and lecturer.
In addition to being a trader, Adam also has deep training in classical music
(piano and composition) and classical French cooking. His perspective is both
deeply quantitative and practical, and he has done extensive personal work
developing his skills as a teacher, coach, and mentor. He is fascinated by the
limits of human knowledge and peak performance—specifically, how do we
get there and stay there, and how to teach others to do the same?
Adam’s relentless focus on trading excellence and self-development
through financial markets has created a unique body of work that has helped
many traders move along the path to trading success.
You can find much more of Adam’s work and teaching on his website,
MarketLife.com.
Table of Contents
1. Frontmatter
2. 1: Chartreading 101
3. 2: Chartreading, Going Deeper
4. 3: Market Structure & Price Action
5. 4: Pullback
6. 5: Anti
7. 6: Failure Test
8. 7: Breakout
9. 8: Pattern Failure
10. 9: Practical Trading Psychology
11. 10: Academic Theories of Market Behavior
12. 11: Tools for Quantitative Analysis of Market Data
13. 12: Test Universe & Methodology
14. 13: Fibonacci Retracements
15. 14: Moving Averages
16. 15: The Opening Range Phenomenon
17. 16: Quantitative Evidence of the Two Forces
18. Backmatter