Dr. Bill Hyde writes:
There's a whole discipline (though scare quotes should be used there) on predicting the coming shape of the market indexes or of a single stock. It is called "technical analysis" and as you can imagine I'm not a big fan of it. Academic studies have shown it to have predictive skill not different from random chance (cf Burton Malkiel's "A random walk down wall street" which gives the results of such studies, among much else - highly recommended reading).
Though the TA types are not doing what you are doing, it might be useful to be familiar with their various techniques (they tend, among other things, to focus on moving averages. When the 50 day moving average crosses the 200 day good or bad things are supposed to happen, depending on the direction. Also, I think, Macbeth gets dethroned).
Brokers mostly don't believe in TA either, but it generates many buy and sell signals, so they're happy when a client believes, and most full service brokerages keep technical types on staff.
Another good book is "Bull" by Maggie Mahar. It's a fun read, and it talks a bit about "Dow theory", which is distillation of what market crashes look like. It seems a bit mystical - the claim is that a bear market will need three down "legs" before it is over. How a leg is defined I don't know. But mainly the book talks about the follies of the 90s, with more than a hint of the follies to come. There's an afterword written about 2004 to which I should have paid more
attention. Much more.
Dr. Thomas Crowley Responds:
Bill, thank you. as usual, your comes are insiteful and succinct. normally I would never engage in this activity on a continuing basis, but our ideas of abrupt transitions made us wonder whether there is any skill at all when it comes to the most extreme events, esp. since we noticed this striking similarity between the present situation and 1929. as you well know, this could be entirely coincidental, although at least we can say we made a semi-quantitative prediction in mid-November that has so far been born out - perhaps the recent actions of the Obama Administration will herald a comeback - I suspect not, not based on technical knowledge but on the suspicion that a depression is an indication of something fundamentally wrong in the wiring diagram of an economy and the only way to fix it is to break it apart and start anew - thus my analogy to the biological concept of punctuated equilibrium, which has some striking parallels in human societal history (eg, revolutions). this could all be blog blather - pet thoughts that have no more merit than that - something nice to stroke. nevertheless, using the climatologist "experimental forecast" terminology, I think one cannot fix something that may be fundamentally broken - although I guess here the real question is whether this is fundamental or truly correctable within the present set of boundary conditions?
Dr. Bill Hyde Responds:
I have a guess, not even at the hypothesis level, based on reading Galbraith's book on economic bubbles (also recommended, and it's very short).
What happens, it seems to me, after a bubble has burst, is that the money is in the wrong hands. In the hands of successful speculators, not investors. And therefore it is not put to productive use.
As an example, after the tulip bulb craze a speculator might find that he now owned, say, a brewery. He has no idea how to run a brewery, and really no desire to learn. His employees will keep it ticking over, but without the kind of attention the former owner gave it, productivity and quality will slide. Any profits from the brewery he will use to finance further speculation, not to improve the business.
It takes years for productive assets to be transferred back to people with the skill and desire to run them. Thus the long depressions that followed the tulip bubble, the south sea bubble, and so on.
How much this matters in big corporations is another question, as they are not run by their owners. Probably less or not at all. But there's a lot of small to medium sized business out there.
Of course this isn't the only structural problem. But I think it matters. The socialist historian AJP Taylor commented on the need for trust in capitalism. The system only works because, within limits, capitalists can trust one another. That trust is broken or at least severely damaged now. Until it is restored, business will be hard to do.
Forecast update to follow...
Monday, 13 April 2009
Friday, 20 March 2009
A Correct Forecast Continued...
Update: by Thomas Crowley & Matthew Unterman
This plot is a 20 March 2008 update of our 6 March figure, assessing effect of recent rise in Dow - although the market has gone up about 800 points in the last two weeks, peak values of about 7400 are still significantly below "plateau" (see further discussion below) values of about 8700. This could of course be just the beginning of a market surge; it could also be what is sometimes picturesquely called on the Street as a "dead cat bounce", before it retreats again.
For what its worth, our forecast stands as before: although factors such as price/earnings ratio are favorable to a market surge, the overall economic situation is still dismal, with respect to very weak banking sector and the $500 trillion* (yes you read that right) in derivatives still "out there". That, plus the ~50% drop, leads us to forecast that the present upsurge will flatten into a "still-stand" before going down again.
Sorry, we would be very happy to be wrong, but this is the way we see it - not as economists but as two people familiar with analysis of abrupt transitions in noisy systems (in our case, the climate system).
PS: In case you were wondering my collaborator Matt Unterman informs me that the "other side of a trillion" is a "quadrillion" - so, in lingo, total derivatives now "out there" would represent about "half a quad".
This plot is a 20 March 2008 update of our 6 March figure, assessing effect of recent rise in Dow - although the market has gone up about 800 points in the last two weeks, peak values of about 7400 are still significantly below "plateau" (see further discussion below) values of about 8700. This could of course be just the beginning of a market surge; it could also be what is sometimes picturesquely called on the Street as a "dead cat bounce", before it retreats again.
For what its worth, our forecast stands as before: although factors such as price/earnings ratio are favorable to a market surge, the overall economic situation is still dismal, with respect to very weak banking sector and the $500 trillion* (yes you read that right) in derivatives still "out there". That, plus the ~50% drop, leads us to forecast that the present upsurge will flatten into a "still-stand" before going down again.
Sorry, we would be very happy to be wrong, but this is the way we see it - not as economists but as two people familiar with analysis of abrupt transitions in noisy systems (in our case, the climate system).
PS: In case you were wondering my collaborator Matt Unterman informs me that the "other side of a trillion" is a "quadrillion" - so, in lingo, total derivatives now "out there" would represent about "half a quad".
Friday, 6 March 2009
Successful Forecast of Present (early March 2009) Dow Jones Drop
By Matthew Unterman & Thomas Crowley
My supervisor, Thomas Crowley, and I go to lunch early, around 11:30 A.M. every morning to try and beat the deluge of students and background noise that flood the lounge around noon time. This daily ritual is in fact an ongoing blog... anything is fair game. We tend to talk about what we have read in the news for the morning, or how god awfully slow I have become in writing up scientific papers... but then there are sometimes moments we just sit in silence, enjoying the 20 or so minutes of tranquility within the University before chaos ensues and its back to the salt mines and my super computer...
There was one time however that we began discussing how it seemed the Dow Jones (DJI) was wiggling about wildly... almost chaotic... a time of increased variability within the stock market… and then it hit. My supervisor recently published a paper in Nature ((Nature 456, 226-230 (13 November 2008)), he and a coauthor conjectured that rapid changes in past climate could sometimes be explained by some "nonlinear" (*) feature of the climate system. They suggested that increasing variability might be one predictor of an abrupt climate change, and that the high variability in climate of the last few hundred thousand years might be an indicator of some future transition to a different, more stable, climate state - permanent midlatitude northern hemisphere glaciation. Consequently our early morning tradition became a time to speculate that perhaps changes in the DJI might also be examined this way... but without the glaciers.
I was able to acquire a long daily time-series of the DJI extending back into the 1920’s from Yahoo Finance and a communication with my father… who happens to be a Wall Street Lawyer… but I digress. Analysis of the variability in the DJI led to ambiguous findings (Fig. 1) - yes there has been a lot of variability lately, but high variability has occurred at other times also just as one would discuss natural variability in the climate system - such as the sharp drop in 1987, when the market recovered fairly quickly. So we just started looking at the raw DJI back to 1927 (Fig. 2) and later, compared the more recent 2007-2008 changes to what happened in 1929. Interestingly, we found that the DJI went through several transitions after 1929, and did not bottom out until 1933 - almost four years later. It is also interesting to point out that the DJI is a perfect proxy for historic events, even in the smaller, less noticeable wiggles in variability (for instance the battle of Shanghai in 1937 is clearly visible and potentially even the battle of Marco Polo Bridge).
Not counting the one day bash of 1987 the magnitude of variability has not been seen since the Great Depression and the years following it.
We noticed that after each of the drops there was a period of stability that we called "backing and filling" and that after a few months of this the 1929-1933 went through another drop. These "stagger steps" did not seem to be a feature of other market drops (Fig 2, Fig 3 - enlarged version of last 15 years), so it could be unique feature of more fundamental market shifts… but this is only speculation.
You can see that we made a prediction back in November on the "Next Transition"...
Thomas’s interpretation of these "back and fill" events is “that the economic shock of the preceding drop takes a while to work its way through the system and may even be countered somewhat by some investors betting that the market has bottomed out. Whatever, in the UK there has been a steady flow of bad news doled out in daily doses over the last couple of months of business closings and unemployment figures. These continued unabated, weakening the already weak banking structure. The recent clustering of bad news from the much-overrated Royal Bank of Scotland, Halifax Bank of Scotland, and AIG were the immediate factors precipitating the next phase of drops, but those big number drops reported by RBS, HBOS, and AIG might not have happened if there had not been a steady stream of smaller bad news preceding. In this respect it is very much like climate theory, where major transitions can be precipitated by small incremental changes crossing "threshold levels", after which there is a big response.”
We therefore tried an experimental forecast approach on November 6 2008, sending the forecast only to my father (refer to dated figure) and trying an experimental forecast* of the next transition, if it were to occur, based on the average time of stability in both the 1929 and 2008 time series (Fig. 4). We predicted a transition, if it were to occur, in the timeframe 1 Feb-Apr 1 2009…. Which, if you are not one of my kitchen flies, is a relatively good prediction thinking of the “now” (Fig. 5). Obviously this agreement by no means indicates proof of the method - but is “better” than failing! Though I wish we were wrong of course…
With at least one forecast to our credit we therefore thought it might be useful to post this information for others to contemplate. Again, caveat emptor.
We will issue another forecast soon, after a little more data are collected - probably end of March early April time frame.
In the meantime, one of us (Thomas Crowley) feels emboldened to state his personal suspicions what is going to transpire:
“There will not be a recovery after the next drop, as many economists are forecasting. Tom thinks this is as much "group think" and wishful thinking as much as anything, as we really don't have other experience (other than the Great Depression) of the market dropping to below 50% of its value. If analysis of abrupt transitions (see above) has any bearing with respect to the future of the DJI, then my experience would be to conjecture that the system has passed the point of no return and will descend into a depression within the next year or year and a half. Beyond that I do not believe that anyone can make any predictions on when the market will recover. I should note that it took the DJI nearly 25 years to surpass the 10 October 1929 peak (Sept 17, 1954), so let us hope that is not a predictor of the future.”
And honestly, I have to agree. It is almost like any news is bad news in the media and that regardless of the trillions of US Dollars being funneled into the system the downward trend might not halt. Was it wise to toss hundreds of billions back into the market to attempt a fix? I’m not sure… I honestly cannot think that I would sit around and do nothing… so I guess something needs to be done. Are we passed the point of no return? Our prediction basically points us in that direction. Yet I am allowed to hope we are wrong… kind of makes me want to scream “stop selling people!” all at once… but then again I don’t want to enforce a motion of no confidence…maybe it takes a catastrophe to make it all better anyway…
More to come…
*TC: A more specific prediction, and even more conjectural, comes from my 1.5 years of living in the UK (I am a US citizen). I personally think the UK is the weak link in the entire global financial market, and that it will be the weak link (not Iceland or Ukraine - something bigger) that is critical rather than the largest economies (US and China). I just do not believe the UK has enough resources to bolster banks much more and that at some point in the next year they are going to collapse on their own weight. Then the walls of Jericho will truly comes tumbling down
*Matthew's background is in computer science and paleoclimate modeling. Application of a 3-day, 5-day, and 10-day basic variability algorithm was applied to the DJI daily average to determine the degrees of variation within the time series.
* This term comes from climate science - Crowley's background, and originated more than 25 years ago with winter forecasts made by the National Weather Service on 1 December of each year. These forecasts have now been more routinely updated and use more numerical methods (rather than primarily vast experience of the forecaster), but the concept is the same - they are experimental forecasts, they are not predictions, and they are by no means at the state where one would want to bet their life savings on. This is for information only, with the "user" taking full responsibility for any actions taken because of the forecast.
My supervisor, Thomas Crowley, and I go to lunch early, around 11:30 A.M. every morning to try and beat the deluge of students and background noise that flood the lounge around noon time. This daily ritual is in fact an ongoing blog... anything is fair game. We tend to talk about what we have read in the news for the morning, or how god awfully slow I have become in writing up scientific papers... but then there are sometimes moments we just sit in silence, enjoying the 20 or so minutes of tranquility within the University before chaos ensues and its back to the salt mines and my super computer...
There was one time however that we began discussing how it seemed the Dow Jones (DJI) was wiggling about wildly... almost chaotic... a time of increased variability within the stock market… and then it hit. My supervisor recently published a paper in Nature ((Nature 456, 226-230 (13 November 2008)), he and a coauthor conjectured that rapid changes in past climate could sometimes be explained by some "nonlinear" (*) feature of the climate system. They suggested that increasing variability might be one predictor of an abrupt climate change, and that the high variability in climate of the last few hundred thousand years might be an indicator of some future transition to a different, more stable, climate state - permanent midlatitude northern hemisphere glaciation. Consequently our early morning tradition became a time to speculate that perhaps changes in the DJI might also be examined this way... but without the glaciers.
I was able to acquire a long daily time-series of the DJI extending back into the 1920’s from Yahoo Finance and a communication with my father… who happens to be a Wall Street Lawyer… but I digress. Analysis of the variability in the DJI led to ambiguous findings (Fig. 1) - yes there has been a lot of variability lately, but high variability has occurred at other times also just as one would discuss natural variability in the climate system - such as the sharp drop in 1987, when the market recovered fairly quickly. So we just started looking at the raw DJI back to 1927 (Fig. 2) and later, compared the more recent 2007-2008 changes to what happened in 1929. Interestingly, we found that the DJI went through several transitions after 1929, and did not bottom out until 1933 - almost four years later. It is also interesting to point out that the DJI is a perfect proxy for historic events, even in the smaller, less noticeable wiggles in variability (for instance the battle of Shanghai in 1937 is clearly visible and potentially even the battle of Marco Polo Bridge).
Not counting the one day bash of 1987 the magnitude of variability has not been seen since the Great Depression and the years following it.
We noticed that after each of the drops there was a period of stability that we called "backing and filling" and that after a few months of this the 1929-1933 went through another drop. These "stagger steps" did not seem to be a feature of other market drops (Fig 2, Fig 3 - enlarged version of last 15 years), so it could be unique feature of more fundamental market shifts… but this is only speculation.
You can see that we made a prediction back in November on the "Next Transition"...
Thomas’s interpretation of these "back and fill" events is “that the economic shock of the preceding drop takes a while to work its way through the system and may even be countered somewhat by some investors betting that the market has bottomed out. Whatever, in the UK there has been a steady flow of bad news doled out in daily doses over the last couple of months of business closings and unemployment figures. These continued unabated, weakening the already weak banking structure. The recent clustering of bad news from the much-overrated Royal Bank of Scotland, Halifax Bank of Scotland, and AIG were the immediate factors precipitating the next phase of drops, but those big number drops reported by RBS, HBOS, and AIG might not have happened if there had not been a steady stream of smaller bad news preceding. In this respect it is very much like climate theory, where major transitions can be precipitated by small incremental changes crossing "threshold levels", after which there is a big response.”
We therefore tried an experimental forecast approach on November 6 2008, sending the forecast only to my father (refer to dated figure) and trying an experimental forecast* of the next transition, if it were to occur, based on the average time of stability in both the 1929 and 2008 time series (Fig. 4). We predicted a transition, if it were to occur, in the timeframe 1 Feb-Apr 1 2009…. Which, if you are not one of my kitchen flies, is a relatively good prediction thinking of the “now” (Fig. 5). Obviously this agreement by no means indicates proof of the method - but is “better” than failing! Though I wish we were wrong of course…
With at least one forecast to our credit we therefore thought it might be useful to post this information for others to contemplate. Again, caveat emptor.
We will issue another forecast soon, after a little more data are collected - probably end of March early April time frame.
In the meantime, one of us (Thomas Crowley) feels emboldened to state his personal suspicions what is going to transpire:
“There will not be a recovery after the next drop, as many economists are forecasting. Tom thinks this is as much "group think" and wishful thinking as much as anything, as we really don't have other experience (other than the Great Depression) of the market dropping to below 50% of its value. If analysis of abrupt transitions (see above) has any bearing with respect to the future of the DJI, then my experience would be to conjecture that the system has passed the point of no return and will descend into a depression within the next year or year and a half. Beyond that I do not believe that anyone can make any predictions on when the market will recover. I should note that it took the DJI nearly 25 years to surpass the 10 October 1929 peak (Sept 17, 1954), so let us hope that is not a predictor of the future.”
And honestly, I have to agree. It is almost like any news is bad news in the media and that regardless of the trillions of US Dollars being funneled into the system the downward trend might not halt. Was it wise to toss hundreds of billions back into the market to attempt a fix? I’m not sure… I honestly cannot think that I would sit around and do nothing… so I guess something needs to be done. Are we passed the point of no return? Our prediction basically points us in that direction. Yet I am allowed to hope we are wrong… kind of makes me want to scream “stop selling people!” all at once… but then again I don’t want to enforce a motion of no confidence…maybe it takes a catastrophe to make it all better anyway…
More to come…
*TC: A more specific prediction, and even more conjectural, comes from my 1.5 years of living in the UK (I am a US citizen). I personally think the UK is the weak link in the entire global financial market, and that it will be the weak link (not Iceland or Ukraine - something bigger) that is critical rather than the largest economies (US and China). I just do not believe the UK has enough resources to bolster banks much more and that at some point in the next year they are going to collapse on their own weight. Then the walls of Jericho will truly comes tumbling down
*Matthew's background is in computer science and paleoclimate modeling. Application of a 3-day, 5-day, and 10-day basic variability algorithm was applied to the DJI daily average to determine the degrees of variation within the time series.
* This term comes from climate science - Crowley's background, and originated more than 25 years ago with winter forecasts made by the National Weather Service on 1 December of each year. These forecasts have now been more routinely updated and use more numerical methods (rather than primarily vast experience of the forecaster), but the concept is the same - they are experimental forecasts, they are not predictions, and they are by no means at the state where one would want to bet their life savings on. This is for information only, with the "user" taking full responsibility for any actions taken because of the forecast.
Subscribe to:
Posts (Atom)