About heteroskedasticity in MT. To check this stuff you can use Spearman test for example. But it won’t help you. The answer and the problem at the same time lay in optimization. To get lower variance (decrease stochastic deviation) you can use factor screening procedure over time and maybe ranks. You ll get not linear dependency changing over time with very high R2. But more likely it wont work in future (fitting problem again).
So the main check is macro and micro economy logic and common sense. And it should go first before any optimization (at least it sets up the limits of you optimization)
MT as far as the non-stationarity concern because to the time series. My understanding is that the problems with non-stationarity occur with time-series.
Your linear regression seemed to have some heteroskedasticity: the realized 12M return regression. You can comment for sure as to the type of data and your statistical conclusions from this data.
Thanks.
Jim
I didn’t chase the goal of maximizing R2 in this regression, it was shown just as example of more or less reliable MT systems in comparison to existing P123 systems.
Yury, now it was my turn to edit my post. 
Just keep going with your thoughts as I am curious about finale.
Thank you.
You really just can’t use non-stationary data in a time series without adjusting. The examples in the text books show high Rs for data that in truth has no correlation whatsoever. A fairly recent Nobel Prize was awarded for techniques to deal with problem.
It has nothing to do with Maximizing R2.
I personally would not try any linear regressions for market timing or times series. I certainly would put no money into it or make any public claims. But that is just me at my level.
I like linear regressions that are similar to our rank performance test. I think this would be cross-sectional data. But I am becoming more aware that any statistical claims are questionable–including R values. This is due to the fact that the data is probably not a normal curve or i.i.d. (thanks Peter and SUpirate1081).
Best,
Jim
Константин, русский? Я думал тут никог нет.
deleted
Market as whole is a more or less stationary process in comparison to individual stocks performance, and that’s the main difference.
As I remember Markov process deals with non-stationary time-series.
Then you know whether a random walk is stationary? Do you think the stock market is or isn’t a random walk?
You mean after you have corrected for any trend? You have to have done that: by definition of stationary.
As I remember stationary means stable first and second momentums for distribution, mean and variance. It was long time ago when I studied in university
So your question is what model to imply for MT? Because my regression model is based on cross sectional rather than time-series data.
Yury,
I actually like what you are doing. Personally, I would refresh my memory before going much further with any time series data.
I will be doing some of this myself but not for market timing. But again that is just me at my level. The overall market is almost certainly non-stationary even if it is not a random walk: any trending (at a minimum) must be corrected for: personally I cannot do that.
Good luck.
Jim
Yep, I forgot many things from that stuff. Maybe later I can say anything for sure regarding stochastic processes
Actually there is no big need to go deep in math. Models that really work are quite simple.
You can correct my memory but my recollection is that a stochastic process is normally distributed by definition. The stock market isn’t.
Steve
I think Jim can make separate theme discussing that things.
I think that you and Jim clearly have too much free time on your hands that could be utilized on other endeavours
Steve
Steve,
Only you get to spend all of your time talking to Yury?
That is called “weakly stationary”. There is also strong stationarity, which basically means that the underlying distribution of the stochastic process does not change over time.
That is not necessary at all. You can model a stochastic process using any kind of distribution.
In timeseries analysis, you want to model/analyse something that is “stable” over time. The simplest example is when there is a trend in the data. For example, the S&P500 index has an upward trend (in the statistical sense). The easiest transformation is to take first differences. That means you wouldn’t take the index as is, but you’d take the difference between the index on each day and the day before that. Now you’ve got data that is confined to relatively narrow range.
However, because the day-to-day changes get larger (in absolute terms) as time goes on, the range of values you’ve got in the data keeps expanding little by little over time. That’s still not “stable”. So in this case we would prefer to transform the index to percentage changes (daily returns). Now we’ve got something that stays in a relatively narrow range: the daily returns of the S&P500 in 1960 were probably similar in magnitude as the daily returns in each of the decades after it for example.
Of course financial data like the S&P500 index exhibits time-varying volatility. Strictly speaking, this means that even the returns are not weakly stationary (the second moment, the variance, changes over time). You could use GARCH models to handle that. But for simple linear models, this is not so important I think. The first moment, the mean, is much more important.
The key here is that using prices or indices as the dependent variable is usually wrong, you need to work with returns instead (as Yury has been doing).
Jim, my MT model is simple expressing in math. Return = MT function(f1(t), f2(t)…) + e,
where is function - deterministic linear function of factors f, f(t) - factors dependent on time, e- stochastic deviation with stable distribution over time (stationary) with zero mean. Also I assume f(t) is constant over time to avoid overfitting. To get higher correlation to future return you can use not constant factors as me, but variable as Hull (it means he add or leave factor over time through screening procedure). That’s all. Nothing you can do with stochastic component. You can try to model it using stochastic processes but there is no need to do that, accuracy of forecasts won’t be higher in practice.
Be carefull with correlation and Regression. they are based on an assumtion (normal Distribution), Stocks are not normal distributed, they
have fat tails, very ofthen a strategy goes for those fat tails, and would not work if the fat tails would not exist.
At least this is the case with my Systems…
Regards
Andreas