A “simple” regression model is simple because it has a single independent variable instead of multiple independent variables. Because simple is in the name, many people make the mistake of thinking they are simple to use. One mistake is to first apply them to their data, without checking to see if the assumptions are met. Here is a useful web page from Duke University called Not-so-simple Regression Models describing a general approach to developing simple linear regression models.
They provide list of types of simple linear regressions, and stress not applying them blindly or mechanically to new data. They propose an initial graphical exploration of the data to determine the qualitative properties of your variables: are they stationary or nonstationary? Do they show linear or compound growth, and is inflation a significant factor? Do they have a constant variance or increasing variance (heteroscedasticity)? Where appropriate rigorous statistical tests are applied.
After establishing the character of the data to the required degree of rigor, the next step is to choose transformations, if any, that would be appropriate to apply before attempting to fit a linear model. For example, when fitting regression models to climatological time series data, it is often helpful to first stationarize the data through a combination of differencing, logging, and/or deflating.
The last thing one does in developing a simple regression model is fitting a straight line to a scatterplot of one variable versus another. Note the most common usage, fitting the independent variable Y to time, is only for the desperate. Here is a summary of their list of transformations:
1. SIMPLE LINEAR REGRESSION MODEL
Assumption: “Y is a linear function of X”.
When to use: when X and Y are stationary time series or cross-sectional (non-time-series) variables, and a scatter plot of Y versus X suggests a significant linear relationship.
2. POWER (a.k.a. MULTIPLICATIVE or CONSTANT-ELASTICITY) MODEL
Assumption: “Y is proportional to some power of X”, or “the elasticity of Y with respect to X is constant”.
When to use: multiplicative regression models are often used to measure elasticities of demand with respect to variables such as price or advertising expenditures.
3. ABSOLUTE CHANGE (DIFFERENCED) MODEL
Assumption: “The absolute change in Y is a linear function of the absolute change in X”.
When to use: when X and Y are nonstationary time series whose that do not exhibit nonlinear trends and/or heteroscedasticity.
4. RELATIVE CHANGE (LOGGED & DIFFERENCED) MODEL
Assumption: “The percentage change in Y is a linear function of the percentage change in X”.
When to use: when X and Y are nonstationary time series with nonlinear trends and/or heteroscedasticity–e.g., series with inflationary or compound growth such as stock prices.
5. FIRST-ORDER AUTOREGRESSIVE (“AR”) MODEL
Assumption: “The level of Y is a linear function of the previous level”.
When to use: when Y is a stationary time series (e.g., a mean-reverting time series) .
6. DIFFERENCED FIRST-ORDER AUTOREGRESSIVE MODEL
Assumption: “The absolute change in Y is a linear function of the previous absolute change”.
When to use: when Y is a nonstationary time series whose first difference is positively autocorrelated or slightly negatively autocorrelated–i.e., a “trend-reverting” series–without pronounced heteroscedasticity.
7. LOGGED AND DIFFERENCED FIRST-ORDER AR MODEL
Assumption: “The percentage change in Y is a linear function of the previous percentage change”. When to use: when Y is a nonstationary time series whose first difference is positively autocorrelated or slightly negatively autocorrelated with pronounced heteroscedasticity—-i.e., a series which is “trend-reverting” in terms of percentage growth.
8. LINEAR TREND MODEL
Assumption: “Y is a linear function of time”. When to use: when you are really desperate for a linear trend projection and have little data to work with.
9. EXPONENTIAL TREND MODEL
Assumption: “Y is an exponential function of time”. When to use: when you are really desperate for an exponential trend projection and have little data to work with.