Simple Linear Regression Models

A “simple” regression model is simple because it has a single independent variable instead of multiple independent variables. Because simple is in the name, many people make the mistake of thinking they are simple to use. One mistake is to first apply them to their data, without checking to see if the assumptions are met. Here is a useful web page from Duke University called Not-so-simple Regression Models describing a general approach to developing simple linear regression models.

They provide list of types of simple linear regressions, and stress not applying them blindly or mechanically to new data. They propose an initial graphical exploration of the data to determine the qualitative properties of your variables: are they stationary or nonstationary? Do they show linear or compound growth, and is inflation a significant factor? Do they have a constant variance or increasing variance (heteroscedasticity)? Where appropriate rigorous statistical tests are applied.

After establishing the character of the data to the required degree of rigor, the next step is to choose transformations, if any, that would be appropriate to apply before attempting to fit a linear model. For example, when fitting regression models to climatological time series data, it is often helpful to first stationarize the data through a combination of differencing, logging, and/or deflating.

The last thing one does in developing a simple regression model is fitting a straight line to a scatterplot of one variable versus another. Note the most common usage, fitting the independent variable Y to time, is only for the desperate. Here is a summary of their list of transformations:

1. SIMPLE LINEAR REGRESSION MODEL
Assumption: “Y is a linear function of X”.
When to use: when X and Y are stationary time series or cross-sectional (non-time-series) variables, and a scatter plot of Y versus X suggests a significant linear relationship.

2. POWER (a.k.a. MULTIPLICATIVE or CONSTANT-ELASTICITY) MODEL
Assumption: “Y is proportional to some power of X”, or “the elasticity of Y with respect to X is constant”.
When to use: multiplicative regression models are often used to measure elasticities of demand with respect to variables such as price or advertising expenditures.

3. ABSOLUTE CHANGE (DIFFERENCED) MODEL
Assumption: “The absolute change in Y is a linear function of the absolute change in X”.
When to use: when X and Y are nonstationary time series whose that do not exhibit nonlinear trends and/or heteroscedasticity.

4. RELATIVE CHANGE (LOGGED & DIFFERENCED) MODEL
Assumption: “The percentage change in Y is a linear function of the percentage change in X”.
When to use: when X and Y are nonstationary time series with nonlinear trends and/or heteroscedasticity–e.g., series with inflationary or compound growth such as stock prices.

5. FIRST-ORDER AUTOREGRESSIVE (“AR”) MODEL
Assumption: “The level of Y is a linear function of the previous level”.
When to use: when Y is a stationary time series (e.g., a mean-reverting time series) .

6. DIFFERENCED FIRST-ORDER AUTOREGRESSIVE MODEL
Assumption: “The absolute change in Y is a linear function of the previous absolute change”.
When to use: when Y is a nonstationary time series whose first difference is positively autocorrelated or slightly negatively autocorrelated–i.e., a “trend-reverting” series–without pronounced heteroscedasticity.

7. LOGGED AND DIFFERENCED FIRST-ORDER AR MODEL
Assumption: “The percentage change in Y is a linear function of the previous percentage change”. When to use: when Y is a nonstationary time series whose first difference is positively autocorrelated or slightly negatively autocorrelated with pronounced heteroscedasticity—-i.e., a series which is “trend-reverting” in terms of percentage growth.

8. LINEAR TREND MODEL
Assumption: “Y is a linear function of time”. When to use: when you are really desperate for a linear trend projection and have little data to work with.

9. EXPONENTIAL TREND MODEL
Assumption: “Y is an exponential function of time”. When to use: when you are really desperate for an exponential trend projection and have little data to work with.

Advertisements

0 thoughts on “Simple Linear Regression Models

  1. Thanks for such a comprehensive list. My data is not stationarity so do I difference both the sides of the equation? And do the interpretations of coefficients change as well? Thanks in advance!

  2. Pingback: pills

  3. Pingback: free startup publicity

  4. Pingback: life insurance policies

  5. Pingback: escort

  6. Pingback: witryna firmowa

  7. Pingback: fmtrader scam

  8. Pingback: go

  9. Pingback: this links

  10. Pingback: ????? ???

  11. Pingback: ????? ?????

  12. Pingback: àìéøï òåáã äéîåøéí.,

  13. Pingback: business network

  14. Pingback: oferta

  15. Pingback: witryna www

  16. Pingback: Custom Rugs

  17. Pingback: review here

  18. Pingback: children's kids entertainment

  19. Pingback: paid backlinks

  20. Pingback: buy cheap uggs

  21. Pingback: carpet cleaning nyc

  22. Pingback: ?? ?????

  23. Pingback: visit website

  24. Pingback: The queen uses us to fix her toilet

  25. Pingback: Birmingham escort agency

  26. Pingback: massage erotique paris

  27. Pingback: orjinal lida

  28. Pingback: orijinal lida

  29. Pingback: strona www

  30. Pingback: zadbane cialo kobiety

  31. Pingback: camilo concha

  32. Pingback: birmingham escorts

  33. Pingback: fearless presentations

  34. Pingback: las vegas escorts

  35. Pingback: see inside

  36. Pingback: las vegas escorts.

  37. Pingback: forex managed accounts

  38. Pingback: Google sniper 3 Reviews

  39. Pingback: code promo

  40. Pingback: buy steroid

  41. Pingback: Watch the review

  42. Pingback: sell your bitcoin

  43. Pingback: Best Movies On Netflix

  44. Pingback: leto 2015 kusadasi

  45. Pingback: kliknij link

  46. Pingback: bforex ??????

  47. Pingback: porno

  48. Pingback: you can try this out

  49. Pingback: Craigslist Posting Service

  50. Pingback: website

  51. Pingback: kliknij

  52. Pingback: scam

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s