Communication Research: Statistical (stepwise) regression

1. Statistical (stepwise) regression

1.1. forward selection
1.2. Backward selection
1.3. Stepwise selection

1. Statistical (stepwise) regression ¶

REGRESSION
  /DEPENDENT api00
  /METHOD=ENTER ell acs_k3 avg_ed meals .

Coefficientsa
Model		Unstandardized Coefficients		Standardized Coefficients	t	Sig.	Correlations
		B	Std. Error	Beta			Zero-order	Partial	Part
1	(Constant)	709.639	56.240		12.618	.000
	ell	-.843	.196	-.147	-4.307	.000	-.766	-.217	-.091
	acs_k3	3.388	2.333	.032	1.452	.147	.185	.075	.031
	avg_ed	29.072	6.924	.156	4.199	.000	.793	.212	.089
	meals	-2.937	.195	-.655	-15.081	.000	-.902	-.615	-.319

From the left column, zero-order (r² ), partial (pr² ), and semi-partial or part (sr² ).
We know that the meal's contribution alone is the biggest in the explanation of Y variance (-.319). Then, ell, avg_ed, acs_k3.
Therefore, the beta (standardized coefficient) value of meals is the largest (We can compare the beta values across the IV since they were standardized).

1.1. forward selection ¶

Forward selection

Forward selection begins with no predictors in the regression equation. The predictor variable that has the highest correlation with the criterion variable is entered into the equation first. The rest variables are entered into the equation depending on the contribution of each predictor.
가장 큰 상관관계를 갖는 meals 변인이 먼저 투입되어 regression이 진행된다. 이 때의 zero-order r squared 값은 -.902 에 달한다. 다른 변인이 고려되지 (투입되지) 않은 상태이므로, partial, part 값들도 zero-order와 동일한 값을 갖는다.
다음 단계에서 상관관계가 다음으로 높은 변인이 투입되어 regression이 진행된다 (partial correlation값에 의해서 판단).

REGRESSION
  /DEPENDENT api00
  /METHOD=FORWARD ell acs_k3 avg_ed meals .

Coefficientsa
Model		Unstandardized Coefficients		Standardized Coefficients	t	Sig.	Correlations
		B	Std. Error	Beta			Zero-order	Partial	Part
1	(Constant)	892.894	6.830		130.731	.000
	meals	-4.046	.100	-.902	-40.531	.000	-.902	-.902	-.902
2	(Constant)	887.826	6.718		132.163	.000
	meals	-3.469	.154	-.773	-22.565	.000	-.902	-.758	-.488
	ell	-.951	.197	-.166	-4.840	.000	-.766	-.242	-.105
3	(Constant)	781.172	27.182		28.738	.000
	meals	-3.004	.189	-.670	-15.855	.000	-.902	-.633	-.336
	ell	-.819	.195	-.143	-4.191	.000	-.766	-.212	-.089
	avg_ed	27.828	6.881	.149	4.044	.000	.793	.204	.086

Excluded Variablesd
Model		Beta In	t	Sig.	Partial Correlation	Collinearity Statistics
						Tolerance
1	ell	-.166a	-4.840	.000	-.242	.398
	acs_k3	.009a	.383	.702	.020	.962
	avg_ed	.175a	4.711	.000	.236	.339
2	acs_k3	.020b	.920	.358	.047	.951
	avg_ed	.149b	4.044	.000	.204	.329
3	acs_k3	.032c	1.452	.147	.075	.936

1.2. Backward selection ¶

Backward selection

Backward elimination begins with all predictor variables in the regression equation and sequentially removes them. Two removal criteria are available.

REGRESSION
  /DEPENDENT api00
  /METHOD=BACKWARD ell acs_k3 avg_ed meals .

All entered
Then, one that is not significantly contributes is removed. . . .

Coefficientsa
Model		Unstandardized Coefficients		Standardized Coefficients	t	Sig.	Correlations
		B	Std. Error	Beta			Zero-order	Partial	Part
1	(Constant)	709.639	56.240		12.618	.000
	ell	-.843	.196	-.147	-4.307	.000	-.766	-.217	-.091
	acs_k3	3.388	2.333	.032	1.452	.147	.185	.075	.031
	avg_ed	29.072	6.924	.156	4.199	.000	.793	.212	.089
	meals	-2.937	.195	-.655	-15.081	.000	-.902	-.615	-.319
2	(Constant)	781.172	27.182		28.738	.000
	ell	-.819	.195	-.143	-4.191	.000	-.766	-.212	-.089
	avg_ed	27.828	6.881	.149	4.044	.000	.793	.204	.086
	meals	-3.004	.189	-.670	-15.855	.000	-.902	-.633	-.336

Excluded Variablesb
Model		Beta In	t	Sig.	Partial Correlation	Collinearity Statistics
						Tolerance
2	acs_k3	.032a	1.452	.147	.075	.936

1.3. Stepwise selection ¶

Stepwise selection

Stepwise selection is a combination of forward and backward procedures.
Step 1

The first predictor variable is selected in the same way as in forward selection. If the probability associated with the test of significance is less than or equal to the default .05, the predictor variable with the largest correlation with the criterion variable enters the equation first.

Step 2

The second variable is selected based on the highest partial correlation. If it can pass the entry requirement (PIN=.05), it also enters the equation.

Step 3

From this point, stepwise selection differs from forward selection: the variables already in the equation are examined for removal according to the removal criterion (POUT=.10) as in backward elimination.

Step 4

Variables not in the equation are examined for entry. Variable selection ends when no more variables meet entry and removal criteria.

REGRESSION
  /DEPENDENT api00
  /METHOD=STEPWISE ell acs_k3 avg_ed meals

Coefficientsa
Model		Unstandardized Coefficients		Standardized Coefficients	t	Sig.	Correlations
		B	Std. Error	Beta			Zero-order	Partial	Part
1	(Constant)	892.894	6.830		130.731	.000
	meals	-4.046	.100	-.902	-40.531	.000	-.902	-.902	-.902
2	(Constant)	887.826	6.718		132.163	.000
	meals	-3.469	.154	-.773	-22.565	.000	-.902	-.758	-.488
	ell	-.951	.197	-.166	-4.840	.000	-.766	-.242	-.105
3	(Constant)	781.172	27.182		28.738	.000
	meals	-3.004	.189	-.670	-15.855	.000	-.902	-.633	-.336
	ell	-.819	.195	-.143	-4.191	.000	-.766	-.212	-.089
	avg_ed	27.828	6.881	.149	4.044	.000	.793	.204	.086

Excluded Variablesd
Model		Beta In	t	Sig.	Partial Correlation	Collinearity Statistics
						Tolerance
1	ell	-.166a	-4.840	.000	-.242	.398
	acs_k3	.009a	.383	.702	.020	.962
	avg_ed	.175a	4.711	.000	.236	.339
2	acs_k3	.020b	.920	.358	.047	.951
	avg_ed	.149b	4.044	.000	.204	.329
3	acs_k3	.032c	1.452	.147	.075	.936

CategoryResearchMethods

Statistical (stepwise) regression

Contents

1. Statistical (stepwise) regression ¶

1.1. forward selection ¶

1.2. Backward selection ¶

1.3. Stepwise selection ¶