Plz choose this assignment only if you how to do it. This assignment is related to counterfactual model and naive estimator (Econometric), all the requirement and report are in the attached files, as well as the lecture slides.

econ803_assignment.pdf

1_the_counterfactual_model.pptx

2_naive_estimator.pptx

Unformatted Attachment Preview

Policy assignment

A review of “the review of the sentence of home detention

2007-2011”

Due date: Friday, March 22th, 12’noon

Hand-in: WY215 or email: [email protected]

The purpose of this assessment is to write a guided review of the Ministry of Justice report

“Home Detention – A review of the sentence of home detention 2007-2011”. The report

has a total word limit of 1500 words and needs to be completed individually.

In 2007 the New Zealand Government introduced home detention as a stand-alone

sentence. The policy change meant that eligible offenders sentenced to short-term

imprisonment could serve their sentence at an approved residence under electronic

monitoring. As part of the 2008 New Zealand general election debate the National Party

committed to an evaluation of the “appropriateness” of home detention sentence. In

October 2011 the Ministry of Justice (MoJ) released the report “Home Detention – A review

of the sentence of home detention 2007-2011”.

The report can be found here:

https://thehub.sia.govt.nz/assets/documents/41113_A_review_of_the_sentence_of_home

_detention_2007-2011_0.pdf

1. (~350 ). Write a short summary of the MoJ report. Please make sure you

include information on:

a. Research question(s)

b. Data

c. Method

d. Conclusion

2. (~150 ). Re-state the (main) MoJ research question (from Q1) using the

notation from the lecture on the counterfactual framework.

3. (~500 ). Discuss the empirical strategy in the MoJ report. Make sure to use

the notation from the lecture on the counterfactual framework. You are encouraged

to derive and sign any possible bias in the MoJ estimate(s) and discuss what

assumptions could justify the MoJ approach.

4. (~500 ). In an imaginary world without any restrictions what would, in your

view, be the ideal experiment to identify the causal effect of home detention?

Note that (~ ) is a suggestion, feel free to distribute the 1500 words differently if

that works better for your answers.

ECONOMETRICS

Econ803

The Counterfactual Model

Peer Ebbesen Skov

Motivation

Consider the following question:

• What is the effect of class size on the educational

achievements?

• What is the effect of an increase in the minimum wage

on employment?

• What is the effect of terrorism on economic growth?

• What is the effect of increased cigarette excise tax on

smoking?

• What is the effect of marriage on crime?

The answers to these questions (and many others which

affect our daily life) involve the identification and

measurement of causal links: an old problem in

philosophy and statistics.

We need a framework to study causality

2

A formal framework to think about causality

We have a population of units; for each unit we observe a

variable and a variable .

We observe that and are correlated. Does correlation

imply causation?

Reference: David Card. Chapter 30 in Handbook of Labor Economics,

1999, vol. 3, Part A, pp 1801-1863. Elsevier.

3

A formal framework to think about causality

Does correlation imply causation?

In general no, because of:

•

•

Confounding factors;

Reverse causality.

We would like to understand in which sense and under

which hypothesis one can conclude from the evidence

that causes .

4

Terminology

index for the units in the population under study

is the (binary) treatment status where

= ቊ

1

ℎ

0 ℎ

indicates the potential outcome according to

treatment where

= ቊ

1

0

The observed outcome for each unit can be written as:

= 1 + 1 − 0

This forces us to think in terms of counterfactuals or socalled potential outcomes.

Note that the terminology is borrowed from experimental

analysis.

5

The fundamental problem of causal inference

DEFINITION: Causal effect

For a unit the treatment has a causal effect on the

outcome if the event = 1 instead of = 0 implies

that = 1 instead of = 0 . In this case the

causal effect of on is

∆ = 1 − 0

The identification and the measurement of this effect is

logically impossible.

PROPOSITION 1: The Fundamental Problem of Causal Inference

It is impossible to observe for the same unit the values

= 1 and = 0 as well as the values 1 and 0

and therefore it is impossible to observe the effect of on

for unit (Holland, 1986)

Another way to phrase this problem is to say that we

cannot infer the effect of a treatment because we do not

have the counterfactual evidence, i.e. what would have

happened in the absence of treatment.

6

The fundamental problem of causal inference

1

0

Treatment status ( = 1)

Treatment status ( = 0)

It is not possible to observe the potential outcome under

the treatment state for those observed in the control

state. Just like you cannot observe the potential outcome

under the control state for those observed in the

treatment state. (Morgan and Winship, 2007).

Yet another way of thinking about it: even if you have

access to all individuals level values of in the

population you only observe half of the information you

need: individuals contribute information only from the

treatment state in which they are observed. (Morgan and

Winship, 2007).

7

Unit homogeneity solution

One “solution” to the missing counterfactual problem is

to assume unit homogeneity:

∆ = ∆ for all

If 1 and 0 are constant across individual

units, then cross-sectional comparisons will

recover ∆ = ∆

If Yi 1 and Yi 0 are constant across time, then

before-and-after comparisons will recover

∆ = ∆i

While this might work in physical sciences the assumption

seems highly unlikely to be realistic in social sciences

Before we get to the statistical ‘solution’ let’s digress for a

moment and consider another challenge…

Note the following notation:

≡ Yi

8

Stable Unit Treatment Value Assumption (SUTVA)

Recall the observed outcome for each unit can be written

as:

= 1 + 1 − 0

This notation implicitly makes the following assumption:

SUTVA:

1, 2 , … , =

′

1′ , 2′ , …,

= ′

In other words:

▪ There is no interference between units:

▪

▪

Potential outcomes for a unit most not be affected

by treatment for any other units.

Spill-over effects, contagion, dilution

No different versions of treatment

▪

Nominally identical treatments are in fact identical

Variable levels of treatment, technical errors

9

Causal Inference without SUTVA

Let = 1 , 2 be a vector of binary treatments for N =

2

How many different values can D possible take?

How many potential outcomes for unit 1?

How many causal effects for unit 1?

How many observed outcomes for unit 1?

1 = 1 , 2 1

Without SUTVA, causal inference becomes exponentially

more difficult as N increases

10

The statistical solution

Statistics proposes to approach the problem by focusing

on the average causal effect for the entire population or

for some interesting sub-groups.

The effect of treatment on a random unit (ATE):

∆ = 1 − 0

= 1

− 0

Or equivalently

1

∆ = 1 −

=1

Note that ATE ∆ is still unidentified. The majority of

this paper is devoted to various assumptions under which

we can identify ATE from observed information.

11

The statistical solution

Statistics proposes to approach the problem by focusing

on the average causal effect for the entire population or

for some interesting sub-groups.

The effect of the treatment on the treated (ATT):

∆ | = 1 = 1 − 0 | = 1

= 1 | = 1 − 0 | = 1

Or equivalently

1

=1

1

1

∆ =

1 − where 1 =

1

When would ≠ ? When and are

associated.

Exercise: define the treatment on control units (ATC).

12

The statistical solution

Statistics proposes to approach the problem by focusing

on the average causal effect for the entire population or

for some interesting sub-groups.

The conditional average treatment effect (CATE)

∆ | = = 1 − 0 | =

13

Illustration: Average Treatment Effect

Suppose we observe a population of 4 units:

i

1

1

3

2

1

1

3

0

0

4

0

1

What is ATE: ∆ = 1

− 0

∆

?

Naïve estimator:

ATE: ∆ = | = 1 − | = 0

(Note this is the observed difference in means)

∆ =

Is this (likely) an unbiased estimate of ATE?

Let’s expand the table

14

Illustration: Average Treatment Effect

Suppose we observe a population of 4 units:

i

1

1

3

2

1

1

3

0

0

4

0

1

What is ATE: ∆ = 1

− 0

∆

?

Naïve estimator is likely biased i.e. we over/under

estimate the average treatment effect.

To obtain an unbiased estimate of the ATE we need

potential outcomes that we do no observe:

But suppose we did – let’s complete the expanded table

15

Illustration: Average Treatment Effect

Suppose we observe a population of 4 units:

i

1

1

3

2

1

1

3

0

0

4

0

1

What is ATE: ∆ = 1

− 0

1

=

0

=

− 0

=

∆ = 1 − 0

=

∆ = 1

∆

?

or

16

Illustration: Average Treatment Effect

Suppose we observe a population of 4 units:

i

1

1

3

2

1

1

3

0

0

4

0

1

∆

What is ATT: ∆ | = 1 = 1 − 0 | = 1

∆ | = 1 =

Why is the = ≠ . Will this

always be the case?

17

Is comparison by treatment status informative?

A comparison of outcome between treatment status (the

naïve estimator) often gives a biased estimate of the ATT:

= 1 | = 1 − 0 | = 1

= 1 | = 1 − 0 | = 0

= 1 | = 1 − 0 | = 1

+ 0 | = 1 − 0 | = 0

= + 0 | = 1 − 0 | = 0

Note = ∆ | = 1 and the second term

0 | = 1 − 0 | = 0 is often referred to

as sample selection bias.

The difference between the left hand side (which we can

estimate) and is the sample selection bias equal to

the difference between the outcomes treated and control

subjects in the counterfactual situation of no treatment

(i.e. at the baseline).

The problem is that the outcome of the treated and the

outcome of the control are not identical in the notreatment situation.

18

Is comparison by treatment status informative?

▪

▪

1.

Causal inference requires a good identification

strategy

The treatment assignment mechanism determines

whether average causal effects are identifiable

Treatment is randomized by the researcher:

1.

2.

3.

2.

Natural experiments

1.

2.

3.

4.

3.

Birthday cut-offs

Weather

Close elections

Arbitrary administrative rules/policy

Treatment is “as-if” random after statistical control

1.

2.

4.

Laboratory experiments

Survey experiments

Field experiments

Marriage (controlling for age ,education and

income)

Earnings (controlling for age, education and

experience)

Treatment is self-selected and no plausible control is

available.

19

ECONOMETRICS

Econ803

The Naïve Estimator

Peer Ebbesen Skov

Livvy Mitchell

Naïve Estimator

Naïve estimator for ATT

Compares outcomes of participants (D=1) and nonparticipants (D=0) as follows:

= = 1 − = 0

It is unbiased under the assumption of no selection bias

(on observed and/or unobserved characteristics)

whereby:

0 = 1 = 0 = 0

Generally we don’t believe that to be the case.

2

Naïve Estimator

Naïve estimator for ATT

The is unbiased under the assumption

of no selection bias on observed and/or unobserved

characteristics:

We generally distinguish between two source of bias

–

Differences in observed characteristics

–

Non-overlap (B1)

–

Different distribution of observables (wrong

weighting scheme) (B2)

–

Selection on unobserved characteristics

–

Omitted variable bias, e.g., ‘ability bias’ (B3)

3

Naïve Estimator

Assessing comparability of groups in terms of

observables

=? =

But, how can you compare the joint empirical distribution

of all the X’s between two samples?

Instead..

(a) Variable-by-variable measures

–

–

Moments: means, variances

Empirical distributions: densities, CDF, boxplots

(b) Overall measures across all X’s

–

–

Or, across some X’s have interaction terms

pstest allows factor variables (e.g., foreign##c.age)

4

Naïve Estimator

Propensity-Score matching (“pstest”)

Use pstest to easily compare the characteristics in two

groups:

pstest var_list , raw treated (treated) scatter

Use it also to quickly graph the non-parametric density or

boxplot of a continuous variable (“var”) for two groups:

pstest var [if] , raw treated (treated) density|box

5

Naïve Estimator

Variable-by-variable: pstest output

6

Naïve Estimator

Summaries: pstest output

7

Naïve Estimator

Continuous variables: pstest output

8

Naïve Estimator

Overall indicators: pstest output

9

Naïve Estimator

How do we get a credible counterfactual?

▪

If no convincing comparison group exists, fancy

statistical work can’t recover the true impact.

▪

Robustness checks

Different methods differ in:

▪

▪

How they construct the counterfactual

▪

Assumptions they make

▪

Data they require

At times: what parameter (ATE, ATT,…) they recover.

Naïve comparisons of e.g., participants and

nonparticipants or simple before-after differences will not

provide the correct counterfactual.

10

Naïve Estimator

Validity concepts

Internal validity is concerned with the validity of the estimates.

• Does the study successfully uncover causal effects for the

sample studied?

• Are the estimates unbiased?

External validity is concerned with the generalisability of the

estimates.

• Do the study’s findings inform us about different

populations?

11

Naïve Estimator

Take-away points

❑ How did the non-treated ‘escape’ treatment?

❑ People deciding to participate and those decided not to are

general fundamentally different.

❑ Assess comparability in terms of observables

❑ Make your life easy by choosing your comparison group

wisely:

How to choose a comparison group

•

Randomly deny program access to a sub-group of

participants (i.e. randomised experiment)

•

Arbitrary rules that locally ‘randomise’ people (i.e.

regression discontinuity design)

•

Sources of natural variation in treatment assignment: two

very similar groups on average, but one ‘randomly’ has more

exposure to treatment (i.e. instrumental variables)

•

Non-participants who look similar to participants (i.e.

regression, matching)

•

Remove selection bias under pre-program data on the two

12

groups (i.e. difference-in-differences)

…

Purchase answer to see full

attachment