Laurie Bassi
This paper summarizes the literature on the impact of employment
and training programs and concludes that they have been neither an
overwhelming success nor a complete failure in terms of their ability to
increase the long-term employment and earnings of disadvantaged workers.
Employment and earnings programs capacity to improve the lot of any
given participant, and the collective economic well—being of the disad-
vantaged, has been modest —— as has been the level of resources devoted
to these programs.
Two decades of non-experimental program evaluation in the
employment and training field have finally taught a less about which
there can be little disagreement: Convincing program evaluation is
going to require continued use of randomized trials. We wish to
emphasize that this is not simply a statement that randomization is a
preferable methodological approach regardless of the field of study.
Instead, we believe the evidence in the study of employment and training
programs overwhelmingly indicates that randomization is essential in
program evaluation in the employment and training field. The difficulty
seems to be that the earnings determination processes of today's workers
and the program selection methods of today's programs interact to make
it nearly impossible to produce reliable estimates of how workers ear-
nings would have behaved in the absence of a program. The evidence to
support this finding comes from both experimental and non—experimental
studies. The non-experimental studies indicate that minor changes in
methods, for which there is no empirical justification, produce large
swings in estimated program effects. The study of experimental findings
indicates that perfectly plausible non—experimental methods may lead to
dramatic errors in inferences about program effects.
This basic finding raises a fundamental question: What is the
proper reaction of policy makers? In our view the appropriate reaction
is for policymakers to begin the development of a credible research and
development effort using randomized clinical trials in a wide variety of
study areas.