Hello, teal-ga!
I have found some very helpful documents which outline the steps
necessary to undertake a Performance Evaluation of public agency
programs. While the programs vary in nature, the steps remain quite
basic and should apply nicely to your task.
==
An overview concerning the need for evaluation:
"There is now a well developed literature on performance measurement,
monitoring and evaluation, and there is considerable experience in
their implementation. Measures range from relatively simple inputs and
workload indicators to consideration of outcomes and effectiveness,
requiring sophisticated systems of strategic planning and evaluation.
Most authorities believe that organizations should choose a number of
measures to `capture? their performance, and that there is no single
best way to do so. Choice will depend on the objective to be met as
well as the interests of those affected by and using the information.
Since the idea of evaluation frequently provokes resistance, it is
essential to gain cooperation through active involvement of all
concerned in constructing measures, gathering information, and
disseminating results."
From "Approaches and Guidelines for Monitoring, Measuring, and
Evaluating Performance in Public Sector Programs." [abstract] by
Gerald & Naomi Caiden. AGDA NewsLetter 9 (3): 4-5
http://www.geocities.com/CollegePark/Library/1705/caiden.html
GUIDELINES FOR PERFORMANCE EVALUATION
======================================
The following document was developed to assist in the evaluation of
community-based programs. While the community program being evaluated
is very different from the Market Townes program, many of the same
questions and techniques for evaluation can be applied:
"Community-Based Project Evaluation Guide." University of Arizona
http://ag.arizona.edu/fcs/cyfernet/cyfar/stst_guide.pdf
* (Please refer to the "Five-Tiered Approach to Program Evaluation")
==
New South Wales has put together some excellent guidelines for
Performance Evaluation of government-funded programs which should be
extremely helpful in outlining your approach:
Please see "A Brief Guide to Evaluation for NSW Drug Summit Programs."
The Cabinet Office (March 2001)
http://www.druginfo.nsw.gov.au/__data/page/229/evaluation.pdf
Chapters include:
Why evaluate your program?
What do you need to do first?
What should you measure?
How do you identify short, medium and long-term outcomes?
Are long-term outcomes useful measures of program impact?
Are there solutions to the limitations of long-term outcomes?
What are the challenges in identifying short and medium-term outcomes?
What tools can you use to measure outcomes?
How should you design an evaluation?
What skills are required to plan and conduct an evaluation?
.....
==
The government of Victoria has produced a slightly different booklet
outlining the basics of Performance Evaluation:
"Guide to Evaluating Drug Prevention Projects in Victoria." Victoria
Premier's Drug Prevention Council (2002)
Download full version from the following page -
http://www.druginfo.adf.org.au/article.asp?id=4796
See the following sections:
Part 3 - Developing an Evaluation Framework
3.3.4 Step 4 - Exploring the Project Issues
This section helps identify the areas that need change.
* "The products of this part of the evaluation will be most useful for
making informed decisions about how the project can be improved."
==
Another excellent model:
"First Nation: Self-Evaluation of Community Programs. A Guidebook on
Performance Measurement." Department of Indian Affairs and Northern
Development Corporate Services. Departmental Audit and Evaluation
Branch
(October, 1998) http://www.ainc-inac.gc.ca/pr/pub/ae/sp/9713_e.pdf
Includes the following chapters:
1. Why Measure Performance?
What is Performance Measurement?
What Does a Program Performance Framework Do?
A Tool for Management
A Tool for Accountability
2. What Makes a Good Performance Framework?
Definitions
Community Goals
Program Plans
Performance Measures
Results Data to be Collected
Performance Targets
Performance Report
3. How to Develop a Community Program Performance Framework
Phase I: Project Planning and Organization
Phase II: Framework Development
Phase III: Implementation
4. Toolkit
==
An excellent evaluation model can be found in the following document:
"From Hubris To Reality: Evaluating Innovative Programs in Public
Institutions," by Gerald Halpern. The Innovation Journal (1998)
http://www.innovation.cc/peer-reviewed/halp4.htm
* make a correct diagnosis of the problem;
* have a reasonable treatment;
* find out whether the program works;
* and why.
"It costs time, effort and money to formally evaluate. It takes
courage to accept that evaluation must be equally open to finding
either failure or success. Careful, formal evaluation however does
have an additional benefit: it yields a greater understanding of why
some programs work and thereby advances the capacity of government to
render its multiple missions."
Read entire document and examine diagrams.
==
Perhaps you can find some useful tools to use in the following booklet:
"A Toolkit for Evaluating Public R&D Investment Models, Methods, and
Findings from ATP's First Decade," by Rosalie Ruegg and Irwin Feller
(July 2003)
http://www.atp.nist.gov/eao/gcr03-857/contents.htm
==
You might want to order the following paper published by the World Bank:
"Evaluating Public Spending: A Framework for Public Expenditure Reviews,"
by Sanja Pradhan (1996)
http://publications.worldbank.org/ecommerce/catalog/product?context=drilldown&item%5fid=202269
A further description of the paper can be found below:
The paper "emphasizes six elements which should be an integral part of
an ongoing exercise to evaluate public spending:
* "Aggregate level of public spending and deficit of the consolidated
public sector must be consistent with the macroeconomic framework"
* "Aggregate spending should be allocated to programs within and
across sectors to maximize social welfare, including the impact on the
poor."
* "The role of the government versus the private sector ought to be a
principal criterion governing the choice of programs for public
financing and provision; public expenditures should complement rather
than substitute for private sector activities."
* "The impact of key programs on the poor, including their incidence
and total costs, should be analyzed."
* "The input mix, or the allocations for capital and recurrent
expenditures, should be analyzed in an integrated manner within
programs and sectors."
* "Budgetary institutions should be analyzed to ensure that the
underlying incentive structure contributes to aggregate fiscal
discipline, allocative efficiency and equity in the composition of
spending, and technical efficiency in the use of budgeted resources."
http://www-wds.worldbank.org/servlet/WDSContentServer/WDSP/IB/1996/05/01/000009265_3961219102516/Rendered/PDF/multi_page.pdf
==
Also see:
"Evaluating Public Expenditure Management Systems: An Experimental
Methodology with an Application to the Australia and New Zealand
Reforms," by Jose Edgardo Campos and Sanjay Pradhan
http://www.adb.org/Documents/Papers/Evaluating_PEMS/default.asp
Complete Paper
http://www.adb.org/Documents/Papers/Evaluating_PEMS/expenditure.pdf
==
MandE News
-------------
You may want to search through the MandE News website:
"A news service focusing on developments in monitoring and evaluation
methods relevant to development projects and programmes with social
development objectives." http://www.mande.co.uk/
==
A few of the papers listed on the following site might prove helpful
although I did not review them. I found that many of the links did not
work. Therefore, I merely copied and pasted the titles into the search
box and found many of the papers that way.
http://econ.worldbank.org/programs/public_services/library/
RECOMMENDING CHANGE
===================
You have asked how to recommend change to public sector programs.
Obviously, this cannot be answered without first identifying the
changes that should be made by doing a performance evaluation. After
potential problem areas are identified, change recommendations will be
very specific to the agency concerned. Most agencies have a designated
task force, working group or section of the board that will listen to
change recommendations. Their format for recommendations will be very
specific.
The Countryside Agency, which works with the Regional Development
Agencies to administer the Market Towns Initiative, is advisory in
nature, so you will have to contact them about "how" to bring forth
recommendations regarding a specific sector. Before you take this
step, you will have to identify "which" recommended changes affect
"what" agency!
"The Countryside Agency is neither a policy maker or a deliverer of
services. We provide well grounded evidence and advice to those who
are. We develop new solutions and spread good practice to shape
policies and the way they are translated into delivery on the ground.
We work in partnership with a wide range of organisations including
local communities, public, private and voluntary organisations,
national, regional and local government and via our eight regional
offices we are able to provide tailored advice/solutions to different
areas."
http://www.countryside.gov.uk/WhoWeAreAndWhatWeDo/HowWeWork/index.asp
The Countryside Agency is extremely broad in scope. The following
page provides a diagram of the statutory functions of the agency:
http://www.countryside.gov.uk/WhoWeAreAndWhatWeDo/HowWeWork/statutoryfunctions.asp
Read more about the Market Towns Initiative on their site:
http://www.countryside.gov.uk/WhoWeAreAndWhatWeDo/HowWeWork/successes.asp
=========
Although your question is very broad in scope, as you mentioned, and
it is still "early in the game," I hope the information I have
provided helps you to understand the steps necessary to undertake a
performance evaluation and outline recommendations for change.
umiat-ga
Search Strategy
evaluating public sector programs
evaluating "public sector" program efficiency
UK +evaluating "public sector" programs
"performance evaluation" of government-funded programs
"performance evaluation" AND community programs
evaluating performance of government-funded programs
recommending improvements in public sector programmes
+evaluating the Market Towns initiative
"Market Towns initiative" AND recommending changes |
,
0 Comments
)