Measuring Effects Across NASA Space Science Education Consortium
Activities Using NSF Impact Categories
Abstract
Measuring the effects of education and communication space science
activities is a challenge due to different audiences, formats, dosage,
and objectives. What claims can be made about the effect of a collection
of different activities? What methods and measures can evaluators
provide to activity managers to collect data that can be aggregated
across activities? A summative evaluation approach was devised based on
the National Science Foundation Framework impact categories of behavior,
attitudes, skills, interest/engagement, and knowledge (BASIK),
“Identification of these categories was based on analysis of project
impacts from a comprehensive review carried out on a representative
sample of Informal Science Education proposals, final reports, and
summative evaluations” ((Friedman, 2008, p. 11). Measurable objectives
were developed for each of the 53 activities offered by the 21
institutions with specific identification of the intended impact
categories. Methods and measures were then complied to evaluate the
effect of those activities on their audiences’ behaviors, attitudes,
skills, interest and/or knowledge. For the knowledge impact, the space
science concepts were identified for each activity from the Science
Literacy Strandmaps (http://nasawavelength.org/strandmaps). This
provided a portfolio overview by concept and level, as well as concept.
Methods and measures were identified by impact category to provide data
across activity on effects. Results from evaluations of projects are
reported in an online portfolio by institutions and summarized across
projects for annual reporting.