Janice Ballou (1)
Science Education for New Civic Engagements and Responsibilities (SENCER) was initiated in 2001 and since then has established and supported an ever-growing community of faculty, students, academic leaders, and others to improve undergraduate STEM (science, technology, engineering and mathematics) education by connecting learning to critical civic questions.(2) SENCER is the signature program of the National Center for Science and Civic Engagement which was founded in affiliation with Harrisburg University of Science and Technology. SENCER empowers students, faculty, and academic leaders and catalyzes large-scale pedagogical and curricular transformation to improve learning and strengthen civic capacity. SENCER goals are to: (1) get more students interested and engaged in learning in STEM courses, (2) help students connect STEM learning to their other studies, and (3) strengthen students’ understanding of science and their capacity for responsible work and citizenship.
SURVEY OVERVIEW
In October 2010 all SENCER program participants who attended at least one national or regional event between 2001 and 2010 were sent an e-mail request to take part in a survey. The web-administered questionnaire(3) was designed to assess the impact of SENCER on a variety of personal and professional goals. The survey provided an opportunity to see from the program participant perspective how well SENCER is meeting its core objectives. The 602 survey participants represent a 45 percent response rate. This summary describes top-line results. More detailed analysis is planned for future reports.
Survey responders were primarily faculty (66.7%). About one-in-five reported having an administrative role on campus such as department chair (11.5%), dean (5.2%), and provost, vice president, president, or chancellor (2.5%). Most were from four-year (85.5%) compared to two-year (14.5%) institutions and similar percentages were from public (55.2%) and private (44.8%) institutions. A majority of responders participated in one or two national or regional SENCER events (54.3%), a third participated in three to six events (33.5%), and one-in-ten experienced more than six SENCER events (11.9%).(4) Examples of these “events” are the intensive SENCER Summer Institute, the three-day DC Symposium and Capitol Hill Poster Session, and various regional training programs and workshops.
SENCER appreciates the work and acknowledges the contributions of the following:
- Dr. Stephanie Knight, Professor of Educational Psychology and Teacher Education at Penn State University, who also serves as director of evaluation and assessment for SENCER,
- Professor Richard Duschl, Waterbury Chair in Secondary Education at Penn State University, who contributed to the design of the survey instrument, and
- the SENCER Assessment and Evaluation Advisory Team, including William E. Bennett, Stephen Carroll, Matthew Fisher, Jeannette Haviland-Jones, and Terry McGuire.
- Janice Ballou consulted on the survey development and prepared the descriptive summary.
SUMMARY OF RESULTS
SENCER Impact Assessment Survey results show that, in general, across all of the questionnaire items, majorities or more have been positively influenced by SENCER experiences. To illustrate key program benefits, examples of results notable for their overwhelming consensus are highlighted. The highlights are followed by a summary of other outcomes that respondents report were influenced by SENCER program participation.
Top Ranking SENCER Influences
These highlights were selected to exemplify areas of notable consensus among the survey respondents. Reports of about eight-in-ten or more survey participants credit SENCER with:
Contributing to…
- Developing courses or programs: 84.9 percent (49.2% a great deal; 35.7% some)
- Setting learning goals for their course(s): 79.8 percent (40.6% a great deal; 39.2% some)
- Refining (SENCERizing) their courses: 79.6 percent (39.4% a great deal; 40.2% some)
Succeeding with…
- New course or new program development: 79.4 percent (42.8% very successful; 36.6% somewhat successful)
Influencing pedagogical practices that increase student opportunities to…
- Make interdisciplinary connections: 85.6 percent (strongly agree 49.8%; agree 35.8%)
- Make connections between science and civic problems/topics: 85.0 percent (strongly agree 51.6%; agree 33.4%)
- Identify scientific problems and questions: 78.1% percent (strongly agree 33.2%; agree 44.9%)
Helping their students achieving 21st Century learning goals, such as…
- Ability to engage in critical thinking: 79.9 percent (37.3% a great deal; 42.6% some)
- Capacity to collaborate or engage in group work: 78.6 percent (42.9% a great deal; 35.7% some)
- Problem solving: 77.7 percent (29.5% a great deal; 48.2% some)
Influencing their image of students as science learners able to:
- Take knowledge gained and apply it in a civic/community setting: 78.9 percent (37.5% strongly agree; 41.4% agree)
Summary of SENCER Influences
From both the perspective of developing courses and the impact on students, survey respondents credit SENCER with transforming their educational practice. Seven areas of impact are summarized below:
Pedagogical practices Six-in-ten respondents or more agree that SENCER participation influenced how they instruct students. Most notable are increasing student opportunities to:
- make interdisciplinary connections (85.6%),
- make connections between science and civic problems/topics (85.0%), and
- identify scientific problems and questions (78.1%).
In addition, as a result of their SENCER program participation, respondents say they have enhanced students’ educational experiences to analyze evidence to determine patterns (67.6%), use evidence patterns and/or models to generate or evaluate explanations (66.1%), conduct measurements and/or observations to develop data sets (65.4%), analyze data sets to determine evidence or the need to conduct more measurements and observations (62.3%), and analyze evidence to construct models (59.5%).
Image of students as science learners Involvement in SENCER has had a noteworthy influence on participants’ views of students as science learners. Seven-in-ten or more agree that students are able to:
- take the knowledge gained and apply it in a civic/community setting (78.9%);
- ask scientifically oriented questions (75%);
- translate the knowledge gained to other courses (74.3%);
- use evidence to develop and evaluate explanations to address scientifically oriented questions (73.6%);
- formulate explanations from evidence to address scientifically oriented questions (73.2%);
- evaluate their explanations in light of alternative explanations (71.1%); and
- reflect on phenomena that do not have unique resolutions (70%).
Fewer agreed on how SENCER influenced their image of students’ abilities to: formulate appropriate criticism of others (60%); respond to criticism from others (59.7%); and seek criticism of their own explanations (58.8%).
Student achievement A substantial two-thirds or more of survey responders acknowledge SENCER participation as helping their students to achieve 21st Century goals. The most notable influences are on students’ abilities to:
- engage in critical thinking (79.9%);
- collaborate or engage in group work (78.6%); and
- problem solving (77.7%).
Other 21st Century goals targeted to enhance student achievement that SENCER participation has led to are: cultivation of a global perspective (69.0%); ability to discern good information from fraudulent claims (68.9%): and quantitative literacy (66.3%).
Course modifications Survey respondents credit participation in SENCER making a great deal or some contribution to:
- developing courses or programs (84.9%);
- setting learning goals (79.8%; and
- refining (SENCERizing) existing courses (79.6%).
More than two-thirds (68.1%) report utilizing formative assessment strategies as a result of the SENCER program.
Institutional change On a broader level, survey responders report that participation in SENCER made them very or somewhat successful with new courses/program development (79.4%) and involving their students in civic engagement activities (75.4%). Moreover, success in strengthening their institution’s civic engagement activities (67.9%) and department/college strategic planning (64.3%) was also attributed to SENCER participation.
Career development Program participants credit SENCER with contributing a great deal or some to important milestones in their careers and scholarship:
- 263 (44.1%) acquired internal grants/funding,
- 197 (33.1%) achieved tenure and promotion,
- 187 (31.4%) acquired external grants/funding, and
- 69 (11.6%) obtained new positions and employment.
Publishing has also been influenced by SENCER participation with 121(20.3%) survey respondents reporting having published in non-refereed journals/outlets and 116 (19.5%) in refereed journals. Eighty (13.4%) authored books or monographs.
Value of SENCER programs and resources The survey invited participants to rate the value to them of nine SENCER programs and resources. The Summer Institute stands out with 85.9 percent who reported it was very valuable or valuable. Majorities were also positive about the value of publications such as Models, Backgrounders, the International Journal, and eNews (60.6%) and implementation awards such as grants (55.0%). On average, a majority of survey responders answered “not applicable” to six other SENCER programs and resources, which suggests that they did not have enough direct experience to assess the value. Among those who were able to give a response, the very valuable and valuable response were the predominate choices for leadership opportunities (42.5%); scholarship opportunities (35.8%); consultations on campus (30.6%); regional symposia (Centers for Innovation) (32%); Washington Symposia and Capitol Hill Poster Sessions (25.5%); and recognition programs (23.8%).
SURVEY METHODOLOGY
In October 2010 a census of 1,685 SENCER program participants who attended at least one national or regional event between 2001 and 2010 were sent e-mail invitations to complete a web-administered questionnaire about their SENCER experience. Of these, 346 e-mails were returned due to bad addresses and five were not eligible due to conflict of interest as evaluators associated with SENCER. Among the 1,334 eligible, 602 (45%) participated in the survey conducted between October 13, 2010 and November 30, 2010. Five e-mail reminders, approximately a week apart, were sent to encourage survey participation. Respondents were given the option to respond anonymously or to give their names and personal contact information.
The 70 item comprehensive questionnaire covered SENCER program experiences, influence of SENCER on faculty teaching, course design, student learning, and institutional impact. In addition to close-ended responses to questions, the questionnaire included open-ended items to provide opportunities for participants to expand upon and deepen the close-ended answers. Particularly noteworthy is that close to 4-in-10 (37.6%) provided rich verbatim answers following the questions about SENCER contributions, compared to a more typical 5-10% response to this type of opportunity.
Upcoming future analysis will focus on the rich data set of both quantitative and qualitative responses. Using SPSS, a computer program used for statistical analysis, relevant sub-groups will be classified to further understand the impact of SENCER on program participants. A review of the detailed verbatim responses will identify potential exemplary SENCER outcomes for possible case studies. Individuals or teams interested in using the survey data for their own analysis should contact the SENCER national office.
Endnotes
(1) Janice Ballou is a nationally recognized survey research methodologist. Ballou recently retired from Mathematica Policy Research in Princeton where she was a Vice President and Senior Fellow. Her prior position was Director of the Rutgers University Center for Public Interest Polling.
(2) This material is based upon work supported by the National Science Foundation under Grant No. 0717407. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.
(3) Additional methodological information follows in the background description.
(4) The characteristics of those who responded mirror the total population of 2001-2010 SENCER program participants in the type of institutions they represent. However, those who responded are more likely to have experienced more than six SENCER events (11.9% compared to 2.0%) and 3 to 6 events (33.5% compared to 8.0%) and less likely to be those participating in 1 or 2 national or regional SENCER events (54.3% compared to 89%). Possible explanations for the survey participation among those who attended fewer events are they had recently completed another survey about a SENCER event and may have not felt they had enough experience with SENCER to answer the survey questions.