The importance of program evaluation to organizations and me

By way of quick introduction, my name is Jennifer Zavaleta and I am a graduate student in the Natural Resource Department at Texas Tech. I was asked to write about the importance of program evaluation and how program evaluation fits in with my future plans after graduation.

My first semester with Ogallala Commons has been a learning and meaningful experience. At the beginning of this internship I was a little unsure of my skills in terms of developing surveys and analyzing their results. However, I have been learning through an independent study with Dr. Todd Brashears in the Agricultural Education Department and now feel much more confidant. In return for helping me with program evaluation for Ogallala Commons, I have worked with him on analyzing surveys, which compare the knowledge and attitudes of meat possessing managers at private and state-operated plants. While the subject matter greatly differs from that which I measured with Ogallala Commons, one fact remains: Program evaluations are essential for program development. Program evaluation is important because it highlights unobvious assumptions, indicates where lessons may need to be bolstered, and provides hard numbers and evidence that an organization is making a difference.

While interning with Ogallala Commons I developed four different surveys for four demographics Ogallala Commons works closely with. This includes landowners who attend management days, teachers who attend playa festival training day, students currently partaking in playa festivals, and former playa festival attendees. Each survey measured knowledge, attitudes and behaviors before and after Ogallala Common’s programming. I have only had the opportunity to analyze the data from the prescribed grazing management day.

The results were, at times, surprising and seemingly contradictory. For example, landowners claimed that management decisions were not primarily driven by economic concerns. However, they were not willing to implement prescribed grazing for financial reasons. After learning this information, I suggested that staff at Ogallala Commons add specific information on the economic advantages of prescribed grazing to better encourage behavioral change. The information I collected helped staff at Ogallala Commons adjust curriculum to better meet the needs of its intended audience, better allocate limited funding, and present hard-data to their board of directors.

The theme of unexpected results is not limited to my experience with Ogallala Commons. Dr. Brashears and I have collected and analyzed data related to organizational climate and culture of meat packing plants in central Mexico. We found that while knowledge-based education is important to improved food safety, it is not sufficient for successful implementation of safety directives. We determined that management’s attitude toward the food safety and work-culture of the plant influenced quality of processed meat. This information changed food safety curriculum. Instead of solely focusing on food safety procedures, now education also includes the importance of food safety and appeals to changing people’s attitude toward safety issues. This small yet meaningful change in curriculum is a significant step toward improving meat quality such that it may be possible to trade with the United States, a goal of Mexican meat production.

Not only are program evaluations valuable because they can show surprising results, they also provide hard numbers as opposed to impressions that people have understood the educational materials and will change their behavior. In our survey to landowners we were able to prove people gained knowledge from management days by asking content-based questions. We were also able to measure that people planned on changing their behavior based on their interactions with Ogallala Commons. The survey asked participants what their behaviors were before the management day and what they hoped to do after. There was a significant intent to change in terms of implementing prescribed grazing, identifying plants on their playa, and providing an adequate buffer zone around their playa. This information demonstrates that Ogallala Commons is having a significant impact in the community. This is powerful information when writing grants and impressive to share with a board of directors. I look forward to analyzing more surveys related to playa festivals once the school year starts.

The program evaluation that I have participated in while working with Ogallala Commons and Dr. Brashears has motivated me to apply for a Fulbright to work in Chile. Chile’s Long Term Socio Economic Research Program has started to link scientists from different universities, field stations, and private industries as well as community members and politicians. This network has the potential to drastically improve collaboration efforts throughout Chile. I would love to be granted the opportunity to work with the LTSER network and help them measure how effective their current efforts are on spreading awareness of the organization and make suggestions for how to improve recognition and collaboration of the network. I will have the opportunity to meet the directors of this program when I attend the Ecological Society of America meeting in Austin this August. I look forward to learning more about the LTSER program and how I can contribute to its success.