Teaching Public RelationsNo.53, Spring 2001
Learning and Retention of the RACE Algorithm
in Public Relations Undergraduate Education
A Comparison of Lecture Instruction, Case Method and Simulation
David O. Woolverton, Ed.D., APR
Eastern Kentucky University
T his article reports the results of an experimental study concerning the educational effects of lecturing, case studies and simulation as teaching methods. The study compared the effects of four different combinations of these teaching methods on students’ ability both (a) to use the RACE formula for solving public relations problems and (b) to retain that ability for use at a later time. For initial learning, the results indicate that a combination of case studies and simulations worked best for initial understanding as compared to lecturing alone or even to case studies or simulations alone. For retention, the results indicate that lecture is the least effective, and there is no significant difference between case studies, simulations or a combination of case studies and simulations.
While school lessons are to be learned, they also must be retained and then applied if professional higher education is to be of value. Simulation, which is administered in various forms, is purported to be a more effective teaching method than lecture. Proponents contend that when simulations, including the scenario-type of simulation, are in a narrative, realistic form, students become active, involved learners rather than passive recipients of information. The term “simulation” is used here to denote a category of teaching methods ofted described by the term “active learning.” The term “scenario” is used to denote a sub-category in which teaching is facilitated by describing a particular situation.
Active learning research generally parallels the method and result of this study. A study by McCarthy and Anderson (2000) found role play in a required history course and student team study in an honors political science course both exceeded results generated from traditional lectures. DeNeve and Heppner (1997) found role playing more beneficial than lecture in a college-level industrial psychology course. A study by Dewey and Meyer (2000) reported that hands-on learning in a college-level climatology course improved grades over traditional lecture instruction.
Research comparing various types of simulations with various other teaching methods (usually lecture) has been conducted for some time. Researchers almost universally find simulation instruction better or not significantly different from other methods. Only a few studies, however, compared simulations with other instructional methods in terms of both initial learning and retention, as does this study.
The case method is used for a number of topics but was developed for teaching business administration (Christensen and Hansen, 1987). Beaudin (1996) contended the case should be real, an actual happening, which should only be fictionalized when original identities cannot be revealed. Lovelock (1986) contended that cases need not be based on real situations. He noted that cases tend to be selective in their reporting in order to fit a relatively short class period and to focus attention on a predefined type of problem. Lovelock’s approach was used in this study.
Similarly, McKeachie (1978) recommended that teachers choose initial cases in which differences are clear before advancing to more complex cases. This approach was used in this study.
Erickson and Strommer (1991) said that in case studies teachers could lead students through a process with step-by-step questioning. Graham and Cline (1980) said this method could be applied to a class as a whole. These approaches were used in this study.
As a result of the findings cited in the literature, the following hypotheses were developed:
HYPOTHESIS 1–A group receiving an introductory lecture and a case method treatment will demonstrate greater initial learning than a group receiving the same amount of time in a lecture setting.
HYPOTHESIS 2–A group receiving an introductory lecture and a scenario exercise will demonstrate greater initial learning than a group receiving the same amount of time in an introductory lecture and case method treatment.
HYPOTHESIS 3–A group receiving an introductory lecture and case method and scenario exercise will demonstrate greater initial learning than a group receiving the same amount of time in an introductory lecture and scenario exercise.
HYPOTHESIS 4–A group receiving an introductory lecture and a case method treatment will demonstrate greater retention than a group receiving the same amount of time in a lecture setting.
HYPOTHESIS 5–A group receiving an introductory lecture and scenario exercise will demonstrate greater retention than a group receiving the same amount of time in an introductory lecture and case method treatment.
HYPOTHESIS 6–A group receiving an introductory lecture and case method and scenario exercise will demonstrate greater retention than a group receiving the same amount of time in an introductory lecture and scenario exercise.
The simulations used in this experiment are simple because the students in this experiment were being exposed to their first experience with a public relations problem-solving algorithm. An earlier pilot test conducted by giving a group in the same course in the previous semester a similar public relations problem established that the students did not have an innate sense of how to use the principles of the algorithm.
Content validation, as called for by Cole (1997), Gredler (1996) and Bowen (1987), was addressed in two ways:
First, content validity of the two simulation assignments used in this experiment comes from their source; they are based on assignments in a public relations textbook by Hendrix, (1997) who holds a doctorate and is professionally accredited by the Public Relations Society of America. His book has also been reviewed by six other public relations professors.
Second, content validity was further assured through an external review as recommended by Popham (1999). That review by four experts produced endorsement of the treatments, assignments and grading criteria. Two of these experts are practicing, accredited public relations professionals who hold leadership positions in a Public Relations Society of America chapter. All endorsed all parts of the presentations and assignments and grading as reflective of actual practice within the field. One of the practicing professionals noted a lack of budget and scheduling information in the simulation but acknowledged that these are too involved to be included in an introductory lesson. One professor compared the initial and retention assignments and found no difference in difficulty.
The design of this experiment was developed after considering the studies cited in the literature, especially five retention studies. The Carney, Dietrich, Freeman & Mott (1995) study used five simulation methods, but failed to discriminate among effects of these methods. The Boettcher (1981) and Vincent (1984) studies compared only one simulation method (computer assisted instruction) to lecture. The Larson (1981) study compared two simulation methods. This experiment most closely follows the pattern of a study by Fishman (1984), which used two simulation methods and compared their effect individually with instruction by lecture.
The four intact groups were each given an identical one-hour introduction on the principles and practices of the public relations problem-solving algorithm. This method, created by Marston (1979) and well known in the profession (Grunig & Hunt, 1984), uses four steps. The steps are research about the situation, action-planning to set an objective, communication designed to accomplish the objective, and evaluation of effectiveness. The presentation included the principles underlying the four steps. The presentation also listed the various techniques used within each step and their characteristics.
The second hour-long treatment was different for each group. This created the four treatment conditions that were compared.
The lecture group received a lecture including videotape examples of application of the RACE algorithm to real public relations efforts and further describing techniques, work environments and specialties within public relations.
The case method group was engaged in an instructor-presented case method treatment. The instructor led the class through the use of the algorithm in a realistic public relations simulation. For each of the four steps, the students wrote individually at their desks their own determination of the best response to various stages of the problem presented, and these responses were discussed. The instructor then presented model responses and explained the reasons for them.
The simulation group engaged in a simulation with the instructor leading them through a realistic public relations scenario. This exercise required them to make choices among alternative methods at each stage of the problem using a latent-image exercise. The latent image exercise provided a written paragraph describing a situation, and let students choose between a variety of options printed below it. The options were chosen by marking a form with a special pen. Marking the form revealed a latent image (invisible ink) response as to the appropriateness of the choice. A key to all appropriate responses was provided at the end of the simulation so the students could score their performance. This simulation was completed individually at the students’ own pace.
The combination group received the same case method treatment as the case method group and the same latent-image simulation treatment the simulation group. (Please contact the author for a copy of the text of the four treatments, if desired.)
In order to assess students’ ability to use and retain this algorithm, one day after the treatment, the lecture group and the case study group were given the assignment of writing individually their solution to a realistic public relations simulation. Two days after the treatment, the scenario group and the combination group were given the same assignment. The times between instruction and the first assignment were dictated by the university calendar. Then, to assess their retention, one week after the initial assignment, all students were asked to write individually their solution to another realistic public relations simulation. All initial assignments were identical, and all retention assignments were identical.
Three public relations faculty members rated each student’s performance on the initial assignment and the retention assignment. The raters did not know the identity of the students whose work they were grading. The rating guides used by the raters measured the students’ ability to use each of the algorithm’s four steps. The interrater reliability was in the “fair to good agreement beyond chance” range according to Fleiss (1981, p. 218). A covariance based on grade point averages was used to allow for comparison of the four intact groups.
The hypotheses were somewhat unusual for this type of study in that they posited a ranking of the results, based on what was found in the literature. Because a ranking was posited, hypotheses were stated in a simpler format than for some other studies, that is, if case method teaching was significantly better than lecture, and simulation teaching was significantly better than case method, then simulation was automatically significantly better than lecture, and so on up the ranking.
Since a ranking was posited, the analysis was done by the planned contrast method of analysis. This method better fits the format of the study and offers a more powerful analysis.
The initial assignment was a realistic public relations problem involving a disagreement between a local, private university and a neighborhood opposing expansion of the campus. The retention assignment was a realistic public relations problem involving a conflict between a city government and an ethnic minority. (Please contact the author for a copy of the two assignments, if desired.)
Each assignment was administered the same way in each group’s classroom. Students were given a written description of the problem and wrote individually what research should be done. That response was presented to the instructor. The instructor initialed across the response so it could not be altered, and then provided the student with a description of the research results. The student then wrote his or her solution to the rest of the problem including the action plan, communication and evaluation steps. All assignments were completed individually, and all students were invited to use their notes.
This method followed the pattern identified as the post test-only control group design by Campbell & Stanley (1963). They noted that this is a common research design in educational practice because teachers frequently experiment with entirely new subject matter, for which pretests in the ordinary sense are impossible.
To assure that the introductory lecture was delivered consistently to all treatment groups, and to avoid possible bias from different levels of teaching abilities from different teachers, the principal investigator conducted all the treatments.
Because the treatments were conducted on intact classes, the statistical analysis was conducted by analysis of covariance to control for any systematic differences in ability level of students across these classes, following criteria from Williams (1979). The covariate was the individuals’ cumulative grade point average, since it reflects cumulative academic achievement at the college level. The ANCOVA was followed by a planned contrast analysis.
Formative evaluation, as called for by Gredler (1996), was conducted. Two pilot studies were performed prior to the main study. One pilot study found no significant innate ability of the students to use the principles in the algorithm prior to the instruction. A second pilot study tested the grading guide and led to revisions in the guide to improve interrater reliability.
The total number of subjects was 133. The generally accepted .05 significance level was chosen for this experiment.
Six hypotheses were developed. Three of these hypotheses deal with comparing initial learning resulting from each of the instructional conditions and are shown in Table 1. Three of these hypotheses deal with comparing retention from each of the instructional conditions and are shown in Table 2.
After the initial assignment was completed, an overall ANCOVA was conducted in order to determine whether there were any statistically significant differences in the initial mean scores across the four incremental treatment conditions taken as a whole. A statistically significant difference was found: F (3, 394)=22.003, MSE 3.231, p=.000). Following the ANCOVA analysis, a Tukey planned contrast analysis was conducted to test the hypotheses.
Analysis by planned contrast found significant difference among all treatment groups for initial learning. The analysis found the effectiveness of the treatment groups ranked in the order hypothesized.
HYPOTHESIS 1–The case method group will demonstrate greater initial learning than the lecture group, was supported, (95 percent confidence level contrast estimate = 1.313 +.472, p.=000.)
HYPOTHESIS 2 – The simulation group will demonstrate greater initial learning than the lecture group, was supported, (95 percent confidence level contrast estimate = .726 +.484, p.=.007.)
HYPOTHESIS 3 – The combination group will demonstrate greater initial learning than the simulation group, was supported, (95 percent confidence level contrast estimate = 1.315 +.1.08, p.=.000.)
After the retention assignment was completed (one week after the initial assignment) an overall ANCOVA was conducted in order to determine whether there were any statistically significant differences for the mean retention scores across the four incremental treatment conditions taken as a whole. A statistically significant difference was found: F (3, 394)=14.345, MSE = 5.454, p=.000). Following the ANCOVA analysis, a Tukey planned contrast analysis was conducted to test the three hypotheses related to the retention assignment. The results of the Tukey planned contrast analyses are reported individually.
An analysis by planned contrast for the retention assignment found a statistically significant difference between the lecture group and the three other, interactive groups. No statistically significant difference was found among the three interactive groups.
HYPOTHESIS 4–The case method treatment group will demonstrate greater retention than the lecture-only treatment group, was supported, (95 percent confidence level contrast estimate = 1.51 +1.22, p=.000.)
HYPOTHESIS 5–The simulation method treatment group will demonstrate greater retention than the lecture and case method treatment group, was not supported, (95 percent confidence level contrast estimate = .045+.09, p=.896.)
HYPOTHESIS 6–The combination treatment group will demonstrate greater retention than the simulation group, was not supported, (95 percent confidence level contrast estimate = .462+.924, p=.200.)
The results for initial learning indicate that either case method or simulation presentation when combined with the introductory lecture, result in more initial learning than lecture alone. The results also indicate that a combination of all methods is better than either of the other two.
For retention, the results are more mixed. The lecture method was found to be least effective, with the retention mean score significantly below the initial mean score for this group. No significant differences in performance on the retention assessments were found between the case method group, the simulation group and the lecture, combination group.
Comparing the differences between the initial assignment and the retention assignment for each group also showed mixed results. The lecture group’s mean score was significantly lower on the retention assignment as compared to the initial assessment (F.=7.61, p.=.007). All other treatment groups’ mean scores improved between the initial assignment and the retention assignment. However, the increase was significant for only the case method group at (F=28.53, p.=.000).
Another observation about instructional method is worth noting. The interactive instructional methods results, as shown in the retention assignment, were all statistically significantly higher than the score of the lecture group. All the interactive methods were clustered closely together in terms of their performance scores. This finding supports the contention of many active learning advocates, who contend that lecturing, as a passive form of learning, not only has low effectiveness for initial learning, but also creates learning in a way that is very difficult for students to retain.
This study is unique from those cited in the literature in that the students in this study were not involved in professional training for their careers. Three of the retention studies cited used nursing students studying nursing skills (Larson, 1981; Boettcher et al., 1981; Vincent, 1984), one used practicing nurses improving their skills (Fishman, 1984) and one used practicing doctors improving diagnostic techniques (Carney et al., 1995). The students in this study had merely enrolled in an elective university course which included public relations along with other mass communications topics. Presumably their motivation level was not as high as that of students learning a professional skill they would use in their careers.
The lack of direct relevance of the RACE algorithm to the students in this study could explain why the mean performance scores in this study reached only 4.5 in a grading scale of 0 through 12. It might also be noted that this was the students’ first exposure to this algorithm, which could account for what might otherwise appear to be low scores.
Generalizations beyond the present study should be made cautiously. However, in combination with other studies cited in this document, the results tend to endorse the value of case method and simulation in combination with lecture as opposed to lecture alone.
Future studies might profitably investigate a broader scope than is usually the case. Most of the studies found in the literature concentrate on small increments of learning, presumably in order to accurately measure results. These, however, do not reflect the reality that several factors are often involved in an actual situation. Feldman (1995) endorses research with complex cases and simulations because they enable students to practice higher levels of abstract analysis of complex problems under conditions of uncertainty. Capstone courses frequently provide this broad application of problem solving principles and skills. Thus capstone courses could be a fertile field for research in teaching the application of public relations skills.
Table 1
Comparison of
Initial Mean Scores
of Treatment Conditions
Treatment Mean
condition Standard
n score deviation
Lecture 39 2.62 1.86
Case Study 26 3.30 1.82
Simulation 36 3.97 1.99
Combination 32 4.32 2.01
Table 2
Comparison of
Retention Mean Scores
of Treatment Conditions
Change
Treatment Mean Standard from
condition score deviation initial test
Lecture 2.41 1.55 -1.56
Case Study 4.11 2.02 +0.81
Simulation 4.19 2.04 +0.22
Combination 4.50 2.12 +0.18
Beadin, B. P. (1996). The case method: Theory and application. (Report ETT-96-02). High Plains Intermountain Center for Agricultural Health and Safety, Fort Collins, CO: Colorado State University.
Boettcher, E. G., Alderson, S. F., & Saccucci, M. S. (1981). A comparison of the effects of computer-assisted instruction versus printed instruction on student learning in the cognitive categories of knowledge and application. Journal of Computer-Based Instruction, 8, 13-17.
Bowen, D. D. (1987). Developing a personal theory of experiential learning. Simulation & Games, 2, 192-206.
Campbell, D. T., & Stanley, J. C. (1963) Experimental and quasi-experimental designs for research. In N. L. Gage (Ed.) Handbook of research on teaching. Chicago: Rand McNally.
Carney, P. A., Dietrich, A. J., Freeman, D. H. & Mott, L. A. (1995). A standardized patient assessment of a continuing medical education program to improve physicians’ cancer-control clinical skills. Academic Medicine, 70, 52-58.
Christensen, C. R. & Hansen, A. J. (1987). Teaching and the case method. Boston, MA: Harvard Business School.
Cole, H. P. (1997). Stories to live by: A narrative approach to health-behavior research and injury prevention. In D. S. Gochman (Ed.), Handbook of health behavior research methods, New York: Polenum.
DeNeve, K. M. & Heppner, M. J. (1997). Role play simulations: The assessment of an active learning technique and comparisons with traditional lectures. Innovative Higher Education, 21, 231-246.
Dewey, K. F. & Meyer, S. J. (2000). Active learning in introductory climatology. Journal of College Science Teaching, 29, 265-271.
Erickson, B. L. & Strommer, D. W. (1981). Teaching college freshmen. San Francisco: Jossey-Bass.
Feldman, H. D. (1995). Computer-based simulation games: A viable educational technique for enterpreneurship classes? Simulation & Gaming, 26, 346-360.
Fishman, D. J. (1984). Development and evaluation of a computer assisted video module for teaching cancer chemotherapy to nurses. Computers in Nursing, 2, 16-23.
Fleiss, J. L. (1981). Statistical methods for rates and proportions (pp. 212-236). New York: John Wiley & Sons, Inc.
Graham, P. T. & Cline, P. C. (1980). The case method: A basic teaching approach. Theory into practice. 19, 112-116.
Gredler, M. E. (1996). Educational games and simulations: A technology in search of a (research) paradigm. In D. H. Jonassen (Ed.) Handbook of research for educational communications and technology (pp. 521-540). New York: Macmillan.
Grunig, J. E. & Hunt, T. (1984). Managing Public Relations. New York, Holt, Rinehart & Winston.
Hendrix, J. A. (1997). Public relations cases. Belmont, CA: Wadsworth.
Larson, D. E. (1981). The use of computer-assisted instruction to teach calculation and regulation of intravenous flow rates to baccalaureate nursing students. Unpublished doctoral thesis, Michigan State University, East Lansing.
Lovelock, C. H. (1986). Teaching with cases. In Lewis, L. H. (Ed.) Experiential and simulation techniques for teaching adults. (pp. 25-35). San Francisco: Jossey-Bass.
Lubbers, C. A., & Gorcyca, D. A. (1997). Using active learning in public relations instructions: Demographic predictors of faculty use. Public Relations Review, 23, 67-80.
McCarthy, J. P. & Anderson, L. (2000). Active learning techniques versus traditional teaching styles: Two experiments from history and political science. Innovative Higher Education, 24, 279-294.
McKeachie, W. J. (1978). Teaching tips. Lexington, MA: D. C. Hatch & Company.
Marston, J. E. (1979). Modern Public Relations. New York: McGraw-Hill. Popham, W. J. (1999). Classroom assessment. Boston, MA: Allyn and Bacon.
Saunders, M. D. & Perrigo, E. (1998). Negotiation as a model for teaching public relations professionalism. Journalism & Mass Communication Educator, 52, 57-65.
Vincent, J. L. (1984). A comparison of learning outcomes of lecture discussion and computer assisted instruction in a baccalaaureate nursing program. Unpublished doctoral dissertation, University of Kansas, Lawrence.
Williams, F. (1979). Reasoning with statistics. New York: Holt, Rinehart and Winston.
[Return to PRD Home Page]
[Return to Teaching Public Relations Index]
April 15, 2001