1. Introduction

The great majority of those who teach operations research or management science at the college or graduate levels have no formal training in teaching and do not regularly read articles or books about teaching. These facts are well known but they raise fundamental questions for anyone interested in scholarly activity related to teaching MS/OR. Why do those of us who teach MS/OR spend so little time learning about teaching? Why is there essentially no literature (until recently) on teaching our subjects? Can there be a scholarship of teaching MS/OR? And the most provocative question: can the scholarship of teaching MS/OR ever deserve to be called research ?

These were some of the questions that were debated within INFORMS some years ago when the idea for what has become the INFORMS Transactions on Education was first proposed. There were strong voices within INFORMS who were opposed to the creation of this publication because, in their eyes, writing about teaching is not research. These questions arose again when the founding editor, Erhan Erkut, stepped down and a new editorial team took up the reins. At that time there were those who advocated that ITE move in the direction of publishing more experimental research, and there were those who objected that essays in which the authors simply shared their experiences about teaching still had value. This group also felt that experimental research about teaching was not as helpful to practicing teachers or even as scientific as it might appear. So questions about the nature and credibility of research on teaching have been central to the evolution of this journal since its inception.

This debate within INFORMS about the scholarship of teaching is to some degree parochial, in that few of the participants are aware of how these questions have been debated in other disciplines. One of the little facts in Weimer's book that jumped out at me was that the Journal of Chemical Education was first published in 1924! Here is a respected discipline that has an education journal in continuous publication for over 75 years. Could it be that there is something useful to say about teaching chemistry but nothing to say about teaching MS/OR? Is it even possible that scholarship about these subjects could be of interest to those who teach MS/OR, or is teaching one subject totally distinct from teaching another?

Weimer's recent book on the scholarship of teaching should be required reading for anyone interested in the scholarship of teaching MS/OR and anyone who participates in the debates on the subject. It provides a compact overview of "published work on teaching and learning authored by college faculty in fields other than education," that is to say, everything published in other fields in journals like ITE. (ITE, by the way, is not mentioned in her book.)

2. The Author

Maryellen Weimer is a professor of communication at Pennsylvania State University-Berks. She has held several administrative and grant-supported positions involved in teaching and learning. She is the editor of the Teaching Professor , a widely-read newsletter on college teaching. Among her many other books, I also recommend the recent Learner-Centered Teaching: Five Key Changes to Practice (Jossey-Bass, 2004).

Why should we in the Management Science community pay heed to a professor of communication? Two reasons stand out: one, some insights about teaching and learning do cross disciplinary boundaries; two, Weimer has done a remarkable service to us by distilling some of the best of the literature spread across hundreds of publications in dozens of fields.

Her own learning process stands as a case in point. She cites five periodicals that have taught her the most over the years about teaching. They are:

• Journal of Chemical Education
• Journal of Engineering Education
• Journal of Management Education
• Teaching Sociology
• Teaching Psychology

Notice that none of these journals comes from Weimer's own teaching specialty. Note also that two of these publications are in engineering or science and one is in management. Despite my own engineering training and experience teaching in a management school, I was only vaguely aware of the existence of these journals and am certainly not a regular reader.

Another important benefit of examining this book is that it provides useful insights into the debates that other professions have gone through around the scholarship of teaching. Weimer cites, for example, articles about editorial standards in the Journal of Engineering Education (Prados, 1996, 1999, 2001), a research forum in the Academy of Management Journal (Frost and Fukami, 1997) on how scholarly work on teaching and learning can match the quality of research journals, and a statistical analysis of the relative status of teaching and research journals in the Journal of Geography in Higher Education (Matthews, 2002).

3. Why and how to look

Weimer begins her review by asking why we should examine the practitioner literature on teaching and how. She points out that most faculty teach with little or no formal training and few read any literature on teaching. Most of us, therefore, learn how to teach from personal experience and the advice of colleagues. At the same time, there is a substantial practitioner literature on teaching and some ideas have been tested and found effective across multiple disciplines: why is this literature not more widely read, and why are these ideas not more widely implemented? Is the literature simply of poor quality and therefore not useful, is it unknown to most teachers, or perhaps is the lack of attention paid to this literature a consequence of the general devaluing of teaching common to higher education? Weimer issues the following call to arms (page 10):

Clearly, this book is motivated by the conviction that college teachers are woefully undereducated about teaching and that the existing literature on teaching, if read, could help change that situation. Even with these convictions, the author does a credible job of presenting the facts in an unbiased manner. One of the great strengths of this book is that Weimer presents literature she feels is exemplary and evaluates the strengths and weaknesses of different categories of literature in an open-minded manner. She allows the reader to make up his or her own mind, while presenting some of the best work across dozens of fields so that those of us with a narrower view of teaching can easily see what is known in other fields.

The heart of the book is Chapters 4 and 5, in which Weimer categorizes and evaluates the major streams of the practitioner literature. She divides this work into two main categories: wisdom-of-practice scholarship and research scholarship . Wisdom-of-practice scholarship refers to experience-based pedagogical writings by practicing teachers. This is the "how-to" literature of teaching. She further divides this category into four parts:

By research scholarship, Weimer means empirical research on teaching practices. This category is also broken down into sub-categories:

For each of these seven sub-categories Weimer provides a description, a critical assessment (what are the quality issues?), several exemplars (what does this work look like at its best?), an analysis of the appropriate standards that should be used when judging writing in this area, and an analysis of the contribution this category makes to practice. In what follows I will touch on just a few of the most interesting aspects of these chapters.

4. The wisdom of practice

The majority of publications on teaching are based on the personal experiences of the author. This raises serious questions about the validity of this literature. Research standards in all fields stress the importance of objectivity; in fact, it is difficult to imagine a research journal in almost any field accepting a paper based primarily on personal experience. Weimer's own view, as one might expect, is more accepting. She cites Schön (Educating the Reflective Practitioner, 1995) on this point:

The proof, in a sense, lies in the actual quality of this category of scholarship. I now provide a brief summary of each of the four sub-categories of this work.

4.1 Personal accounts of change

This is the typical paper we see published in Interfaces or ITE that gives an account of changes made by the author and colleagues to a specific course or collection of courses. Usually the story relates how it was done in the old days, why we decided to make changes, how we implemented those changes, and how the results lead to improvements. (I don't recall reading a paper of this type that concluded that the experiment was a failure.)

Weimer points out that there are two principal objections to this type of paper: its questionable objectivity, and inadequate referencing. The first point is inescapable: any teacher who sets out to make changes to a course is a biased observer of the outcome of those changes. This is a serious flaw, but one that can be addressed, Indeed, some journals have begun to require objective measures of success for such papers. Here is an eloquent statement from Journalism and Mass Communication Educator :

The second problem Weimer cites is the lack of references to the relevant literature. For example, a paper about changes that involve active learning or cooperative learning should cite the literature behind those approaches. This problem too can be fixed and it has drawn the attention of journals. The cause would seem to be that many who write in this area are not aware of the literature that relates to their experiences. This is a natural role for journal editors, who either should know the literature that needs to be cited or at the least should ensure that reviewers recommend relevant references.

Here is one of the exemplars Weimer provides for this group of papers:

Paulson, D.R. (1999) "Active Learning and Cooperative Learning in the Organic Chemistry Class," Journal of Chemical Education , Vol. 76, No. 8, 1, pp. 136-140.

What standards should apply to this literature? Here is Weimer's list:

• Recognize the knowledge base that justifies the change
• Critically analyze the change in depth
• Contain evidence that adaptation occurred
• Draw appropriately bounded implications
• Use relevant and rigorous assessment methods

While not a fully specified set of criteria, this list certainly should be applied to papers submitted to ITE in this category.

4.2 Recommended-practices reports

This category of literature offers advice on teaching methods. The advice is typically based on a combination of personal experience, collective experience, and empirical research. Weimer raises one major concern about this literature: who is qualified to give advice? In practice, pedagogical advice can be offered based on limited classroom experience and/or on a superficial understanding of previous work. Journals do not seem to have recognized this flaw. Weimer is pessimistic about correcting this problem, too pessimistic in my view. While journals are in no position to require quantitative proof of competence to advise, they certainly can require some evidence. For example, advice based primarily on personal classroom experience should be based on more than one or two years of teaching. Furthermore, advice should always be connected to the relevant literature. For example, a paper advocating active learning in the management science course should cite and summarize the literature on this approach and how it has worked in other disciplines.

Here is one of the exemplars Weimer provides for this group of papers:

Herreid, C.F. (1994) "Case Studies in Science – A Novel Method of Science Education," Journal of College Science Teaching , Vol. 23, No. 4, pp. 221-229.

What standards should apply to this literature? Here is Weimer's list:

• Relates to a meaningful aspect of instruction
• Offers "good" advice
• Communicates constructively

4.3 Recommended-content reports

These are advice-giving papers that focus on the content of a course, not the teaching methods. Some journals are entirely devoted to this class of paper, although I cannot remember more than one or two related to MS/OR. Weimer cites one major problem with this literature: it may send the implicit message that content is more important to successful teaching and learning than method. In fact, she offers this strong opinion (page 80) which I interpret as a challenge to the MS/OR community:

4.4 Personal narratives

These are reports on individual opinions, views, ideas, concerns, and positions related to teaching. While papers in this category may be based in part on classroom experience or other literature, the focus is on what the author thinks and feels. This work often has a strong emotional element, and authors often advocate for or against a particular policy or practice.

Credibility is a major issue with papers in this category. Not only are they based on personal experience, they are explicitly individual and emotional. In no other field are personal perspectives considered valid scholarship. Thus, one either rejects this category entirely or one must redefine one's notion of scholarship. Weimer suggests we do the latter, pointing out that the majority of the papers that have most influenced her own teaching are in this category. That is, she justifies this line of scholarship based on its impact on her personally, but that merely redefines the problem. How is the editor of ITE to judge a submission in this category: by its impact on him or her personally? I don't have an answer to these questions. Best to read some of the exemplars offered by Weimer and learn more about this style of paper.

Here's one paper that sounds worth reading:

Bailey, J. (2000) "Students as Clients in a Professional/Client Relationship," Journal of Management Education, Vol. 24, No. 3, pp. 353-365.

What standards should apply to this literature? Here is Weimer's list:

• In-depth, critical self-reflection
• Transcendence of personal relevance and application
• Originality of insights
• Excellence of writing

5. Research scholarship

The second major category of pedagogical literature compares much more closely to the research done in most disciplines, which is empirical and objective rather than experience-based and subjective. Weimer asks five challenging questions about this scholarship (page 92):

• What is good pedagogical research? Work that discovers, that interprets, or that explores? Work that is quantitative, qualitative, or descriptive?
• Can practitioners operating out of disciplines other than education do quality pedagogical research?
• What are the implications of studying teaching and learning with perspectives and methods borrowed from the disciplines?
• What is the likelihood of research scholarship adding to the knowledge base and improving instructional practice?
• Is pedagogical research inherently better than experience-based work? Is it more scholarly, more intellectually rigorous, and more likely to affect practice and gain respect for the profession?

5.1 Quantitative investigations

This is experimental work that uses research designs involving treatment and control groups and a manipulation of variables across them. To this extent it is taken directly from formal research methods in the social sciences. This apparently is its great appeal: Weimer believes that the superiority of this style of research is virtually unchallenged in the journals. This work is closely related to research done by educational researchers. The only important difference is that the researchers here are practitioners in their own disciplines.

But is this approach to pedagogical scholarship really the highest? Weimer raises a number of critical points. From the viewpoint of contributors to ITE , this is the most critical part of the book.

Weimer's first critique is that many of these studies are poorly designed, presumably because the researchers are not trained social scientists. This may seem like a fixable flaw, but the question then arises as to how feasible it is to train practicing teachers in the research methods of the social sciences? In addition to teaching their courses, doing research in their disciplines, and performing service for their institutions, are practicing teachers really likely to master the research methods required for quantitative studies? If not, how poor will the results be?

The second point Weimer makes is that these studies often mimic the methods of the underlying discipline. It seems on its face dangerous to assume that teaching and learning function the same way as the phenomena studied by the discipline itself. As Weimer says (page 96):

Weimer's third and potentially most devastating critique is that the methods of the social sciences are not designed to study education in action. As she says (page 97): "Classrooms are dynamic milieu with far too many variables to control for effectively." Or as Nummendal, Benison, and Chew (2002) write:

It these critiques hold, it hardly seems justified to give this class of studies the uncritical acceptance it generally receives.

Here's one exemplar from this work:

Bacon, D. R., Stewart, K. A., and Stewart-Belle, S. (1998) "Exploring Predictors of Student Team Project Performance," Journal of Marketing Education, Vol. 20, No. 1, pp. 63-71.

What standards should apply to this literature? Here is Weimer's list:

• A good research design
• Appropriate use of research methods
• Viable research question
• Accurately address issues of implication and application

5.2 Qualitative studies

Qualitative research is controversial even in the social sciences, and it may be unfamiliar to many readers. Here is one definition (Denzin and Lincoln, 1994):

This research, which is based on ethnography, is sometimes difficult to distinguish from experience-based approaches. As such, it suffers from many of the problems of those approaches. So while some dismiss this work as unscientific ("mere journalism"), others dismiss it as falsely rigorous. Weimer's own opinion is that the few studies of this type that have appeared are generally excellent, perhaps because the authors are well-trained in the methods. She also notes that unlike many quantitative methods, these methods fit pedagogical settings like the classroom.

Here's one exemplar from this work:

Haller, C. R., Gallagher, V. J., Weldon, T. L., and Felder, R. M. (2000) "Dynamics of Peer Education in Cooperative Learning Workgroups," Journal of Engineering Education, Vol. 89, No. 3, pp. 285-293.

5.3 Descriptive research

The final sub-category of pedagogical research is the largest and least controversial. This type of research attempts to describe what is , usually through surveys of students, faculty, or both. The overall quality of this work is high, although the questions asked are rather basic. Survey research always begs the question of whether what people say actually reflects what they do. And survey research can never establish what's best , which is ultimately what the practitioner wants to know.

Here's one of Weimer's exemplars from this work:

Deeter-Schmelz, D. R., Kennedy, K. N., and Ramsey, R. P. (2002) "Enriching Our Understanding of Team Effectiveness," Journal of Marketing Education, Vol. 24, No. 2, pp. 114-124.

What standards should apply to this work?

• Good descriptive research design and analysis
• Tied to relevant theory and connected to related research
• Addresses important topics
• Proposes appropriate implications

6. Summary

The remainder of Weimer's book examines some promising options for innovative pedagogical scholarship (books, newsletters, and so on) and offers advice to faculty contemplating entering this area of research and to administrators evaluating such research. While these chapters are interesting in their own right, I will not attempt to summarize them here. For me, the heart of the book is the chapters that categorize and assess both the wisdom of practice and the research scholarship literatures.

Weimer has broken important new ground for all us interested in the scholarship of teaching but unsure just what that means. Her book is not the last word but the first word. It offers a very helpful categorization of the literature and a well-reasoned and open assessment of the strengths and weaknesses of each category. She does not shy away from expressing her opinions, but she makes clear the basis for those opinions. While I do not agree with everything she says, I know where she stands and why.

Much of what she says applies with full force to the situation of pedagogical scholarship in the MS/OR community. In this highly scientific and methodologically sophisticated community, research is king. Teaching is something most of us do but few devote precious research time to it. And teaching is not strongly rewarded by those who assess us professionally. So the hurdles that writing on teaching must overcome to be valued (and read) by this community are high. So too is the skepticism that may always attend a journal like ITE that is devoted to teaching. But surely we can improve the journal and its reputation if we can better understand the various types of pedagogical scholarship and learn which have been successful in other fields. Weimer's book provides a strong first step in that direction.

References

Denzin, N. K., and Lincoln, Y. S. (2000)(eds.). Handbook of Qualitative Research, (2nd Ed.) Thousand Oaks, Calif.: Sage.

Frost, P. J., and Fukami, C. V. (1997) "Teaching Effectiveness in the Organizational Sciences: Recognizing and Enhancing the Scholarship of Teaching," Academy of Management Journal, Vol. 40, No. 6, pp. 1271-1281.

Matthews, H. (2002) "Editorial I: Pedagogy, Research and Quality Publishing," Journal of Geography in Higher Education, Vol. 26, No. 1, pp. 5-11.

Nummendal, S. G., Bension, J. B., and Chew, S. L. (2002) "Disciplinary Styles in the Scholarship of Teaching and Learning." In M. T. Huber and S. P. Morreale (eds.), Disciplinary Styles in the Scholarship of Teaching and Learning: Exploring Common Ground, Washington, D. C.: American Association for Higher Education and Carnegie Foundation for the Advancement of Teaching.

Prados, J. W. (1996) "The Editor's Page," Journal of Engineering Education, Vol. 85, No. 3, p. 173.

Prados, J. W. (1999) "The Editor's Page," Journal of Engineering Education, Vol. 88, No. 2, p. 139.

Prados, J. W. (2001) "The Editor's Page," Journal of Engineering Education, Vol. 90, No. 2, p. 169.

Weimer, M. (2004) Learner-Centered Teaching, Jossey-Bass, San Francisco, CA.

Weimer, M. (2006) Enhancing Scholarly Work on Teaching and Learning, Jossey-Bass, San Francisco, CA.

	To download a printable version (pdf) of this paper, click here. To download the Adobe Acrobat reader for viewing and printing pdf files, click here.
	To reference this paper, please use:  Powell, S.G. (2006), "Review of Maryellen Weimer: Enhancing Scholarly Work on Teaching and Learning (Jossey-Bass, 2006)," INFORMS Transactions on Education, Vol. 6, No 3,  http://ite.pubs.informs.org/Vol6No3/Powell/