Wednesday, January 19, 2005

Concerns 1: Honesty

Perhaps my biggest concern as a University faculty member is issues of intellectual honesty . I am posting a summary sheet of a study I helped to supervise here so that it can be commented on by students, staff, and community members anonymously. Before reading what is below please keep the following in mind. To my knowledge, there is no evidence that students at this University are exceptional when it comes to intellectually dishonest behaviors. Information published on other Universities put this one well within the range of "normal" behavior. Data from other Universities do not not exist for all behaviors listed here. Conversely, just because similar behaviors exist elsewhere does not mean there existence here is not cause for concern.

Overview of intellectual honesty issues within the disciplines of chemistry and biochemistry.

A study was carried out over the summer looking at the extent of “cookbook” use and plagiarism. The study was carried out by a student under the guidance a Prof. in education and Dr. Hanley. Here, the results found in that study have been supplemented with a separate test of an anti-plagiarism software program using essays turned in electronically.

The following were the major findings broken down into self-reported student behaviour, students perceptions of other students’ behaviour, faculty perceptions of student behaviour, student ratings of incentives, and summary of results using the anti-plagiarism software.

Self-reported student behaviour:

1) 94.4% participating in the study reported that they used cookbooks. Cookbook use was universal among the level II and III students participating in the study.
2) 36.1% reported participating in copy and paste type plagiarism.
3) Self reported use of cookbooks increases over the course of the degree program. Lowest use is in prelim (1.79), next highest is year I (1.88), next highest is year II (2.23), and highest use is in year III (2.56). The scale from1-4 used here was: 1 – did not use cook books at all, 2 – used cookbooks occasionally, 3 – used cookbooks often, and 4 – used cookbooks nearly all the time.
4) The 5 highest levels of cookbook use occur in the final year chemistry courses. Within these five courses are the final year courses at the core of the chemistry degree program.

Student perceptions of other students’ behaviour:

1) Students estimated on average that ~70-80% of students use cookbooks to complete labs or assignments. This underestimates the self-reported value of 94.4%.
2) On a relative scale (1-5), students rank cookbook prevalence 4.42, copy and paste plagiarism (3.14), and data falsification (3.59). If the trend between the relative scale and the self-reported values continues (e.g.: 4.42 corresponds to 94.4% for cookbook use, and 3.14 corresponds to 36.1% for plagiarism), ~50% of students are falsifying data.

Lecturer perceptions of students’ behaviour:

1) On average lecturers participating believed ~50% of students were using cookbooks. It is worth noting, that the largest group of lecturers (42.9%) think 0-20% are using cookbooks.
2) As a group, teaching staff consistently perceived a lower prevalence of all categories of dishonest behaviours than did students. The categories included: copying colleagues assignments, data falsification, downloading entire essays, and copy and paste plagiarism. Many lecturers underestimate the extent of the problem and some seem to be completely disconnected from what their students are doing.

Student ratings of incentives:

1) On a relative scale (1-5), students indicated the major incentives to use cookbooks are a poor understanding of theory (3.64), course load too heavy (3.57), and labs identical year after year (3.53).
2) Least important of the proposed incentives (2.02) was lack of lecturer approachability.

Summary of results using anti-plagiarism software:

The results obtained using anti-plagiarism software (EVE) indicate that 7 out 32 (~20%) reports scanned showed evidence of a high degree of plagiarism. Among these reports, as much as 40.69% of the total report was copied from internet sites. While less convenient, careful use of GOOGLE is more effective at locating plagiarized web sites than EVE. Further, the level of plagiarism indicated was highly variable, a characteristic of this program which has been noted previously.[i]

Conclusions:

While these results are not flattering, in comparison to other studies on plagiarism at other universities, this university is in the higher end of prior reports. Reported values range from ~10% (Universities with honor codes)[ii] to ~42%.[iii] Although this university does not appear exceptional, the 36.1% is of considerable concern. Plagiarism has been proven to harm the grades of honest students1 and may have long-lasting effects on this university's reputation.

We were unable to find reports of “cookbook” use in the education literature so a comparison was not possible.

However, there are several points of deep concern.

1) Cookbook use appears to be learned over the course of study in chemistry at this university. Rather than being a bad habit from secondary school that is progressively reduced by attendance at this university, the opposite appears to be the case. Exceptions to this trend are biochemistry and analytical chemistry. Both of these disciplines exhibit low levels of cookbook use over all years.
2) If the trends from cookbook use and copy and paste plagiarism hold for students’ perception of data falsification as well, there could be a very serious problem that should be dealt with carefully.
3) Lecturers seem to greatly underestimate the extent of these behaviours.

Finally, it is notable that the disciplines with the lowest prevalence of cookbook use benefit from lecturers who regularly revise their lab manuals. While students do not perceive the reuse of the same labs year after year as the highest incentive to use cookbooks, the fact remains that there will be no cookbooks available to students for a new or newly revised lab module. Complete revision of all labs each year does not seem to be needed. Substantive changes to existing experiments appears to work well or better.

[i] Braumoeller, B.F., and Gaines Brian J. (2001) Actions do speak louder than words: Deterring Plagiarism with the Use of Plagiarism Detection Software. Political Science and Politics 34 835-39.

[ii] McCabe, D. L., L. K. Trevino, and K. D. Butterfield (2001). Cheating in Academic Institutions: A Decade of Research. Ethics and Behavior 11(3):219-232.

[iii] Newstead, S. A. Franklyn-Stokes, P. Armstead,. (1996) Individual differences in student cheating Journal of Educational psychology 88 229-241.

No comments: