PURPOSE OF THE STUDENT EVALUATION OF INSTRUCTION

University Rules require that students be given the opportunity to evaluate the quality of instruction provided in each of their courses (University Rule 3335-3-35). The purpose of the Student Evaluation of Instruction (SEI) is to provide a standardized survey instrument for the collection of student feedback on the quality of instruction. This feedback is used as one of several methods of assessing teaching effectiveness when making personnel decisions regarding promotion, tenure, and merit pay. The SEI has several characteristics that make it particularly useful for such assessment. First, it is applicable to a wide variety of instructional settings and focuses on dimensions of teaching performance that are generally acknowledged as being important. Second, the content of the questionnaire and certain aspects of its administration are standardized. Third, an instructor's scores on a global question can be compared with those of others teaching similar courses.

Despite the utility of the SEI, it is important to keep in mind several considerations:

• The SEI is not a precise instrument. Therefore, the interpretation of its results
  should focus on extreme patterns, not on trivial differences in mean values.

• Although student evaluation of teaching is required, departments are not
  required to use the SEI. Instead, each department may design and use its own
  instrument for collecting student feedback, provided that the department uses it
  in a consistent manner. Faculty & TA Development (FTAD) is available to
  assist in the development of alternative instruments.

• Student feedback is only one source of data for assessing teaching
  effectiveness. SEI results should be interpreted in conjunction with data from
  other assessment methods, in particular peer evaluation of teaching and self-
  assessments.

The Office of Academic Affairs Guidelines for both student and peer evaluation of instruction are set forth in Section IV of the Office of Academic Affairs Policies and Procedures Handbook (http://oaa.ohio-state.edu/handbook/iv_instruction.html).

SEI DEPARTMENTAL ADMINISTRATION INSTRUCTIONS

SEI forms are automatically provided to all departments, unless a department has indicated that it has designed an alternative method of collecting student feedback. Forms are provided for all courses with enrollments of 5 or more, except for courses missing an instructor ID, course numbers 998, 999 or ending in 93, and courses with section type A, E, F, I, P, or Y depicted in the MARX course scheduling system. The number of forms provided is based on the number of students registered in a course. Questions about course enrollment should be referred to the Scheduling Office (292-1616).

• SEI forms will be delivered to the same departmental distribution point as class rosters. If the materials are not received by the end of the eight week of classes (or the end of the third week of first and second term summer term classes), contact the University Registrar at 292-2241 or by e-mail at seiadmin@osu.edu.

• As soon as the forms are received, departmental distributors should take the following steps:

  • Check that each course section received a minimum of one return envelope, and that sections with more than 150 students received one additional envelope for each increment of 150. A colored information sheet divides the course sections.

  • Contact the Office of the University Registrar if the forms need to be corrected for any reason, including if no instructor name is printed for a section, if the instructor name is incorrect, or if you did not receive enough forms. If you did not receive enough forms, please do not photocopy additional forms. Photocopying the forms will not work because the copied forms will not be able to be scanned by the computer.

  • Distribute materials to instructors upon receipt to allow for sufficient time for
    completion prior to the final exam period.

• Forms are computer scanned and therefore should not be folded or stapled.

• Please also contact the Office of the University Registrar if you received forms for courses in which the SEI is not used so that they are not produced in the future.

CLASSROOM ADMINISTRATION INSTRUCTIONS

• The SEI should be administered prior to the final exam period.

• The instructor should make certain that the forms have the correct course and call numbers pre-printed on them. The pre-printed course and call numbers are essential for processing the forms and cannot be changed by writing on the forms. Incorrect forms must be reprinted.*

• If multiple instructors are teaching the course, the students should be told that the
  evaluation is only for the instructor whose name appears on the form. Separate forms are provided for every instructor noted on the Master Schedule file.

• Students must complete the forms in class using #2 pencils (instructors should bring extra pencils).

• If the department does not assign a proctor, then the instructor must select a responsible proctor (student volunteer or staff member) to administer the instrument. The instructor should not be present during survey administration.

• The proctor should collect completed forms and place them in the return envelopes so that they all face one direction (not folded or stapled). Unused forms should go on the bottom of the stack.

• The envelope should then be sealed and the proctor should sign across the seal.
  The proctor should then place the sealed envelope in campus mail, or return it
  to the department office to be forwarded to the Office of the University
  Registrar. Forms returned after the envelope has been sealed must be sent
  separately to the Office of the University Registrar.

• Forms will be scanned immediately following the quarter. Forms received later than one week after the close of finals week will be scanned only upon special request as a result of unique circumstances, according the discretion of the instructor's department.

• If evaluation forms other than the SEI are used, such as open-ended comment sheets, these should be put in a separate envelope and returned to the instructor or department coordinator, according to departmental procedures.

*In the special case of cross-listed courses, the instructor should contact the SEI
Coordinator at the Office of the University Registrar prior to the seventh week of the quarter to have forms printed so that the course can be considered as one.

SEI SUMMARY REPORTS

SEI Summary Reports are confidential and are sent to instructors in the fourth week of the quarter following the evaluation. In the case of departments that have adopted the SEI system as part of their official pattern of administration, a copy of each report may also be distributed to the department chair on request. The following is a description of the information included in a SEI Summary Report:

Section I
The first table displays the percentage of students in the course that responded at each level (1-5 or Not Applicable) on each of the ten items. This table is useful for determining if there are outlier scores and may be most useful for small courses in which one student's response on an item may greatly influence the mean.

Section II
The second table on the report displays means and standard deviations for the following categories:

Instructor: The instructor's means and standard deviations on the ten items for the course section.

Comparison Group by College: These means and standard deviations are based on all of the courses in the instructor's college from the previous four quarters that are in the same Comparison Group as the course being evaluated. Comparison Groups are determined according to the course electivity and size. The appropriate Comparison Group for the course being evaluated is given in the box above the second table. These groupings control for course characteristics that can affect SEI scores. For more information on Comparison Groups, email seiadmin@osu.edu.

Comparison Group by University: These means and standard deviations are based on all of the course in the university from the previous four quarters that are in the same Comparison Group as the course being evaluated. Note that, in order to preserve confidentiality of other instructors' ratings and to ensure meaningful comparisons, calculations based on data from fewer than 5 instructors are not reported.

Course-Offering Unit*: The course-offering units are the same divisions of departments and schools that are used in the course bulletin. The means and standard deviations are calculated from the four years preceding the quarter that is being recorded. Depending on the size and duration of existence, these calculations might be based on a small amount of data or might be unavailable. These scores are not calculated according to Comparison Groups because there is not sufficient data to do so.

*The course-offering unit is the department or school under which the course is listed in the master schedule.

Section III
The third section of the report is a graph showing the distribution of mean scores, for the previous four quarters, on the overall rating (item 10 on the SEI form) for all of the courses in the university that are in the same Comparison Group as the course being evaluated. The graph of all university ratings, included on previous reports, has been omitted. The decision that the comparison with the entire university is not valid was made by the SEI Faculty Oversight Committee and confirmed by CAA. Henceforth, that graph will not be included, and earlier university-level graphs should not be considered in the evaluation process.

INTERPRETATION OF SEI SUMMARY REPORTS

Despite the utility and effectiveness of the SEI, please keep in mind that it is a blunt instrument and that its results should not be regarded as highly precise. Please also keep in mind that the SEI is only one source of data for assessing teaching. When interpreting results, the following advice should therefore be considered:

• Consider the results within the context of other less quantifiable information, such as the usual performance of the instructor and special circumstances surrounding the particular offering that might have influenced student opinion.

• Check response rate to determine if the scores reflect the opinion of a substantial part of the course enrollment. For small section sizes, look at frequencies as well as mean scores, since outliers can greatly influence the mean rating.

• The focus should be on patterns of responses and general comparisons rather than on trivial differences in mean values. To dwell on a trivial difference in mean values is inappropriate as a basis for comparing one instructor with another. Differences of a few tenths of a point should not be the basis for personnel decisions.

• Please keep in mind that SEI scores are generally high. Scores close to the mean of the Comparison Group indicate teaching quality that is perceived as satisfactory by the students. Scores at the extreme ends of the range provide the most significant information. Scores that are at the extreme high end are signals that students found the teaching to be extremely effective. Those at the extreme low end indicate that students perceive problems with teaching performance.

• In addition to looking at an instructor's scores on one summary report, please also look at the scores that an instructor has received over a period of time if they are available. For instance, you might want to consider patterns of improvement over time.

• The SEI is only one component of the evaluation of teaching. Data from other assessment techniques must also be taken into consideration. In particular, peer evaluation of teaching is required of all departments. Guidelines for peer evaluation are set forth in Section IV (Evaluation of Instruction) of the Office of Academic Affairs Policies and Procedures Handbook. Among other assessments of teaching effectiveness that might be also considered include student progress patterns and self-reports of the instructor.

• Faculty and TA Development is available to consult with instructors or departments regarding the interpretation of the SEI Summary Reports.

Please email seiadmin@osu.edu if you have concerns about the accuracy of the SEI Summary Report.

SPECIAL DATA REQUESTS

Instructors who wish to compare their scores with comparable courses on any of the variables collected on the SEI instrument that are not included in the summary report may request specialized reports from the Office of the University Registrar seiadmin@osu.edu). Requests for specialized reports require the approval of the department chair. A minimum of ten working days should be allowed for special reports. Departments may request data to perform their own analyses. However, the release of data is subject to the confidentiality policy stated above. Specialized reports on variables related to Response Information, Instructor Information, and Course Information are available.

THE DEVELOPMENT OF THE SEI

The SEI questionnaire and procedures for its administration resulted from a thorough development effort over several years. It began with a review of the literature in several disciplines related to student evaluations of teaching and, in a more general sense, to the measurement of human attitudes, opinion, and behavior. In addition, three SEI development committees conducted approximately 60 hours of in-depth interviews with faculty, 15 hours of in-depth interviews with deans and department chairs, two waves of the OSU Poll of faculty opinion, and an OSU Poll of student opinion.

Results of the SEI have also been systematically studied on three occasions to determine if there are course characteristics not relevant to the quality of teaching that affect the scores. These studies found that two extraneous factors most directly impact the results: (1) the reason why the student enrolled in the course (with elective courses more positively evaluated than required courses) and (2) class size (with small sections producing higher scores than classes in which more than 60 students are enrolled). To offset the influence of these characteristics, the SEI Summary Reports display the scores for a given course in relationship to the average mean scores of courses that that are of similar size and have a similar profile with respect to required/elective status (the "Comparison Group").

If you are interested in learning more about the findings of the studies used to develop the SEI or other research on the student evaluation process, please contact Faculty and TA Development, 292-3644.