Gathering and acting on feedback
Gathering your own Student Feedback
Peer observation of teaching and other collaborative activities
Responding to student feedback
Collecting feedback and reading it is only the first step in using student feedback. Academic staff may wish to consult the ITL staff on their interpretation of the feedback or the action planned on the basis of such feedback. Such consultation is part of the ITL's usual role with staff and is of course entirely confidential.
Gathering your own Student Feedback
1. Quick & Easy Feedback Strategies
While many staff already collect student feedback at the end of semester using questionnaires and student group interviews, it is usually collected too late to allow staff to respond by implementing changes to improve the teaching of the students offering the feedback.
Staff who would like to be able to monitor their teaching during the semester in order to act immediately to improve their teaching might find collection of student feedback during the semester a useful strategy. As with all aspects of feedback and evaluation of teaching, it is useful to draw upon a range of different yet complementary sources of data.
While formal feedback strategies can be adapted to collect feedback during the semester, there are a range of strategies suitable for gathering informal feedback from students. Perhaps the most obvious way is to ask the students questions directly. If classes are small and interactive and there is mutual trust and respect on the part of the students and teachers, then posing questions directly to a class may be feasible. However, there is no scope for anonymity of student responses and some students may feel uncomfortable voicing less popular opinions in such a public manner. There are various strategies which have been developed to address these issues yet still permit quick and easy collection of feedback from students. Information on some of these is included in this section.
Three open questions
This strategy can be adapted to different modalities and purposes. There is nothing magical about the number three however this number of questions usually provides a manageable amount of feedback without taxing student or staff resources unnecessarily. This uses students' individual written responses to open questions asked at the end of the class. The questions might be written on the board or an overhead transparency or they may be provided as an actual questionnaire. The questions you choose to ask will be determined by what you want to gather feedback on. However they are typically along the lines of the following-:
- "What was the most useful thing you learned today?"
"What was the best thing about today's class?"
"How could I change my teaching to help students to learn more from this class?"
In some cases it might be appropriate to ask very specific questions such as;
- "Which of the set pre-readings was most helpful in preparing
for today's class?"
"How did today's computer learning task help you understand the concept of...?"
In larger classes the time required to read written responses from every student can be a barrier to using this technique. In such cases sampling procedures can be helpful. There are many ways to select a random sample of students. One simple technique is ;
- "Please pass these questionnaires along the row, could every fifth person take one and fill out the questionnaire. In a few minutes I'll ask you to pass your questionnaire back along the row to the end so I can collect them. While those people are filling out the questionnaire would the rest of you...."
Other techniques used include selection based on sub-groups in the class, for example "All those in a Tuesday tutorial group....". If sampling is used then the usual caveats apply in that there is only a probability that the results of your sample are representative of the whole population.
Group responses: An alternative to sampling in larger classes is to divide the class into groups of 5-10 students and collect the collated responses to the questions from each group of students after the individual students have contributed to a group discussion.
- "I want you each to write down your answers to these three questions. In a few minutes I want you to discuss your answers in your groups. Once everybody has had their say I'd like one person in each group to write down their group's response to each question."
This technique is useful as it ensures discussion and some degree of consensus amongst the students before they respond. At the same time it streamlines the amount of feedback that the teacher has to read. In some settings, this technique can be equally well adapted to verbal presentation of group responses.
Critical learning statement
This is a useful technique which gathers feedback on student learning as a way of investigating the effectiveness of teaching. In this strategy students are asked to write down the three critical points they have most clearly learned from the class and the three points they are still most unclear about. This strategy is also known as three clear and three muddy points.
A consideration of the responses and identification of any recurring themes is an effective way of identifying where teaching has worked well in terms of student learning and where it could be improved. It also provides feedback which can be acted upon quickly as the teacher can review any particularly difficult points in the following session.
A variant of this strategy involves asking students to write down three questions they still have on the topic of the class. This can provide pointers to any remaining areas of confusion for students and also provide staff with an indication of what students are interested in learning more about. This can of course also be adapted for group responses rather than individual responses.
2. Student Group Interviews
In units of study with small enrolments it is also possible to obtain student feedback using structured group interviews. In this process the staff member meets with an agreed interviewer (e.g. fellow staff member or RA) to devise a suitable interview schedule. On the appointed day, the interviewer visits the class and the staff member leaves. The class is then interviewed and the responses to the preset questions as well as an indication of the extent of support for different opinions and views within the group are documented by the interviewer. Comments and suggestions from the students can be followed-up during the interview and this additional information is included in the report. The interview usually takes about 45 minutes. A report is written and sent to the staff member.
This process is particularly suitable for small classes of less than 20 students. However, the process can be easily adapted for use with larger groups. The structured group interview is particularly useful when information about the reasons for the students' opinions or perceptions is required. The additional information typically volunteered by students can prove a valuable source of data for new staff, new units of study or when seeking feedback on innovations in existing teaching practices.
3. Focus Group Interviews
Focus Groups are a variant of the Structured Group Interview method adopted for surveying larger classes. Students are asked to nominate / self select a group of approximately 15 students to participate in a structured group interview. The process is then the same as that of the structured group interview however it is conducted with this subgroup of the complete class. On the day of the interview the interviewer visits the class, the interview group of students remain and the rest of the class leaves. The interview is then completed. A report is written and forwarded to the staff member. It is recommended that the feedback from the report be validated with the student group as a whole in a subsequent class to ensure that the comments are representative.
This process is an effective way of obtaining qualitative data from large student groups when particularly detailed information is required as in a formal unit of study review or evaluation.
Lizzio et al. (2002) have developed focus group protocols for investigating students’ experiences of workload and assessment at the unit of study level. These appear in the appendices of the following paper:
Lizzio, A., Wilson, K., & Simons, R. (2002) University students’ perceptions of the learning environment and academic outcomes: Implications for theory and practice. Studies in Higher Education, 27, 27-51.
4. Getting Feedback Online
If your Unit of Study includes online materials and activities, you might consider gathering student feedback on their learning experience via online means. You could invite comments on particular topics and tasks via
- private e-mail (in WebCT: 'Mail'),
- a public discussion list (in WebCT, 'Discussion'),
- a public chat session. (in WebCT: 'Chat').
Using one-to-one e-mail (eg. WebCT Mail) guarantees a certain privacy for each student, in that the exchange will be between you and that student only. You could invite your students to e-mail you privately at particular points in the Unit with their questions, and some brief notes or comments about what they find useful or aren't sure about. Your availability and the feedback you provide via this private channel will be appreciated by many of your students.
In a discussion list all messages and posts to the list are likely to be read by everyone enrolled in your Unit (unless you set it up otherwise). In WebCT, the 'Discussion' tool is structured around discussion forums called 'Topics'. You could create an 'Evaluation' Topic in the WebCT Discussion/Topic area. Here students could be encouraged to post feedback on specific aspects of the teaching-learning process. They might discuss, for example, what they understood were the key points in a particular talk, topic or reading. This kind of discussion will enable you to determine whether the teaching-learning processes are effective in supporting and developing the students' understanding.
The composition of the group which accesses the 'Evaluation' Topic discussion will influence the nature of what is posted. Therefore you will need to indicate to your students whether the 'Evaluation' Topic area is public and accessible by all, including yourself and other teaching staff. You might consider setting up 'private' group-based discussions in the WebCT Discussion/Topic area. After a period of online discussion, one student representative from each group could e-mail you a summary of the group's discussion. Or group representatives could post their summaries to a public Discussion/Topic area for you and others to discuss.
Online chat rooms (eg. WebCT Chat) offer a third online feedback option. Because chat sessions are synchronous and must run at a set time, they are not necessarily convenient for all students. However, you may decide to set up regular or occasional chat sessions, to handle student queries and issues. (This is very much like opening your office door at set times to your students). You could pose open-ended questions to the students in the chat room about specific or general aspects of the learning experience in that Unit of Study.
Peer observation of teaching and other collaborative activities
5. Peer Observation of Teaching
Student feedback on teaching and units of study provides only a limited perspective for the evaluation of teaching. There are aspects of units of study and teaching on which students are sometimes not in a position to provide feedback. In addition to data drawn from students it is often beneficial to gather feedback from academic colleagues. This may be done by inviting a colleague to visit your class to observe a session prior to your engaging in a reflective discussion with them. Colleagues may be peers in the same department, colleagues in a different field or staff from the ITL. Please read Teaching Insight 6 for further information.
6. Collaborative Peer Teaching Activities
An alternative to engaging in peer observation of teaching is to ask a colleague to participate in the
- collaborative planning of a class
- collaborative teaching of a class
review of unit of study documentation
(ie. unit of study outline, assessment tasks and criteria etc.)
Such collaboration and reflection on teaching activities can provide you with additional insights or a different perspective on your teaching practice.
Collaborative planning of a class or teaching activity
Inviting a colleague to participate in the joint planning of a class or similar teaching activity can provide useful feedback on this aspect of your teaching practice. It is also an opportunity to share ideas and teaching strategies and can be a rewarding learning experience for both people.
Inviting a colleague to engage as a critical partner in planning a class can encourage you to reflect on and consider-:
- Your reasons for teaching the way you do.
- The effectiveness of the teaching strategies you select.
- Possible alternative teaching strategies and approaches.
In the joint preparation of a class it may be useful to consider at least the following issues-:
- What are the intended learning outcomes for the class and how do these relate to the aims and objectives of the unit of study as a whole?
- What teaching and learning activities have been selected to achieve these outcomes?
- How will the teaching and learning tasks or activities selected achieve these learning outcomes?
- How will the student learning be monitored?
The role of the critical partner is to prompt a consideration of these and similar issues, to support their colleague's reflection and contribute to the development of alternative teaching strategies.
- Such feedback and reflection may be usefully documented in the form of a learning outline with accompanying rationale and a list of potential alternative teaching strategies or ideas to follow up at a later date. Evidence of such reflection and subsequent action to improve teaching can usefully support claims of quality teaching.
Collaborative teaching of a class or teaching activity
University teachers often comment on the usefulness of co-teaching experiences in investigating and reflecting on their own teaching. Given current staffing levels, most academics do not have the luxury of routinely co-teaching classes even when there are several staff involved in teaching a unit of study. Negotiating with colleagues to co-teach a class however can usually be arranged. As a strategy for supporting reflection on teaching, some staff report this to be a less confronting yet equally useful variant of peer observation of teaching.
In inviting a colleague to collaborate in the teaching of a class it is useful to consider the following:
- Select a co-teacher from the same unit of study team or department or somebody with an interest / expertise in the topic of the class.
- Select a class that provides interesting scope for co-teaching activities.
- Agree that the observation and discussion will remain confidential and clarify the purposes of the co-teaching exercise.
- Meet with your colleague beforehand to discuss the topic of the class and to prepare for the teaching. Such collaborative planning of teaching is a worthwhile source of feedback on teaching in its own right - see collaborative planning of a class or teaching activity. Clarify your individual roles and responsibilities within the session and share your ideas about how you might teach the session.
- During the co-teaching session try to note how you and your colleague approach and handle different teaching tasks. What different strategies are employed and how do students respond? Consider the possible implications of different teaching strategies and techniques for student learning.
- After the session meet to review the class and share your observations and insights on each other's teaching. Often such a discussion is best initiated by a reflection on your own teaching before seeking additional observation and input from your co-teacher. Note aspects of your teaching that you could modify to further facilitate student learning, and jointly generate possible strategies to address these issues.
- Both staff members may find it useful to document the issues raised during the discussion and any possible actions or initiatives that are proposed to address relevant teaching and learning issues. Evidence of such reflection and subsequent action to improve teaching can usefully support claims of quality teaching.
Collaborative review of unit of study documentation
Collaborative critical review of unit of study documentation can also provide useful feedback to staff on aspects of their teaching practice. There are many different examples of documentation which can provide a useful basis for collaborative review. Some examples include:
- Unit of study Outlines
- Unit of study aims, objectives or learning outcomes.
- Assessment tasks and assessment criteria, (eg exam papers, essay questions)
- Lecture handouts and overheads
- Tutorial questions
- Tutors' / Demonstrators' teaching guides
- Student laboratory and practical manuals.
- Sets of prepared readings etc.
Review questions might encompass the following issues:
- What is the intended purpose of the documented aspect of teaching that is to be reviewed? How does it contribute to student learning?
- Against what criteria would you decide how well it achieves the intended purpose?
- Based on these criteria, what are its shortcomings and strengths with respect to promoting student learning?
- How could this aspect of teaching be improved?
Feedback from this process of collaborative review and discussion can be usefully documented under the questions noted above. Action statements and possible improvements arising from the process of collaborative review can also be documented. Evidence of such reflection and subsequent action to improve teaching can usefully support claims of quality teaching.
Responding to student feedback
7. Interpretation of the feedback
General Student Feedback
The following guidelines may assist you in interpreting the results. Please also see Teaching Insight 9 for further information.
(i) You should consider the representativeness of the responses. If the number of students completing the questionnaire as a percentage of the number of students enrolled in the unit is less than about 60% the responses may not be representative of the perceptions of those students who have experienced your teaching.
(ii) The statistical report gives the response analyses for all individual questions from your questionnaire. For each question the analysis shows:
- the number of students who gave each of the possible responses
- the percentage of students who gave each possible response (calculated over the number answering the question, which may be less than the number completing the questionnaire)
- a graphical representation of those percentages
- the mean and standard deviation
A summary of the teaching and learning practices considered to characterise each scale is attached to the report.
(iii) The students' responses to the open ended questions have been returned with the statistical report. The open-ended comments usually provide additional information worth considering when interpreting the statistical report. As the responses to the open ended questions typically include the most critical positive and negative aspects of the students' experience of the teaching and learning process, they are often the form of student feedback most useful for indicating potential improvements in teaching.
With your knowledge of the unit and its material you will be in a position to interpret the results of your particular survey, however you should bear in mind that student perceptions are shaped by various aspects of the teaching and curriculum context. For instance, a unit with a small number of students taught predominantly in small groups and using continuous assessment will invariably be rated more highly than a unit with large student numbers in which lectures are the predominant mode of delivery and where students are assessed solely on the results of a final unseen examination. Optional units tend to be rated more highly than compulsory sections of a program.
It should also be noted that students assign "ratings" individually in the absence of any agreed upon criteria for what constitutes "satisfactory". For example, a rating of "4" may be assigned by different students for different reasons and reflect different standards.
Standard Open Response Questionnaires
Our usual practice with the results of handwritten student surveys, is to hold the forms until after the completion of the examination period. The Institute for Teaching and Learning does not routinely provide written interpretive analysis of open comment feedback such as that contained in these surveys.
Student responses to open-ended questions such as those included in this survey typically include the most critical positive and negative aspects of the students' experiences of the teaching and learning process and are well worth considering. Such responses have been noted by academic staff to be the form of student feedback most useful for indicating potential improvements in teaching.
Student evaluation by questionnaire is only one source and method of collecting information. Other sources such as peers and other methods such as group discussion also provide valuable information and should be considered in course or teaching evaluation.
If you would like to discuss the interpretation of the feedback, a member of the academic staff of the ITL would be happy to meet with you once you have considered the comments and report. Please do not hesitate to contact the ITL on extension 13725 or email firstname.lastname@example.org to arrange an appointment. Staff at the Institute for Teaching and Learning are available to discuss any aspect of your teaching or of the unit curriculum which you would care to develop further as a result of the students' feedback.
8. Feedback Cycle
Staff are encouraged to reflect on the results of their student feedback:
You may wish to use the format similar to the Unit of Study Evaluation template. Feedback should be obtained from a range of sources. This could include your own reflections of the teaching and learning in the unit of study, as well as any collaborations with colleagues.
- Context/Unit of Study Description/Background
- Key Issues Identified by Students
- Key Peer Review Issues
- Self Review Issues
9. How to tell students what you are going to do as a result of their feedback
Students commonly report that they rarely see any action arising from their feedback and consequently, they come to perceive their opinions and views as being of little real interest to staff. In order to recognise the value to staff of constructive student feedback, it is recommended that students be advised of actions that will or will not be taken on the basis of their feedback. This can often be done at the beginning of a subsequent class or via student notice boards. One framework for feeding such information back to students is to tell students:
- Which suggestions will be acted upon and how
- Which suggestions you would like to act on but are unable to and the reasons why
- Which suggestions you will not be acting on and why
For further details see Responding to Student Feedback