THE SCEQ FACTOR SCALES

The University of Sydney Student Course Experience Questionnaire is based upon the items included in the nationally administered Course Experience Questionnaire (CEQ). These items have been shown to cluster together to form factor scales: Good Teaching, Clear Goals and Standards, Appropriate Assessment, Appropriate Workload, Generic Skills, and Learning Community. There are other factor scales in the extended CEQ. Only some items from these additional factor scales are included in the SCEQ. There are other items included in the SCEQ which do not contribute to these factors but which are particularly relevant to the University of Sydney context, for example "I benefited from being in contact with active researchers". The scale that each item relates to is indicated on the reports (* denotes a reversed item).

TEACHING PRACTICES CHARACTERISING RESPONSES FOR EACH SCALE

The Good Teaching Scale. SCEQ items 3, 8,16, 19, 21 & 27. The scale is characterised by teaching practices which include the following: providing useful and timely feedback, clear explanations, motivating students, making the course interesting, and understanding students' problems. Lower scores on this scale are associated with the perception by students that such practices occur less frequently.

The Clear Goals and Standards Scale. SCEQ items 6, 12, 25 & 29*. Practices characteristic of this scale relate to the establishing of clear aims and objectives for a course and clear expectations of the standard of work expected from students. It is possible to employ the good teaching practices described under the Good Teaching Scale, without implementing practices characteristic of the Clear Goals and Standards Scale.

The Appropriate Assessment Scale. SCEQ items 13*, 20* & 26*. This scale deals with the extent to which assessment measures higher order thinking and understanding rather than simple factual recall. This scale does not probe other important aspects of assessment practices such as the congruence of the assessment with the material actually taught, the level of difficulty and the consistency of the quality of the assessment./

The Appropriate Workload Scale. SCEQ items 2*, 4*, 15, & 24*. Higher scores on this scale indicate a perception of reasonable workloads. Heavy workloads do not necessarily equate to high standards and expectations so the wording of the items probes the extent to which heavy workloads interfere with student learning. Heavy workloads tend to preclude students from engaging with and understanding the material they are learning. Instead, many students adopt surface approaches to learning as a strategy for dealing with high workloads.

The Generic Skills Scale. SCEQ items 5, 9, 10, 18, 22 & 23. This scale reflects the extent to which students perceive their studies to have fostered the development of the generic skills recognised by the university as being a valuable outcome of university education, in addition to discipline specific skills and knowledge. These skills are detailed in the Academic Board document 'Generic Attributes of Graduates of the University of Sydney'.

Learning Community Scale. SCEQ items 28, 31, 33, 34 & 35. This scale concerns student perceptions of the social experience of learning at university. It indicates the students' sense of belonging to a community where learning with other people is a priority.

At least one item from each of the following extended CEQ scales is also included in the current SCEQ:

Learning Resources Scale. SCEQ items 14 & 32

Graduate Qualities Scale. SCEQ item 11

Intellectual Motivation Scale. SCEQ item 1

 

The On Campus Experience (2001 - 2003) SCEQ items 39, 40, 41, 42, 43, 44. In 2001 six additional items (39 - 44) were piloted. These items gather data on the contribution of students' on-campus experiences to the development of broader university learning outcomes such as graduate attributes. The pilot items were developed on the basis of the current literature on the on-campus experience. The items ask students to use the five point scale to rate the contribution to their overall personal and intellectual development of various key aspects of their on-campus experience. In the 2002 and 2003 surveys, the core item of the On Campus Experience (q39. The extra-curricular activities at the University of Sydney have helped me develop as a person) was retained.

Graduate Attributes (2005) SCEQ items 36, 37, 39, 42, 48. In 2005 five additional items were piloted to gather data on the contribution of students' degree experiences to the development of University of Sydney graduate attributes. The pilot items were developed on the basis of University of Sydney policies around these attributes.

Research-Enriched Learning and Teaching (2005) SCEQ items 40, 44, 46, 47, 49. In 2005 five additional items were piloted to gather data on the contribution of research-enriched learning and teaching activities to students' degree experiences. The pilot items were developed on the basis of University of Sydney policies around such activities.

Information and Communication Technologies in Teaching and Learning (2005) SCEQ items 14, 38, 41, 43, 45. In the 1999 to 2003 surveys, item 14 ("Where it was used, information technology helped me to learn") was used as an indication of quality of e-Learning. In 2005 four additional items were piloted to broaden the scope of quality assurance of this aspect of students' degree experiences. The pilot items were developed based on University of Sydney policies around e-Learning.

DATA REPORTING CONVENTIONS

The questionnaire asked students to respond to statements using a 5 point Likert scale to indicate the extent to which they agreed or disagreed with each item. The scale ranged from 1=strongly disagree to 5=strongly agree with a neutral mid-point. Prior to calculation of factor scale scores, the student responses to reversed items (items 2, 4, 13, 17, 20, 24, 26 & 29) were adjusted appropriately and all responses were re-calibrated using a -100 to +100 scale to facilitate reporting. This is the standard CEQ reporting procedure.

The following data is included in the various reports:

OVERALL REPORT
Faculty/Degree Average: This is the average of the student responses for each item or factor scale.
Broad Agreement: This is the proportion of students who responded indicating they 'broadly agreed', ie did not 'disagree' or 'strongly disagree', with the item statement. For the support and administration services items, the proportion of students who did not rate the services they used as 'poor' or 'very poor' are reported. This is a standard CEQ report format.
Agreement: This is the proportion of students who either strongly agreed or agreed with the SCEQ item statement once the reversed items had been adjusted. This figure removes the neutral responses from the measure of 'Broad Agreement' described above.
Disagreement: This is the proportion of students who either strongly disagreed or disagreed with the SCEQ item statements once the reversed items had been adjusted.
(Change): The figures in brackets represent the change in the value from the either the previous year's figure or the overall change since the first SCEQ survey in 1999. They are positive or negative depending on whether there has been an improvement in the score over the intervening year. (NB: All negatively worded items are reversed prior to calculating factor scale scores).

SUMMARY REPORT
Mean: The figure reported is the average of the students' ratings for each item, recalibrated to + 100 to -100 scale. The midpoint zero, corresponds to the neutral rating of 3 on the original Likert scale, +100 corresponds to five and -100 corresponds to 1.
Standard Deviation: This is a measure of how the scores cluster around the mean. The greater the variability in scores the greater the standard deviation.
N: This is the total number of responses (sample size) for that particular item.
(Change):The figures in brackets represent the change in the value from the either the previous year's figure or the overall change since the first SCEQ survey in 1999. They are positive or negative depending on whether there has been an improvement in the score over the intervening year. When interpreting the change figures it is important to note that some SCEQ items are negatively worded. These items are always indicated with an asterisk* in the reports. For these items a negative change might be an indication of improvement.
The data is also presented graphically as the percentage who agreed (4&5), were neutral (3) or disagreed (1&2) with each item.

DETAILED REPORT
%: This is the proportion of the students who assigned each rating (Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree) for the item.
Value: This is the recalibration of the 5 point Likert scale (Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree) to the +100 to -100 scale.
N: This is the total number of responses (sample size) for that particular item.

AGREEMENT REPORT:
Broad Agreement: This figure is the proportion of students who responded indicating they 'broadly agreed', ie did not 'disagree' or 'strongly disagree', with each item statement. It is the sum of the Neutral, Agree and Strongly Agree ratings, expressed as a percentage of the total number of responses for that item. The percentage Broad Agreement is also presented as a bar graph.
Agreement: This figure is the proportion of students who responded indicating they 'agreed', ie 'agreed' or 'strongly agreed', with each item statement. It is the sum of the Agree and Strongly Agree ratings, expressed as a percentage of the total number of responses for that item. The percentage Agreement is also presented as a bargraph.
N: This is the total number of responses (sample size) for that particular item.
(Change): The figures in brackets represent the change in the value from the either the previous year's figure or the overall change since the first SCEQ survey in 1999. They are positive or negative depending on whether there has been an improvement in the score over the intervening year. When interpreting the change figures it is important to note that some SCEQ items are negatively worded. These items are always indicated with an asterisk* in the reports. For these items a negative change might be an indication of improvement.

MINIMUM SAMPLE SIZE FOR REPORTING

SCEQ reports are available at university, college, faculty, and, in the case of undergraduate students, aggregated degree levels.

Aggregated degrees are groupings of similar degrees, which are used by the Office of Planning and Statistics for administrative purposes.

The minimum recommended sample size for SCEQ reporting is 20 valid responses. This is the same convention applied to reporting the CEQ. In aggregated degrees where less than 20 valid responses have been received a report is still provided however a warning message notes that the results should be interpreted with caution.

Reports are not provided for samples with fewer than 5 respondents. This is necessary to ensure anonymity of respondents.

OPEN RESPONSE COMMENTS

Each year the open response comments are made available to faculties to support the interpretation of the SCEQ data. They are typed up and returned to Faculty Deans and Associate Deans Teaching & Learning.