Criterion 4 – Student Assessment
AUNQA requirements for Criterion 4
TRSU QA uses each one of AUNQA criterion requirements below to assess the relevant QA practice of a program.
4.1 Assessment alignment
A variety of assessment methods are shown to be used and are shown to be constructively aligned to achieving the expected learning outcomes and the teaching and learning objectives.
Diagnostic questions
- To what extent do the assessments and examinations cover the content of the courses and the programme (content validity)?
- To what extent do the assessments and examinations cover the objectives of the courses and of the programme as a whole (construct validity)?
- Are a variety of assessment methods used? What are they?
- Are the assessment methods constructively aligned to achieving expected learning outcomes? How are the constructive alignment evaluated?
4.2 Assessment and appeal policies
The assessment and assessment-appeal policies are shown to be explicit, communicated to students, and applied consistently.
Diagnostic questions
- Are there clear regulations for assessment or examination? How are they communicated to students?
- Are they applied consistently?
- Are the students satisfied with the assessment procedures? Do students have complaints?
- What is the appeal procedure? Did the students appeal? What is the cause?
4.3 Assessment for completion
The assessment standards and procedures for student progression and degree completion, are shown to be explicit, communicated to students, and applied consistently.
Diagnostic questions
- Is there an assessment for students to progress to a higher level? When and how is it conducted?
- Is there an assessment for students to complete a degree and receive an award? How is it applied?
4.4 Assessment criteria
The assessments methods are shown to include rubrics, marking schemes, timelines, and regulations, and these are shown to ensure validity, reliability, and fairness in assessment.
Diagnostic questions
- Is the assessment criterion-referenced?
- Are all the courses required to have rubrics and marking scheme for the assessment?
- How are validity, reliability and fairness assured in assessment?
- Are there safeguards in place to ensure objectivity?
4.5 Measuring achievement
The assessment methods are shown to measure the achievement of the expected learning outcomes of the programme and its courses.
Diagnostic questions
- What are the assessment methods for the achievement of course learning outcomes? How are the achievement measured?
- What are the assessment methods for the achievement of program learning outcomes? How are the achievement measured?
- Is there a relationship between the achievement of courses and program learning outcomes? How is this explained?
4.6 Assessment feedback
Feedback of student assessment is shown to be provided in a timely manner.
Diagnostic questions
- Is feedback of student assessment required by every course? When and how is the feedback provided to students?
- Does the feedback help improve student performance in the final assessment? Why or why not?
4.7 Assessment review
The student assessment and its processes are shown to be continuously reviewed and improved to ensure their relevance to the needs of industry and alignment to the expected learning outcomes.
Diagnostic questions
- When is the student assessment and processes reviewed? What is the review procedure?
- What aspects of student assessment are subject to review or improvement?
- How is the student assessment considered relevant to the need of industry?
- How is the alignment of student assessment to the ELOs evaluated? How might it be improved?
Scoring for TRSU QA
| TRSU-QA score | 1 | 2 | 3 | 4 | 5 |
| Items showing QA practice | 1 item | 2 items | 3 items | 4-5 items | 6-7 items |
AUNQA scoring rubrics
A seven-point rating scale used for AUNQA assessment is described below.
1. Absolutely inadequate
The QA practice to fulfil the criterion is not implemented.
There are no plans, documents, evidences or results available.
Immediate improvement must be made.
2. Inadequate and improvement is necessary
The QA practice to fulfil the criterion is still at its planning stage or is inadequate where improvement is necessary.
There is little document or evidence available.
Performance of the QA practice shows little or poor results.
3. Inadequate but minor improvement will make it adequate
The QA practice to fulfil the criterion is defined and implemented but minor improvement is needed to fully meet them. Documents are available but no clear evidence to support that they have been fully used. Performance of the QA practice shows inconsistent or some results.
4. Adequate as expected
The QA practice to fulfil the criterion is adequate and evidences support that it has been fully implemented. Performance of the QA practice shows consistent results as expected.
5. Better than adequate
The QA practice to fulfil the criterion is better than adequate. Evidence supports that it (the QA practice) has been efficiently implemented. Performance of the QA practice shows good results and positive improvement trend.
6. Example of best practice
The QA practice to fulfil the criterion is considered to be example of best practices in the field. Evidences support that it has been effectively implemented.
Performance of QA practice shows very good results and positive improvement trend.
7. Excellent (example of world-class or leading practice)
The QA practice to fulfil the criterion is considered to be excellent or is an example of world-class practices in the field. Evidences support that it has been innovatively implemented.
Performance of the QA practice shows excellent results and outstanding improvement trends.
How to write a response to the criterion requirements
- The response should describe a QA practice that the program do to fulfil the criterion. The QA may be a PDCA (plan-do-check-act) process to ensure the quality, and include documentation, evidence and result of the implementation.
- The response should illustrate clearly what the QA mechanisms or instruments are; when, where, by who, and how they are implemented and managed to fulfil the criteria.
- The response should identify and evaluate problems or issues that are not addressed, and indicate how the problems will be dealt with. The diagnostic questions in each of the criterion requirements may help identify the problems.
- To be concise and factual, the response should focus on information or data (or objective evidence) that directly addresses the criteria, and provide trends and statistics to show achievement and performance.
How the QA practice fulfil the criterion
Quality assurance practice should have QA plan that evaluates and may modify organization’s procedures to ensure that they provide the desired results. It includes documentation for the planning, implementation and assessment procedures for a project, process or any QA activity. To fulfil requirement 1.1 Outcomes formulation, for example, the QA plan should provide information to answer the following questions.
- What procedure does the program use to formulate the expected learning outcomes?
- Is a learning taxonomy used to write the statement of LOs? What taxonomy and how is it used?
- Is the taxonomy used for program or course learning outcomes, or both? Why?
- What are the objectives in terms of the quality of the learning outcomes (e.g. transparency, ease of assessment, the extent of alignment, enabling hierarchy, effectiveness, clarity to the stakeholders)?
- What is the process used to ensure that the quality objectives are met?
- What is a method used to measure the achievement of the quality objectives?
Operational Result
4.1 A variety of assessment methods are shown to be used and are shown to be constructively aligned to achieving the expected learning outcomes and the teaching and learning objectives.
In order to make sure that students can meet the learning outcomes, the program examines them prior to enrollment, during the study, and prior to graduation. Students are interviewed by the instructors to ascertain their commitment, motivation, and attitude.
At the end of each course, students participate in a variety of formative and summative assessments, including performances, quizzes, exercises, assignments, work presentations, and open-book exams. The purpose of these tests is to evaluate their knowledge and abilities in relation to the PLOs and CLOs. Students receive comments and feedback from the instructors to help them get better. Students who fail the exam have to retake it until they achieve the highest level of proficiency. Prior to continuing with their study, they must show that they can interpret their data in summative exams across many courses (e.g., business research, seminar, etc.).
Students must finish writing their thesis and publish at least one article in an international peer-reviewed publication in order to graduate from the program. The topic, proposal, and final thesis presentation rubric assessment forms are in good alignment with the PLOs and CLOs.
4.2 The assessment and assessment-appeal policies are shown to be explicit, communicated to students, and applied consistently.
Policies for assessments are thoroughly recorded. They outline the what, how, and when of assessment. The assessment data are utilized to enhance student performance as well as the teaching and learning process. The program orientation manual contains information on all official assessments, grading, criteria, passing level, and forms. These are also posted on a regular basis via email, Google Classroom, and Line. Every semester, the staff makes sure that students are fully aware of the assessments. Additionally, assessments are measured objectively and individually. Students are continuously monitored, evaluated and communicated by the main adviser and co-advisors to ensure that the schedule, procedures, and rules are adhered to. Every three months, students are required to produce a progress report for their supervisors.
If students have any formal concerns or objections regarding the study’s evaluations, they may file an appeal with the university and the Faculty of Business Administration. Three options are presented to them by our program, first, they can submit their appeal in person or via email to the program head, the instructors, and their advisor(s); second, students can file a complaint by contacting the registration department and completing the form, finally, students have the option to send an automoted email with their remarks or requests. The support staff regularly check for any appeals and report them to the committee. Staff members respond to appeals by either providing comments right away or during regular meetings, depending on the urgency of the matter, in order to find a way to resolve the issue and communicate the students.
4.3 The assessment standards and procedures for student progression and degree completion, are shown to be explicit, communicated to students, and applied consistently.
In every subject, student achievement is evaluated using a criterion-reference system. For every grade level, there is a predetermined minimum passing level, marking scheme, and score range.
To evaluate students’ participation in each class activity, a rubric assessment is created for coursework that includes a detailed explanation of each score level. The teaching staff who teach each topic are responsible for preparing the exam questions for the midterm or final exams. The exam paper is mainly structured with answer guides and marking schemes, including keywords of the answers being prepared beforehand. Before they are utilized, the program committee evaluates the test questions and determines whether they are appropriate in terms of reflecting learning outcomes, level of difficulty, language used, clarity of the questions, and objectivity. The teaching staff who created the question checks the exam paper, and the program committee reviews all of the question scores before assigning grades.
During the first week of the course and in the orientation book, all assessments are described in detail. All assessments are applied fair and impartial manner to all students and scores are given. These scores are tools to feedback to all students.
Every semester, thesis advisers are required to keep an eye on each advisee’s thesis credits and provide tailored input following the proposal and final defense. Additionally, students must submit and adhere to a progress plan in advance.
Thesis defense usually takes place once the students publish an article related to their thesis in a designated journal.
4.4 The assessments methods are shown to include rubrics, marking schemes, timelines, and regulations, and these are shown to ensure validity, reliability, and fairness in assessment.
The course syllabus includes and presents the timeline for each assessment. The assessments’ rubrics were created with PLO alignment in mind. Experts in the field, including invited examiners and other teaching staff, also provide qualitative remarks.
At the end of the academic year, the program committee often reviews the methods of assessment in order to make improvements. Every year, there is a discussion about the evaluation rubrics and marking methods. Focus group discussions with current students are conducted to get their viewpoints on the assessment methods and scales.
Written examination procedure.
- Two reviewers review examination questions
- Reviewers comment send it back to the lecturer (if there’s any issue)
- Students take an examination.
- Lecturers check examination paper.
- Lecturers grade the papers
- Registrar reviews the result and publish it ontime
- Re-take the course if fail.
- Program committee review exam procedure.
Students get to know the results of the coursework (summative examination + formative assessment) within two weeks of the examination through MIS. The students can appeal against the result in two weeks of result publication. Students who take presentation examinations receive a summary of comments from their advisors. They can also discuss with the examiners in person. Students have to respond to the comments and submit them to their advisors in two weeks.
4.5 The assessment methods are shown to measure the achievement of the expected learning outcomes of the programme and its courses.
During class, instructors always ask questions to evaluate how much students understands the lessons which reflect the CLOs and PLOs. Teachers discuss with them and point them in the right direction. During lectures and exam periods, students are free to express their opinions and ask questions.
Additionally, after each class students are invited to rate their understanding of the program and suggest improvements throughout class. Lecturers for every course evaluate students based on CLOs during their coursework. In exchange, at the end of the course, students are able to assess their own performance on CLOs. The students’ progress and endeavors are evaluated throughout the research proposal development, data gathering and analysis, manuscript preparation and thesis production processes. Furthermore, the students who successfully complete the thesis defense will assess PLO accomplishments and provide program feedback. The program committee meets regularly to monitor, analyze, and use the results of all feedback, PLOs, and CLOs achievement scores for improvement.
4.6 Feedback of student assessment is shown to be provided in a timely manner.
Regular formative assessments are conducted at the end of each class session and during one-on-one student supervision to ensure that feedback is provided promptly. Feedback is given to students at the end of class or later, if necessary. The registration department release the grade within 2 weeks of final exam date. At the end of the course, lecturers get feedback regarding their instruction. However, the lecturers have an option to response if there are some urgent issues and/or any specific reason for deviation of the performance from expected level.
During the consultation of the student’s proposal development, the thesis adviser offers comments to the advisee in order to assist them to enhance their efforts. If there is a significant delay in the gathering and analysis of data or in the writing of the publication, comments and recommendations will always be provided. Following the thesis presentation, all of the external committees’ comments are compiled and given to the students immediately. After a test, students receive feedback from their instructors one week later.
Google Classroom is one of the web-based platforms that the program has created for use in teaching and learning activities. Using this platform, teaching staff members can give rapid feedback. The program has utilized google classroom as one of the web-based learning platforms for teaching and learning activities. Teaching staff are able to provide immediate feedbacks on this platform.
4.7 The student assessment and its processes are shown to be continuously reviewed and improved to ensure their relevance to the needs of industry and alignment to the expected learning outcomes.
Students fill out a survey about the instructor’s instruction after each class. The instructor also provide continuous assessment score for each student’s participation. The student and instructor then receive the scores and comments based on CLOs at the end of the semester. Quizzes and exams are used to evaluate students’ progress in their coursework. During the thesis research process, students are evaluated based on whether they have completed the necessary milestones (e.g., passing the proposal defense, getting ethical approval for the study, finishing data collection, publishing research findings, and passing the thesis defense).
In order to examine the MBA students’ qualifying, proposal, comprehensive, and thesis defense, we have created a rubric and disclosed the evaluation standards.
Based on PLOs, stakeholder needs, and student achievement, student evaluations are routinely reviewed. All students’ feedbacks are responded by the teaching committee. The course and expected learning outcomes are considered to modify and improved accordingly, if appropriate.
Evidence
| ID_Evidence | Name_Evidence |
| 4.1-1 | Student assessment flow at different levels |
| 4.1-2 | Assessment plan in TQF3 |
| 4.2-1 | Appeal procedure |
| 4.3-1 | Orientation handbook |
| 4.4-1 | MIS- Exam results |
| 4.4-2 | Scoring rubric |
| 4.5-1 | TQF3- Assessment plan |
| 4.7-1 | Theses evaluation rubric |
Self-Assessment
| Requirements | Result | Score |
| 4.1 A variety of assessment methods are shown to be used and are shown to be constructively aligned to achieving the expected learning outcomes and the teaching and learning objectives. | / | 1 |
| 4.2 The assessment and assessment-appeal policies are shown to be explicit, communicated to students, and applied consistently. | / | 1 |
| 4.3 The assessment standards and procedures for student progression and degree completion, are shown to be explicit, communicated to students, and applied consistently. | / | 1 |
| 4.7 The student assessment and its processes are shown to be continuously reviewed and improved to ensure their relevance to the needs of industry and alignment to the expected learning outcomes. | ||
| 4.4 The assessments methods are shown to include rubrics, marking schemes, timelines, and regulations, and these are shown to ensure validity, reliability, and fairness in assessment. | / | 1 |
| 4.5 The assessment methods are shown to measure the achievement of the expected learning outcomes of the programme and its courses. | / | |
| 4.6 Feedback of student assessment is shown to be provided in a timely manner. | / | 1 |
| Overall | 5 | |