Criterion 4 – Student Assessment
AUNQA requirements for Criterion 4
TRSU QA uses each one of AUNQA criterion requirements below to assess the relevant QA practice of a program.
4.1 Assessment alignment
A variety of assessment methods are shown to be used and are shown to be constructively aligned to achieving the expected learning outcomes and the teaching and learning objectives.
Diagnostic questions
- To what extent do the assessments and examinations cover the content of the courses and the programme (content validity)?
- To what extent do the assessments and examinations cover the objectives of the courses and of the programme as a whole (construct validity)?
- Are a variety of assessment methods used? What are they?
- Are the assessment methods constructively aligned to achieving expected learning outcomes? How are the constructive alignment evaluated?
4.2 Assessment and appeal policies
The assessment and assessment-appeal policies are shown to be explicit, communicated to students, and applied consistently.
Diagnostic questions
- Are there clear regulations for assessment or examination? How are they communicated to students?
- Are they applied consistently?
- Are the students satisfied with the assessment procedures? Do students have complaints?
- What is the appeal procedure? Did the students appeal? What is the cause?
4.3 Assessment for completion
The assessment standards and procedures for student progression and degree completion, are shown to be explicit, communicated to students, and applied consistently.
Diagnostic questions
- Is there an assessment for students to progress to a higher level? When and how is it conducted?
- Is there an assessment for students to complete a degree and receive an award? How is it applied?
4.4 Assessment criteria
The assessments methods are shown to include rubrics, marking schemes, timelines, and regulations, and these are shown to ensure validity, reliability, and fairness in assessment.
Diagnostic questions
- Is the assessment criterion-referenced?
- Are all the courses required to have rubrics and marking scheme for the assessment?
- How are validity, reliability and fairness assured in assessment?
- Are there safeguards in place to ensure objectivity?
4.5 Measuring achievement
The assessment methods are shown to measure the achievement of the expected learning outcomes of the programme and its courses.
Diagnostic questions
- What are the assessment methods for the achievement of course learning outcomes? How are the achievement measured?
- What are the assessment methods for the achievement of program learning outcomes? How are the achievement measured?
- Is there a relationship between the achievement of courses and program learning outcomes? How is this explained?
4.6 Assessment feedback
Feedback of student assessment is shown to be provided in a timely manner.
Diagnostic questions
- Is feedback of student assessment required by every course? When and how is the feedback provided to students?
- Does the feedback help improve student performance in the final assessment? Why or why not?
4.7 Assessment review
The student assessment and its processes are shown to be continuously reviewed and improved to ensure their relevance to the needs of industry and alignment to the expected learning outcomes.
Diagnostic questions
- When is the student assessment and processes reviewed? What is the review procedure?
- What aspects of student assessment are subject to review or improvement?
- How is the student assessment considered relevant to the need of industry?
- How is the alignment of student assessment to the ELOs evaluated? How might it be improved?
Scoring for TRSU QA
| TRSU-QA score | 1 | 2 | 3 | 4 | 5 |
| Items showing QA practice | 1 item | 2 items | 3 items | 4-5 items | 6-7 items |
AUNQA scoring rubrics
A seven-point rating scale used for AUNQA assessment is described below.
1. Absolutely inadequate
The QA practice to fulfil the criterion is not implemented.
There are no plans, documents, evidences or results available.
Immediate improvement must be made.
2. Inadequate and improvement is necessary
The QA practice to fulfil the criterion is still at its planning stage or is inadequate where improvement is necessary.
There is little document or evidence available.
Performance of the QA practice shows little or poor results.
3. Inadequate but minor improvement will make it adequate
The QA practice to fulfil the criterion is defined and implemented but minor improvement is needed to fully meet them. Documents are available but no clear evidence to support that they have been fully used. Performance of the QA practice shows inconsistent or some results.
4. Adequate as expected
The QA practice to fulfil the criterion is adequate and evidences support that it has been fully implemented. Performance of the QA practice shows consistent results as expected.
5. Better than adequate
The QA practice to fulfil the criterion is better than adequate. Evidence supports that it (the QA practice) has been efficiently implemented. Performance of the QA practice shows good results and positive improvement trend.
6. Example of best practice
The QA practice to fulfil the criterion is considered to be example of best practices in the field. Evidences support that it has been effectively implemented.
Performance of QA practice shows very good results and positive improvement trend.
7. Excellent (example of world-class or leading practice)
The QA practice to fulfil the criterion is considered to be excellent or is an example of world-class practices in the field. Evidences support that it has been innovatively implemented.
Performance of the QA practice shows excellent results and outstanding improvement trends.
How to write a response to the criterion requirements
- The response should describe a QA practice that the program do to fulfil the criterion. The QA may be a PDCA (plan-do-check-act) process to ensure the quality, and include documentation, evidence and result of the implementation.
- The response should illustrate clearly what the QA mechanisms or instruments are; when, where, by who, and how they are implemented and managed to fulfil the criteria.
- The response should identify and evaluate problems or issues that are not addressed, and indicate how the problems will be dealt with. The diagnostic questions in each of the criterion requirements may help identify the problems.
- To be concise and factual, the response should focus on information or data (or objective evidence) that directly addresses the criteria, and provide trends and statistics to show achievement and performance.
How the QA practice fulfil the criterion
Quality assurance practice should have QA plan that evaluates and may modify organization’s procedures to ensure that they provide the desired results. It includes documentation for the planning, implementation and assessment procedures for a project, process or any QA activity. To fulfil requirement 1.1 Outcomes formulation, for example, the QA plan should provide information to answer the following questions.
- What procedure does the program use to formulate the expected learning outcomes?
- Is a learning taxonomy used to write the statement of LOs? What taxonomy and how is it used?
- Is the taxonomy used for program or course learning outcomes, or both? Why?
- What are the objectives in terms of the quality of the learning outcomes (e.g. transparency, ease of assessment, the extent of alignment, enabling hierarchy, effectiveness, clarity to the stakeholders)?
- What is the process used to ensure that the quality objectives are met?
- What is a method used to measure the achievement of the quality objectives?
Operational Result
4.1 A variety of assessment methods are shown to be used and are shown to be constructively aligned to achieving the expected learning outcomes and the teaching and learning objectives.
The performances of students are assessed fairly, just and reasonably. Individual learner can learn in many ways and a variety of assessment method has to be explored to motivate them to learn, Each learner’s achievement of Airline Business program’s expected learning outcomes are continuously assessed from time to time during the study period. Accomplishments of course learning outcomes, which in turn aligned with the programs‟ expected learning outcomes, are assessed by individual courses stipulated in the TQF3 Section 4.2 (Append 4.1.1)
The programs also employ none-coursework assessments such as progress report (every semester), comprehensive examination both oral and written, mini-research, projects and etc. Overall summary of the assessment schemes employed throughout the program duration is presented earlier in Table Component 4.2.
4.2 The assessment and assessment-appeal policies are shown to be explicit, communicated to students, and applied consistently.
General student appeal and procedure follows the rules of St Teresa International University (TRSU) Faculty of Business Administration Airline Business Program as reflected in the Students Regulation Manual (Append 4.2.1) and /or course coordinators as explicitly stated in the course syllabus (Append 4.2.2) which is distributed during onsite classes or even in the online classroom (Append 4.2.3) as well as to the program head.
4.3 The assessment standards and procedures for student progression and degree completion, are shown to be explicit, communicated to students, and applied consistently.
The Airline Business program used Bell Curve Grading as a standardized assessment tools to give fair evaluation of every students. The MIS is the online platform designed by the institution to let each full-time lecturer key-in their individual assessment on time. The system will process and calculate the scores of each student and it will be presented to the Final Grade Evaluation Committee (Append 4.3.1).
Student assessments including timelines, methods, regulations, weight distribution, rubrics and grading criteria are explicitly described in the program specifications as well as in the course syllabus distributed to all students at the beginning of the semester/course see example in (Evidence: Course Syllabi). (Append 4.3.2) applying Taxonomy’s Higher Order Thinking Skills
After approval by the Grades Committee, it will be release and each student can check their grades online. However, before they can see their course grade, they have to evaluate first the subject and how each lecturer delivers the subject during their classroom studies.
4.4 The assessments methods are shown to include rubrics, marking schemes, timelines, and regulations, and these are shown to ensure validity, reliability, and fairness in assessment.
The Airline Business program student assessments including timelines, methods, regulations, weight distribution, rubrics and grading criteria are explicitly described in the program specifications (Append 4.4.1) as well as in the course syllabusdistributed to all students at the beginning of the semester/course see example in (Evidence example of CS) issue from the program‟ staff and online in the Management of Information System (MIS) at the Airline Business office.
The timeline of all assessments are included and presented in the course syllabus posted in the Google classroom on or before the start of the semester. The rubrics assessments were constructed considering the alignment with PLOs and these assessment criteria were developed in 2020 to be used in the Continuous Assessment, midterm and final examination, for students. Qualitative comments are additionally given by invited examiners and other teaching staff who are experts in that field for fair, valid and reliable assessment.
The AB program committee usually notes the assessment methods at the end of the academic year for enhancement. The marking schemes and rubrics assessment are discussed every year. Focus group discussions with current students are conducted to get their viewpoints on the assessment methods and scales in the beginning period of the second semester of each year. The online feedback system has also been developed for students to give their feedback on the course management and assessment systems to improve their further learning-teaching process and activities.
Written examination procedure.
- The AB Program committee review examination questions together and provide feedback
- Rubric assessment is announced to the lecturers and students
- The students take an examination by checking on their e-mails
- Lecturers check examination paper and record results in the MIS
- Lecturers grading together and reviewed by the Academic committee prior to its release
- Support staff announces the release of score to student during advising meeting
- Re-grade if
- Program committee review exam
First-year students know the results of the coursework summative examination within two weeks by e-mail or through agreed online platform. The teaching staff appoints students who do not pass the exam on discussing and give advice. The staff discusses the answers to the examination questions if needed. Any failed students may re-grade to improve their understanding of the subject as well as improve their performance. The students have to respond to the comments and submit them to their advisors in two weeks. Any student who fails to achieve the standard CGPA will delay his completion, therefore measures or steps has to be foreseen on time.
4.5 The assessment methods are shown to measure the achievement of the expected learning outcomes of the programme and its courses.
The St Teresa International University specifically the Airline Business program ensures that all assessments undertaken follow standardized methods to observe validity, reliability and fairness of student evaluation and to measure the achievement of expected learning outcomes (Append 4.5.1) like 100 203 Communication and Human Relations subject. For knowledge-based (core and elective) courses, assessments are typically in the form of written examination (midterm and final composes objective and subjective types of questions) with Test Blueprint (Evidence: TB) incorporating Taxonomy’s higher order thinking skills. The only exception is 216 497 Research and Trends in Airline Industry that the exam questions are evaluated by the rubrics method. Before grading of the essay examination, lecturers come up with the answer key and marking schemes. For evaluation of soft skills, assessment rubrics are used. Example of the guidelines and rubrics used for assessing student seminar is presented in Course Syllabus or TQF3 in the MIS.
4.6 Feedback of student assessment is shown to be provided in a timely manner.
The Management Information System is the college platform to showcase all the updated information online used by the school administrators, academic staff and non-teaching staff to retrieve information that has something to do with our students (Append 4.6.1)
Each student can log in to their own Portfolio (Append 4.6.2) and browse all the details regarding their studies and progress. They can log in anytime and find out how well they perform on their subjects per term. Several of our courses, especially those emphasizing soft skill competencies, provide immediate feedbacks to students for improvement.
The Google classroom is the learning and feedback avenue for each student/ For examples, in the 216 314 Airline Ticketing and Reservation courses, feedbacks on the strengths and weaknesses of students‟ demonstration are made in both verbal and written comments at the end of the class. or workshop. Similar approaches are implemented for certain courses that have group assignments, small group discussion or group report presentation by students. Progress report conducted every semester also allow assessment of students‟ progression on their duration of study.
Immediate feedback/comments on certain aspects of the teaching-learning process or technical problems are provided to the students.
4.7 The student assessment and its processes are shown to be continuously reviewed and improved to ensure their relevance to the needs of industry and alignment to the expected learning outcomes.
The Airline Business program has a system and mechanism to review student’s performance to cater the changing needs of the industry and alignment with the expected learning outcomes.
When completed, each course coordinator provided the course evaluation such as TQF5 in Section 5.1 and 5.2 by stakeholders including students, instructors, and employers in the practice settings. The assessment plan was a part of the course evaluation, including assessment tools, assessment methods, weight distribution, and alignment of assessments to CLOs. The contribution to PLOs would be reviewed by course coordinators and instructors. The course coordinators wrote the course specification report and plan for improvement and submitted it online. (Append 4.7.1)
The usual practice is that after each class, students fill out a score sheet and remark about the instructor’s teaching-learning methods. The lecturer also fills out a score sheet on each student’s participation and class interaction. The student and instructor then receive the scores and comments based on CLOs at the end of the semester. In each subject, there is one discussion session after every 5-6 classes, similar to recitation sessions at other schools. The sessions provide an opportunity to learners to ask queries to clarify technical contents or doubts, and for lecturers to receive feedback and recommendations for enhancement. Lecturers also note students’ strengths and weaknesses during this time and discuss their observations and potential approaches for student and instructor improvements at the weekly advising meetings. The learners are assessed on the coursework progress by quizzes and minor and major examinations and during the internship process by whether they have reached required milestones (e.g., passing internship provider expectations and written examinations, and passing their internship presentation.
To evaluate the continuous assessment such as quizzes, projects, reports, homework and written comprehensive examinations for A.B. students, the AB program committee have developed a rubric and made public announcements of the assessment criteria to the students. Student assessments are reviewed regularly based on students’ performance, stakeholders’ needs and PLOs. The improvement of student assessment process is summarized in Table 2. Feedbacks from stakeholders are obtained every 1-2 years while students give feedback about the course at the end of each term before they can check their course grade online on the MIS. All students’ feedbacks are responded by the teaching committee. The course and expected learning outcomes are considered to modify and improved accordingly, if appropriate.
Table 1 Improvement of student assessment process from year 2015-2022
|
No |
Year 2015-2019 |
Year 2019-2022 |
|
1 |
Coursework assessment without CLOs assessment forms. |
Coursework assessment with CLOs assessment forms. |
|
2 |
Continuous Assessment, Midterm and Final Examination, without rubric assessments. |
– Create and use rubric assessment for Continuous Assessment, Midterm and Final Examination in year 2019. – Create and rubric assessment for Preparation for Field Experience and Internship in year 2019. |
|
3 |
No Basic Research Methodology in Aviation Business and Preparation for Professional Experience |
Distribute manual for research checklist to students. |
|
5 |
3 hours of midterm examination and 3 hours for final examination |
2 hours of midterm examination 3 hours for final examination |
|
6 |
Teacher centered approach |
Student centered Approach with Internship |
|
7 |
No students’ submission of internship report to main advisor |
– Work in Progress (WIP) every month (all students can write online progress report and learn from other students’ condition): lesson learn from benchmarking process with other interns |
|
8 |
Use hard copy of assessment form. |
Use online assessment form. |
|
9 |
Program assessment without PLOs assessment form. |
Create and use evaluation of achievement of PLOs form. |
Evidence
Self-Assessment
| Requirements | Result | Score |
| 4.1 A variety of assessment methods are shown to be used and are shown to be constructively aligned to achieving the expected learning outcomes and the teaching and learning objectives. | / | 1 |
| 4.2 The assessment and assessment-appeal policies are shown to be explicit, communicated to students, and applied consistently. | / | 1 |
| 4.3 The assessment standards and procedures for student progression and degree completion, are shown to be explicit, communicated to students, and applied consistently. | / | 1 |
| 4.7 The student assessment and its processes are shown to be continuously reviewed and improved to ensure their relevance to the needs of industry and alignment to the expected learning outcomes. | / | |
| 4.4 The assessments methods are shown to include rubrics, marking schemes, timelines, and regulations, and these are shown to ensure validity, reliability, and fairness in assessment. | / | 1 |
| 4.5 The assessment methods are shown to measure the achievement of the expected learning outcomes of the programme and its courses. | / | |
| 4.6 Feedback of student assessment is shown to be provided in a timely manner. | / | 1 |
| Overall | 5 | |