Pair Assessment with Coach and Expert Feedback
All Course Assessments
Overview of the Course
What is the subject context of the course?
Theoretical basics and fundamental concepts of Geographic Information Science (GIS) are implemented and further applied using ArcGIS Software. These concepts are introduced to the students with practical examples and real-world exercises related to their field of study.
What should students learn and be able to do at the end of the course?
Students should be able to:
- explain the theoretical and conceptual basics of geographic information systems (GIS);
- perform routine GIS tasks independently using a (commercial) GIS software;
- identify geographic problems and formulate possible workflows to solve them;
- identify and acquire geographical data from different sources and use it effectively to solve geographical problems
Why was the specific assessment format chosen?
During the COVID-19 pandemic, the final exam was replaced with an ungraded project taking place at the end of the semester. The project represents intensive work for both the students and for us. However, we still felt that the students did not fully benefit from our feedback, and when the project was over, there was little engagement in trying to understand what could have been done better – the students’ focus had already shifted to the next challenge (e.g. exams, other projects, next semester).
We realized that the feedback was not being applied and was therefore easily forgotten, which matched with the feedback from our students, who often told us that they would appreciate having further opportunities to apply the inputs given. That is why we decided to split the project into two parts, focusing on providing feedback after the first part and being available to discuss not only the feedback but also the application during the second part.
How are students prepared for the assessment?
We have a classic lecture with interactive interactions and a practical class where we offer hands-on training using GIS software to solve spatial problems. The exercises are pedagogically developed in different topics and take place in person. After the theoretical class on the topic, where the basics of the topic are introduced, the exercise is published, and the students have a time frame of two weeks to complete the exercises using the software. During this time, students can choose to be coached in practical classes, or to work independently at their own pace.
Thus, students are continuously prepared for the project throughout the semester by participating in the exercises and by solving eight mandatory assignments for each topic (Moodle quizzes with proposed solutions).
The course is designed to support the development of independence in software skills and to introduce new skills sequentially throughout the semester. The first part of the project evaluates the skills acquired up to the middle of the semester, and the second part deals with the remaining skills, with the possibility of reviewing the first part.
Although the two parts are independent, they accompany the class topics in a progressive manner.
Course Fact Sheet
Master Students
D-USYS: Environmental Sciences Bachelor, For the Specialization in Forest and Landscape, Eligible for credits
D-USYS: Environmental Sciences Bachelor, Environmental Planning, Eligible for credits
Competencies Fact Sheet
Assessment Fact Sheet
Support for the teaching/learning process (formative)
(Feasible for 10 - 75)
Submission
2 weeks to complete the conceptional and logical data modelling, data collection and follow the project guidelines and submit the project according to a predefined procedure
Project work, part 2:
3 weeks to complete the conceptual data analysis (without software)
The reason for the feasibility for 10 – 75 candidates:
- The maximum number of 75 students is based on our available resources. The limiting resources are computer availability and staff availability. With more staff and rooms for lectures and exercises, the assessment format can be implemented for larger cohorts of students.
Resources
Computer rooms or normal rooms, when students use their own devices
BYOD
Moodle + network folders to store the data from the exercises and the project
- to submit the 8 quizzes
- to submit the project work
Pre-structured Word document for project work which also contains reflection questions and pre-structured ArcGIS Pro project to facilitate correction.
Student teaching assistants
Grading and Feedback
Pair performance
Rule-based software
Student teaching assistant
Students receive three different types of feedback:
- During the project work, students receive coaching feedback in the practical sessions.
- After the first and second part, they receive expert feedback:
Student teaching assistants correct the first and the second part of the project work, using a scoring rubric. A GlS-specialist (administrative-technical employee with teaching function) supervises this work and forwards the written feedback via Moodle to the students. - In addition to the project work, each student must complete 8 Moodie quizzes between week 4-13 in order to pass the course. Students receive automated and personal feedback on the Moodie quizzes (depending on the questions). The Moodie quizzes ensure that every student engages with the content.
Staff Workload (66 Candidates)
| Time | Staff | |
|---|---|---|
| Student Coaching | 2 x 2 Lessons | 2 Lecturers |
| 6 x 2 Practice Lessons | 1 Lecturer and 2 - 3 Student Teaching Assistants | |
| Rubric Creation | 2 - 4 d | 2 Lecturers |
| Evaluation & Writing Feedback on Project Parts 1 and 2 | 132 h (1 h per Part) | 2 - 3 Student Teaching Assistants |
| Supervising TAs | 20 h | 1 Lecturer |
| Overview on Written Feedback | 0.5 - 1 d | 2 Lecturers |
| Writing Feedback | 2 h | 1 Lecturer |
| Providing Oral Feedback | 1 Lesson | 1 Lecturer |
| 4 h for Individual Feedback | 2 Lecturers |
Student Coaching
Rubric Creation
Evaluation & Writing Feedback on Project Parts 1 and 2
Supervising TAs
Overview on Written Feedback
Writing Feedback
Providing Oral Feedback
Extra Information
- Rubric Creation
The effort of 2-4 days is required if a template is already available and can be built upon. If a rubric has to be created from scratch, the effort is higher. - Providing Oral Feedback
The written feedback on part 1 and 2 is provided via Moodle:- For the detailed feedback on the first part, students have the opportunity to either ask questions during a practice session or even review the project and the correction together. Around 6 groups took advantage of this opportunity this year.
- The feedback for the second part is more general. Students who would like a more detailed explanation can always contact us.
Shared Experience
How many times has the assessment been conducted in this format?
The assessment of the project has been conducted for three years with good feedback from the students. Autumn 2024 was the first year that the project was split into two parts. The feedback was predominantly positive.
What contributed to the success?
The project provides an open topic, in which the students have to formulate their own geographical question to be answered. It gives students the freedom to pursue their own interests while using the knowledge they have acquired during the semester to look at the world from a geographical perspective. Besides practical formulation, the project guides them into a practical application of the basic concepts, providing the opportunity to explore real-world problems and common challenges such as data acquisition and preparation. Students are made aware of the effort and time involved in data acquisition, which is often underestimated. These insights, gained under supervision, are very valuable for later Bachelor’s and Master’s thesis and for their future endeavors.
The coaching throughout the project allows space to discuss ideas and to make sure there is continuous progress. Since the project is only conceptual, we can really evaluate if the basic concepts are understood and where more support is needed, without focusing too much on the technical skills which are evaluated in the exercise quizzes.
Although the overall outcome has been very positive so far, and the projects show very creative ideas on spatial issues. There are usually 1-2 projects per semester that do not meet the expectations. These groups have the chance to revise the project after feedback.
What were the challenges and how were they overcome?
We realized that our feedback at the end of the project did not have much impact on the students’ learning, either because their minds were already in the next challenge or because there was no actual application of the feedback given. By splitting the project into two parts and giving the first feedback after the first part and discussing it in class, the feedback is well received and often applied in the second part of the project, making the understanding of the concepts much more effective.
We knew from the beginning that we wanted the project to have an open topic, so that students could apply the content of the course to their own interests and have a higher engagement. However, this brought a number of challenges, as we needed to make sure that the assessment was fair and that the feedback involved still a manageable amount of work. The solution found was to create a pre-structured Word document that provides the structure for the project and also works as a guide for their process. Inevitably, this document took a lot of thought and effort to make it easy to correct, fair to evaluate and yet a guideline for a wide range of topics.
Are there any further developments planned?
Over time, we have noticed that the course has a lower priority because it is not graded. This does not necessarily mean that students are not interested in the course, because we do feel there is an engagement and even interest in pursuing further work (e.g. Bachelors’ and Masters’ theses) in Geographic Information Systems (GIS). And although GIS is currently an elective course, our previous students point out that GIS is one of the most used skills after graduation. However, between a graded course and an ungraded course, the ungraded course often gets less attention simply because the amount of effort invested past a certain point does not make a difference in their outcome in terms of grades. This is a paradigm that we have observed over the last few years, and it is the reason why we are planning to introduce grading of the project work. Additionally, because of changing the course into a graded version, we are currently discussing whether to replace part 2 of the project work with an exam in which students must apply their theoretical knowledge by interpreting practical exercises. This would ensure that the evaluation (grade) is fair for all students and that the learning objectives are achieved by each student individually.
We are also planning to make our content even more competence-based, and to that end we will be making adjustments to the lecture material in particular.
What tips would you give lecturers who are planning a similar assessment?
The project involves a lot of effort and given the resources available, the project must be carried out in groups of two. We recommend dividing the students into groups as soon as it is clear who will definitely be participating in the course. In the past, we have let students create their own groups, but this often leads to discussion. So just be clear from the beginning to give them space to organize their schedules.
Another tip we can give is to create conceptual projects if possible. This takes the focus off the outcome and puts it on the process. While it can sound strange that the result is not the deciding factor in the end, it adds a lot of value from the pedagogical side. Having a guided process organized in this way takes the stress off the software application of the workflow and ensures that the students think through their solution and organize their data structure in a logical way, which is very valuable for their future work.
Downloads & Links
The students liked the fact that they had to think through the given steps themselves and get feedback in the lecture.