An Evidence-Based Study on Teaching Computer Aided Design in Higher Education during the COVID-19 Pandemic

Abstract

The pandemic has had a major effect on engineering education, transforming both current and future teaching practice. The physical meetings between student and teacher have during the pandemic been replaced by online contact and recordings of lectures and demonstrations. In this paper, the focus is on computer aided design (CAD) teaching for first-year engineering students. CAD is a topic usually characterized by a close contact by student and teacher, with hands-on instruction at the computer using the CAD software. In the paper, the experiences and learnings from the rapid shift to on-line teaching in CAD are summarized and discussed, and learnings and takeaways for a redesign of future CAD teaching are discussed. Both the students’ learning and their mental wellbeing are evaluated. It is found that on a general level, the students were satisfied with the online teaching and rated it as better or equal to traditional teaching. However, there is still room for improvement, since some students found the situation stressful and pointed out the difficulty to ask questions online. The findings are based on a student survey, existing literature, and the authors own teaching practices during the pandemic.

1. Introduction

The effects of the global outbreak of COVID-19 on higher engineering education were extensive. The teaching needed to be completely transformed from classroom teaching to distance teaching with very short notice. In order to gain knowledge from this and to form a future learning environment, which most likely will be hybrid learning in a mix of online and physical classrooms, it is important to analyze the effects from online teaching on learning and mental wellbeing of students and teachers. In this paper, those aspects are analyzed based on the statistical analysis of a student survey in a course teaching basic computer aided design (CAD) and geometric dimensioning and tolerancing (GD&T) at a Swedish university.

The pandemic’s effect on the education system has been discussed by [1]. It is recognized that the shift towards online learning environment raises pedagogical challenges, but that it also is a possibility to adopt the education system to online learning, with increased availability and resilience as a result. In [2], it was shown that online teaching tends to reduce student engagement with respect to collaborative learning and taking part in discussions with other students. On the other hand, students from online courses were more prone to engage in quantitative reasoning. Warfvinge et al. [3] point out that Swedish students seem less satisfied and that they receive less feedback from the teachers during the online teaching. Gonzalez et al. [4] on the other hand showed in a Spanish study that the students adapted well to the new online learning and increased their study efficiency and that there was a positive effect from the online education on student learning. In [5], a large study of Indian and Turkish students showed no significant difference in learning outcome between online and offline teaching. In another large study [6] with data from 62 countries, it was reported that students experienced a higher workload during the pandemic, that they were bored, frustrated and worried about their future career. It was also shown that lack of computer skills was a problem and that European students showed grater satisfaction with their education during the pandemic than students from other parts of the world. The negative effects of the lockdown on students’ mental health were highlighted in [7]. Cox et al. [8] showed that second year students were more affected than the first year students. Some research, like [9] and to some extent [1], identified problems with internet access as a problem for online education in underdeveloped countries. In [10], the problems with technical tools and equipment for online teaching from a teacher perspective were discussed. As stated by [5], online learning is not just one problem but a collection of different obstacles.

It can be noted from the literature review that there is not a common view on the effect from online teaching on student learning. Obviously, the student’s learning experience is related to course topic, student background, and how the online course is organized. Focusing on CAD courses, those courses generally seem to be quite successful in an online setting. CAD courses often have a large share of computer demonstrations and computer exercises, and not so many theoretical lectures. With a good adaption to online teaching, CAD can be taught with greater success online than onsite [11,12]. Collaboration via Zoom or the equivalent is often seen as a good way of sharing knowledge [13]. It has been reported that videos used to demonstrate CAD functionality should be relatively short, preferably around 4–5 minutes, to best serve the students [14]. Quizzes are found to be a useful tool for CAD teaching, and in [15] it is concluded that online lectures with corresponding quizzes will also be used after the pandemic, while physical activities in the lab of course need to be done on campus.

Scope of the Paper

In this paper, experiences from online teaching in CAD and GD&T for first-year students in a mechanical engineering program in Sweden are discussed. Answers from a survey among the students are statistically analyzed to evaluate differences in learning and experienced stress. Those results, together with general experiences and insights from a teacher point of view, form the basis for a suggestion of future hybrid teaching practice for CAD and GD&T.

In Section 2, the materials and methods used as a basis for this paper are described. Section 3 focuses on the analysis of the online survey given to the students. In Section 4, the teachers’ observations and reflections are summarized. The lessons learned are discussed in Section 5 and conclusions are drawn in Section 6.

2. Materials and Methods

The observations in this paper are based on a course in CAD given to first-year students in a mechanical engineering program at Chalmers University of Technology in Sweden. The course contains 4.5 higher education credits and covers eight weeks. The main content is exercises in Catia V5, used for CAD. A smaller part of the course is also aimed to give an introduction to geometric dimensioning and tolerancing (GD&T). This part consists mainly of lectures and exercises done by hand on paper.

The course has been given with more or less the same structure in the CAD part for about ten years. Traditionally, the teachers have been supporting the students in computer halls, helping them with a number of CAD exercises. There have also been short demonstrations by the teachers. The number of students in the course is around 180, but the students are separated into groups of about 20 students in each group. The course is usually appreciated among the students, with average grades above four out of five in general student course surveys.

During the pandemic, Zoom has been used for communication with the students. The course material has been available via the learning management system Canvas, used by thousands of universities [16] worldwide.

2.1. Description of the CAD Part of the Course

The CAD part of the course consists of four hours of teaching per week. In addition, there are teachers available for two additional hours per week to answer questions if needed. During those two hours no new material is introduced. The four hours cover:

• Week 1: Basic interface, get started. Sketches in 2D.
• Week 2: Part workbench. Learn how to form 3D geometries.
• Week 3: Part workbench, continued. Dress-up functions, transformations, and functions. There is also an exercise about geometry analysis.
• Week 4: Assembly workbench. Ways to import parts, different constraints, methods to control and analyze assemblies.
• Week 5: Drafting workbench. How to do 2D drawings including scenes, views, dimensions, tolerances, etc., as well as assembly drawings.
• Week 6–7: Project work.
• Week 8: Exam.

In addition to this, there is a lecture in the beginning introducing the course content and its context, and a summary lecture, given week 5 to summarize the course content and introduce the project. The project is done in pairs of two students and covers all workbenches introduced during week 1–5.

The study material for the course covers detailed descriptions of different commands in Catia, including screen dumps from the software. There is also a number of exercises to encourage the students to try and practice the different functionalities in Catia. In addition to this, there is a separate practice material guiding the students through the different steps in designing a CAD model of a small toy helicopter, as shown in Figure 1. This task is voluntary, but done by most of the students, and usually quite appreciated.

During fall 2020 the course was held completely online, and a number of short videos were produced to support the students. The videos correspond to the short demonstrations previously held by the teachers in each computer hall. The teachers were available in a number of Zoom-rooms during the lecture times. Attendance was not mandatory. The attendance in the Zoom-rooms was lower compared to the attendance in the computer halls before the pandemic and the teachers got not as many questions in Zoom as they used to in the computer halls before the pandemic. One explanation, based on discussions with the students, was that the students were working off-line, sometimes in groups helping each other, and just dropped by the Zoom-rooms if they ran into problems. Before the pandemic, this common work among the students took place in the computer halls and the teachers could monitor the students’ work. It was also perceived as easier to just ask a teacher in the room about something than logging into Zoom for asking a quick question.

The CAD part of the course is examined based on the mandatory project work during study week 6–7 and an exam in the end of the course. The project work consists of a simplified version of a Stirling engine, where the students are supposed to do CAD models of fourteen parts, assemble those parts with some already provided parts and use the drafting workbench in Catia to design a drawing of one of the parts as well as an assembly drawing of the complete product. The project is approved if all parts and the drawings are correct and if the parts are correctly assembled into a functional product in Catia.

The CAD exam consists of three tasks covering part modelling, drafting and assembly in Catia. The exam is done individually in computer halls and the students have three hours to solve the given tasks. During the pandemic, the exam was done via Zoom and monitored by special personnel to make sure the students did not cooperate in solving the tasks. The maximum number of points in the exam is 30, and the grading is based on the number of achieved points. Less than 12 points means not passed, 12–17.5 points corresponds to grade 3, 18–23.5 points to grade 4, and 24–30 points to grade 5, which is the best possible grade in the Swedish system.

2.2. Description of the GD&T Part of the Course

The GD&T part of the course is of smaller scope than the CAD part and run during the first three weeks of the course. It consists in total of three lectures and three exercise classes.

During the pandemic, the lectures were pre-recorded and made available to the students. The videos consisted of PowerPoint slides together with a small picture-in-picture video of the teacher. There were also a few recorded interviews with industrial experts about different aspects of GD&T.

During the exercises, the students were assigned a number of different GD&T tasks and the teachers were available via Zoom for answering questions and discuss with the students. Just as in the CAD-part it was a relatively low number of students attending the Zoom-rooms all the time. The students seemed to perceive the Zoom meeting as a room they entered when they needed help. Otherwise, they were either working together physically or connected through Discord or other similar tools on their own preferences.

This part of the course was examined based on a smaller exam with a “go/no go” grading given in the fourth week of the course. The GD&T part of the course covers approximately 20% of the complete course content, while the remaining 80% is focused on the CAD part. To pass the course, the students must be approved on the GD&T exam, the CAD project, and the CAD exam. The grading in the course is based on the result on the CAD exam.

2.3. Online Survey

To explore the students’ perception of the situation with online teaching, an online survey was conducted after the course using Google forms. There were 54 answers whereof 3 were incomplete, leaving 51 answers for the analysis. The questions in the survey can be seen in

Table 1. List of questions in the survey.

(Q1a) CAD: How was your learning affected by online teaching compared to what you think it would have been like with traditional classroom teaching? (grade 1–5)

(Q1b) CAD: What were the problems with respect to learning and online teaching? (multiple choice)

(Q2a) CAD: How were your experienced stress and mental health affected by online teaching compared to what you think it would have been like with traditional classroom teaching? (grade 1–5)

(Q2b) CAD: What were the problems with respect to stress/mental health and online teaching? (multiple choice)

(Q3a) GD&T: How was your learning affected by online teaching compared to what you think it would have been like with traditional classroom teaching? (grade 1–5)

(Q3b) GD&T: What were the problems with respect to learning and online teaching? (multiple choice)

(Q4a) GD&T: How were your experienced stress and mental health affected by online teaching compared to what you think it would have been like with traditional classroom teaching? (grade 1–5)

(Q4b) GD&T: What were the problems with respect to stress/mental health and online teaching? (multiple choice)

(Q5) Have you used any CAD software before this course? (yes/no)

(Q6) Where have you most often been during your studies? (at home/at campus)

(Q7) Do you have any work life experience before? (No/1–6 months/more than 6 months)

. Question 1–4 consist of two strongly connected sub-questions (a and b), while questions 5 and 6 cover background parameters. The survey was completely anonymous and there were no incentives for the students to participate in the survey, other than contributing to improve the teaching and the course.

For questions (1a), (2a), (3a), and (4a) a Likert scale were used. The possible answers can be seen in

Table 2. Possible answers to choose from in questions (1a), (2a), (3a), and (4a).

1	Worse
2	Slightly worse
3	Comparable
4	Slightly better
5	Better

.

Questions (1b), (2b), (3b), and (4b) were multiple choice questions. The possible values can be seen in

Table 3. Multiple choice alternatives for questions (1b), (2b), (3b), and (4b).

(a) Not enough communication with the teachers

(b) Not enough possibilities to ask questions

(c) There were possibilities to ask questions, but I did hesitate since it was on-line

(d) Problems with the on-line tools (Zoom, Canvas)

(e) Problems with the Catia setup

(f) My working environment (ergonomics, privacy, etc.)

(g) Too few discussions and social interactions with the other students

(h) Difficult with self-discipline and self-motivation

. The multiple-choice alternatives were chosen to cover different aspects of the course and the students’ learning perception, such as course administration, technical aspects, and physical and psychosocial work environment.

It should be noted that for a correct comparison and analysis it would have been desirable that the students had taken the course both in a more traditional setting with classroom teaching and in the online setting. This is however obviously not possible, but based on their previous experiences of traditional teaching, they are assumed to be able to envisage the course in a traditional setting.

In addition to the questions in Table 1, there were also free-form text data, where the students could comment on their answers. Questions 5–7 were posed to allow for analysis of the relation between student background and their answers to questions 1–4.

The purpose of this study was to evaluate how the students experienced the online teaching compared to traditional teaching with respect to learning outcome and mental well-being.

3. Analysis of Survey Results

This section covers the analysis of the survey given to the students after the course.

The answers to the survey introduced in the previous section have been analyzed. The answers to questions 1a, 2a, 3a, and 4a can be seen in Figure 2.

If the answers are grouped into the sets NOK = {Much worse, Worse} and OK = {Comparable, Better, Much better} it can be seen that for the four questions, the share rating the experience as OK is 76%, 69%, 55%, and 67%, respectively. The Likert scale, shown in Table 2, produces ordinal data, since the distance between categories cannot be assumed to be equal, i.e., the difference between “comparable” and “better” cannot be assumed to be equal to the difference between “much worse” and “worse”. Therefore, non-parametric statistical procedures must be used for significance testing. A binomial exact test [17] was used to find out if there is significant evidence that the majority of the students found the experience to be comparable or better than traditional teaching, i.e., to test if the probability p that a random student belongs to the set OK is larger than 0.5. This was formulated as:

$$H_0:p\le 0.5$$

(1)

$$H_1:p>0.5$$

(2)

The p-values for the test for questions 1a to 4a are 0.01%, 0.54%, 29%, and 1.2%. Comparing this to a critical level of α = 5% indicates that the majority of the students find the online teaching to be comparable or better than what they would have expected the traditional teaching to be, for questions 1a, 2a and 4a. They are less positive when it comes to learning GD&T (question 3a).

Answers to the questions 1b, 2b, 3b, and 4b about problems related to the online studies, can be seen in Figure 3. The answers differ a bit for the four questions, but it can be noticed that the lack of discussions and social interactions with other students were seen as a problem in all the four questions, but it was especially affecting the stress and mental well-being for the CAD part of the course. Many of the students also pointed out that there existed possibilities to ask questions, but it was harder to ask questions online than they would have expected if in direct contact with the teacher in a traditional classroom setting.

It can be seen from Figure 3 that the students missed the social interactions and discussions. This seems to have been the main cause of stress during the CAD part. It was also one of the main obstacles for learning CAD, together with lack of study discipline and motivation and the fact that students had the possibility to ask questions but found it uncomfortable to do online. For the GD&T part of the course, the learning obstacles were study discipline/motivation and absence of discussions and social interactions. When it comes to stress in the GD&T part of the course, the most popular answer was “everything was good”.

For each question, there were possibilities to motivate or comment on the answers in a free-form text box. The comments confirmed to a large extent the conclusions in Figure 3. On the positive side, they liked the video clips, the weekly newsletter, and the possibility to plan their own day and do the exercises in their own pace. On the negative side, they mentioned that it was more difficult to focus when working from home from the same computer they used for gaming and entertainment, it was difficult to know if you kept the right pace without seeing what the other students were doing, and it was embarrassing to ask questions in front of the other students in the Zoom sessions.

How a student experiences the online teaching in CAD can of course be affected by the student’s background. To investigate this, the background questions 5–7 were used.

For questions 1–4, the responses have been divided based on the answers to questions 5–7 and the results can be seen in Figure 4, Figure 5, Figure 6 and Figure 7. No obvious trends can be seen from the studying the distribution of the different groups in those graphs. The students with work life experience exceeding six months might be slightly happier with the online studies than their fellow students. However, when a multinomial regression for ordinal responses were set up to see if the background variables 5–7 had a significant impact on the answers to questions 1–4, no significant impact could be shown.

4. General Findings

From the teachers’ perspective, the transition to online teaching was demanding. Special efforts were made to produce a number of video clips to describe different CAD software functionalities. There was also at least one video clip for each CAD exercise. There were also videos about installing Catia and some more traditional lectures for introduction and summaries.

The preparations for using the online tools could perhaps have been more extensive. For example, during the first lecture, the students were divided into four different Zoom rooms. It turned out that as the Zoom rooms were set up, the first room closed when the third one opened due to some system limitations. These kinds of unpleasant surprises created extra stress for both students and teachers.

During the CAD exercises, there were several Zoom rooms open. The students were in most cases only present when they needed to ask questions. From a teacher perspective, this made it more difficult to get an overview of the progress of the class, compared to gathering all students in computer halls together with the teachers. Some students seemed to prefer to ask their questions in front of the other students in the Zoom room, and some wrote in the chat asking for help in a private session in a break-out room. During the course, the teachers believed that this was an acceptable set-up, but based on the comments in the survey some students found it difficult to ask questions in front of the other students and probably also embarrassing to ask for a private session. Maybe this is especially difficult for first-year students, as in this course.

After some confusion and discussions during the first week of the course, a weekly newsletter was sent out in the beginning of each week during the remaining part of course. This newsletter contained more or less the same information as already stated in the course description, but for just one week at the time. There were also some reminders of what the students needed to do that special week. This was appreciated among the students and seemed to give them some stability and a feeling of security.

The course contains a project, to be done in groups of two students in the second half of the course as described in Section 2. This was thought to benefit discussions between students, but as seen in Figure 2 the low rate of social interactions and discussions with other students was still seen as a problem.

The CAD exam was also held online using Zoom. It was monitored via camera by special personnel and the students could ask questions to the examiner in a special break out room. The results for the exam are shown in Figure 8. The results are quite satisfying with a large portion of approved students. For the approved students, grades from 3 to 5 are given where 5 is the best possible grade. It should be noted that this is a basic course in CAD without any requirement of previous knowledge. A large portion of the students have however had some kind of CAD education during high school. Therefore, the large share of high grades (5) is not very surprising. From a teacher point of view, it was of course easier to answer questions during the exam via the computer compared to visiting a large number of computer rooms.

5. Lessons Learned

In this section, some of the lessons learned from giving a CAD course completely online during a pandemic are summarized. Based on experiences, the survey, and discussions among teachers and with the students, the following lessons learned can be concluded:

• Be predictable: The students do not cope well with uncertainties, so do not change practice from week to week. A weekly newsletter/instruction worked well.
• Be extremely clear in all communication, leave no room for personal interpretations. All teachers must convey the same message. In the online setting, there is a lack of interaction between students hindering information transfer between them, which probably can explain this need.
• Be quick: Changes must be communicated without delays to avoid stress.
• Make sure the online tools are working smoothly. This is quite obvious, but some flaws are only noticed with a large number of users and might therefore be difficult to identify with single user tests. If possible, stress-test the course setup in advance.
• Preparations and tests are vital. From a teacher perspective, it is much more difficult to improvise when you don’t have a face-to-face contact with the students.
• In video clips, some small mistakes can be intentionally included. For example, to click on the wrong button and see its effect, and then explain why it was not the best method and how we should do it instead can be a good learning experience for the students.
• Find activities to encourage discussions among the students early on in the course. A solution can be to form teams of students (on a voluntary basis) in the beginning of the course. Each team can have their own chat and get some exercises to solve. Another solution can be to encourage the use of the course chat room and discussion board to a larger extent. In an online setting, it is even more important that the teacher is “nice” to the students to bridge the distance to the students [18], create a friendly environment, encouraging discussions and two-way communication.

A return to a situation with all students in classrooms at campus all the time seems unlikely. Instead, the course studied in this paper, and probably many other courses in higher education, should evolve towards a hybrid learning experience [19] where the students can choose if they want to spend time in classrooms at campus or if they want to take part in online teaching. Hybrid and/or blended learning will go from being a nice extra feature in a course to what is expected from a standard course. Based on the learnings from the survey described in previous sections and experiences from teaching during the pandemic, the following actions are suggested to transform a higher education course like the one described in this paper. The lessons learned in the list above are also implemented. The assumed concept is a hybrid course, with a mix of online and onsite participating and teaching. Some conclusions are specific for CAD or GD&T courses, marked with CAD/GD&T in the list below.

• A good atmosphere, encouraging students to ask questions also online must be created. Tools for asking anonymous general questions can be used and a good system for online queues to private question sessions should be established. In Zoom, private questions can be sent to the teacher who can repeat them in class. In such a way, the anonymity of the students is retained. If there are a lot of questions, Google forms or similar can be used for creating a short specification of the question in a waiting list.
• Technical setup. With hybrid teaching, it is very important to make sure the students participating online can see and hear the teacher just as good as the students in the lecture hall can.
• The teachers and examiner need to get an understanding of how well the students follow. Short anonymous quizzes can be a good tool for checking this. This works both online and onsite and is a way of treating a class as a whole, independently if they follow the teaching online or onsite.
• CAD/GD&T: The concept with short videos explaining functionalities in the CAD software should be kept. This is a good basis for freedom of choice and flexibility in higher education. It also creates a more resilient teaching practice, enabling an easy way for students that missed class to catch up.

In [20], an overview of different approaches to increasing student’s creativity in engineering education is presented. Projects and other student-centered activities are ways to foster creativity.

In Sweden, the students are usually equipped with laptops, PCs and smart phones, and internet access/bandwidth is usually not a problem. Those issues have therefore not been addressed in this study. They can however be vital in other settings.

6. Conclusions

This paper has used a CAD and GD&T course as a case study to analyze pros and cons of online teaching in higher education with respect to students’ learning and mental well-being. This situation arose during the COVID-19 pandemic, but general lessons learned can be drawn with bearing on future hybrid teaching in higher education. The importance of an atmosphere where students dare to ask questions, the use of short videos, and predictability in the teaching and course setup to avoid stressful situations are highlighted. From an online survey among the students in the course, it was concluded that the students were generally satisfied with online teaching, but a lack of social interactions and discussions made them feel more stressed which affected their mental well-being. Previous experience of CAD, previous work life experience, or if the students worked from home or from campus did not affect their learning experience or stress significantly.

For future work, it would be interesting to re-run the survey when the main part of the course is given on-site again and compare the results.