Abstract
Let's dive to the topic of distance learning, which became a hit during 2020. All industries and academias pay now attention to this issue. We would like to focus on the most important and effective aspects of E-learning. The objective of this paper is to develop a practical E-learning framework with implementation of case method and project-based learning. In paper we obtain the following results. Content of the massive open online course (MOOC) “Safety and Security of Control Systems” is analyzed. This MOOC was introduced in 2017 for master students program “Cybersecurity” at National Aerospace University “KhAI” (Kharkiv, Ukraine). A core part of this MOOC is a project devoted to safety and security assessment of real systems and software. Taxonomy for indicators of E-learning effectiveness is proposed. Case study was done in Ukraine between students of Cybersecurity program after finish of study of the MOOC “Safety and Security of Control Systems”. A sample includes 40 master students involved in the course learning during 2017-2019. Case study results confirmed a set of hypotheses related with E-learning effectiveness when case method and project-based learning are implemented.
Introduction
Online learning also named as electronic learning (E-learning) means 100% virtual education via internet media with support of information technologies (IT). E-learning entails a huge transformation of education which becomes more and more distance as well as personalized. A modern approach to E-learning implementation is consists in development of massive open online courses (MOOCs). Despite numerous well known advantages and disadvantages of E-learning we put emphasis on the challenges and opportunities related with application of this relatively new education technology, such as: individual approaches to students with opportunities to build individual learning trajectories based in student-oriented approach; application of Learning Management System (LMS) as centralized environment for administration, documentation, tracking, reporting, and delivery of learning courses; needs in essential scope of relevant studies and data, that would provide a strong background for empirical based analysis; choice of relevant indicators to make qualitative and quantitative assessment of E-learning; high degree of importance of students’ homework which can be organized in both individual and collaborative manner; needs in choice of effective teaching methods depending on features of courses and sciences; in the paper we discuss the case method as well as the project-based learning applied for the course devoted to safety and security of control systems.
It is needed to notice concerning some gaps in researches devoted to the above points. The actual research is directed to fulfill some of these gaps. Based on the above, we develop a concept for our research area (Fig. 1). This concept recognizes dramatic importance of students’ homework for successful implementation of the whole E-learning process. After that, we take into account learning course defendant features and discuss in this paper the course “Safety and Security of Control Systems” which is provided since 2017 by the Department of Computer Systems, Networks and Cybersecurity of National Aerospace University “KhAI” (Kharkiv, Ukraine). Also, we discuss the additional values which are provided by application of the case method as well as project-based learning for E-learning.
Fig. 1. A Concept of the paper: an influence of a learning course, the case method and the project-oriented approach to homework during E-learning
Objective and tasks
We recognize some gaps in area of research related with case-based and project-based E-learning. These gaps are in the area of some lack of empirical data as well as a lack of theoretical basis for development of MOOC supported with PBL and the case method. The objective of this paper is to develop a practical E-learning framework with implementation of case method and PBL. To achieve the paper objective we perform the following research steps:
- Firstly, we analyze theoretical and practical content of the MOOC “Safety and Security of Control Systems”;
- Secondly, we propose taxonomy for indicators of E-learning effectiveness;
- After that, we do setting a field questioner based study of experience of students involved in E-learning of the MOOC “Safety and Security of Control Systems”;
- Finally we develop a practical E-learning framework with implementation of case method and PBL.
Analysis of the MOOC “Safety and Security of Control Systems”
The MOOC “Safety and Security of Control Systems” was introduced in 2017 for master students program “Cybersecurity” by the Department of Computer Systems, Networks and Cybersecurity at National Aerospace University “KhAI” (Kharkiv, Ukraine). The goal was initially to make the course as a problem solving, so it is aimed at solving a practical problem: preparing for safety assessment against the requirements of the standard IEC 61508 “Functional safety of electrical/ electronic/ programmable electronic safety-related systems” or other similar safety standards. Another purpose of this course was to make it fit for industrial training.
Thus, the course participants received the whole set of advantages of MOOC users, like learning with small information blocks, study at a convenient time, if a student did not understand something or missed, then he “scroll through” it again, etc. In the subject area, the benchmark was made primarily for Industrial Control Systems (ICS), but other control system architectures are also considered like embedded systems, and Internet of Things ( IoT). A look at security was directed from the point of view of safety assurance. However, where appropriate, a relation between safety and security is demonstrated, for example, a common life cycle of safety and security. It was considered which techniques of safety assurance can increase the level of security.
The course includes six lectures or, in MOOC terminology, six weeks of study. Primary, the lectures were posted on the YouTube channel in the form of the following playlists (see Fig. 2): Lecture 1. Introduction; Lecture 2. Requirements of safety standards; Lecture 3. Functional safety management; Lecture 4. Safety and security life cycle; Lecture 5. Quantitative assessment of safety; Lecture 6. Techniques and measures of safety assurance.
Fig. 2. Structure of the MOOC “Safety and Security of Control Systems”
Every lecture is supported with a set of 5-10 minutes videos in quantity from four to seven per one lecture as well as with lecture slides and homework items. Students’ homework includes the following items: video lectures and slides of lecture for independent learning of theory; answer to test questions (quiz); reading of recommended literature, including literature included in the curriculum specially for mandatory individual learning; study of additional materials exceeding the scope of the course program; the course project, which is the most important part of the course practicum. The course learning contains the following techniques: asynchronous interactive learning with video lectures during a primary study of theoretical material; self-paced independent learning with slides of lectures and recommended literature during a deep study of theoretical material; collaborative learning during implementation of the course project (Fig. 3); synchronous learning during consultancy leaded by a teacher.
Fig. 3. Structure of the Assurance Case template for performance of the course project
The course project performance is the main objective and result of the course learning. First of all, the project is necessary to solve an applied problem based on the acquired knowledge. The main task of the project is to develop a document covering the analysis and assessment of safety and security related with the lecture material. This document is called the Assurance Case in accordance with the actual practice of safety and security assessment. The development of the Assurance Case is now used in the practice of assessment and certification against safety and security requirements. The components of ICS like controllers, actuators and sensors are offered as subjects of assessment. Students are given the Assurance Case template, and they fill it out in stages, based on the material in each lecture. During the project implementation we assign project teams with two, three or four students depending from their preferences. The structure of the Assurance Case template associated with the lecture content for the course project performance is represented on Fig. 3.
Taxonomy of E-Learning Effectiveness Indicators
The proposed three-level taxonomy of E-learning effectiveness indicators is presented in Table 1. This taxonomy is a good starting point to analyze the known empirical researches, but also the taxonomy can be enriched by results of new experiments.
Table 1. Taxonomy of E-learning effectiveness indicators
Researches, which are analyzed above, deal mainly with the education performance indicators defined by teachers and students in the learning process. It should be noted that assessment of the learning process by students is quite subjective, since students do not fully possess objective information about the actual usefulness of the acquired skills and knowledge. The same applies to some extent to teachers. Complaints about the lag of teachers’ experience from life's realities are justified in many cases. Therefore, if teachers are working in the framework of the curriculum only, they can also not always fully appreciate the objective usefulness of the knowledge and skills taught. Therefore, in our opinion, it is necessary to take into account two more important categories of indicators. The first category is effectiveness indicators determined by graduates during some time after the end of the graduation, and the second category is effectiveness indicators determined by the employer. A challenge is to take valid and relevant data to estimate these kinds of indicators, but modern online technology can help to resolve this issue.
Case Study of E-learning Effectiveness
Our research setting is Ukrainian National Aerospace University “KhAI”. Computer Systems, Networks and Cybersecurity Department introduced a new education program “Cybersecurity” in 2017. During 2017-2019, four groups of master students have been graduated in accordance with the Cybersecurity program. One of the courses of this program, named “Safety and Security of Control Systems” was provided on completely E-learning basis as a MOOC with use YouTube for video files disposition as well as Google Drive for distribution of all other course materials. A non-random sample covers all general population of 40 students, which is presented in Table 2 with gender details. It should be noticed that in 2017 we taught two groups of students because the Cybersecurity program update and shifting of the considered MOOC to other semester. The groups of students were 8 to 12 people, so it was easy to distribute and to collect the study questionnaire since individual communications have been established between teacher and students.
Table 2. A sample of students for study
Study design is based on a set of the following hypotheses related with E-learning effectiveness (see Fig. 4): Hypothesis 1 (H1): Students are satisfied with E-learning process; Hypothesis 2 (H2): Students prefer E-learning format faster than traditional format; Hypothesis 3 (H3): Theoretical E-learning materials can be presented in a form which is understandable for students; Hypothesis 4 (H4): Practical E-learning exercises can be presented in a form which is understandable for students; Hypothesis 5 (H5): E-learning motivates students to get new knowledge.
Fig. 4. A structure of the hypotheses network
All students were asked to fulfill the questionnaire after the end of the course study. The questionnaire contains five parts related with five stated hypotheses. Fivedegree scale is used to estimate students’ agreement with the questionnaire statements. Also we asked students to indicate the positive issues of the course as well as the issues which can be improved. For this part we calculate quantity of students who are agree with one or the other statement. One student can choose more than one statement. Results of the study are presented in Table 3. We also include average exam grades in ECTS points to Table 3.
Table 3. Results of case study
Results of review confirmed the truth of the hypotheses H1-H5. An average satisfaction degree is 4.2, so H1 “Students are satisfied with E-learning process” is confirmed. An average confidence level for E-learning preference is 4.1, so H2 “Students prefer E-learning format faster than traditional format” is confirmed. An average understanding level of the lecture materials is 4.0, so H3 “Theoretical E-learning materials can be presented in a form which is understandable for students” is confirmed. An average understanding level of the project materials is 3.9, so H4 “Practical Elearning exercises can be presented in a form which is understandable for students” is confirmed. We notice, the degree for project materials understanding is the lowest because the project is really the most difficult and challenging part of the course. An average motivation degree is 4.3, so H5 “E-learning motivates students to get new knowledge” is confirmed. All ratings from students are in the range from 3.8 to 4.5 what mean a low level of the dispersion. An average exam degree is 77.7. Three groups of students got high level degree more than 80 on the average, and only the second group at 2017 got 64.3, what affect dramatically the degree for all four groups. The most important issues for students are opportunities to be free in a choice of a place and a time of study (80% of respondents). After that students emphasize the importance of a new view for security issues (62.5%), availability of all the course materials at one place (57.5%), and an opportunity to drive a real world project (47.5%). The main concern of students is a lack of teacher support (90% of respondents). Concerning this point it is worth to notice, that it is the main challenge which face students during E-learning, and also this is the main difference between E-learning and traditional “face-to-face” learning. We understand a lack of the selfdriven behavior is a big challenge during the project performance. It is also highlighted with other students’ issues such as the requests for more detailed steps for the project implementations (82.5%) and for more transparent expectations and acceptance criteria for the project (67.5%).
We continue to work for more transparent and understandable project descriptions. The degree of project material understandability increased from 3.8-3.9 in 2017 to 4.1 in 2019. The same, the number of students who need more explanations for the project steps decreased from 12 (100%) in 2017 to 5 (62.5%) in 2019, and the number of students who expect more transparency and acceptance criteria decreased from 10 (83.3%) in 2017 to 5 (33.3%) in 2019. It is also important for students to have the project subject in the field related with the students’ experience in software engineering (32.5%), including ICS and IoT.
E-learning Framework with Case Method and Project-Based Learning
An integral part of this research joints all outputs in a view of E-learning framework supported with case method and PBL. This framework is presented on Fig. 5 Below we briefly discuss the main parts of the framework.
Fig. 5. A structure of the E-learning framework supported with case method and PBL
PBL principles are related with approach to choose and drive the project as the core practical activity of the course. We adopt the traditional approach with the following principles:
- Projects are the central item of the course and the central teaching strategy, because students learn the course via the project;
- Projects involve students in a special kind of investigation directed to knowledge construction and transformation;
- Students play the main role in projects performance, and projects are realistic.
Principles of case method implementation are close related with PBL approach. The main idea is to propose for students a real software-hardware product and real documents covering safety and security assessment and assurance. Successful case method implementation suggests consideration of all best practices related with PBL and homework organization. Homework organization includes supplying of students with all materials and instructions. Principles of homework are the following: collection of online materials in the LMS area; combination, depending on the material being studied, of various forms of homework (such as classroom individual or group study, extracurricular individual or group study); student-centered approach with the formation of an individual trajectory of the education and homework depending on the individual preferences of students and the recommendations of teachers; focus aimed at the development of creative and research competencies; monitoring by teacher in order to analyze adoption of learning material by students; analysis of the opportunities to improve the homework process; taking into account curriculums of other interrelated academic courses.
MOOCs development and improvement bases on using of the modern design tools and LMS. Other MOOCs related issues lay in students feedback monitoring and continuous improvement. For E-learning effectiveness monitoring teachers should choose a set of appropriate effectiveness indicators. An example of taxonomy is presented in Table 1. Results of effectiveness monitoring are outputs of a case study organized to supply stakeholders with relevant empirical data. Periodical field review can be a method to implement a case study. Also other relevant case study can be included in data collection to support cases meta-analysis.
Conclusions
Twenty years after beginning of intensive implementation of E-learning we continue to follow technologies rather than theory like “the cart has been placed before the horse”. However, pedagogic innovations are such important like technological innovations especially, if a lack of students’ motivation is founded. In this paper we obtain the following results. Content of the MOOC “Safety and Security of Control Systems” is analyzed. This MOOC was introduced in 2017 for master students program “Cybersecurity” by the Department of Computer Systems, Networks and Cybersecurity at National Aerospace University “KhAI” (Kharkiv, Ukraine). A core part of this MOOC is a project devoted to safety and security assessment of real systems and software. Taxonomy for indicators of E-learning effectiveness is proposed. This taxonomy contain three the following levels: stakeholder who defines indicators including teachers, students, graduates and employers, group of indicators and single indicators.
Case study was done in Ukraine between students of Cybersecurity program after finish of study of the MOOC “Safety and Security of Control Systems”. A sample includes 40 master students involved in the course learning during 2017-2019. Case study confirmed a set of hypotheses related with E-learning effectiveness when case method and PBL are implemented. We found the truth of five hypotheses, H1-H5 (see Fig. 4). Also we checked the main issues which increase or decrease quality of E-learning. The students’ opinion is the most important advantage of E-learning is an opportunity to be free in a choice of a place and a time of study (80% of respondents). After that students emphasize the importance of a new view for security issues (62.5%), availability of all the course materials at one place (57.5%), and an opportunity to drive a real world project (47.5%). Students are concerned about a lack of teacher support (90%), needs for more detailed explanation concerning steps of the project implementations (82.5%) and more explanations for the lecture materials (70%), more transparent expectations and acceptance criteria for the project (67.5) and more the project subjects from software engineering area (32.2%).
Finally, we develop a practical E-learning framework with implementation of case method and PBL, which includes the following entities: principles of PBL implementation, principles of case method implementation, principles of homework organization, MOOCs development and improvement, E-learning effectiveness monitoring, and case study collection. A practical value of the paper is determined by development of a framework for effective E-learning implementation based on the principles of PBL and case method. A theoretical novelty of the paper lies in a technique of Elearning effectiveness analysis supported with new experimental data.
No comments:
Post a Comment