CC BY
0
doi https://doi.org/10.5937/vojtehg73-54803
Application of different hardware and software in the concept of digital pilot «
training and selection
Aleksandar Z. Knezevica, Aleksandar G. Bukvicb, Branimir B. Krsticc ju
<u
University of Defense in Belgrade, Military Academy, Belgrade, Republic of Serbia
a Department of Air Force, ^
e-mail: [email protected], corresponding author, ORCID iD: https://orcid.org/0000-0001-6786-4530 b Department of Military Mechanical Engineering, o
e-mail: [email protected], corresponding author, _
ORCID iD: https://orcid.org/0000-0002-3025-8446 B
g
a Department of Military Mechanical Engineering, u
e-mail: [email protected], ORCID iD: https://orcid.org/0000-0002-9612-6104
p
p
e c n o c
FIELD: mechanical engineering ARTICLE TYPE: original scientific paper
Abstract: 2
Introduction/purpose: In many military air forces and civil aviation organizations that use or own training aircraft equipped with a digital cockpit, training tools such as trainers and simulators are understood to a varying extent. Special attention is paid to the ratio of hours spent training on the real aircraft to the number of hours spent on the simulator. This approach in training professional circles and the public is called Live virtual constructive. For all other teaching aids used for preparation for training, neither the methodology nor the extent of their use has been specifically investigated. The research aims to develop instruments that will measure erff and evaluate the success and qualitative progress in pilot training achieved t5 by using the digital cabin space, based on which a decision can be made about further training for a specific aircraft type.
Methods: In conjunction with a PC-based aviation training device and advanced statistical analyses, an eye movement tracking device will help arrive at a reference pattern of instrument observation to train pilots and select pilot candidates.
a
ACKNOWLEDGMENT: The authors are grateful for the financial support through three ,0
projects from the Ministry of Defense of the Republic of Serbia (project name: VA-TT/5/13- >
15 „Application of digital cabin space in pilot training" and VA-TT-2/19-21 „Investigation of >n perceptual and motor characteristics of pilot cadets in instrument flying on piston and jet aircraft", VA-TT/2/24-26 "Application of concept of digital training and pilot's selection").
<D tn
m
PO
"(5 >
LO <N
0 <N
01 yy
0£ ZD
o o
-J
<
o
X
o
LU
I—
>-
CC <
I—
< -J
CD >o
X LU I—
o
o >
Results: The proposed concept's research includes detailed descriptions of the application of trainers and teaching aids with different profiles, which significantly impact the quality of the implementation of the digital cabin space in the training of cadet pilots.
Conclusion: Given that the Military Academy educates cadet pilots who continue their training in the operational flying units of the Serbian Armed Forces (SAF), it is essential that during the period spent at the Military Academy, their quality theoretical preparation be carried out.
Keywords: flight training, simulation, hardware, software, eye tracking.
Introduction
Flight simulation systems come in various forms and provide varying assistance in the training of candidate pilots and trained pilots. Flight simulators and trainers which are known today can be classified into the following groups:
- Full Flight Simulator,
- Flight Training Device,
- Flight and Navigation Procedures Trainer, and
- Basic Instrument Training Device.
The flight simulation and training device can be any of the above simulator/trainer types. All types of simulators and trainers have a common name: Flight Simulation Training Device.
For each of the mentioned types of devices for flight simulation and training, there is a more detailed division according to the EASA (European Aviation Safety Agency), which goes along with the corresponding characteristics that each device must meet (EASA, 2018). The breakdown is given in Table 1.
At the end of the 20th century, a device based on the PCATD (PCATD- Personal Computer-Based Aviation Training Device) became available to pilot candidates and their instructors for training on the ground. These devices consist of a display console, a PC monitor that houses the instruments of the aircraft being flown and a controller used for control.
At the beginning of the 21st century, there was an increased emphasis on using PCATD personal computers in flight training. Today's computers can realistically represent high-quality situations and instruments on board an airplane in life-size. Current technology has also succeeded in providing aerodynamic models and near-realistic flight controllers used by the pilot in flight and having the same high accuracy
as the devices intended for training a particular type of aircraft. The U.S. Federal Aviation Administration (FAA) has approved using these devices in training as the equivalent of real flight time, but only for a specific type of flight and a certain number of hours defined by the FAA (FAA, 1997).
CD O CNI CO
oo !±
Table 1 - Distribution of types of flight simulation and training devices according to the o
EASA -f
(Flight Navigation and Procedures Trainer - FNTP)
ro
O)
ü
CT T3
<u o c o o
(D
ro
EASA FNTP Level I
EASA FNTP Level
EASA FNTP Level
MCC (multi crew cooperation). Not a true "level" of qualification, but an addon that allows any level of FNTp to be used for multi crew cooperation training._
Flight Training Devices (FTD)
EASA FTD Level 1
EASA FTD Level 2
EASA FTD Level 3 (Helicopter only)
Full Flight Simulators (FFS)
EASA FFS Level A
EASA FFS Level B
EASA FFS Level C
EASA FFS Level D
Personal computers must have special software for simulating flights, and the most suitable for use on the market is "Microsoft Flight Simulator," as an amateur and gaming software for flight simulators. Pilots can practice procedures for the types of aircraft they are trained in. The PCATD provides capabilities and functions for practicing instrument and instrument flying tasks. Deviations are observed in exercises for flight in instrument flight conditions (IFR) which cannot be performed as well as in other simulators or, of course, in real aircraft. o
This paper concerns the concept of digital training and the selection of cadet pilots at the Military Academy. The technical solution implemented "§ has introduced and applied the concept of pilot training for digital cabin space, which, in addition to trainers and simulators, envisages other methods and synthetic means of different profiles.
The optimal pilot training profile was created according to the needs of the Air Force and Air Defense. This model can be modified and adapted to the needs of civil aviation organizations in training pilots who use aircraft equipped with digital cockpits.
<D tn
CO
"o
>
LO CM o CM
of
yy
0£ ZD
o o
-J
<
o
X
o
LU
I—
>-
Q1 <
Î0 <
-J
CD >o
X LU I—
o
o >
&
The Air Force Command has issued a certificate of use of the Trainer for the aircraft of the Serbian Armed Forces.
Research methods have been developed to evaluate the success and progress of cadet pilot training achieved by applying digital cabin space. On this basis, a decision can be made on the further education of each candidate for a particular aircraft type. This technical solution was developed in two projects funded by the Ministry of Defense, namely:
1. VA-TT/5/13-15 "Application of Digital Cabin Space in Pilot Training" Decision of the Senate of the University of Defense, No. 9/11 of 15 November 2012.
2. VA-TT-2/19-21 "Research of Perceptual and Motor Characteristics of Cadet Pilots in Instrument Flying on Piston and Jet Airplanes" Decision of the Senate of the University of Defense, No. 3/82 of 5 December 2018.
Methodology Materials
Interactive learning materials
To master theoretical knowledge in aviation, cadet pilots use CAE Oxford Aviation Academy interactive materials in compliance with the EASA. The materials are interactive books and multimedia (CBT Computer Based Training). Each lesson is accompanied by clear and precise narrations that guide the cadet-pilot through the material. During the many years of implementation of CBT, cadet pilots have shown better results in preparations for practical flights on the Lasta aircraft. The Lasta training plane features a cockpit built by the American company Garmin. At the Military Academy, Garmin software and a hardware simulator for the Garmin 500/600 are used, Figure 1. Cadet pilots learn to navigate Garmin systems using Flight Plan pages and GPS direct-to-head courses. Generations who have used these training tools have shown better results in the initial stages of basic flight training.
Figure 1 - Garmin 500, the instrument found in the Lasta aircraft (left), and a desktop trainer with Garmin software used to train cadet pilots
PCATD (Flight Simulator based on commercial software)
Simulators imitate parts of the aircraft and components with which the pilot has direct contact: the cockpit with the control units, instruments, the environment in which the use is carried out, and more. The use of simulators achieves significant savings because it avoids the consumption of aircraft resources, fuel, and ordnance (if it is combat training), does not pollute the environment, reduces the risk of any injuries or accidents, and it is possible to practice emergency procedures that are difficult or risky to carry out in reality. Simulator training cannot fully recreate all the effects and conditions that occur in reality, and that must be practiced on actual aircraft.
The Lasta aircraft trainer is a tool used to train cadet pilots in visual and instrumental flying using commercially available hardware and software developed at the Military Academy within the project "Application of Digital Cabin Space in Pilot Training." The original software add-on (Figure 2), which is the result of the work of the research team of the Military Academy, has the following characteristics:
1. High precision in geometry,
2. A large number of polygons (63813), and
3. High-resolution external textures (4K 3480 x 1280 pixels).
The virtual cabin space has the following characteristics:
1. High precision in geometry,
2. A large number of polygons (49490),
3. High-resolution textures (4K 3480 x 1280),
4. Fully "clickable",
5. Equipped with the appropriate avionics, and
6. Brightness and intensity in line with reality.
CD O C\l CO 00
.p p
c el le
a g
ol ip
al ti
igi di
p
e c n o c
er ar
d n a
<D
ar
w
dr ar
e
ffi
di f
o
a
ci lp
p
<
t e
<
■o
iv e
>N
e
Figure 2 - Software add-on for the Lasta aircraft in Flight Simulator 10
e u s
CO
"o
V
5,
2
O
2 ,R
EI RI
ZD
O C L A C
X
C E T
>-
R A
S A L
CD >Q
X
E T O
O
>
For the Flight Simulator 10 software programming environment, a simulator containing a simulation model of the Lasta aircraft has been created. This simulator is classified as a "Personal Computer-Based Aviation Training Device (PCATD)". It is based on the computer platform for flight simulation "Microsoft Flight Simulator X," with a field of view of 90 degrees (Figure 3) and a field of view of 270 degrees (Figure 4). This configuration allows for the exact visualization as in a real aircraft. All the characteristics of the flight dynamics of the aircraft are defined through the "XML" code that is implemented on the computer platform.
Figure 3 - PCA TD trainer for the Lasta aircraft with a 90° field of view
The flight controls used on this trainer are identical in shape, color and function to the controls in a real aircraft (Figure 5).
Figure 5 - Controls in the cockpit of a Lasta aircraft (left) and in the physical setup of the
PCATD trainer (right)
This way, adapting to the cabin space and creating a mental image of the working environment is faster, easier, and cheaper. Adding the Martin Baker seat, built into the flight simulator, creates an additional positive impression of the working space in the aircraft (Figure 3).
The control surfaces include a joystick, a rudder control, and a throttle control with a USB input for a computer. These components are intended to control the aircraft around the longitudinal, lateral, and vertical axes, as well as the engine control (power and pitch), on which additional switches are used to control the landing gear and flaps. Before the flight, meteorological conditions can be defined, as well as failures and the time of occurrence of failures during the flight. Speed, altitude, course, time, and positions on the map can be read at the end of the flight.
A trainer of this type is not of the highest technological complexity. Still, it provides training in instrument flying procedures, the use of devices and instruments, and regular and emergency procedures.
The flight simulator with software add-on for the Lasta aircraft consists of three high-resolution (HD) projectors (Figure 6 position 10) which project their image onto three projection screens (Figure 6 position 11), and one high-resolution (HD) monitor. The projectors are positioned at an angle to capture a field of view of 90° x 60° and thus give an overall image of 3480 x 1280 pixels. The software add-on allows them to spherically rotate the image they display on the canvases with the help of an IR sensor to track head movements.
CD O C\l CO 00
!±
O <D
CO O)
'EL
T3
<U o c o o
<D
ro
T3 c ro
<D
ro
"a ro
<u 4=
T3
M—
O
CO
O <
To
(D <
■o <u
>N <U
e u s
CO
"o
V
5,
2
O
2 ,R
EI RI
ZD
O C L A C
X
C E T
>-
R A
S A L
CD >Q
X
E T O
O
>
Figure 6 - Arrangement of the projection surfaces and the projectors on the PCATD
simulator
A view of the cabin area is given on a single touchscreen which presents the pilot with only the instrument panel. The left and right consoles with switches found in a real aircraft are absent.
With the modernization of the Lasta aircraft trainer, the field of view has been expanded. It is provided by three 4K resolution projectors (short throw projectors) for projecting images from a short distance and a cylindrical screen measuring 270° x 60° (Figure 7).
Figure 7 - Modernized PCA TD trainer for the Lasta aircraft and the position of the short-throw projectors (rounded positions)
This improvement has eliminated the use of IR sensors which existed in the previous variant of the trainer, and increased the angle to which it is possible to see the aircraft surroundings up to 270°. This is important for the training of fighter pilots, where the dynamics of the movement of the gaze is far more intense and where it is insisted that the relations between the projection of the wings and the orientation on the ground (visual orientation) are compared. This is due to the possibility of a 270° field of view (Figure 8). The Lasta aircraft trainer exceeds the criteria defined for the width of the field of view on the category D FFS simulator (180°x60°) and the category 3 FTD trainer (150°x60°) as defined by the EASA (European Union Aviation Safety Agency (EASA, 2018).
Figure 8 - Extension of the field of view to the PCATD trainer for the Lasta aircraft
An eye-tracking device was demonstrated as part of this trainer (Figure 9, position 6). The schedule of attention distribution (shifting view from instrument to instrument) in the glass cockpit configuration was recorded and compared with the established observation scheme of analog instruments. The experiment was conducted with cadet pilots as well as experienced pilots.
CD O C\l CO 00
!± Ci
O <D
<0 O)
Ei
m
T3
Ci
<u o c o o
<D
ro
T3 c ro
<D
ro
"a ro
<u ¡fc T3
M—
o
ro o
Ci <
Tô
(D <
■o >
<u
>N
<u
Figure 9 - Eye-tracking device
e u s
CO
"o
>
lo
2
0
2
01 yy
0£ ZD
o o
L A
o
X
o
LU
I—
>-
CC A
The concept of digital training and the selection of pilots for training purposes has been extended to the needs of training helicopter pilots. A version of the PCATD trainer has been created, which has the width of the field of view of the FTD trainer of the third category (150°x60°), a projection screen in the form of a spherical section, and a pilot seat on which the pilot controls for the helicopter are installed (the so-called cyclic and collective stick) (Figure 10).
iO A L
CD >o
X LU I—
o
o >
&
Figure 10 - PCATD trainer for the Gazelle helicopter with analog instruments and with a projection surface in the form of a spherical section measuring 150° x 60°
The characteristic cockpit view of the helicopter in Figure 10 satisfies the required projection width in contrast to the aircraft training needs. Analog instruments found on older types of helicopters are especially characteristic.
The simulator allows the flight to be recorded and later analyzed by incorporating the FS recorder plug-in, an add-on to the basic software. Flight parameter data and footage can then be reproduced as a separate flight or within a next-flight scenario. This option is used when analyzing flights with cadet pilots, although it is not part of the standard simulation software.
The main advantage of this solution is the multifunctionality of the trainers' applications depending on the need, whether it is the training of cadet pilots, fighter pilots or psychological and medical selection from the aspect of checking future pilots.
Virtual reality (VR) devices
In 2021, the EASA announced that it had approved a virtual reality training solution for a helicopter (EASA, 2021), which is a big step for this method of training.
The VR environment is simple and easy to implement on a PCATD. High-reliability sensors in VR devices play a significant role in their success. Virtual interaction itself increases the interest of cadet pilots. The Military Academy uses Oculus Rift VR device glasses. Implementing the existing software add-on for the Lasta aircraft, which is used in the VR environment, includes the software that bridges the connection between the Oculus Rift VR device and the Microsoft Flight Simulator X.
During the training at the Military Academy, adverse effects were observed when using VR glasses, which were not mentioned in previous research and impacted the training itself. Sponges on VR glasses, which make the device comfortable, when used at high temperatures in the room, interfere with the user and affect the result of training and the execution of tasks because the glasses fog up from sweat. An additional problem arises if users wear glasses because VR glasses cannot be adapted to different diopters. In the paper entitled "A Model for the Adoption of Virtual Reality Technology for Dynamic Learning (Fussell & Troung, 2022) some interesting facts were presented about the difficulties of using VR glasses. Figure 11 shows the use of the PCATD with the VR environment, which is used at the Military Academy for the initial training of cadets.
Figure 11 - PCATD with the VR environment, the Lasta aircraft accessory
Other hardware and software used in pilot training
Sky Demon is one of the most well-known flight planning and tracking software. This software is also known as Electronic Flight Bag Software. It allows you to plan your flight as well as GPS navigation during the flight. Its unique options allow you to update aeronautical data in real time. Within its unique features, Sky Demon also has an internal simulator called X-plane. When the simulation mode of this software is started, it can be networked with the Microsoft Flight Simulator X software with the help of a
CD O C\l CO 00
!±
O <D
<0 O)
'EL
T3
<U o c o o
<D
ro
T3 c ro
<D
ro
"a ro
<u 4=
T3
M—
O
<0
O <
To
(D <
■o <u
>N <U
<D tn
CO
"o
>
LO CM o CM
of
yy
0£ ZD
o o
-J
<
o
X
o
LU
I—
>-
Q1 <
Î0 <
-J
CD >o
X LU I—
o
o >
0
smaller software called FSX2Sky Demon located on the Google Play platform. The software takes data from the simulation currently running on the computer and plots the flight path on Google Maps (Figure 12). At the same time, it transmits speed and course data to Sky Demon. It receives feedback from Sky Demon in the form of winds that are obtained from weather stations in real time and has an impact on the flight of the model in the simulation once recorded and saved in the Google Map extension, .gpx. It can be subsequently analyzed in 3D and 2D mode (Figure 13).
Figure 12 - Using the SKY Demon software (bottom left part of the image) with the addition of the FSX2 Sky Demon app (upper left part of the image)
Figure 13 - Recorded trajectory from a simulated flight in the SKY Demon app and the FSX add-on for the Lasta aircraft. The trajectory is recorded in the Google Map
extension .gpx
Methods
Analysis of the flight parameter data and the eye movement recording obtained from the simulator
This chapter presents an experiment conducted with cadet pilots of the Military Academy in flights in simulators with different visual environments or fidelity. The task is to spot any differences in the flight performance and presentation data in the best way. This and similar studies have been carried out using free software for these purposes. One group of cadets practiced taking off and landing with the Lasta aircraft on the school circuit in a conventional simulated environment that involved projecting the environment of the aircraft on the projection screens and the aircraft cabin on a monitor in front of the eyes of the examinees. This group of cadets served as a control group in the trial. The second group of cadets practiced taking off and landing with the Lasta aircraft on a simulator with a virtual cabin space and the aircraft surroundings. This group served as an experimental group. The first and last control flights were flown in a conventional simulated environment to examine the effect of changing the simulated environment on the flight performance and instrument observation strategies. The experiment was designed to determine whether there is a transfer of skills from one simulation environment to another.
For the experiment in which the flight parameters were compared (project number VA-TT/2/24-26), the PC flight simulator used in earlier research (projects numbered VA-TT/5/13-15 and VA-TT-2/19-21) by researchers from the Military Academy and described in the study (Vlacic et al, 2020) has been specially upgraded with Simpit Technologies' Centurion 270 PRO cylindrical 270° vision system to increase its fidelity in the field of view. The experiment used an add-on for the Lasta aircraft, a military pilot training aircraft, which falls into the category of technologically advanced aircraft due to its integrated electronic instruments (EFIS).
A new hardware configuration of the computer commercially available during the experiment allowed for a better multi-frame operation and a curved projection of an external scenario with a resolution of 3080 x 1024 pixels. Figure 14 shows the trainer with the settings from the experiment and a device that records eye movements in the background.
e u s
CO
"o
>
lo
2
0
2
01 yy
0£ ZD
o o
-J
A
o
X
0
LU
I—
>-
01 A
iO A
-J
CD >Q
X LU I—
o
Figure 14 - Lasta aircraft trainer with 270° field of view and the eye tracking device
(rounded position)
The Microsoft FSX software add-on for the Lasta aircraft has been enhanced with additional software called FlyInside FSX by FlyInside Inc. 2018. This made it possible to set up a model of the Last aircraft in virtual reality with the help of an Oculus Rift Virtual Reality headset. The Flyinside software uses the features of the Microsoft Flight Simulator and adds an interface to the VR environment. The flight dynamics and features of the original software remain the same. Figure 11 shows the exterior of the VR flight simulator, while Figure 15 shows both the flight scenery and the VR simulation presentation in the cockpit.
o >
0
Figure 15 - View from the cockpit of the VR flight simulator for the Lasta aircraft
During the flight, a demonstration of the traffic pattern typically used in training to practice take-off and landing on the Lasta aircraft was used for testing. The subjects who participated in a comparison of the flight performance and instrument eye scanning strategies on the simulators of different fidelity practiced the same traffic pattern.
Matthias Neusinger (2007-2010) created the FS Recorder module to record flight performance data from the simulator used in the study by Vlacic et al. (2020). FS Recorder is free software that adds flight recording and playback functions to the core software. This software records flight data parameters during all flights in a conventional simulated environment and VR. It can record and reproduce flights of almost unlimited length. It stores data in its own frc data format. In addition, the FS Recorder converter created by Matthias Neusinger (2007-2010) was used to convert the FS Recorder frc files into text files. After conversion, the actual flight data is formatted into rows and columns, where each row contains a single set of parameters (from now on referred to as sample data), and each column represents a specific parameter. The lines represent a single sample of data and consist of a series of values separated by space (tabs or spaces). These values must correspond to the definitions of the columns (i.e., latitude, longitude, altitude, pitch, heading, flap position, instrument airspeed (IAS). Indicated Airspeed, Actual Airspeed (TAS), and True Airspeed). The most important column for synchronizing eye-tracking and flight performance footage is the "timestamp" column, which records the time interval in seconds from the start of the recording for each parameter that makes up that row of data (i.e., latitude, longitude, altitude, pitch). In this way, it is possible to compare the events in the simulation with a timer that measures time within the eye-tracking software, as they start working almost instantly.
Collection of the flight data parameters
To enter as much data as possible about the flight parameters in specific flight segments in the studio, the Proracun function was created in the R programming language. In this function, the flight performance data by segments of the traffic pattern is extracted and analyzed based on the pre-inserted part of the pilot's knowledge and experience. One part evaluates the performance of the flight in the traffic pattern, and the other part evaluates the grade according to the size of the standards that are evaluated and compared according to the official methodological manuals for training. In this study, a very important detail is the number of flights given for training as a norm defined by methodological manuals. It
represents a series of knowledge transferred and practiced from flight to flight.
Each traffic pattern flight practice contains six flights with six take-offs and landings. After the csv files for one of the respondents are imported ° into the R environment, to begin with; it is essential to set aside the time in frame in which each traffic pattern takes place. This is achieved by filtering S based on a variable that contains a value of 0 if the aircraft is in the air or ^ 1 if the aircraft is on the ground. Since the aircraft is on the ground for a ^ short time, filtering it based on a value of 1 is more convenient. The final =3 value of the column is obtained - a timestamp, between which there is an
0 interrupted time series that represents the beginning or end of a school
< circle. The function filters this variable and writes the data to the Times ° table, which has a separate appendix for each cadet pilot and each of his
flights.
After that, certain required segments are reached based on filtering £ by several set criteria for the given time frames in which the traffic pattern
< takes place. The example below shows how the mean square deviation from the flight course of a given value in the segment of the school circle from the first turn to the second turn (Crosswind) is calculated.
_ if (l=="crosswind") { iO
p2<- abs(cww$heading- 32)#daje promenljivoj p vrednost razlike kursa i zadanog
« kursa#
^
>o prenos2<- data.frame(p2)
1 rms_crs_head<-sqrt(mean(prenos2$p2A2))
O Zapis_crosswind$head_second_circle<<- prenos2$p2
—>
O >
&
O Zapis_crosswind$rms[2]<<-rms_crs_head
>
print("rms_crs_head")
print(rms_crs_head)
Collection of the data obtained from an eye-tracking recorder
Gaze transition strategies are defined as eye movements that an individual can perform and that are relevant to flying an airplane. Specifically, the two relevant eye movements are fixations (i.e., the time that elapses while one's gaze is focused on one particular location or the number of glances at that same point) and saccade (i.e., the rapid movement of the eyes from one point of fixation to another) (Ziv, 2016). In practice, at least when it comes to aviation, the point at which the number of views is measured is defined as an area of interest (AOI).
A sequence of gaze movements can start from any other AOI and is a return scan path or "revisit" with any other AOI. A study by Lefrancois et al. (2016) provides research results between visual patterns and flight performance with experienced pilots. It explains how they did not divide efficiently their visual attention compared to the reference crew. They assume that prerecorded experts' visual pattern could help novice pilots to adopt appropriate gaze strategies.
A study by Ayala et al. (2023) provides the results of the influence of the difficulty of the flight task on the instrument observation strategy of pilots with less flight experience. This study performed an entropy-based analysis at 10 AOI in a simulated landing scenario. A series of fixation locations were then generated for each flight. The necessary codes are written in Python to calculate the Static Gaze Entropy (SGE) and the Gaze Transition Entropy (GTE). The SGE is a measure of a view's dispersion calculated over a given viewing period. The more evenly distributed the fixations across the AOI (i.e., the wider the dispersion of the gaze), the higher the entropy. The view transition entropy (GTE) examines the complexity of a fixation sequence through the analysis of gaze transition matrices.
A similar approach to the measure of the entropy of the transition of gaze was developed by Mandal & Kang (2018) in their study. A network eye movement analysis was applied for dynamic tasks, where different AOIs were represented as network vertices (nodes), and the transition of views between them was represented by directed paths between the vertices of the network. The development of grid-based visualization and other similar attempts are an effort to replace traditional measures of transition views such as the number and duration of fixations, which did not provide a sufficiently reliable indication of the difference in observation strategies, to be more reliable.
Grid-based visualization and measurement of gaze transition requires first creating an AOI transition matrix, which is a tabular representation of the eye fixation transitions that occur between different AOI pairs (Table 2). The information about the transition matrix is used to develop a web presentation which visualizes the visual scanning strategy of the respondent.
TO AOI
FROM AOI AST AVST EI OC
AST 0 1 2 6
AVST 1 0 1 8
EI 1 0 0 3
OC 7 8 2 0
A way of presenting data for a static network as in Mandai et al. (2016) with some adjustments was used in the paper of Vlacic et al. (2020). The proposed four AOIs on the simulated cockpit of the Lasta aircraft for this study can be seen in Figure 16.
Figure 16 - Appearance of the cockpit of the Lasta aircraft in the Microsoft Flight Simulator and 4 AOIs for which the instrument scanning schedule was recorded in the
paper Vlacic et al. (2020)
Eye movement recording was performed using the GP3 Desktop Eye Tracker from Gazepoint (Figure 9) with a sampling rate of 60 Hz and an angular accuracy of 0.5° —1°. When the simulation and the FS Recorder software are started, the recording function in the Gazepoint Analysis software is also started so that the timeline of recording flight parameters and view transition can be compared. It is interesting to note that the difference between the number of AOIs in the example from the Ayala et al. (2023) study (where there were 10) comes from the fact that the cockpit of the Lasta aircraft is equipped with electronic instruments that combine the display of several parameters in one space, which is exactly what was
200
stated in the introduction as the need to conduct research. This study used an igraph package (Csardi & Nepusz, 2005) in the R programming language to convert AOI transition matrices (Table 1) into diagrams. There is a difference in the proposed color of the knots compared to the proposed principle in Mandal et al. (2016). Based on the RGB palette of the RColorBrewer package in the R programming language, it has been defined that: red means a low fixation duration, orange means a longer fixation duration, and yellow means the longest fixation duration, as shown in Figure 17.
Figure 17 - Visualization of the instrument observation strategies using the network method (an example for participant No. 5)
To statistically check the different strategies visualized in this way, calculations of the centrality of the nodes within the diagram were carried out using the Igraph package in the R programming language. These measurements inherent in diagrams determine the relative importance of the node in relation to the diagram. In this study, these were measures of the degree of centrality (Indegree), centrality by closeness (Closeness), and Relational Centrality (Betweenness). In addition, these measures have been normalized to determine how far each of these measures is from the maximum recorded value within a given time interval, or in this case the flight phase.
Results
The data obtained from the recordings from the flight recorder files in conventional and virtual environments reveal a distinct and characteristic progress of the group that rehearsed the traffic pattern in the virtual environment. Better results from this group were recorded even when they
CD O C\l CO 00
!±
O <D
CO O)
'EL
T3
<u o c o o
<D
ro
T3 c ro
<D
ro
"a ro
<u 4=
T3
M—
O
<0
O <
To
(D <
■O >
<u
>N <U
CO
r-
Ô >
in
CN O CN
Ql LJU
ZD
o o
-J
<c o
x o m
i>-
en <c
(D <C
.J
O
sz o
X
m
O >
&
were returned for testing on a conventional simulator together with the control group after practice, as was envisaged by the experiment setup. An example of a more precise runway alignment in the direction of landing, i.e., a more minor deviation from the ideal trajectory of the given course in the final approach for each of the subjects, is given in Figure 18. The trend of progression and differences in the mean square deviation from the ideal trajectory of the final view for the groups of subjects is shown in Figure 19. The graphical representation is given using the functions of the ggplot package and the RColorBrewer package in the R programming language.
Final heading RMS error boxplot and trend accross flights
trendlines for participants
flight
Figure 18 - Summary statistics for the mean squared deviation from the final approach course for the experimental group (VR)- and the control group (Simpit)-, with a trendline
by flight
Final heading RMS error trend accross flights
Figure 19 - Size and trend of the mean square deviation from the final approach heading for the experimental group (VR)-down and the control group (Simpit)-up, with the
trendline by flight
The analysis of the eye movement imaging device results revealed that the control group had a weaker strategy for observing the instruments than the experimental group. When looking at traditional measures for assessing eye movement, the number and duration of fixations, it was observed that the control group had acquired a habit of observing more instruments that show inclination, angle of climb, and velocity (Bank AOI, Pitch AOI, IAS AOI-, shown in Figure 20).
CD O CNI CO
oo !±
O <U
<0 O)
m
CT T3
<u o c o o
(U
ro
Figure 20 - Summary statistics of the average fixation time in the phase of the final approach across the school circle (Final) in each group of participants for all defined AOI
Values of Indegree
T3 c ro
<D <0
"a ro
25
iS 20 Ü 15 S 10
S 5 J2 0 o
-Q
(B <U (L
M eu ■a c
Bank
Pitch
IAS
ALT
RMI
Figure 21 ■
test AOI
Simpit VR
Display of the absolute values and the trend of the Indegree measure for the final approach phase for each of the AOIs by groups of subjects
<u ¡fc T3
M—
O
<0 O
"ü
CP <
To
(D <
■O >
<u
>N <U
EI
<u tn
CO
"o
>
LO CM
0 CM
01 yy
0£ ZD O o
-J
<
o
X
0
LU
I—
>-
01 <
CO <
_J
CD >o
X LU I—
o
o >
Also, the measures of the centrality of nodes within the diagram using the Igraph package in the R programming language showed that subjects in the conventional simulation environment had higher centrality values for the same AOI as in traditional measures (Bank AOI, Pitch AOI, IAS AOI-, Figure 21).
In contrast, the subjects who flew in the VR environment had a higher degree of centrality for the areas of interest in height, Altitude AOI, RMI AOi, and the AOI engine (Figure 21), which proves that these two groups of subjects have developed different strategies for observing instruments.
Conclusion
Basic flight training is considered the most crucial stage of pilot training and the basis for a successful process of producing future pilots. This training phase has undergone significant changes with the introduction of the Lasta aircraft which belongs to the category of technically advanced aircraft. This aircraft is characterized by a high degree of digitalization of the cockpit and other systems. Due to these characteristics, paying close attention to flight safety and training is necessary. To do this, attention must be paid to the theoretical training stage with synthetic means.
The aircraft manufacturer Lasta did not provide funding for training and simulators. The Military Academy's research team has acquired and developed many training tools to make the training process more efficient. The concept of digital training includes a series of digital devices designed to enhance flight training.
The assessment of flight instructors and examiners from the Military Academy after flights with cadet pilots of the first generation of pilots-cadets who began flight training after practicing on these trainers is that the results in training, especially in the early stages of basic training, are better. The simulators and trainers based on COTS components are the core of these devices.
VR offered opportunities to improve the flight training process— cadets learn faster and more efficiently, and their memory of flight tasks is better.
The GP3 Desktop Eye Tracker proved to be a valuable tool for evaluating results in a simulated environment. It is also fully applicable as an additional tool for selecting piles.
With its packages and functions, open-source software, such as the R environment, allows you to work with large amounts of data, such as logs from flight parameter recorders, either from a real aircraft or a
References
simulator. In the foreground comes the possibility of detailed statistical processing of data, which can prove a hypothesis or compare groups of respondents. In this way, so-called quick tools can be created to contribute to decision making regarding how experiments should be continued. On the other hand, the possibility of graphical representations and simulations of processes can serve to understand better the progress of complete processes for those who need to become users of data processing services.
Although digital training is not mandatory in the current pilot training model at the Military Academy, it is successfully implemented. Its potential is significant and can, to some extent, affect the very nature of the basic training process.
CP
m
T3
Ayala, N., Zafar, A., Kearns, S., Irving, E., Cao, S. & Niechwiej-Szwedo, E. 2023. The effects of task difficulty on gaze behavior during landing with visual flight rules in low-time pilots. Journal of Eye Movement Research, 16(1). Available § at: https://doi.org/10.16910/jemr.16.1.3.
Csardi, G. & Nepusz, T. 2005. The igraph software package for complex network research. ResearchGate, November [online]. Available at: https://www.researchgate.net/publication/221995787_The_Igraph_Software_Pa g ckage_for_Complex_Network_Research#fullTextFileContent [Accessed: 16 November 2024].
-EASA. 2021. EASA approves the first Virtual Reality (VR) based Flight Simulation Training Device. EASA (European Union Aviation Safety Agency), 26 April [online]. Available at: https://www.easa.europa.eu/en/newsroom-and-events/press-releases/easa-approves-first-virtual-reality-vr-based-flight-simulation [Accessed: 16 November 2024].
-EASA. 2018. CS-FSTD(A) Issue 2. EASA (European Union Aviation Safety Agency), 03 May [online]. Available at: https://www.easa.europa.eu/document-library/certification-specifications/cs-fstda-issue-2#group-read-more [Accessed: 16 November 2024].
-FAA. 1997. Approval of flight simulators and flight training devices. Washington, DC: United States Department of Transportation, Federal Aviation S Administration.
Fussell, S.G. & Troung, D. 2022. Using virtual reality for dynamic learning: < an extended technology acceptance model. Virtual Reality, 26, pp.249-267. ^ Available at: https://doi.org/10.1007/s10055-021-00554-x.
Lefrancois, O., Matton, N., Gourinat, Y., Peysakhovich, V. & Causse, M.
<u
o
2016. The role of Pilots' monitoring strategies in flight performance. In: European >
<u
Association for Aviation Psychology Conference EAAP32, Cascais, Portugal, >n ffhal-01405262, September 26-30 [online]. Available at: https://enac.hal.science/hal-01405262v1 [Accessed: 16 November 2024].
<D tn
CO
"o
>
Mandal, S. & Kang, Z. 2018. Using Eye Movement Data Visualization to Enhance Training of Air Traffic Controllers: A Dynamic Network Approach. Journal of Eye Movement Research, 11(4). Available at: https://doi.org/10.16910/jemr.11.4.1.
Mandal, S., Kang, Z. & Millan, A. 2016. Data Visualization of Complex Eye Movements Using Directed Weighted Networks: A Case Study on a Multi-Element & Target Tracking Task. Proceedings of the Human Factors and Ergonomics ° Society Annual Meeting, 60(1), pp.106-110. Available at: oc https://doi.org/10.1177/1541931213601024.
^ Vlacic, S.I., Knezevic, A.Z., Mandal, S., Rodenkov, S. & Vitsas, P. 2020.
g Improving the pilot selection process by using eye-tracking tools. Journal of Eye
0 Movement Research, 12(3). Available at: https://doi.org/10.16910/jemr.12.3.4. < Ziv, G. 2016. Gaze behavior and visual attention: A review of eye tracking ^ studies in aviation. The International Journal of Aviation Psychology, 26(3-4),
1 pp.75-104. Available at. https://doi.org/10.1080/10508414.2017.1313096.
o
LU
d£ Aplicación de distintos equipos y programas informáticos en el
concepto de formación y selección de pilotos digitales
Aleksandar Z. Knezevic3, Aleksandar G. Bukvicb, Branimir B. Krsticb
Universidad de Defensa de Belgrado, Academia Militar, ^ Belgrado, República de Serbia
a Departamento de la Fuerza Aérea, autor de correspondencia b Departamento de Ingeniería Mecánica Militar
I
CAMPO: ingeniería mecánica J TIPO DE ARTÍCULO: artículo científico original
ET
o Resumen:
o Introducción/objetivo: En muchas fuerzas aéreas militares y
organizaciones de aviación civil que utilizan o poseen aviones de entrenamiento equipados con una cabina digital, las herramientas de entrenamiento como entrenadores y simuladores se entienden en mayor o menor medida. Se presta especial atención a la proporción de horas de entrenamiento en el avión real con respecto al número de horas de entrenamiento en el simulador. Este enfoque en la formación de los círculos profesionales y del público se denomina en directo virtuales constructivas. Para todos los demás medios didácticos utilizados en la preparación para la formación, no se ha investigado específicamente ni la metodología ni el alcance de su uso. La investigación tiene como objetivo desarrollar instrumentos que midan y evalúen el éxito y el progreso cualitativo en la formación de pilotos logrados mediante el uso del espacio de cabina digital, en función de los cuales se pueda tomar una decisión sobre la formación adicional para un tipo de avión específico.
Palabras claves: entrenamiento de vuelo, simulación, hardware, software, seguimiento ocular.
РУБРИКА ГРНТИ: 78.25.13 Военная авиационная техника и вооружение ВИД СТАТЬИ: оригинальная научная статья
Резюме:
Métodos: Junto con un dispositivo de entrenamiento de aviación basado en PC y análisis estadísticos avanzados, un dispositivo de seguimiento del movimiento ocular ayudará a llegar a un patrón de referencia de observación de instrumentos para capacitar a los pilotos y seleccionar candidatos a piloto.
Resultados: La investigación del concepto propuesto incluye descripciones detalladas de la aplicación de entrenadores y medios didácticos con diferentes perfiles, que impactan significativamente en la calidad de la implementación del espacio de cabina digital en la formación de pilotos cadetes. ?
Conclusión: Dado que la Academia Militar forma pilotos cadetes que continúan su formación en las unidades operativas de vuelo de las Fuerzas Armadas de Serbia (SAF), es fundamental que durante el período de estancia en la Academia Militar se lleve a cabo su preparación teórica de calidad. H
"O
Ф O
Ф
Применение различных аппаратных и программных средств в концепции 8 цифрового обучения и отбора пилотов
Александар З. Кнежевича, Александар Г. Буквич6, Бранимир Б. Крстич6
Университет обороны в г. Белград, Военная академия, <5
г. Белград, Республика Сербия а кафедра военной авиации, корреспондент
"О
б департамент военного механического инжиниринга §
ф го
"О
го
Введение/цель: В военной авиации и организациях гражданской авиации, которые пользуются или владеют учебно-
тренеровочными самолетами, оснащенными цифровой кабиной о пилота, средства обучения, такие как тренажеры и симуляторы,
используются в разной степени. Особое внимание уделяется го
соотношению часов, затраченных на обучение на настоящем Ц
воздушном судне, с количеством часов, проведенных на <
тренажере. Такой подход в профессиональной среде принято ю
называть конструктивным моделированием в режиме реального ф
времени. Однако методология и степень использования других < учебных средств для подготовки к учениям отдельно не
изучались. $
ф
Методы: Использование устройств отслеживания движения глаз в сочетании с высокоточным пилотажным тренажером
ф (Л
со
7
было исследовано с помощью расширенного статистического анализа с целью разработки эталонной модели осмотра приборов для обучения курсантов и отбора кандидатов в пилоты.
с5 Результаты: Исследование предлагаемой концепции включает в
себя подробное описание применения тренажеров и учебных средств различного профиля, которые существенно влияют на качество внедрения цифрового пространства кабины при ш обучении будущих пилотов.
^ Вывод: Учитывая тот факт, что Военная академия готовит
о военных летчиков, которые продолжат свое обучение в
оперативных летных подразделениях Вооруженных сил Сербии (ВСС), необходимо провести качественную теоретическую подготовку в течение обучения в Военной академии.
^ Ключевые слова: летная подготовка, имитационное
моделирование, аппаратное обеспечение, программное
к обеспечение, отслеживание глаз.
< _
Примена различитог хардвера и софтвера у концепту дигиталне обуке и селекци]е пилота
_ Александар З. Кнежеви1а, Александар Г. Букви1б, Бранимир Б. Крсти1б
<л
< Универзитет одбране у Београду, Во]на академи]а,
О Београд, Република Срби]а
^ а Катедра во]ног ваздухопловства, аутор за преписку
б Катедра во]номашинског инженерства
х ш
[- ОБЛАСТ: машинство
КАТЕГОРИJА (ТИП) ЧЛАНКА: оригинални научни рад
> Сажетак:
0 Увод/цигь: У многим во]ним ваздухопловствима, као и цивилним
ваздухопловним организаци]ама ко}е користе или поседу}у школске авионе опремъене дигиталним кабинским простором, подразумева се употреба средстава за обуку као што су тренажери и сумулатори различитог профила. Посебна пажша прида]е се односу бро}а часова проведених у обуци на авиону и бро}а часова на симулатору. Ова] приступ у обуци у стручним круговима и}авности познат }е као Жива виртуелна конструктивна. За сва остала наставна средства ко}а се користе у сврху припреме за обуку ни}е посебно истражена методологи}а нити обим шихове употребе. Методе: Употреба уре^а за праПеше покрета ока у спрези са тренажером одре/еног степена верности истражена }е помогу напредних статистичких анализа како би се дошло до референтног
шаблона осматра^а инструмената щи би служио за обуку пилота g
С\1 со 00
и селекц^у кандидата за пилоте.
Резултати: У оквиру истраживаъа предложеног концепта детал>но jе описана примена тренажера и наставних средстава различитог профила, ща имаjу значаjан утицаj на квалитет имплементац^е дигиталног кабинског простора у обуци кадета-пилота.
Закъучак: С обзиром на то да се на Воjноj академии образуjу кадети-пилоти ко\и своjу обуку наставл>аjу у летачким jединицама я Воjске Србще (ВС), врло jе важно да се у току периода проведеног на школоваъу изврши ъихова квалитетна теор^ска припрема.
Къучне речи: летачка обука, симулац^а, хардвер, софтвер,
"Й
прапе^е покрета ока.
.5? Ъ
EDITORIAL NOTE: The third author of this article, Branimir Krstic, is a current member of the Editorial Board of the Military Technical Courier. Therefore, the Editorial Team has ensured that the double blind reviewing process was even more transparent and more rigorous. The Team made additional effort to maintain the integrity of the review and to minimize any bias by having another associate editor handle the review procedure " independently of the editor - author in a completely transparent process. The Editorial Team has taken special care that the referee did not recognize the author's identity, thus avoiding the conflict of interest. a>
ro
Paper received on: 16.07.2024 Manuscript corrections submitted on: 29.01.2025. "g
Paper accepted for publishing on: 30.01.2025. <o
2025 The Authors. Published by Vojnotehnicki glasnik / Military Technical Courier g (www.vtg.mod.gov.rs, BTr.MO.ynp.cp6). This article is an open access article distributed under the % terms and conditions of the Creative Commons Attribution license ro (http://creativecommons.org/licenses/by/3.0/rs/)
e
ffi
di f
0 n
01 it a
ci lp
p
< "rô
t e
<
■o
iv e
>N
e
