Military, corporate, and A&P trainers are adopting game-based, first-person technologies as fast as they can. These virtual technologies engage students and supplement lectures and manuals. But the jury is still out on which approaches are best suited to which training tasks.
Technologies range from virtual maintenance trainers (VMTs) to virtual reality (VR) and augmented reality (AR). Found in classrooms, VMTs – desktop-based simulators — offer students 3D training scenarios through which to learn aircraft systems and troubleshooting.
AR and VR require the use of goggles to present images of objects or whole environments. AR overlays images and information on a real object and presents virtual objects while students are still anchored in the here and now, whereas VR calls up a completely virtual world.
VR can be good for beginners, as part of familiarization training. FL Technics, for example, is introducing VR into the basic training of aviation mechanics, the company says. Its first module covers the opening of the reverse thrust engine of a 737NG.
VR is also suited to tasks that are complex and tricky enough to be hard to convey by traditional means, that are too difficult or expensive to use the real equipment to train on, or that require environmental elements such as high noise levels, says Pete Boeskov, chief technologist with Boeing’s Training & Professional Services unit.
Boeing, for example, has fielded a VR procedures trainer for aviation maintenance to a military customer. “The use case tends to be where you want to simulate an environment with a certain level of chaos to prepare you for the real thing,” Boeskov says.
VMT content is also deployable. DiSTI, for example, provides a subset of its U.S. Army UH-72 Lakota VMT for remote use. Segments loaded onto an iPad or Windows-based tablet can be launched in the field as refresher training via “autoplay lessons,” says Kevin Mikalsen, director of global marketing.
Prior to performing a task, users can go through the lesson, interact with it, show they have completed it, and prove that they’re competent and capable of executing the procedure, he says. DiSTI’s VE Studio framework allows content to be pushed to desktop/mobile, VR, or AR devices.
Whether VR/AR training in civil aviation is the equivalent of classroom training is still a gray area, says Allan Bachan, a vice president with ICF. Frequently regulators wait until some level of maturity can be demonstrated, he says.
VR for maintenance is still in the innovation arena, Bachan says. It’s one thing to service a landing gear in a lab but another to do it in a hangar with all the heat, dirt, noise, and heavy components. In the lab you don’t get wires scratching your fingers or glycol spilled on your overalls. And you don’t find yourself bent over in a funky position, he says. So while productivity might be great in the lab, it would drop in the real world.
L3Harris asserts that maintenance training costs are decreased by 20 percent through the use of AR/VR/MR (mixed reality) training technologies vs. traditional methods. The company says it has quadrupled student throughput based on the use of virtual training. With further enhancements to its learning management system (LMS), it expects to achieve 30-50 percent reductions in training times, adds Lenny Genna, president of L3Harris’s Military Training Sector. (LMS is the software that helps instructors track student progress.)
Boeing has done some controlled studies concerning the use of virtual approaches. “Even things like desktop simulators bring that learning-by-doing element,” Boeskov says. And the technologies can reduce the cost of training. “We’ve demonstrated we can get the same outcomes without having to spend as much time at the actual airplane.”
Karen Jo Johnson, an associate professor in the Department of Aviation Technologies at Southern Illinois University Carbondale, is doing a doctorate on ways to teach troubleshooting to A&Ps. With the help of a former student, Gary Tippin, Johnson has created an animation of hydraulic schematics. The simulation can be activated by a smart phone or operated from a web site. You can interact with it and introduce different scenarios into it. The animation allows you to see the fluid flow that makes the components move.
Johnson hasn’t designed the structure of her experiment yet, but the crux of the dissertation is to judge the effectiveness of the technology in teaching skills such as troubleshooting. She hopes to include all of the students in the school’s aviation technology program – 110 to 130 men and women – in her study.“People know that students don’t go home and read textbooks,” she says. “I think that’s why you’re really seeing a lot of this technology come into the system.” She’s “trying to see if it works.”
Augmented Reality
A collaborative environment where students in a class are looking at the same virtual landing gear – something Boeing has demo’d — is a good use case for AR, Boeskov says. The students can interact with it, actuate it, move it, and highlight things. And it makes sense in the real maintenance world “where you’re providing knowhow at the point of use.”
But there are challenges, such as field-of-view and visibility in daylight conditions. In AR, which mixes virtual and real worlds, the two have to be aligned, Boeskov says. “It’s got to be seamless.”
L3Harris is also working on AR with “reachback” capability, so that someone deployed in the desert can connect with an expert and send photos back and forth to refresh the maintainer about how to perform a task like a helicopter blade replacement. “It’s like a rehearsal,” Genna says. The company’s Unified Maintenance Solution (UMS), which combines VMT, LMS, AR, VR, MR, and reachback, has not yet been sold to aviation customers.
GE Aviation, partnered with Upskill, ran an AR trial a few years ago, using the enterprise edition of Google Glass. The idea was to see whether the technology could help mechanics in engine assembly tasks. The experiment integrated a wireless torque wrench from Atlas Copco and provided a real-time display of how much torque mechanics were applying to different fittings, nuts, or bolts, says Ted Robertson, systems engineering program leader. When they hit the right torque, the system would take a photo so that there was an “historical record of them torqueing the right thing at the right time.”
The project looked at 15 mechanics performing tasks on a borrowed CF34 engine. In the morning a mechanic would do two maintenance tasks with traditional tools and GE would measure the time it took. In the afternoon the mechanic would perform the same two tasks using the virtual assist and have the time measured. GE found that the technology decreased task performance time by about 10 percent, Robertson says. He got very positive feedback from the workers, about 85 percent of whom thought the system would reduce errors in the workplace.
DiSTI has done proof-of-concept VR and AR training modules for the EC145, the commercial version of the UH-72, involving tail rotor replacement. The AR project translates “training” to “performing” the task, guided by the images in a HoloLens AR headset, Mikalsen says.
Naval Aviation Maintenance
The Navy has developed the Multipurpose Reconfigurable Training System (MRTS) 3D, which is based on large, multitouch flat panel screens that run a family of simulation applications. Using gaming technology, it immerses students in a 3D, first-person virtual environment. Content also can be injected into headsets for VR-type training, says Tyson Griffin, head of advanced technology products for the Naval Air Warfare Center’s Training Systems Division.
One aviation application involves the mobile electric power plant (MEPP) used to start aircraft. Sailors learning how to operate and fix the MEPP first interact with the equipment in a simulated environment, explains Capt. Tim Hill, the commanding officer of the Training Systems Division.
The division has also performed research, looking at the repair-and-replace procedure for the alternator of E-28 arresting gear used at airfields, says Randy Astwood, head of the Human Performance Science & Technology Group. The research suggests that people made more substantial errors during desktop training vs. VR training. The VMT and the VR systems were shown to be equally immersive, but the VR students had better recall, perhaps because they couldn’t “just click through the training – they had to pay attention and work through it.
There are cost savings, as well, Capt. Hill says. Although a MRTS application costs “somewhere less than a million dollars,” a trainer using real aircraft parts costs much more, and the virtual trainer will be able to “have every failure mode that’s possible.” Training via virtual technology can be less expensive, faster, and more engaging.
Another Navy initiative, Ready Relevant Learning, focuses on giving sailors just the right training at just the right time rather than jamming 90 percent of their training into their first six to 18 months in the service, Capt. Hill says. An associated concept is the use of job aids, such as tablets or VR/AR gear throughout sailors’ careers, so when they encounter a task for the first time, they have something to help them through it.
For the P-8 the Navy is looking at VR and AR for complex tasks such as removing and replacing pods, Griffin says. But so far it has not fielded any VR or AR maintenance trainers. VR can provide good situational awareness and an understanding of the size and scope of the equipment you’re dealing with, Griffin says. But for fine motor movements, it’s not mature enough to provide the level of precision the Navy needs.
Among the challenges of delivering training to deployed personnel are logistics, equipment ruggedness, and transmission issues such as emissions, information assurance, and bandwidth.
Boeing
Boeing emphasizes content reusability. Its Extended Reality Learning Framework (XRLF) architecture “allows the reuse of assets across a broad spectrum of delivery modes,” Boeskov says.
Boeing’s VMT, Microsim, is tied into XRLF. Targeted at tablets and laptops, Microsim is available at the company’s training centers and is licensable to customers. It does not require a server.
Boeing has some streaming proof-of-concepts. A cloud-based solution would “give us the scale, reach, and bandwidth to hit a large number of users and endpoints,” he says. He expects to see hosted streaming applications in two to three years.
VMTs
VMTs account for most virtual training today and are effective at providing a lot of information in an immersive manner. VR has a role in tasks such as familiarization training – navigating around an aircraft to find a specific LRU panel — says Phil Perey, head of technology for CAE’s defense business.
But VR can’t provide the same depth of systems information or diagnostics understanding as a VMT can. “When you put the VR on, you lose so much,” such as screens with the maintenance manual, training tasks, and cockpit view, Perey says.
CAE provides the VMT for the A400M. The primary goal of the VMT was to avoid the need for a real aircraft to get crews up to speed, Perey says. To this day the forces using the A400M “don’t typically have a dedicated aircraft” for crews to train on. VMTs can be extended to VR – “you’re basically mapping the desktop into a VR headset,” he says.
UDC
The University of D.C. Community College’s aviation program has been using an L3Harris 737-800 VMT since January, thanks to a grant and internal funds. There is a large projection screen up front and a multimonitor computer system for each pair of students, says Bill Russo, aviation program director.
The virtual airplane “does everything exactly the way the real airplane would,” he says. In the hydraulics class students can see how the fluids move. They also can see the bigger picture – how the hydraulic system works with the landing gear, for example. Because students are learning aircraft systems on a 737-800 VMT, they have knowledge that is “more in tune with the current needs of the industry,” Russo says. “Almost no A&P schools teach commercial aviation maintenance.”
He can insert faults like the failure of an avionics cooling fan and have students troubleshoot it. Students interact with an electronic version of the manual and the aircraft system, so they’re learning both. They can remove and replace components per the manual. The first time through, Russo shows them the procedure and they follow along; the second time two students come up to the instructor station and demonstrate to the class; and the third time they do it on their own.
The VMT can replicate inflight problems like a TCAS alarm to show students how it presents to the pilot. The VMT also can simulate an in-flight TCAS failure, which triggers a fault code that they can troubleshoot.
Although the school hasn’t been using the VMT very long, Russo says that students that have used the technology have scored much higher on quizzes, projects, and finals than similar students the previous year.
But VMTs are pricey for A&P schools, says Crystal Maguire, executive director of ATEC, the Aviation Technician Education Council. “They are more the exception than the rule.” There are also some things – like welding and fabric – that you can’t teach with high tech.
GE Aviation
GE Aviation is planning a training technology upgrade. Today students at its Customer Technical Education Center (CTEC) are issued tablets with PDF-formatted materials and a PDF editing tool while instructors operate “smart boards” – large, interactive white boards. These can display models of engines that can be disassembled and reassembled virtually.
As soon as 2020 GE aims to introduce a homegrown VMT, coupled with a new learning management system that employs cloud delivery, so that lessons can be streamed to remote locations.
GE’s Chief Engineer’s office is using VR for analysis, says Brandon Richards, senior engineer. For VR the office uses GE content in its cloud-based distribution system, GE App Dash; F110 maintenance awareness, which provides 3D, immersive content for the F110 engine; the Siemens Teamcenter Visualization Mockup; and the HTC Vive headset.
Dynamic Adaptation
Another trend is the dynamic customization of training to individual needs. Charles River Analytics (CRA), partnering with DiSTI, is developing an intelligent VMT, known as MAGPIE, under a contract with the Air Force Research Lab. The “intelligent” part involves the system’s ability to monitor students’ behavior and, based on their grasp of the material, to dynamically adapt how the course is being delivered, Mikalsen says.
a more traditional style of hands-on training utilizing real propeller products. APS image.
MAGPIE software can actually track the student’s performance in reference to a given standard or set of steps typically taken and can diagnose and adjust the scenario flow and content, as needed, to help the student progress, explains Winston Bennett, readiness product line lead, at AFRL’s Warfighter Readiness Research Division. It can understand where student’s gaps are and move the student back in the curriculum to ensure the “building blocks” are built, he says.
The “intelligent tutor” uses various AI algorithms to dynamically track performance and guide training exercises, explains Sean Guarino, CRA’s principal scientist for human effectiveness. Ongoing work will revise scenario parameters — including faults and environmental stressors — based on the skill level of the trainee, he says.
Smaller Scale
Texas State Technical College is working on developing labs using X-Plane flight simulation software running on Alienware gaming hardware with Occulus Rift VR, says Kelly Filgo, executive director of special projects. The software includes a startup sequence for the 737-800.
“There’s no substitute for the real deal,” he says, and the school has a PT-6 and other airplanes that students can start up. But no A&P school can afford a 737. The lab – not yet in official classroom use – would give students a better understanding of how individual systems work together and make them better troubleshooters.
At WSU Tech, part of Wichita State University, Diana Holladay, a lead instructional designer, created an interactive animation of the axes of an airplane. Although it was “cartoonish,” it would be appropriate for an introductory lesson about how an airplane functions, she says. Instructors liked the idea, which could be a step toward making worksheets interactive. She focuses on starting small and using free apps on ubiquitous devices like smart phones.
We’re at the beginning, Holladay says. The software has to be customized to the aviation niche. She hopes to partner with other aviation schools or companies or make a trainer “specific to us.” Her department is already gaining AR experience, using zSpace to develop a virtual crime scene for police science instruction.
Prop U
High tech notwithstanding, there’s still good value to traditional approaches. Aircraft Propeller Service, a full-service propeller shop, offers its own Propeller University, or Prop U, for training employees of the operators it supports.
“We have actual products,” says Joe Mayer, chief inspector. With enough lead time, APS can teach students using their company’s products. The training, based on OEM technical data, can help lower customers’ operating costs, he says. “Each class is unique.”