- Augmented Reality & Virtual Reality in STEM
What is STEM ?
STEM is not just a subject at school it’s a way of thinking and doing, and an important skill set that could serve your generation working together to solve some of the world’s greatest challenges. STEM stands for science, technology, engineering, and mathematics, but it’s not about being an expert of all those things. Instead it’s about harnessing the essential transferable skills behind them. You won’t have to master all of these skills, teamwork is an essential STEM skill. STEM careers can start small, then change the world. Engaging in your high school classes, or learning to code, could be your first step.
What is the relationship between STEM and Augmented Reality & Virtual Reality ?
There are still a lot of false assumptions and stereotypes about virtual and augmented reality (VR & AR). Do you still feel like it’s just a silly new thing used only by gamers? Or maybe you just can’t see how it can be used outside of the entertainment industry?
You are not alone. A lot of people seem to put these new technologies in a box of just more fun gadgets that no one takes seriously. So, we will try to show you a different side of immersive reality. But contrary to such popular assumptions, project by project and app by app, it is constantly being proven that VR & AR can be incredibly powerful tools in education too. And guess which field of education is on everyone’s mind right now? STEM!
Governments, businesses and education experts try really hard to come up with innovative ways to teach STEM subjects. But how can VR & AR help them?
Why VR & AR important in STEM ?
How exactly does immersive technology add value and create new possibilities in STEM education? VR and AR provide several benefits that apply to any field of education, but there are certainly some that apply particularly to STEM:
- VR & AR help to accommodate a risk-free environment in school classes. A lot of traditional experiments in STEM involve potential risks. Fire, electricity or corrosive substances make the teacher limit the extent of experiments or even skip them altogether. With the help of immersive technology, teachers can show all of these great things in a safe environment.
- Use of VR & AR enormously cuts costs. The costs of developing immersive reality are nothing compared to building a professional laboratory. In this way high quality STEM education is not limited to affluent areas and wealthy educational institutions.
- Cutting costs leads to one of the most important issues in education now — democratization of education. VR & AR can provide quality access and opportunities to a huge mass of students, that would be impossible to reach otherwise.
- STEM education, especially biology and medical research, happen to get involved with ethical dilemmas. Animal testing or patients rights are one of the examples of how traditional education can be morally questionable. With VR & AR students can get involved with a lot of experiments in a cruelty-free and ethical environment.
- A lot of STEM scenarios are just impossible to show in traditional education. There’s no way to take students on a trip to Mars or put them inside a blood cell. With VR & AR impossible is nothing!
- Immersive technology can change STEM perception. For a lot of students math, physics or biology are boring and difficult to understand. With immersive technology we can change the image of those subjects so that they become much more attractive to our kids.
What makes VR & AR unique?
AR/VR technologies are a promising addition to the “edtech” space due to their immersive nature, ability to share information in new and engaging ways, and potential to offer virtual experiences that can mitigate barriers from cost or distance.
Although still in its early stages, there are many promising examples of this technology already in use in K-12, higher education, and teacher training. Applications range from STEM education and technical training to arts and humanities.
AR/VR can provide K-12 educators with interactive and engaging tools for classroom learning. These include libraries of immersive content, experiences for specific subjects or learning objectives, and tools for students with learning disabilities.
In higher education, AR/VR can help learners grasp abstract concepts and gain hands-on experience in low-risk virtual settings. This can enhance STEM courses, medical simulations, arts and humanities materials, and technical education.
Congress should direct the Department of Education to invest in programs, resources, and initiatives that will guide the development of AR/VR educational solutions and encourage further innovations in this field.
WHAT ARE THE TECHNOLOGIES USED IN AR/VR ?
AUGMENTED REALITY
Vuforia
Vuforia is the SDK for building augmented reality applications which provides users with excellent computer vision technology and the opportunity to work with simple 3D objects in real time. Vuforia also gives users with the possibility to recognize flat and dimensional markers.
WikiTude
It is the cross-platform SDK for building location-based, marker or markerless AR experiences. This SDK gives developers a chance to create virtual reality applications for a wide range of operating systems and devices. The latest version of WikiTude promises markerless 3D objects tracking, object/image recognition, compatibility with ARKit and ARCore, instant tracking.
EasyAR
EasyAR is the SDK that supports 3D object recognition, environment perception, cloud recognition, smart glass solution, screen recording, app cloud packaging, content and integration support.
ARKit
It is one of the most advanced SDKs in the world today for AR apps building created by Apple for all iOS users. ARKit gives users unparalleled AR experience with the help of vertical tracking and amazing 3D graphics. ARKit is the chance to have non-interrupted AR experiences and share them with other users. It’s possible to incorporate real-world objects into your AR experiences, giving your users even greater immersive opportunities.
ARCore
It is the top SDK created by Google for all Android 7.0 users. The number of devices that support ARCore is growing rapidly every day. This SDK enables APIs with all the important AR features, such as motion tracking, environmental understanding, and light estimation. With these deliverables, it’s possible to build amazing AR experiences and enhance existing applications with AR features.
Face Recognition
It is the technology that allows recognizing a human face using a 3D model, and then comparing it to the face from the database for identification. Most of them use neural networks and machine learning technologies to achieve this effect. Face recognition helps to overlay digital content over a human face with high precision for numerous AR applications, such as makeup apps.
VIRTUAL REALITY APPLICATIONS
Samsung Gear VR
Mobile VR is the most accessible and the most comfortable VR experience for the majority of users. It’s easy to build virtual reality applications for Samsung Gear VR using Unity. Samsung Gear VR is designed to work with Samsung’s flagship smartphones to make virtual reality application available to the majority of Android users.
Google Daydream
Google Daydream stands aside from all other HMDs because it was first of all invented by Google and also because it’s definitely an easy-to-use headset that can compete with Samsung Gear VR. Building apps for Google Daydream means reaching hundreds of millions of users with the most interactive and the most powerful applications.
HTC Vive & Oculus Rift
High-end VR headsets, such as HTC Vive and Oculus Rift, give users the sense of total immersion. So if users want to experience alternative dimensions created with the help high-quality 3D graphics they need to choose these devices as long as they have powerful PCs.
As for today, HTC Vive is the best HMD in terms of resolution, audio, the level of immersion that is why all those who want an excellent level of presence place a bet on HTC Vive. Oculus, created by Facebook can boast with 2160 x 1200 resolution, OLED display ( the same as Vive, besides) and higher affordability.
HTC Focus and Oculus HMDs
Standalone VR HMDs are getting more and more popular today for one simple reason — you can use them anywhere and you don’t need a powerful PC to start them. Users don’t only want absolute immersion and presence, they also need comfort today.
Wireless HMDs, such as HTC Focus and Oculus Go, provide users with this comfort and make the display of VR content possible during big events. Building apps for standalone headsets are most effective using Unity, Unreal. Building Apps for standalone HMDs means giving a global audience the chance to have extraordinary VR experiences.
Example of challenge in this field
There are numerous limitations with AR/VR devices currently on the market. Perhaps the field of view (FOV) is the biggest. Today, these devices have a FOV of up to 90 degrees, compared to the 190 degrees horizontal and 120 degrees vertical for normal human vision. For these devices to create the immersive experiences they aim to, they must capture as much of the FOV as possible.
AR/VR devices must show the projected image in a large FOV for the human eye to make the experience more immersive, which requires devices like headsets to be bulky. In their current status, the size makes prolonged usage of these devices unlikely and uncomfortable. As work continues in this arena, there are many specifications that need to be achieved to overcome AR/VR device limitations, including work on the weight, brightness, display quality, FOV, latency and finally the user experience. I believe the industry will overcome these challenges in the next five to 10 years and produce great devices at the right price point.
Positives and Negatives in augmented reality virtual reality
Virtual Reality
Positives:
- VR, when implemented properly, can be an incredibly engaging sensory ride. Using computer-generated imagery (CGI), there are no limits aside from money and imagination when creating other worlds, product demos, or spaces in novel and interesting ways.
- When applied to education, VR makes learning easier and more comfortable.
- Virtual reality users can experiment with artificial environments
Negatives:
- VR is a fragmented market. Headset pricing ranges from about $15 USD (Google Cardboard using a smartphone) to $1,500 USD (HTC Vive Pro) and many choices with widely varying capabilities in between. VR standards are in the early adoption stage and content created for one platform will usually not work with another.
- Content creation tends to be customized and, oftentimes, expensive. Best practices for effective and engaging content creation are still being worked out as well.
- VR is often an isolating, individual experience — it takes you somewhere else, a place removed from the existing environment. This is the opposite effect of events where one of the main goals is to bring people together and interact in a group.
- VR is slow for demos. It takes time to configure the headset, to put on/adjust the headset, explain the controls, and for the user to view the content. Even if the content is only two to three minutes long, an exhibitor would be lucky to get 15–20 people per hour through the system.
Augmented Reality
Positives:
- AR, when used properly, can provide very useful and engaging information layered onto a real-world scene.
- Basic AR apps using smartphones are well-established.
- Apple’s latest phones/tablets and AR developer kit have some very significant new capabilities. Newer Google Android phones also have strong AR functionality. Smartphone users can see furniture in their own home before buying (IKEA), find their way to an airport gate (American Airlines AR), play games (Monster Park), see restaurant menu options in rotating 3D before ordering (KabaQ), measure distances very precisely (AR measuring tape), and much more. As these newer phones become widely used, it is likely that AR apps will see mainstream use with lots of opportunities for exhibitions and events.
- New tools have been created to help doctors during surgeries by allowing them to be constantly aware of patient data during the procedure.
Negatives:
- AR/MR headsets are pricey (Hololens: $3,500 USD; Magic Leap $2,295 USD) and have some serious limitations in their current forms. Both have a limited range of fields and the gesture controls are somewhat difficult to use. We will not see widespread consumer use until costs lower substantially and the form factor improves.
- AR/MR headsets are geeky looking and will be used only for specific applications; they will not be meant for everyday use in the foreseeable future.
Sources:
