Skip to content

Latest commit

 

History

History
161 lines (146 loc) · 49.4 KB

Augmented_reality.md

File metadata and controls

161 lines (146 loc) · 49.4 KB
Raw

  • Virtual Fixtures – first A.R. system, 1992, U.S. Air Force, WPAFB * Augmented Reality (AR) is an interactive experience of a real-world environment whereby the objects that reside in the real-world are "augmented" by computer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory, and olfactory.The overlaid sensory information can be constructive (i.e. additive to the natural environment) or destructive (i.e. masking of the natural environment) and is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment. In this way, augmented reality alters one’s ongoing perception of a real world environment, whereas virtual reality completely replaces the user's real world environment with a simulated one.[3][4] Augmented reality is related to two largely synonymous terms: mixed reality and computer-mediated reality. The primary value of augmented reality is that it brings components of the digital world into a person's perception of the real world, and does so not as a simple display of data, but through the integration of immersive sensations that are perceived as natural parts of an environment. The first functional AR systems that provided immersive mixed reality experiences for users were invented in the early 1990s, starting with the Virtual Fixtures system developed at the U.S. Air Force's Armstrong Laboratory in 1992.[2][5][6][7] The first commercial augmented reality experiences were used largely in the entertainment and gaming businesses, but now other industries are also getting interested about AR's possibilities for example in knowledge sharing, educating, managing the information flood and organizing distant meetings. Augmented reality is also transforming the world of education, where content may be accessed by scanning or viewing an image with a mobile device.[8] Another example is an AR helmet for construction workers which display information about the construction sites. Augmented Reality (AR) is used to enhance natural environments or situations and offer perceptually enriched experiences. With the help of advanced AR technologies (e.g. adding computer vision and object recognition) the information about the surrounding real world of the user becomes interactive and digitally manipulable. Information about the environment and its objects is overlaid on the real world. This information can be virtual or real, e.g. seeing other real sensed or measured information such as electromagnetic radio waves overlaid in exact alignment with where they actually are in space.[15][16][17] Augmented reality also has a lot of potential in the gathering and sharing of tacit knowledge. Augmentation techniques are typically performed in real time and in semantic context with environmental elements. Immersive perceptual information is sometimes combined with supplemental information like scores over a live video feed of a sporting event. This combines the benefits of both augmented reality technology and heads up display technology (HUD).

  • Technology *

Hardware Hardware components for augmented reality are: processor, display, sensors and input devices. Modern mobile computing devices like smartphones and tablet computers contain these elements which often include a camera and MEMS sensors such as accelerometer, GPS, and solid state compass, making them suitable AR platforms. Display Various technologies are used in augmented reality rendering, including optical projection systems, monitors, handheld devices, and display systems worn on the human body. A head-mounted display (HMD) is a display device worn on the forehead, such as a harness or helmet. HMDs place images of both the physical world and virtual objects over the user's field of view. Modern HMDs often employ sensors for six degrees of freedom monitoring that allow the system to align virtual information to the physical world and adjust accordingly with the user's head movements. HMDs can provide VR users with mobile and collaborative experiences. Specific providers, such as uSens and Gestigon, include gesture controls for full virtual immersion. In January 2015, Meta launched a project led by Horizons Ventures, Tim Draper, Alexis Ohanian, BOE Optoelectronics and Garry Tan.On February 17, 2016, Meta announced their second-generation product at TED, Meta 2. The Meta 2 head-mounted display headset uses a sensory array for hand interactions and positional tracking, visual field view of 90 degrees (diagonal), and resolution display of 2560 x 1440 (20 pixels per degree), which is considered the largest field of view (FOV) currently available. Eyeglasses Vuzix AR3000 AugmentedReality SmartGlasses AR displays can be rendered on devices resembling eyeglasses. Versions include eyewear that employs cameras to intercept the real world view and re-display its augmented view through the eyepieces and devices in which the AR imagery is projected through or reflected off the surfaces of the eyewear's lenspieces. HUD

Headset computer A head-up display (HUD) is a transparent display that presents data without requiring users to look away from their usual viewpoints. A precursor technology to augmented reality, heads-up displays were first developed for pilots in the 1950s, projecting simple flight data into their line of sight, thereby enabling them to keep their "heads up" and not look down at the instruments. Near-eye augmented reality devices can be used as portable head-up displays as they can show data, information, and images while the user views the real world. Many definitions of augmented reality only define it as overlaying the information.This is basically what a head-up display does; however, practically speaking, augmented reality is expected to include registration and tracking between the superimposed perceptions, sensations, information, data, and images and some portion of the real world. CrowdOptic, an existing app for smartphones, applies algorithms and triangulation techniques to photo metadata including GPS position, compass heading, and a time stamp to arrive at a relative significance value for photo objects.[39] CrowdOptic technology can be used by Google Glass users to learn where to look at a given point in time.[40] A number of smartglasses have been launched for augmented reality. Due to encumbered control, smartglasses are primarily designed for micro-interaction like reading a text message but still far from more well-rounded applications of augmented reality. In January 2015, Microsoft introduced HoloLens, an independent smartglasses unit. Brian Blau, Research Director of Consumer Technology and Markets at Gartner, said that "Out of all the head-mounted displays that I've tried in the past couple of decades, the HoloLens was the best in its class." First impressions and opinions were generally that HoloLens is a superior device to the Google Glass, and manages to do several things "right" in which Glass failed. Contact lenses Contact lenses that display AR imaging are in development. These bionic contact lenses might contain the elements for display embedded into the lens including integrated circuitry, LEDs and an antenna for wireless communication. The first contact lens display was reported in 1999, then 11 years later in 2010-2011. Another version of contact lenses, in development for the U.S. military, is designed to function with AR spectacles, allowing soldiers to focus on close-to-the-eye AR images on the spectacles and distant real world objects at the same time. The futuristic short film Sight features contact lens-like augmented reality devices. Many scientists have been working on contact lenses capable of many different technological feats. The company Samsung has been working on a contact lens as well. This lens, when finished, is meant to have a built-in camera on the lens itself. The design is intended to have you blink to control its interface for recording purposes. It is also intended to be linked with your smartphone to review footage, and control it separately. When successful, the lens would feature a camera, or sensor inside of it. It is said that it could be anything from a light sensor, to a temperature sensor. In Augmented Reality, the distinction is made between two distinct modes of tracking, known as ''marker'' and ''markerless''. Marker are visual cues which trigger the display of the virtual information.A piece of paper with some distinct geometries can be used. The camera recognizes the geometries by identifying specific points in the drawing. Markerless also called instant tracking does not use marker. Instead the user positions the object in the camera view preferably in an horizontal plane.It uses sensors in mobile devices to accurately detect the real-world environment, such as the locations of walls and points of intersection. Virtual retinal display A virtual retinal display (VRD) is a personal display device under development at the University of Washington's Human Interface Technology Laboratory under Dr. Thomas A. Furness III. With this technology, a display is scanned directly onto the retina of a viewer's eye. This results in bright images with high revolution and high contrast. The viewer sees what appears to be a conventional display floating in space. Several of tests were done in order to analyze the safety of the VRD. In one test, patients with partial loss of vision were selected to view images using the technology having either macular degeneration (a disease that degenerates the retina) or keratoconus. In the macular degeneration group, 5 out of 8 subjects preferred the VRD images to the CRT or paper images and thought they were better and brighter and were able to see equal or better resolution levels. The Kerocunus patients could all resolve smaller lines in several line tests using the VDR as opposed to their own correction. They also found the VDR images to be easier to view and sharper. As a result of these several tests, virtual retinal display is considered safe technology. Virtual retinal display creates images that can be seen in ambient daylight and ambient roomlight. The VRD is considered a preferred candidate to use in a surgical display due to its combination of high resolution and high contrast and brightness. Additional tests show high potential for VRD to be used as a display technology for patients that have low vision. EyeTap The EyeTap (also known as Generation-2 Glass) captures rays of light that would otherwise pass through the center of the lens of the eye of the wearer, and substitutes synthetic computer-controlled light for each ray of real light. The Generation-4 Glass(Laser EyeTap) is similar to the VRD (i.e. it uses a computer-controlled laser light source) except that it also has infinite depth of focus and causes the eye itself to, in effect, function as both a camera and a display by way of exact alignment with the eye and resynthesis (in laser light) of rays of light entering the eye. Handheld A Handheld display employs a small display that fits in a user's hand. All handheld AR solutions to date opt for video see-through. Initially handheld AR employed fiducial markers, and later GPS units and MEMS sensors such as digital compasses and six degrees of freedom accelerometer–gyroscope. Today SLAM markerless trackers such as PTAM are starting to come into use. Handheld display AR promises to be the first commercial success for AR technologies. The two main advantages of handheld AR are the portable nature of handheld devices and the ubiquitous nature of camera phones. The disadvantages are the physical constraints of the user having to hold the handheld device out in front of them at all times, as well as the distorting effect of classically wide-angled mobile phone cameras when compared to the real world as viewed through the eye.[62] The issues arising from the user having to hold the handheld device (manipulability) and perceiving the visualisation correctly (comprehensibility) have been summarised into the HARUS usability questionnaire. Games such as Pokémon Go and Ingress utilize an Image Linked Map (ILM) interface, where approved geotagged locations appear on a stylized map for the user to interact with. Spatial Spatial augmented reality (SAR) augments real-world objects and scenes without the use of special displays such as monitors, head-mounted displays or hand-held devices. SAR makes use of digital projectors to display graphical information onto physical objects. The key difference in SAR is that the display is separated from the users of the system. Because the displays are not associated with each user, SAR scales naturally up to groups of users, thus allowing for collocated collaboration between users. Examples include shader lamps, mobile projectors, virtual tables, and smart projectors. Shader lamps mimic and augment reality by projecting imagery onto neutral objects, providing the opportunity to enhance the object's appearance with materials of a simple unit - a projector, camera, and sensor. Other applications include table and wall projections. One innovation, the Extended Virtual Table, separates the virtual from the real by including beam-splitter mirrors attached to the ceiling at an adjustable angle. Virtual showcases, which employ beam-splitter mirrors together with multiple graphics displays, provide an interactive means of simultaneously engaging with the virtual and the real. Many more implementations and configurations make spatial augmented reality display an increasingly attractive interactive alternative. An SAR system can display on any number of surfaces of an indoor setting at once. SAR supports both a graphical visualization and passive haptic sensation for the end users. Users are able to touch physical objects in a process that provides passive haptic sensation. Tracking Modern mobile augmented-reality systems use one or more of the following tracking technologies: digital cameras and/or other optical sensors, accelerometers, GPS, gyroscopes, solid state compasses, RFID. These technologies offer varying levels of accuracy and precision. The most important is the position and orientation of the user's head. Tracking the user's hand(s) or a handheld input device can provide a 6DOF interaction technique. Networking Mobile augmented reality applications are gaining popularity due to the wide adoption of mobile and especially wearable devices. However, they often rely on computationally intensive computer vision algorithms with extreme latency requirements. To compensate for the lack of computing power, offloading data processing to a distant machine is often desired. Computation offloading introduces new constraints in applications, especially in terms of latency and bandwidth. Although there are a plethora of real-time multimedia transport protocols, there is a need for support from network infrastructure as well. Input devices Techniques include speech recognition systems that translate a user's spoken words into computer instructions, and gesture recognition systems that interpret a user's body movements by visual detection or from sensors embedded in a peripheral device such as a wand, stylus, pointer, glove or other body wear. Products which are trying to serve as a controller of AR headsets include Wave by Seebright Inc. and Nimble by Intugine Technologies. Computer The computer analyzes the sensed visual and other data to synthesize and position augmentations. Computers are responsible for the graphics that go with augmented reality. Augmented reality uses a computer-generated image and it has an striking effect on the way the real world is shown. With the improvement of technology and computers, augmented reality is going to have a drastic change on our perspective of the real world.[76] According to Time Magazine, in about 15–20 years it is predicted that Augmented reality and virtual reality are going to become the primary use for computer interactions. Computers are improving at a very fast rate, which means that we are figuring out new ways to improve other technology. The more that computers progress, augmented reality will become more flexible and more common in our society. Computers are the core of augmented reality. The Computer receives data from the sensors which determine the relative position of objects surface. This translates to an input to the computer which then outputs to the users by adding something that would otherwise not be there. The computer comprises memory and a processor. The computer takes the scanned environment then generates images or a video and puts it on the receiver for the observer to see. The fixed marks on an objects surface are stored in the memory of a computer. The computer also withdrawals from its memory to present images realistically to the onlooker. The best example of this is of the Pepsi Max AR Bus Shelter.[ Software and algorithms A key measure of AR systems is how realistically they integrate augmentations with the real world. The software must derive real world coordinates, independent from the camera, from camera images. That process is called image registration, and uses different methods of computer vision, mostly related to video tracking.Many computer vision methods of augmented reality are inherited from visual odometry. Usually those methods consist of two parts. The first stage is to detect interest points, fiducial markers or optical flow in the camera images. This step can use feature detection methods like corner detection, blob detection, edge detection or thresholding, and other image processing methods.The second stage restores a real world coordinate system from the data obtained in the first stage. Some methods assume objects with known geometry (or fiducial markers) are present in the scene. In some of those cases the scene 3D structure should be precalculated beforehand. If part of the scene is unknown simultaneous localization and mapping (SLAM) can map relative positions. If no information about scene geometry is available, structure from motion methods like bundle adjustment are used. Mathematical methods used in the second stage include projective (epipolar) geometry, geometric algebra, rotation representation with exponential map, kalman and particle filters, nonlinear optimization, robust statistics. Augmented Reality Markup Language (ARML) is a data standard developed within the Open Geospatial Consortium (OGC),which consists of XML grammar to describe the location and appearance of virtual objects in the scene, as well as ECMAScript bindings to allow dynamic access to properties of virtual objects. To enable rapid development of augmented reality applications, some software development kits (SDKs) have emerged. A few SDKs such as CloudRidAR leverage cloud computing for performance improvement. AR SDKs are offered by Vuforia,[89] ARToolKit, Catchoom CraftAR[90] Mobinett AR, Wikitude,Blippar Layar, Meta.and ARLab. Development The implementation of Augmented Reality in consumer products requires considering the design of the applications and the related constraints of the technology platform. Since AR system rely heavily on the immersion of the user and the interaction between the user and the system, design can facilitate the adoption of virtuality. For most Augmented Reality systems, a similar design guideline can be followed. The following lists some considerations for designing Augmented Reality applications: Environmental/context design Context Design focuses on the end-user’s physical surrounding, spatial space, and accessibility that may play a role when using the AR system. Designers should be aware of the possible physical scenarios the end-user may be in such as: Public, in which the users uses their whole body to interact with the software Personal, in which the user uses a smartphone in a public space Intimate, in which the user is sitting with a desktop and is not really in movement Private, in which the user has on a wearable. By evaluating each physical scenario, potential safety hazard can be avoided and changes can be made to greater improve the end-user’s immersion. UX designers will have to define user journeys for the relevant physical scenarios and define how the interface will react to each. Especially in AR systems, it is vital to also consider the spatial space and the surrounding elements that change the effectiveness of the AR technology. Environmental elements such as lighting, and sound can prevent the sensor of AR devices from detecting necessary data and ruin the immersion of the end-user. Another aspect of context design involves the design of the system’s functionality and its ability to accommodate for user preferences.[100][101] While accessibility tools are common in basic application design, some consideration should be made when designing time-limited prompts (to prevent unintentional operations), audio cues and overall engagement time. It is important to note that in some situations, the application’s functionality may hinder the user’s ability. For example, applications that is used for driving should reduce the amount of user interaction and user audio cues instead. Interaction design Interaction design in augmented reality technology centers on the user’s engagement with the end product to improve the overall user experience and enjoyment. The purpose of Interaction Design is to avoid alienating or confusing the user by organising the information presented. Since user interaction relies on the user’s input, designers must make system controls easier to understand and accessible. A common technique to improve usability for augmented reality applications is by discovering the frequently accessed areas in the device’s touch display and design the application to match those areas of control.[102] It is also important to structure the user journey maps and the flow of information presented which reduce the system’s overall cognitive load and greatly improves the learning curve of the application.[103] In interaction design, it is important for developers to utilize augmented reality technology that complement the system’s function or purpose.[104] For instance, the utilization of exciting AR filters and the design of the unique sharing platform in Snapchat enables users to better the user’s social interactions. In other applications that require users to understand the focus and intent, designers can employ a reticle or raycast from the device.[100] Moreover, augmented reality developers may find it appropriate to have digital elements scale or react to the direction of the camera and the context of objects that can are detected.[99]
The most exciting factor of augmented reality technology is the ability to utilize the introduction of 3D space. This means that a user can potentially access multiple copies of 2D interfaces within a single AR application.[99] AR applications are collaborative, a user can also connect to another’s device and view or manipulate virtual objects in the other person’s context. Visual design In general, visual design is the appearance of the developing application that engages the user. To improve the graphic interface elements and user interaction, developers may use visual cues to inform user what elements of UI are designed to interact with and how to interact with them. Since navigating in AR application may appear difficult and seem frustrating, visual cues design can make interactions seem more natural. In some augmented reality applications that uses a 2D device as an interactive surface, the 2D control environment does not translate well in 3D space making users hesitant to explore their surroundings. To solve this issue, designers should apply visual cues to assist and encourage users to explore their surroundings. It is important to note the two main objects in AR when developing VR applications: 3D volumetric objects that are manipulatable and realistically interact with light and shadow; and animated media imagery such as images and videos which are mostly traditional 2D media rendered in a new context for augmented reality.[98] When virtual objects are projected onto a real environment, it is challenging for augmented reality application designers to ensure a perfectly seamless integration relative to the real world environment, especially with 2D objects. As such, designers can add weight to objects, use depths maps, and choose different material properties that highlight the object’s presence in the real world. Another visual design that can be applied is using different lighting techniques or casting shadows to improve overall depth judgment. For instance, a common lighting technique is simply placing a light source overhead at the 12 o’clock position, to create shadows upon virtual objects.* Virtual Fixtures – first A.R. system, 1992, U.S. Air Force, WPAFB * Augmented Reality (AR) is an interactive experience of a real-world environment whereby the objects that reside in the real-world are "augmented" by computer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory, and olfactory.The overlaid sensory information can be constructive (i.e. additive to the natural environment) or destructive (i.e. masking of the natural environment) and is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment. In this way, augmented reality alters one’s ongoing perception of a real world environment, whereas virtual reality completely replaces the user's real world environment with a simulated one.[3][4] Augmented reality is related to two largely synonymous terms: mixed reality and computer-mediated reality. The primary value of augmented reality is that it brings components of the digital world into a person's perception of the real world, and does so not as a simple display of data, but through the integration of immersive sensations that are perceived as natural parts of an environment. The first functional AR systems that provided immersive mixed reality experiences for users were invented in the early 1990s, starting with the Virtual Fixtures system developed at the U.S. Air Force's Armstrong Laboratory in 1992.[2][5][6][7] The first commercial augmented reality experiences were used largely in the entertainment and gaming businesses, but now other industries are also getting interested about AR's possibilities for example in knowledge sharing, educating, managing the information flood and organizing distant meetings. Augmented reality is also transforming the world of education, where content may be accessed by scanning or viewing an image with a mobile device.[8] Another example is an AR helmet for construction workers which display information about the construction sites. Augmented Reality (AR) is used to enhance natural environments or situations and offer perceptually enriched experiences. With the help of advanced AR technologies (e.g. adding computer vision and object recognition) the information about the surrounding real world of the user becomes interactive and digitally manipulable. Information about the environment and its objects is overlaid on the real world. This information can be virtual or real, e.g. seeing other real sensed or measured information such as electromagnetic radio waves overlaid in exact alignment with where they actually are in space.[15][16][17] Augmented reality also has a lot of potential in the gathering and sharing of tacit knowledge. Augmentation techniques are typically performed in real time and in semantic context with environmental elements. Immersive perceptual information is sometimes combined with supplemental information like scores over a live video feed of a sporting event. This combines the benefits of both augmented reality technology and heads up display technology (HUD).

  • Technology *

Hardware Hardware components for augmented reality are: processor, display, sensors and input devices. Modern mobile computing devices like smartphones and tablet computers contain these elements which often include a camera and MEMS sensors such as accelerometer, GPS, and solid state compass, making them suitable AR platforms. Display Various technologies are used in augmented reality rendering, including optical projection systems, monitors, handheld devices, and display systems worn on the human body. A head-mounted display (HMD) is a display device worn on the forehead, such as a harness or helmet. HMDs place images of both the physical world and virtual objects over the user's field of view. Modern HMDs often employ sensors for six degrees of freedom monitoring that allow the system to align virtual information to the physical world and adjust accordingly with the user's head movements. HMDs can provide VR users with mobile and collaborative experiences. Specific providers, such as uSens and Gestigon, include gesture controls for full virtual immersion. In January 2015, Meta launched a project led by Horizons Ventures, Tim Draper, Alexis Ohanian, BOE Optoelectronics and Garry Tan.On February 17, 2016, Meta announced their second-generation product at TED, Meta 2. The Meta 2 head-mounted display headset uses a sensory array for hand interactions and positional tracking, visual field view of 90 degrees (diagonal), and resolution display of 2560 x 1440 (20 pixels per degree), which is considered the largest field of view (FOV) currently available. Eyeglasses Vuzix AR3000 AugmentedReality SmartGlasses AR displays can be rendered on devices resembling eyeglasses. Versions include eyewear that employs cameras to intercept the real world view and re-display its augmented view through the eyepieces and devices in which the AR imagery is projected through or reflected off the surfaces of the eyewear's lenspieces. HUD

Headset computer A head-up display (HUD) is a transparent display that presents data without requiring users to look away from their usual viewpoints. A precursor technology to augmented reality, heads-up displays were first developed for pilots in the 1950s, projecting simple flight data into their line of sight, thereby enabling them to keep their "heads up" and not look down at the instruments. Near-eye augmented reality devices can be used as portable head-up displays as they can show data, information, and images while the user views the real world. Many definitions of augmented reality only define it as overlaying the information.This is basically what a head-up display does; however, practically speaking, augmented reality is expected to include registration and tracking between the superimposed perceptions, sensations, information, data, and images and some portion of the real world. CrowdOptic, an existing app for smartphones, applies algorithms and triangulation techniques to photo metadata including GPS position, compass heading, and a time stamp to arrive at a relative significance value for photo objects.[39] CrowdOptic technology can be used by Google Glass users to learn where to look at a given point in time.[40] A number of smartglasses have been launched for augmented reality. Due to encumbered control, smartglasses are primarily designed for micro-interaction like reading a text message but still far from more well-rounded applications of augmented reality. In January 2015, Microsoft introduced HoloLens, an independent smartglasses unit. Brian Blau, Research Director of Consumer Technology and Markets at Gartner, said that "Out of all the head-mounted displays that I've tried in the past couple of decades, the HoloLens was the best in its class." First impressions and opinions were generally that HoloLens is a superior device to the Google Glass, and manages to do several things "right" in which Glass failed. Contact lenses Contact lenses that display AR imaging are in development. These bionic contact lenses might contain the elements for display embedded into the lens including integrated circuitry, LEDs and an antenna for wireless communication. The first contact lens display was reported in 1999, then 11 years later in 2010-2011. Another version of contact lenses, in development for the U.S. military, is designed to function with AR spectacles, allowing soldiers to focus on close-to-the-eye AR images on the spectacles and distant real world objects at the same time. The futuristic short film Sight features contact lens-like augmented reality devices. Many scientists have been working on contact lenses capable of many different technological feats. The company Samsung has been working on a contact lens as well. This lens, when finished, is meant to have a built-in camera on the lens itself. The design is intended to have you blink to control its interface for recording purposes. It is also intended to be linked with your smartphone to review footage, and control it separately. When successful, the lens would feature a camera, or sensor inside of it. It is said that it could be anything from a light sensor, to a temperature sensor.

In Augmented Reality, the distinction is made between two distinct modes of tracking, known as ''marker'' and ''markerless''. Marker are visual cues which trigger the display of the virtual information.A piece of paper with some distinct geometries can be used. The camera recognizes the geometries by identifying specific points in the drawing. Markerless also called instant tracking does not use marker. Instead the user positions the object in the camera view preferably in an horizontal plane.It uses sensors in mobile devices to accurately detect the real-world environment, such as the locations of walls and points of intersection. Virtual retinal display A virtual retinal display (VRD) is a personal display device under development at the University of Washington's Human Interface Technology Laboratory under Dr. Thomas A. Furness III. With this technology, a display is scanned directly onto the retina of a viewer's eye. This results in bright images with high revolution and high contrast. The viewer sees what appears to be a conventional display floating in space. Several of tests were done in order to analyze the safety of the VRD. In one test, patients with partial loss of vision were selected to view images using the technology having either macular degeneration (a disease that degenerates the retina) or keratoconus. In the macular degeneration group, 5 out of 8 subjects preferred the VRD images to the CRT or paper images and thought they were better and brighter and were able to see equal or better resolution levels. The Kerocunus patients could all resolve smaller lines in several line tests using the VDR as opposed to their own correction. They also found the VDR images to be easier to view and sharper. As a result of these several tests, virtual retinal display is considered safe technology. Virtual retinal display creates images that can be seen in ambient daylight and ambient roomlight. The VRD is considered a preferred candidate to use in a surgical display due to its combination of high resolution and high contrast and brightness. Additional tests show high potential for VRD to be used as a display technology for patients that have low vision. EyeTap The EyeTap (also known as Generation-2 Glass) captures rays of light that would otherwise pass through the center of the lens of the eye of the wearer, and substitutes synthetic computer-controlled light for each ray of real light. The Generation-4 Glass(Laser EyeTap) is similar to the VRD (i.e. it uses a computer-controlled laser light source) except that it also has infinite depth of focus and causes the eye itself to, in effect, function as both a camera and a display by way of exact alignment with the eye and resynthesis (in laser light) of rays of light entering the eye. Handheld A Handheld display employs a small display that fits in a user's hand. All handheld AR solutions to date opt for video see-through. Initially handheld AR employed fiducial markers, and later GPS units and MEMS sensors such as digital compasses and six degrees of freedom accelerometer–gyroscope. Today SLAM markerless trackers such as PTAM are starting to come into use. Handheld display AR promises to be the first commercial success for AR technologies. The two main advantages of handheld AR are the portable nature of handheld devices and the ubiquitous nature of camera phones. The disadvantages are the physical constraints of the user having to hold the handheld device out in front of them at all times, as well as the distorting effect of classically wide-angled mobile phone cameras when compared to the real world as viewed through the eye.[62] The issues arising from the user having to hold the handheld device (manipulability) and perceiving the visualisation correctly (comprehensibility) have been summarised into the HARUS usability questionnaire. Games such as Pokémon Go and Ingress utilize an Image Linked Map (ILM) interface, where approved geotagged locations appear on a stylized map for the user to interact with. Spatial Spatial augmented reality (SAR) augments real-world objects and scenes without the use of special displays such as monitors, head-mounted displays or hand-held devices. SAR makes use of digital projectors to display graphical information onto physical objects. The key difference in SAR is that the display is separated from the users of the system. Because the displays are not associated with each user, SAR scales naturally up to groups of users, thus allowing for collocated collaboration between users. Examples include shader lamps, mobile projectors, virtual tables, and smart projectors. Shader lamps mimic and augment reality by projecting imagery onto neutral objects, providing the opportunity to enhance the object's appearance with materials of a simple unit - a projector, camera, and sensor. Other applications include table and wall projections. One innovation, the Extended Virtual Table, separates the virtual from the real by including beam-splitter mirrors attached to the ceiling at an adjustable angle. Virtual showcases, which employ beam-splitter mirrors together with multiple graphics displays, provide an interactive means of simultaneously engaging with the virtual and the real. Many more implementations and configurations make spatial augmented reality display an increasingly attractive interactive alternative. An SAR system can display on any number of surfaces of an indoor setting at once. SAR supports both a graphical visualization and passive haptic sensation for the end users. Users are able to touch physical objects in a process that provides passive haptic sensation. Tracking Modern mobile augmented-reality systems use one or more of the following tracking technologies: digital cameras and/or other optical sensors, accelerometers, GPS, gyroscopes, solid state compasses, RFID. These technologies offer varying levels of accuracy and precision. The most important is the position and orientation of the user's head. Tracking the user's hand(s) or a handheld input device can provide a 6DOF interaction technique. Networking Mobile augmented reality applications are gaining popularity due to the wide adoption of mobile and especially wearable devices. However, they often rely on computationally intensive computer vision algorithms with extreme latency requirements. To compensate for the lack of computing power, offloading data processing to a distant machine is often desired. Computation offloading introduces new constraints in applications, especially in terms of latency and bandwidth. Although there are a plethora of real-time multimedia transport protocols, there is a need for support from network infrastructure as well. Input devices Techniques include speech recognition systems that translate a user's spoken words into computer instructions, and gesture recognition systems that interpret a user's body movements by visual detection or from sensors embedded in a peripheral device such as a wand, stylus, pointer, glove or other body wear. Products which are trying to serve as a controller of AR headsets include Wave by Seebright Inc. and Nimble by Intugine Technologies. Computer The computer analyzes the sensed visual and other data to synthesize and position augmentations. Computers are responsible for the graphics that go with augmented reality. Augmented reality uses a computer-generated image and it has an striking effect on the way the real world is shown. With the improvement of technology and computers, augmented reality is going to have a drastic change on our perspective of the real world. According to Time Magazine, in about 15–20 years it is predicted that Augmented reality and virtual reality are going to become the primary use for computer interactions. Computers are improving at a very fast rate, which means that we are figuring out new ways to improve other technology. The more that computers progress, augmented reality will become more flexible and more common in our society. Computers are the core of augmented reality. The Computer receives data from the sensors which determine the relative position of objects surface. This translates to an input to the computer which then outputs to the users by adding something that would otherwise not be there. The computer comprises memory and a processor. The computer takes the scanned environment then generates images or a video and puts it on the receiver for the observer to see. The fixed marks on an objects surface are stored in the memory of a computer. The computer also withdrawals from its memory to present images realistically to the onlooker. The best example of this is of the Pepsi Max AR Bus Shelter.[ Software and algorithms A key measure of AR systems is how realistically they integrate augmentations with the real world. The software must derive real world coordinates, independent from the camera, from camera images. That process is called image registration, and uses different methods of computer vision, mostly related to video tracking.Many computer vision methods of augmented reality are inherited from visual odometry. Usually those methods consist of two parts. The first stage is to detect interest points, fiducial markers or optical flow in the camera images. This step can use feature detection methods like corner detection, blob detection, edge detection or thresholding, and other image processing methods.The second stage restores a real world coordinate system from the data obtained in the first stage. Some methods assume objects with known geometry (or fiducial markers) are present in the scene. In some of those cases the scene 3D structure should be precalculated beforehand. If part of the scene is unknown simultaneous localization and mapping (SLAM) can map relative positions. If no information about scene geometry is available, structure from motion methods like bundle adjustment are used. Mathematical methods used in the second stage include projective (epipolar) geometry, geometric algebra, rotation representation with exponential map, kalman and particle filters, nonlinear optimization, robust statistics. Augmented Reality Markup Language (ARML) is a data standard developed within the Open Geospatial Consortium (OGC),which consists of XML grammar to describe the location and appearance of virtual objects in the scene, as well as ECMAScript bindings to allow dynamic access to properties of virtual objects. To enable rapid development of augmented reality applications, some software development kits (SDKs) have emerged. A few SDKs such as CloudRidAR leverage cloud computing for performance improvement. AR SDKs are offered by Vuforia,[89] ARToolKit, Catchoom CraftAR[90] Mobinett AR, Wikitude,Blippar Layar, Meta.and ARLab. Development The implementation of Augmented Reality in consumer products requires considering the design of the applications and the related constraints of the technology platform. Since AR system rely heavily on the immersion of the user and the interaction between the user and the system, design can facilitate the adoption of virtuality. For most Augmented Reality systems, a similar design guideline can be followed. The following lists some considerations for designing Augmented Reality applications: Environmental/context design Context Design focuses on the end-user’s physical surrounding, spatial space, and accessibility that may play a role when using the AR system. Designers should be aware of the possible physical scenarios the end-user may be in such as: Public, in which the users uses their whole body to interact with the software Personal, in which the user uses a smartphone in a public space Intimate, in which the user is sitting with a desktop and is not really in movement Private, in which the user has on a wearable. By evaluating each physical scenario, potential safety hazard can be avoided and changes can be made to greater improve the end-user’s immersion. UX designers will have to define user journeys for the relevant physical scenarios and define how the interface will react to each. Especially in AR systems, it is vital to also consider the spatial space and the surrounding elements that change the effectiveness of the AR technology. Environmental elements such as lighting, and sound can prevent the sensor of AR devices from detecting necessary data and ruin the immersion of the end-user. Another aspect of context design involves the design of the system’s functionality and its ability to accommodate for user preferences.[100][101] While accessibility tools are common in basic application design, some consideration should be made when designing time-limited prompts (to prevent unintentional operations), audio cues and overall engagement time. It is important to note that in some situations, the application’s functionality may hinder the user’s ability. For example, applications that is used for driving should reduce the amount of user interaction and user audio cues instead. Interaction design Interaction design in augmented reality technology centers on the user’s engagement with the end product to improve the overall user experience and enjoyment. The purpose of Interaction Design is to avoid alienating or confusing the user by organising the information presented. Since user interaction relies on the user’s input, designers must make system controls easier to understand and accessible. A common technique to improve usability for augmented reality applications is by discovering the frequently accessed areas in the device’s touch display and design the application to match those areas of control.[102] It is also important to structure the user journey maps and the flow of information presented which reduce the system’s overall cognitive load and greatly improves the learning curve of the application.[103] In interaction design, it is important for developers to utilize augmented reality technology that complement the system’s function or purpose.[104] For instance, the utilization of exciting AR filters and the design of the unique sharing platform in Snapchat enables users to better the user’s social interactions. In other applications that require users to understand the focus and intent, designers can employ a reticle or raycast from the device.Moreover, augmented reality developers may find it appropriate to have digital elements scale or react to the direction of the camera and the context of objects that can are detected.[99]
The most exciting factor of augmented reality technology is the ability to utilize the introduction of 3D space. This means that a user can potentially access multiple copies of 2D interfaces within a single AR application.[99] AR applications are collaborative, a user can also connect to another’s device and view or manipulate virtual objects in the other person’s context. Visual design In general, visual design is the appearance of the developing application that engages the user. To improve the graphic interface elements and user interaction, developers may use visual cues to inform user what elements of UI are designed to interact with and how to interact with them. Since navigating in AR application may appear difficult and seem frustrating, visual cues design can make interactions seem more natural. In some augmented reality applications that uses a 2D device as an interactive surface, the 2D control environment does not translate well in 3D space making users hesitant to explore their surroundings. To solve this issue, designers should apply visual cues to assist and encourage users to explore their surroundings. It is important to note the two main objects in AR when developing VR applications: 3D volumetric objects that are manipulatable and realistically interact with light and shadow; and animated media imagery such as images and videos which are mostly traditional 2D media rendered in a new context for augmented reality.[98] When virtual objects are projected onto a real environment, it is challenging for augmented reality application designers to ensure a perfectly seamless integration relative to the real world environment, especially with 2D objects. As such, designers can add weight to objects, use depths maps, and choose different material properties that highlight the object’s presence in the real world. Another visual design that can be applied is using different lighting techniques or casting shadows to improve overall depth judgment. For instance, a common lighting technique is simply placing a light source overhead at the 12 o’clock position, to create shadows upon virtual objects. img src="https://www.bing.com/images/search?view=detailV2&ccid=tk6N6jCh&id=1AF92216EC7041CDCA5B7DF71AA2D0C961DF192D&thid=OIP.hbFWBc3nAz2hsef1A8nWUQHaEc&mediaurl=http%3a%2f%2fstatic.guim.co.uk%2fsys-images%2fGuardian%2fPix%2fpictures%2f2010%2f3%2f20%2f1269125712641%2fAugmented-reality-001.jpg&exph=276&expw=460&q=augmented+reality++images&simid=608006727057345054&selectedIndex=2" Arnab Shrivastav [email protected]