Using Augmented Reality (AR) as Assistive Technology

Apple had released their ARKit upon the world which allows developers to create apps which take advantage of the iOS device’s camera and sensors to help create experiences blending the real world and digital content. These apps will work on devices supported by the A9 chip and running iOS 11. There are some AR experiences, however, that you can build right now using what you have.


Metaverse is an AR experience building platform with a fairly user-friendly studio that can be run from your web browser. You can request access through their main landing page and start building. Pros of using Metaverse include their companion apps for both Andriod and iOS as well as a friendly onramp. They have also recently added blocks of code especially those using Google Vision which attempts to identify objects, text, or emotions in the picture taken by the user. For more advanced users, Metaverse supports 3rd party APIs.

metaverse studio layout
Metaverse Studio Layout


ARIS has been around for some time and came out of Wisconsin University and is highly robust. This platform allows you to build entire experiences that can be triggered by QR codes, locations, or alphanumeric codes. They also recently released an AR component. User accounts are free and you can start building right away. ARIS is also well documented with support forums available. Unfortunately, there is currently only an iOS app available for accessing the content you created.

ARIS studio layout
ARIS Studio Layout

Possible Uses

Beyond creating fun scavenger hunts and games, these tools can also be used to help support independent transportation in the community. Support cues and triggers can be embedded along a user’s bus route or walking path. These tools can also help support data tracking and enhance engagement using a badge and token system. The use of QR triggers within the home or school can be used to activate video modeling of directions or sequence of steps.

What About VR?

Virtual reality has also entered the world of rehabilitation and regaining function. Some of its uses have included creating memorable experiences for those with dementia and simulating movement experiences for those with mobility impairments. RAPAEL, which created a hand rehabilitation glove is now blending VR exercises with a sensor based glove to detect finger, wrist, and forearm movements which impact the VR world. Here is research study they cite: Effects of virtual reality- based rehabilitation on distal upper extremity function and health- related quality of life: a single-blinded, randomized controlled trial. 

A chapter on VR and Occupational Therapy is also included as part of the Occupational Therapy – Occupation Focused Holistic Practice in Rehabilitation open textbook.

It’s a brave new world for assistive technology. Time to dive in.



Autodraw: Word Prediction for Drawing

autodraw iconGoogle has recently released the Autodraw AI for some fun and games for the casual artist. This tool, however, can serve as power assistive technology for students with disabilities who struggle with drawing and expressing, themselves especially in the early pre-writing days. For those who haven’t tried it out, Autodraw lets the user start to draw some basic shapes or lines and the AI will attempt to guess what the user is trying to draw providing options of icons. The user can then tap on the desired icon to insert it into their drawing.

For young learners, especially those with visual- motor, perceptual or motor planning difficulties drawing can be quite a frustrating challenge. Most early grade writing standards have some requirement of drawing and labeling the drawing based on a classroom topic or reading. The Autodraw tool can be a great support. Just check out the results:

drawing attempt of a banana
Original drawing
icon drawing of a banana
Icon selection based on the drawing.