Using Augmented Reality (AR) as Assistive Technology

Apple had released their ARKit upon the world which allows developers to create apps which take advantage of the iOS device’s camera and sensors to help create experiences blending the real world and digital content. These apps will work on devices supported by the A9 chip and running iOS 11. There are some AR experiences, however, that you can build right now using what you have.


Metaverse is an AR experience building platform with a fairly user-friendly studio that can be run from your web browser. You can request access through their main landing page and start building. Pros of using Metaverse include their companion apps for both Andriod and iOS as well as a friendly onramp. They have also recently added blocks of code especially those using Google Vision which attempts to identify objects, text, or emotions in the picture taken by the user. For more advanced users, Metaverse supports 3rd party APIs.

metaverse studio layout
Metaverse Studio Layout


ARIS has been around for some time and came out of Wisconsin University and is highly robust. This platform allows you to build entire experiences that can be triggered by QR codes, locations, or alphanumeric codes. They also recently released an AR component. User accounts are free and you can start building right away. ARIS is also well documented with support forums available. Unfortunately, there is currently only an iOS app available for accessing the content you created.

ARIS studio layout
ARIS Studio Layout

Possible Uses

Beyond creating fun scavenger hunts and games, these tools can also be used to help support independent transportation in the community. Support cues and triggers can be embedded along a user’s bus route or walking path. These tools can also help support data tracking and enhance engagement using a badge and token system. The use of QR triggers within the home or school can be used to activate video modeling of directions or sequence of steps.

What About VR?

Virtual reality has also entered the world of rehabilitation and regaining function. Some of its uses have included creating memorable experiences for those with dementia and simulating movement experiences for those with mobility impairments. RAPAEL, which created a hand rehabilitation glove is now blending VR exercises with a sensor based glove to detect finger, wrist, and forearm movements which impact the VR world. Here is research study they cite: Effects of virtual reality- based rehabilitation on distal upper extremity function and health- related quality of life: a single-blinded, randomized controlled trial. 

A chapter on VR and Occupational Therapy is also included as part of the Occupational Therapy – Occupation Focused Holistic Practice in Rehabilitation open textbook.

It’s a brave new world for assistive technology. Time to dive in.



Autodraw: Word Prediction for Drawing

autodraw iconGoogle has recently released the Autodraw AI for some fun and games for the casual artist. This tool, however, can serve as power assistive technology for students with disabilities who struggle with drawing and expressing, themselves especially in the early pre-writing days. For those who haven’t tried it out, Autodraw lets the user start to draw some basic shapes or lines and the AI will attempt to guess what the user is trying to draw providing options of icons. The user can then tap on the desired icon to insert it into their drawing.

For young learners, especially those with visual- motor, perceptual or motor planning difficulties drawing can be quite a frustrating challenge. Most early grade writing standards have some requirement of drawing and labeling the drawing based on a classroom topic or reading. The Autodraw tool can be a great support. Just check out the results:

drawing attempt of a banana
Original drawing
icon drawing of a banana
Icon selection based on the drawing.

Your Library Card = Goldmine!

library-cardAn often over looked resource is your local library. Be sure to check out your local library’s resources page for subscriptions to online databases that often charge a subscription fee. Entrance to these sites is through your local library’s website using your library card credentials.

The following is a list of student related resources available through the Brooklyn Public Library Website (a complete list of available resources can be found HERE):


  • BookFlix– (Requires card)- Video based books for young readers with read along text. Site uses Flash for content and may not be accessible on iOS devices unless using Flash friendly browser apps.
  • Flipster– (Requires card)- Provides access to popular magazines either through a web browser, Android, Kindle and iOS apps. Kids resources include Ask (teacher companion), Ranger Rick Jr, Discovery Girls, and Highlights.
  • Tumblebooks– (Requires card)- Early readers video based books with a variety of topics. Books are also available in multiple languages with human narration and sentence highlighting.

History & Culture

  • American History– (Requires Card)- Comprehensive resource library and research tools related to American history broken up by periods. Detailed summaries are available for each period with text to speech and translation support.
  • CultureGrams– (Requires Card)- Learn about the culture and day to day life of people across the word. There are US states, the world, and kids editions available. Content is readily broken down by category (places, history, people, cultural notes, and fun facts).  Text to speech enabled.
  • FreedomFlix– (Requires card)- Video and text companion on topics related to US history and civics. The available text has human narration with word highlighting as it reads and two available speeds.
  • Global Voices– Curates and translates current events news from around the world.
  • Mango Languages– (Requires card)- Learn a new language with video and activity cards.
  • Opposing Viewpoints– Collection of articles, media, images, and statistics based on topic. Text to speech and translation supports available.


General Research

  • Encyclopedia Britannica Online for Kids– (Requires card)
  • Kids InfoBits– (Requires card)- Easy to navigate picture based research for early grades. Text to speech and translation support.
  • Scholastic Go
  • SIRS Discover– (Requires card)- Materials have text to speech and translation supports. Content can also be filtered by Lexile reading level.
  • TrueFlix– (Requires card)- Video and accompanying text around a variety of topics (picture based category search). The text has human narration with word highlighting supports.

Instructional Resources

New Found Tools for OCR with Auditory Feedback and Phonetic Word Prediction (FREE)

Microsoft OCR with Immersed Reading

office-lens-iconIn a bid to continue supporting struggling student, Microsoft has introduced OCR functionality with the added benefit of Text to Speech (TTS) to its document scanning app, Office Lens . Office Lens works in tandem with your other Microsoft apps and requires an account. At this time, users do not need a Microsoft 365 subscription and can use a free account.

The scanning app is straightforward to use and provides options to scan a business card, photo, document, or whiteboard.  Automatic page detection is available however the user still needs to press the camera button to capture the image.  The next step is to select the “Immersive Reader” option. The app will then OCR (Object Character Recognize) the image and produce a screen with the text extracted from the image.

Some standout features of this app include its simple user interface and the option for “Wide Text Spacing” which increases the line spacing for easier reading.  TTS is easily activated through the play button and reading speed is adjusted through a slider in the toolbar.  While reading, the screen is grayed out while the word being read is highlighted. This allows for easier scanning and reading. On tried scanning, the OCR accuracy proved significantly accurate.

Drawbacks of the app include limited ability to modify the font style and color options. Additionally, navigating through the text is a bit cumbersome. Original formatting of the document is removed from the scan which might be disorienting to readers.

See the video below for a quick walkthrough:

Phonetic Word Prediction Using Google Input Tool

Word prediction has become a commonly available tool on current mobile devices used to increase typing speed using the onscreen keyboard. This functionally, however, has been used by students with dyslexia for many years especially using flexible spelling or phonetic spelling predictors. Most operating system and mass market word prediction rely on the context of the writing versus phonetic spelling patterns and often do not support students with significant dyslexia.

Google’s  Input Tool is both a setting and a Chrome extension that helps support this spelling pattern. The traditional use of this tool is for keyboarding in multiple languages with transliteration available for some languages. This support, for example, allows a Hebrew writer to keyboard “shalom” using the roman keyboard while the Input Tool will then translate the entry to “שלןם”. This same support can be used while writing in English within Google Drive apps.

To activate this support:

  • Visit: My Account- Input Tools
  • Click “Select Languages”
  • Select English from the checkbox menu and save the settings
  • To test: open a Google Doc, upon loading you should see “En” icon at the end of the toolbar to the right of the “Tx” icon. Select the suggestion icon.
  • When typing, you should begin to see a suggested word list that follows the cursor and adapts to the letters being written.
  • To enter the desired word, either click on the word or enter the corresponding number.

The accuracy of this support for phonetic spelling has been  fairly accurate for longer words.  It falters however for homophones or shorter words. These, however, can often be corrected by Google Doc’s built-in spelling and grammar checker.

There is a bit of a lag between when the tool is enabled and the time it shows up within the Google Apps. An additional workaround to try is adding an additional language and then removing it once the tool is available.

See the video below for a quick walkthrough:

It’s Time to Open Source EdTech

This week another email in my inbox has popped up of yet another edtech company being bought out by another. Zaption, an online video annotation service was bought by Workday. As a supporter of special education students, Zaption held great promise in providing students the opportunities to view instructional videos which are embedded with additional content and comprehension checks. Workday, however, appears to be in the market of providing HR and financial support to businesses. Another favorite tool of mine which recently closed shop due to an unsustainable business model was Curriculet. This site provided fiction and non- fiction text with multimedia annotations as well as comprehension checks along the way.

Edtech companies have every right to a profitable business model but educators have a right to innovation, consistency, and reliability. Some of these platforms depend on educators to create content and grow its product base only to have the service become inaccessible. Hours of work by the educator are lost and the search for a replacement ensues. Having to learn a new platform, reconnect students to a new system, and provide instruction again, on how to use this new tool. This, in the long run, is inefficient for educators and may ultimately continue being a barrier to integrating technology in the classroom.

Open source software has the potential to help support educators in their needs and developers for this community are actively needed. Open source software has the potential of providing educators with stable and accessible tools which can be built to meet the direct needs of educators. The potential for an organic community consisting of educator and developers can have ultimate benefits to students without the barrier of traditional business models and profit generating. Make and keep the tools educators need! Open source has come a long way in terms of usability. My first experience a few years ago involved an installation of a vector drawing tool and having to access my computers terminal commands, install X11 and other somewhat confusing directions. This same tool is now a similar install process as any other software which a lay person can easily access.

Open sourced software may ultimately become THE accessible option for users with disabilities. Building tools in collaboration with a community can lead to a conversation about who is using it and what they need. Take, for example, the story of Julius Sweetland who built OptiKey, an open source eye gaze communication system. My personal wishes? High-quality text to speech technology baked in for any text, high-quality word prediction for writing tasks, keyboard navigation, and robust visual supports. Many of these two tools are often used by students with disabilities. So much so that they have become part of mobile operating systems.

Now the challenge to educators and developers/ programmers to connect and make a change!

Co:Writer 3rd Party Keyboard is Here!

Co:Writer’s 3rd Party Keyboard
topic dictionary

Third party keyboards have given iOS users the ability to use customized keyboards across apps. Until recently, the powerful Co:Writer word prediction had been locked within it’s own app limiting the user’s ability to use the support on any app. With a recent update, Co:Writer Universal now provides a 3rd party keyboard option giving writers the ability to use this support across any app (Notability, Notes, Google Docs, Safari, etc).

The settings for auditory feedback carry over from the Co:Writer app as well as the topic dictionaries created. These can be toggled on and off directly within the keyboard.

This is a great and well needed update! For more information on how to enable 3rd party keyboard see this Apple support documentation.

Chrome as Assistive Tech Infographic

The variety of web apps and Chrome extensions has grown by leaps and bounds over the last few years. Many of these tools are available to students either for free or offer lower cost alternatives to what traditionally was considered cost prohibitive support. I complied a Coggle infographic to organize  some of the apps, extensions, and add-ons I use most frequently to support my students. Know of a good tool that’s not on the list? Be sure to comment.

As a side bar… most of these tools require a working internet connection. This on its own is prohibitive for many families. Everyone On (  is an organization that works in low income communities and schools to provide a router and affordable internet connection for household that qualify.

Using Table of Content Add-ons to Support Executive Skills in Google Docs

More schools are beginning to integrate the Google Apps for Education (GAFE) tools into their curriculum and instruction which is a great step forward for students who use assistive technology. No longer are these students the only ones keyboarding in class, having to modify classroom materials to meet their learning needs. While this is tremendously helpful, some of these students continue to struggle with organization and executive skills. Having a Drive full of random documents is slightly better than a backpack full of misfiled papers. Much the way that students are unable to put away physical papers into correct folders the same begins to happen in digital form.

Microsoft’s OneNote platform is able to provide students with a binder equivalent of file managment. The student only needs to open one file (Notebook) and all the subjects for that semester are laid out in tabs and pages. This is a tremendously helpful system for students who struggle with organization by giving them one central location for most of their writing. This organizational support however is lacking (hopefully only for now) for students on the GAFE platform.

Binder layout with subjects and pages
OneNote Binder Layout

There is however a workaround using Google Doc’s Table of Contents structure and available add-ons which place the organization structure in the tools tab on the righthand side of the document. Currently, there are two add-on available Table of Contents and Document Navigator. If students are able to use the heading structures effectively in Google Docs, these tools provide easy navigation across one document for multiple subjects. Here is a starter doc where the classroom subject holds the ‘Heading 1’ formatting and each classroom task holds the ‘Heading 2’ formatting. Subsequent dated entries can hold the ‘Heading 3’ formatting and so on. Color coding can also be applied by subject.

Google doc with color coded heading and organized outline in tools
Google Doc with Table of Contents Add-on

Good luck!

Educational Technology Models and Application for Assistive Technology

The field of educational technology is growing at a steady clip with new tools, apps, and services cropping up. Many of these tools are used to enhance learning and engagement in the classroom. Some facilitate project based learning or encourage easy data tracking for the teacher. As teachers are becoming more familiar with integrating technology into their classroom a window to incorporating assistive technology opens.

Currently, assistive technology is generally IEP driven and is meant to accommodate for an identified disability. These supports can range from magnifiers for visual disabilities, augmentative communication devices for the non-verbal, or text to speech for those struggling with learning disabilities. Very often however, teachers are faced with students in their classroom who for multiple reasons do not end up being identified for support nor receive the needed help. Additionally, abandonment of equipment is a significant concern within the assistive technology community. Finding the correct fit between the student, the teacher, and demands in the classroom become great hurdles to overcome despite the vested interest of all parties.

Universal Design for Learning (UDL)

UDL is a model evolving from the field of architecture, universal design. The most often referenced analogy is one of the ramp. Buildings originally designed with ramps not only allow access to a building for those in wheelchairs but also assist parents with strollers and delivery personnel. UDL aims to encourage curriculum planning and environmental design to create content that can be accessible to all students in the classroom. This may include having text to speech books available for all students or creating a classroom color coded organization system for notebooks and folders. Students are able to select the tools that best help them meet their learning goals. UDL is most powerful when implemented at the planning stages versus having to retrofit pre-existing curriculum. There are three main components to UDL which will be briefly reviewed given that this model is familiar to professionals in the assistive technology field. For further details be sure to visit

  • Multiple Means of Representation- Known as the recognition network, this area explores the “what” aspect of learning. It encourages providing a variety of modalities when explaining concepts.
  • Multiple Means of Expression- Known as the strategic network it represents the “how” of learning. This principle encourages a variety in how a learner can express their knowledge base.
  • Multiple Means of Engagement- Known as the affective network it represents the “why” of learning. This principle encourages looking at the learner’s motivation and desires for learning.

SAMR Model

SAMR is a model that has been developed to create a hierarchy of how technology is being integrated in the classroom. It starts with the most simplistic use of technology and graduates to more complex encouraging deep learning skills. Jonathan Brubaker had created this infographic illustrating the various SAMR levels comparing them to Starbucks offerings.

Assistive technology providers often focus on finding technology that can either serve in the Substitution or Augmentation portions of this model. If a student cannot write using pencil and paper, perhaps a portable word processor can substitute? A student is unable to read? Provide her with text to speech for reading materials. AT providers may however be in a unique position to encourage teachers to branch their pedagogy to the Modification and Redefinition segments of this model. Asking a teacher about the learning goals and outcomes of a unit or lesson may prove a more powerful question than what tasks need to be completed by the student daily. After all, successful completion of the tasks does not automatically result in learning. Supporting a teacher in redefining, might result in a scenario in which one group of students produce a video while another group produces a written report.


Reproduced by permission of the publisher

Another model currently being implemented in educational technology integration is the Technology- Pedagogy- Content model. The illustration below provides an overview of the three main segments a teacher needs to consider as well as overlapping domains. Classroom teachers need to establish competence in not only the technical use of available hardware and software but also how this technology can meet content requirements. Analysis of how pedagogically sound a technology is another pivotal role a classroom teacher takes on. When examining this model, the teacher’s level of competence in each of these areas can effect the successful integration of assistive technology for a student. A teacher who does not posses comfort in using technology nor have the skills to problem solve may be more reluctant to integrate AT tools. Likewise, if a teacher does not perceive that an AT tool meets pedagogical standard, its implementation may also falter. Supporting teachers in each of these three areas and their overlapping domains may ultimately help students with mandated assistive technology integrate this support and decrease abandonment.

Understanding the various educational pedagogy currently being applied in educational technology can help assistive technology providers in successfully supporting their students and beyond. Providers must also play a key role in demanding that many of the tools being used for the disabled population become readily available within the technology being developed for the classroom. Text to speech, word prediction (flexible spelling), speech to text, and writing with auditory feedback should not be fringe add-ons to a program. These tools often recommended should be integrated as a given. Assistive technology professionals are also uniquely qualified in understand where the gaps in learning exist for a particular learner and may better understand how content should be modified. Involvement in the development of educational tools are key especially in the area of adaptive learning technology. Many learning platforms are being developed to provide customization for learners. Although in its infancy, this adaptive technology mostly consists of moving a student up or down a hierarchy of difficulty given their response to a question. Providing better insight into how content can be modified to meet the needs of struggling learners can help improve this technology.