Blog

Using Augmented Reality (AR) as Assistive Technology

Apple had released their ARKit upon the world which allows developers to create apps which take advantage of the iOS device’s camera and sensors to help create experiences blending the real world and digital content. These apps will work on devices supported by the A9 chip and running iOS 11. There are some AR experiences, however, that you can build right now using what you have.

Metaverse

Metaverse is an AR experience building platform with a fairly user-friendly studio that can be run from your web browser. You can request access through their main landing page and start building. Pros of using Metaverse include their companion apps for both Andriod and iOS as well as a friendly onramp. They have also recently added blocks of code especially those using Google Vision which attempts to identify objects, text, or emotions in the picture taken by the user. For more advanced users, Metaverse supports 3rd party APIs.

metaverse studio layout
Metaverse Studio Layout

ARIS

ARIS has been around for some time and came out of Wisconsin University and is highly robust. This platform allows you to build entire experiences that can be triggered by QR codes, locations, or alphanumeric codes. They also recently released an AR component. User accounts are free and you can start building right away. ARIS is also well documented with support forums available. Unfortunately, there is currently only an iOS app available for accessing the content you created.

ARIS studio layout
ARIS Studio Layout

Possible Uses

Beyond creating fun scavenger hunts and games, these tools can also be used to help support independent transportation in the community. Support cues and triggers can be embedded along a user’s bus route or walking path. These tools can also help support data tracking and enhance engagement using a badge and token system. The use of QR triggers within the home or school can be used to activate video modeling of directions or sequence of steps.

What About VR?

Virtual reality has also entered the world of rehabilitation and regaining function. Some of its uses have included creating memorable experiences for those with dementia and simulating movement experiences for those with mobility impairments. RAPAEL, which created a hand rehabilitation glove is now blending VR exercises with a sensor based glove to detect finger, wrist, and forearm movements which impact the VR world. Here is research study they cite: Effects of virtual reality- based rehabilitation on distal upper extremity function and health- related quality of life: a single-blinded, randomized controlled trial. 

A chapter on VR and Occupational Therapy is also included as part of the Occupational Therapy – Occupation Focused Holistic Practice in Rehabilitation open textbook.

It’s a brave new world for assistive technology. Time to dive in.

 

Advertisement

Autodraw: Word Prediction for Drawing

autodraw iconGoogle has recently released the Autodraw AI for some fun and games for the casual artist. This tool, however, can serve as power assistive technology for students with disabilities who struggle with drawing and expressing, themselves especially in the early pre-writing days. For those who haven’t tried it out, Autodraw lets the user start to draw some basic shapes or lines and the AI will attempt to guess what the user is trying to draw providing options of icons. The user can then tap on the desired icon to insert it into their drawing.

For young learners, especially those with visual- motor, perceptual or motor planning difficulties drawing can be quite a frustrating challenge. Most early grade writing standards have some requirement of drawing and labeling the drawing based on a classroom topic or reading. The Autodraw tool can be a great support. Just check out the results:

drawing attempt of a banana
Original drawing
icon drawing of a banana
Icon selection based on the drawing.

Your Library Card = Goldmine!

library-cardAn often over looked resource is your local library. Be sure to check out your local library’s resources page for subscriptions to online databases that often charge a subscription fee. Entrance to these sites is through your local library’s website using your library card credentials.

The following is a list of student related resources available through the Brooklyn Public Library Website (a complete list of available resources can be found HERE):

Fiction

  • BookFlix– (Requires card)- Video based books for young readers with read along text. Site uses Flash for content and may not be accessible on iOS devices unless using Flash friendly browser apps.
  • Flipster– (Requires card)- Provides access to popular magazines either through a web browser, Android, Kindle and iOS apps. Kids resources include Ask (teacher companion), Ranger Rick Jr, Discovery Girls, and Highlights.
  • Tumblebooks– (Requires card)- Early readers video based books with a variety of topics. Books are also available in multiple languages with human narration and sentence highlighting.

History & Culture

  • American History– (Requires Card)- Comprehensive resource library and research tools related to American history broken up by periods. Detailed summaries are available for each period with text to speech and translation support.
  • CultureGrams– (Requires Card)- Learn about the culture and day to day life of people across the word. There are US states, the world, and kids editions available. Content is readily broken down by category (places, history, people, cultural notes, and fun facts).  Text to speech enabled.
  • FreedomFlix– (Requires card)- Video and text companion on topics related to US history and civics. The available text has human narration with word highlighting as it reads and two available speeds.
  • Global Voices– Curates and translates current events news from around the world.
  • Mango Languages– (Requires card)- Learn a new language with video and activity cards.
  • Opposing Viewpoints– Collection of articles, media, images, and statistics based on topic. Text to speech and translation supports available.

Science

General Research

  • Encyclopedia Britannica Online for Kids– (Requires card)
  • Kids InfoBits– (Requires card)- Easy to navigate picture based research for early grades. Text to speech and translation support.
  • Scholastic Go
  • SIRS Discover– (Requires card)- Materials have text to speech and translation supports. Content can also be filtered by Lexile reading level.
  • TrueFlix– (Requires card)- Video and accompanying text around a variety of topics (picture based category search). The text has human narration with word highlighting supports.

Instructional Resources

New Found Tools for OCR with Auditory Feedback and Phonetic Word Prediction (FREE)

Microsoft OCR with Immersed Reading

office-lens-iconIn a bid to continue supporting struggling student, Microsoft has introduced OCR functionality with the added benefit of Text to Speech (TTS) to its document scanning app, Office Lens . Office Lens works in tandem with your other Microsoft apps and requires an account. At this time, users do not need a Microsoft 365 subscription and can use a free outlook.com account.

The scanning app is straightforward to use and provides options to scan a business card, photo, document, or whiteboard.  Automatic page detection is available however the user still needs to press the camera button to capture the image.  The next step is to select the “Immersive Reader” option. The app will then OCR (Object Character Recognize) the image and produce a screen with the text extracted from the image.

Some standout features of this app include its simple user interface and the option for “Wide Text Spacing” which increases the line spacing for easier reading.  TTS is easily activated through the play button and reading speed is adjusted through a slider in the toolbar.  While reading, the screen is grayed out while the word being read is highlighted. This allows for easier scanning and reading. On tried scanning, the OCR accuracy proved significantly accurate.

Drawbacks of the app include limited ability to modify the font style and color options. Additionally, navigating through the text is a bit cumbersome. Original formatting of the document is removed from the scan which might be disorienting to readers.

See the video below for a quick walkthrough:

Phonetic Word Prediction Using Google Input Tool

Word prediction has become a commonly available tool on current mobile devices used to increase typing speed using the onscreen keyboard. This functionally, however, has been used by students with dyslexia for many years especially using flexible spelling or phonetic spelling predictors. Most operating system and mass market word prediction rely on the context of the writing versus phonetic spelling patterns and often do not support students with significant dyslexia.

Google’s  Input Tool is both a setting and a Chrome extension that helps support this spelling pattern. The traditional use of this tool is for keyboarding in multiple languages with transliteration available for some languages. This support, for example, allows a Hebrew writer to keyboard “shalom” using the roman keyboard while the Input Tool will then translate the entry to “שלןם”. This same support can be used while writing in English within Google Drive apps.

To activate this support:

  • Visit: My Account- Input Tools
  • Click “Select Languages”
  • Select English from the checkbox menu and save the settings
  • To test: open a Google Doc, upon loading you should see “En” icon at the end of the toolbar to the right of the “Tx” icon. Select the suggestion icon.
  • When typing, you should begin to see a suggested word list that follows the cursor and adapts to the letters being written.
  • To enter the desired word, either click on the word or enter the corresponding number.

The accuracy of this support for phonetic spelling has been  fairly accurate for longer words.  It falters however for homophones or shorter words. These, however, can often be corrected by Google Doc’s built-in spelling and grammar checker.

There is a bit of a lag between when the tool is enabled and the time it shows up within the Google Apps. An additional workaround to try is adding an additional language and then removing it once the tool is available.

See the video below for a quick walkthrough:

iOS 10 Accessibility Now Offers Writing with Auditory Feedback and More

If you haven’t installed the update to iOS released this week, this might be a reason to do so. Besides general user experience updates, iOS has added some nice features to their accessibility menu.

Typing Feedback

iOS devices now come with the ability to type with auditory feedback across apps. The setting is enabled via the Speech Accessibility section (General > Accessibility > Speech> Typing Feedback).

To activate this feature toggle on the “Speak Words” and “Hold to Speak Predictions” options. Character feedback is also available however may prove to be confusing for a more proficient typer. A user now also has the ability to tap and hold the word prediction list to hear each recommended word. The touch is a little tricky, however. Press too long, the word is inserted into the sentence.

The update has also improved the ability to have the text read back. A press and hold on any word in the sentence provides the option of having the sentence read back. This no longer requires the user to highlight the entire sentence to have it read back. One feature that is not yet available but hopefully coming, is having the sentence read back after a punctuation mark. Currently, users who use dictation and would like to proof the outcome of the dictation with auditory feedback need to use the press and hold method.

See it in action here:

High-Quality Voice Options

With the iOS update, a larger variety of high-quality voices are now available for use. Previously a user was relegated to Alex however a greater variety is now available as an additional download.

Hear some of the voices here:

Contextual Word Prediction

The word prediction built into the on-screen keyboard has also improved somewhat in its intelligence. It still does not support phonetic spelling but is better able to suggest words based on the context of the sentence you are writing. Let’s hope one day there will be a phonetic spelling support option.

See a demo here:

Color Filters

Within the Display Accommodations, users can now activate Color Filters which changes the intensity and hue of the screen across all apps. This helps support users whose eyes are affected by screen glare and fatigue. Other options include color inversion (night mode) and reducing white point (color intensity). Try these settings here: General> Accessibility> Display Accommodations.

It’s Time to Open Source EdTech

This week another email in my inbox has popped up of yet another edtech company being bought out by another. Zaption, an online video annotation service was bought by Workday. As a supporter of special education students, Zaption held great promise in providing students the opportunities to view instructional videos which are embedded with additional content and comprehension checks. Workday, however, appears to be in the market of providing HR and financial support to businesses. Another favorite tool of mine which recently closed shop due to an unsustainable business model was Curriculet. This site provided fiction and non- fiction text with multimedia annotations as well as comprehension checks along the way.

Edtech companies have every right to a profitable business model but educators have a right to innovation, consistency, and reliability. Some of these platforms depend on educators to create content and grow its product base only to have the service become inaccessible. Hours of work by the educator are lost and the search for a replacement ensues. Having to learn a new platform, reconnect students to a new system, and provide instruction again, on how to use this new tool. This, in the long run, is inefficient for educators and may ultimately continue being a barrier to integrating technology in the classroom.

Open source software has the potential to help support educators in their needs and developers for this community are actively needed. Open source software has the potential of providing educators with stable and accessible tools which can be built to meet the direct needs of educators. The potential for an organic community consisting of educator and developers can have ultimate benefits to students without the barrier of traditional business models and profit generating. Make and keep the tools educators need! Open source has come a long way in terms of usability. My first experience a few years ago involved an installation of a vector drawing tool and having to access my computers terminal commands, install X11 and other somewhat confusing directions. This same tool is now a similar install process as any other software which a lay person can easily access.

Open sourced software may ultimately become THE accessible option for users with disabilities. Building tools in collaboration with a community can lead to a conversation about who is using it and what they need. Take, for example, the story of Julius Sweetland who built OptiKey, an open source eye gaze communication system. My personal wishes? High-quality text to speech technology baked in for any text, high-quality word prediction for writing tasks, keyboard navigation, and robust visual supports. Many of these two tools are often used by students with disabilities. So much so that they have become part of mobile operating systems.

Now the challenge to educators and developers/ programmers to connect and make a change!

Zap yourself Organized with the Rocketbook Wave

Occasionally I like to support crowd sourcing projects that won’t break the bank and may have the potential of providing a new tool to my toolbox. The Rocketbook Wave has been just that kind of project and has become a staple in my work bag. It can also serve as a powerful tool to those students who struggle with organization but prefer the low tech pencil and paper.

Here’s how it Works

The notebook is available for purchase in two sizes and comes with a free accompanying app. The user then uses heat sensitive Frixion pens to take notes as needed.The best part is that a student can use one notebook across multiple subjects and does not need to worry about skipping pages. Each page comes with 7 icons at the bottom. The student can then “tag” which subject the note belongs to by coloring in the appropriate tag.
When finished note taking, the student thenscans the notes using a mobile device. Scanning is actually extrememly fastand effective given the notebook’s darkbrown border. No cropping or re-sizing needed like other scanning apps. Once all the pages are scanned, the student then sends the notes to its given cloud storage folder in either a PDF or JPG format. If writing using multicolor pens, these are preservedin the scan as well.
Now here comes the best part, when the notebook is filled up, the student can then place the entire notebook in the microwave, the ink will disappear, and the notebook is reusable once again!
//www.youtube.com/get_player
I have been using my Rocketbook Wave for the past 2 months and fell in love with it. It has been a great tool for helping me organize my notes. Notetaking has been one area where I still treasure the kinesthetic pencil on paper experience and no touchscreen stylus will replace that feeling. Not yet atleast. A variety of writing implements are available through Amazon ranging from multicolor thicker markers to .7mm and .5mm pens. The markers also come with an eraser (friction heat) for on the spot editing.

 

The first wave of funding through Indigogo is closed however a new Kickstarter campain is open until April 1st. A $27 contibution will get you one Wave notebook and a Frixion pen. Check out the project and more details here: https://www.kickstarter.com/projects/642311833/rocketbook-wave-cloud-connected-microwavable-noteb

Co:Writer 3rd Party Keyboard is Here!

Co:Writer’s 3rd Party Keyboard
topic dictionary

Third party keyboards have given iOS users the ability to use customized keyboards across apps. Until recently, the powerful Co:Writer word prediction had been locked within it’s own app limiting the user’s ability to use the support on any app. With a recent update, Co:Writer Universal now provides a 3rd party keyboard option giving writers the ability to use this support across any app (Notability, Notes, Google Docs, Safari, etc).

The settings for auditory feedback carry over from the Co:Writer app as well as the topic dictionaries created. These can be toggled on and off directly within the keyboard.

This is a great and well needed update! For more information on how to enable 3rd party keyboard see this Apple support documentation.

Chrome as Assistive Tech Infographic

The variety of web apps and Chrome extensions has grown by leaps and bounds over the last few years. Many of these tools are available to students either for free or offer lower cost alternatives to what traditionally was considered cost prohibitive support. I complied a Coggle infographic to organize  some of the apps, extensions, and add-ons I use most frequently to support my students. Know of a good tool that’s not on the list? Be sure to comment.

As a side bar… most of these tools require a working internet connection. This on its own is prohibitive for many families. Everyone On (everyoneon.org)  is an organization that works in low income communities and schools to provide a router and affordable internet connection for household that qualify.
https://embed.coggle.it/diagram/56aeb2334f77943445e639df/44a4c419ece1f77b3e76a86fe3e5c7102ae75e859bcc15d3c720a0d5aa68e81e

Using Table of Content Add-ons to Support Executive Skills in Google Docs

More schools are beginning to integrate the Google Apps for Education (GAFE) tools into their curriculum and instruction which is a great step forward for students who use assistive technology. No longer are these students the only ones keyboarding in class, having to modify classroom materials to meet their learning needs. While this is tremendously helpful, some of these students continue to struggle with organization and executive skills. Having a Drive full of random documents is slightly better than a backpack full of misfiled papers. Much the way that students are unable to put away physical papers into correct folders the same begins to happen in digital form.

Microsoft’s OneNote platform is able to provide students with a binder equivalent of file managment. The student only needs to open one file (Notebook) and all the subjects for that semester are laid out in tabs and pages. This is a tremendously helpful system for students who struggle with organization by giving them one central location for most of their writing. This organizational support however is lacking (hopefully only for now) for students on the GAFE platform.

Binder layout with subjects and pages
OneNote Binder Layout

There is however a workaround using Google Doc’s Table of Contents structure and available add-ons which place the organization structure in the tools tab on the righthand side of the document. Currently, there are two add-on available Table of Contents and Document Navigator. If students are able to use the heading structures effectively in Google Docs, these tools provide easy navigation across one document for multiple subjects. Here is a starter doc where the classroom subject holds the ‘Heading 1’ formatting and each classroom task holds the ‘Heading 2’ formatting. Subsequent dated entries can hold the ‘Heading 3’ formatting and so on. Color coding can also be applied by subject.

Google doc with color coded heading and organized outline in tools
Google Doc with Table of Contents Add-on

Good luck!