thinkAR - an 🤯 AR Brainstorming Mobile App

An Augmented Reality & Artificial Intelligence Mobile Application for Design Thinking

thinkAR is an Android and iOS mobile application based on Unity’s hybrid AR Foundation framework with a Node-RED and IBM Cloudant back end including IBM Watson services. The result demonstrates the feasibility of designing usable AR brainstorming and design thinking applications. The prototype may lay the groundwork for subsequent user acceptance research examining the relationship between AR and creativity.

Context & Background

Recent transitions to working from home exposed the challenges surrounding remote collab- orative brainstorming. Existing ideation tools have concentrated predominantly on variants of digital whiteboard designs. Despite research suggesting that brainstorming in open settings with three-dimensional objects may yield more creative results, the role of Augmented Reality (AR) in ideation tools has remained largely unexplored. This project aims to investigate the extent to which AR can facilitate the design thinking process. A prototype realised a three-dimensional interpretation of core design thinking concepts based on AR design principles. thinkAR is an Android and iOS mobile application based on Unity’s hybrid AR Foundation framework with a Node-RED and IBM Cloudant back end including IBM Watson services. The result demonstrates the feasibility of designing usable AR brainstorming and design thinking applications. The prototype may lay the groundwork for subsequent user acceptance research examining the relationship between AR and creativity.

🤓 Authors

MSc Computer Science Thesis with IBM

• @benediktkau

🥳 Screenshots

3D Persona Avatars can be added, edited and moved within the AR environment.

The concept of design thinking empathy maps has been translated into a 3D AR environment.

Virtual post-its can be added, edited and moved in the AR environment.

😱 Even more screenshots

🔥 Deployment

📱 Front end

1. Create an empty Unity 3D project
2. Install all packages from the manifest.json
3. Follow all steps to set up the AR Foundation environment
4. Import Prefabs, Scenes, Scripts and UI into your empty project
5. Import the Microsoft Rocketbox avatars into a Unity Resources folder.
6. Import animations into a Unity Resources folder.
7. Import the Watson Developer Cloud Unity SDK.
8. Import the Unity SDK Core.
9. Import emojis for the avatars.
10. Import icons to beautify the user interface. After the backend has been set up:
11. Find the Login GameObject and enter the API URL into the Serialized Field api of MainMenu.cs.
12. Find the SpeechToText GameObject and enter the IAM authentication credentials for the Watson service into the Serialized Fields.
13. Find the ToneAnalyzer GameObject and enter the IAM authentication credentials for the Watson service into the Serialized Fields.

🔨 Back end

1. Create an IBM Cloud account.
2. Set up a Node-RED instance as a Cloud Foundry app on IBM Cloud. This tutorial might be helpful.
3. The Node-RED instance will automatically set up and connect to an IBM Cloudant instance.
4. Import flows.json into Node-RED.
5. In the Cloudant dashboard, create databases with the names: users, projects, empathise, ideate. You may use names different from these, but do not forget to change the names in the Node-RED Cloudant nodes 😉.
6. For each Cloudant database, create a search index and copy & paste each the corresponding design documents (not the documents themselves) from Section C.2
7. Create an IBM Watson Speech to Text application on IBM Cloud and share the credentials with Unity (see front end description)
8. Create an IBM Watson Tone Analyzer application on IBM Cloud and share the credentials with Unity (see front end description)

🚀 About Me

UCL MSc Computer Science • Thesis @IBM • 💙 Product Fellow @Tech4Germany • Previously Digitalisation @Deutsche Bahn & Business Development @VW New Mobility Services Beijing