Developing an Augmented Reality Menu Application for Group Dining

The source code for the application is hosted on GitHub, found here.

Application demo

As part of my Engineering Physics project at the University of British Columbia, I worked with my teammate to develop an Augmented Reality (AR) menu application. Our goal was to improve the group dining experience by leveraging AR technology to visualize menu items and streamline the ordering process.

Project Overview

The project aimed to create an Android application that would allow diners to:

1. View 3D models of menu items on their table using AR
2. Interact with these virtual dishes in real-time
3. Share the AR experience across multiple devices
4. Automatically calculate the total bill as items are selected

Technical Implementation

System design

We used several key technologies to bring this project to life:

• Unity: We chose Unity as our development platform due to its robust 3D capabilities and cross-platform support.
• AR Foundation and ARCore: These frameworks provided the core AR functionality, enabling features like surface detection and object placement.
• Photon PUN 2: We implemented this networking solution to enable real-time synchronization across multiple devices.

The application workflow is as follows:

1. The app detects the table surface using AR capabilities.
2. Users can select menu items, which appear as 3D models on the detected surface.
3. These virtual dishes can be moved and manipulated in the AR environment.
4. All connected devices share the same AR space, updating in real-time as changes are made.
5. The app keeps a running total of the bill based on selected items.

Challenges and Learning Experiences

Developing this application presented several challenges that provided valuable learning experiences:

1. Coordinate System Alignment: Ensuring that all devices shared the same coordinate system proved challenging. We implemented a workaround by having all devices initialize from the same surface, but this is an area for future improvement.

2. Real-time Networking: Implementing Photon PUN 2 for real-time updates across devices was complex but crucial for the shared AR experience.

3. UI/UX Design for AR: Designing an intuitive user interface for an AR application required careful consideration of how users interact with virtual objects in a real-world space.

Future Improvements

While we successfully created a working prototype, there are several areas where the application could be enhanced:

1. Implementing a shared coordinate system using methods like image recognition or cloud anchors.
2. Expanding the menu backend to handle more extensive and dynamic menu data.
3. Enhancing the user interface to display more detailed information about menu items.
4. Optimizing the networking to handle multiple groups simultaneously.

Conclusion

This project provided hands-on experience in AR development, real-time networking, and user interface design for spatially-aware applications. It demonstrated the potential of AR technology to enhance everyday experiences like dining out, while also highlighting the complexities involved in creating shared AR experiences.

The skills and knowledge gained from this project have deepened my understanding of software engineering principles and AR technology, preparing me for future challenges in this exciting field.