Echoes of the Lighthouse is an AR game that invites players to uncover the mysteries of an abandoned lighthouse through an innovative storytelling experience. Using Spectacles wearable glasses, players interact with a mysterious machine at the heart of the lighthouse, feeding it combinations of three narrative cards to unlock the story's secrets. Each unique combination triggers AI-generated story fragments complete with original artwork and voice narration, steering the narrative in unexpected directions. Beginning with an initial story crafted by the designers, players shape their own journey through the cards they choose and combine, ensuring that every playthrough offers a distinct and surprising adventure with its own unique ending.

The experience is powered by a blend of AR and AI technologies working in harmony to generate content live as players make their choices. When players combine cards, Gemini generates both the narrative description and accompanying artwork in real-time, with textures that maintain visual consistency with the established card style. ElevenLabs then brings these newly created story fragments to life through customized voice narration, immersing players in an evolving tale that responds dynamically to their choices. This seamless integration of generative AI with AR technology transforms a simple card-mixing mechanic into a rich, personalized storytelling engine where the lighthouse's secrets unfold differently for each player.

More in-depth technical pipeline:

There are a set of cards each with a description and a texture. The player puts three cards into the three slots and then turns the wheel and we create a completely new card based on the player’s choice. In the background, we first use Gemini to mix the card descriptions with the back story of the game and create a new card description. We then use the description and a reference style image and a style prompt to generate a texture for the new card using Gemini. At the same time, we also call ElevenLabs to create the narrative voice for the card, which is played when picked up. We use AR+AI features such as gesture interaction, surface detection and placement, particle effects, and custom shader graph in Lens studio.

I made this game with my teammate during Junction 2025 hackathon and we won the 3rd place in Snap challenge.