Our First Try at Building an AR Experience: What We Learned

Building a basic AR experience involves four stages: defining your concept, sourcing or creating a 3D asset, developing the scene in a platform like Unity with AR Foundation, and testing across real devices in varied environments. We learned that even a simple AR prototype involves more detail than it appears, particularly around lighting realism, surface tracking in challenging conditions, and performance optimisation for lower-end phones. This is an honest account of what that process looked like for us, what worked well, what we struggled with, and what we'd do differently next time.

Augmented reality (AR) has been on our radar for a while, but this was our first real attempt to build something from scratch. We wanted to explore what it actually takes to create an AR experience people can hold in their hands, and whether all the hype lives up to the reality.

Here's what we discovered.

Why did we decide to try building an AR experience?

We decided to try AR because it's no longer a novelty reserved for major brands; it's becoming part of how brands, educators, and creators genuinely connect with audiences.

We've all seen AR in action: trying on glasses virtually, seeing how furniture fits in a living room, catching digital creatures in games. We decided to challenge ourselves to build a simple AR prototype, a 3D object that appears on any flat surface and responds to user interaction, to understand what the process actually involves from the inside.

How We Built Our First AR Prototype

We built our first AR prototype using Unity with AR Foundation, which supports both iOS (ARKit) and Android (ARCore), letting us test across devices without starting over each time. Our process ran through four stages: concept, 3D asset sourcing, development, and real-device testing.

1. Concept: A floating, animated object the user could place in their environment and rotate by touch
2. 3D asset: We sourced a model from Sketchfab, a free repository of 3D objects
3. Development: We built the scene in Unity, set up plane detection, anchors, and interaction scripts
4. Testing: We deployed to smartphones and tablets to see how the experience performed in real environments

We quickly learned that even a simple AR app involves more layers of detail than it appears from the outside.

What Worked Well When Building An AR Prototype

Three things worked better than expected: real-time plane detection, touch-based interaction, and the device deployment pipeline.

Real-time plane detection felt almost magical. Watching the system map out surfaces and place objects with accuracy was genuinely exciting, and it gave us an immediate sense of why AR holds so much potential for product-based brand experiences.

Interaction was intuitive. Touch controls to rotate and scale the object made the experience feel responsive and natural, even at a basic prototype level.

Deploying to devices was faster than expected. Once the build pipeline was set up, iterating between changes and testing on real phones became significantly easier, a relief given how critical real-world testing turned out to be.

What Were The Biggest Challenges While Building An AR Experience

The three biggest challenges were lighting and realism, surface tracking in difficult conditions, and performance on lower-end devices.

Lighting and realism were harder than expected. Our object often looked out of place because the virtual lighting didn't match the room. We learned that light estimation, using ARKit/ARCore features to read and mirror real-world lighting, is critical to blending virtual and real elements convincingly. Without it, the experience breaks the illusion quickly.

Surface tracking wasn't perfect. In low light or on reflective floors, plane detection flickered or failed entirely. We had to experiment across different environments to achieve stable tracking, something that's easy to miss when testing in ideal conditions.

Performance needs active management. Even modest 3D models and shaders can slow down lower-end phones significantly. Optimising assets, reducing polygon counts, and simplifying materials, turned out to be as important as the creative work itself.

What We Would Do Differently Next Time

We'd start simpler, use performance-friendly assets from the beginning, and test on real devices much earlier in the process.

• Start with a single, focused interaction. Our first ambition was too broad. Starting with one clear visualisation or interaction point would have given us a stronger foundation before adding complexity.
• Use baked lighting and simpler materials. High-quality PBR textures look great in demos but can kill frame rates on real devices, a trade-off we'd factor in from day one next time.
• Test early and often in real spaces. Simulators don't tell you the whole story. Real-world testing is the only way to catch tracking and lighting issues before they become late-stage problems.

FAQ

What platform should beginners use to build their first AR experience?

Unity with AR Foundation is the most practical starting point for teams new to AR development. AR Foundation supports both iOS (ARKit) and Android (ARCore) from a single codebase, which means you can test across devices without rebuilding separately for each platform. Unity also has the largest community and tutorial library, which significantly reduces the barrier to entry.

How long does it take to build a basic AR prototype?

A simple AR prototype, a 3D object that detects surfaces, anchors in real space, and responds to basic touch interaction, can be built in a matter of days by a developer already familiar with Unity. For teams starting from scratch, expect one to three weeks to learn the tools, source or create a 3D asset, and go through enough real-device testing to catch the most common issues (lighting, tracking, performance).

What is light estimation in AR, and why does it matter?

Light estimation is an ARKit/ARCore feature that reads the lighting conditions in a real-world environment and applies similar lighting to virtual objects placed in that space. Without it, 3D objects look artificially lit, obviously out of place in the real environment. Enabling light estimation is one of the most important steps in making an AR experience feel convincing rather than like a clearly digital overlay.

Why does AR surface tracking fail in some environments?

Surface tracking relies on the device camera detecting visual features on a surface to map it accurately. Low light reduces the camera's ability to detect those features. Shiny, reflective, or plain surfaces (like polished floors or white tables) offer fewer visual features for the algorithm to read, causing tracking to flicker or fail. Testing in varied real-world environments, not just ideal demo conditions, is essential to catching these issues before launch.

How do you optimise a 3D model for AR on mobile devices?

The key optimisations are reducing polygon count (the number of surfaces in the 3D model), simplifying or baking lighting rather than using real-time PBR materials, compressing textures to reduce memory load, and avoiding complex shader effects that strain the GPU. Even a visually impressive model can cause frame rate drops on mid-range or lower-end phones without these optimisations, which affect the majority of real-world users.

What is a realistic budget for a brand's first AR experience?

A simple AR social media filter can be produced for a few hundred to a few thousand pounds. A custom AR prototype like the one we built, a 3D object with surface detection and touch interaction, is achievable as an internal project with free tools (Unity, Sketchfab) but requires developer time. A fully-produced, consumer-facing AR experience for a brand campaign typically ranges from £5,000 to £30,000+ depending on complexity, testing requirements, and whether it's built for a standalone app or an existing platform like Instagram or Snapchat.

Final Thoughts

Trying AR for the first time was eye-opening. It’s powerful, but it demands careful planning, design, and testing to feel refined. We walked away with a deeper appreciation for the teams that make seamless AR experiences look effortless.

If you’re considering your own AR project, our advice is: start small, test in real spaces, and don’t be afraid to experiment.

Do you have an AR project you want to work on? Let's work together! Send us a DM at: hello@childcreativestudio.com, let's collaborate!