Fusing AI with AR: Custom Markers
summary
Demonstrates on-device object detection using TensorFlow Lite + MobileNet to get bounding boxes and then uses ARCore with Sceneform to hit-test centers, create anchors, and render 3D assets — the demo produces an APK with a required native tensorflow_demo shared library and a separate tracking library that stops color-box flicker. Performance notes: battery and GPU load are high, object scaling and anchor choice matter, transfer-learning retraining is recommended for custom classes, and the live demo successfully places Android models on real chairs.
AI-written summary · about
chapters (7)
- 0:00Start
- 5:22Establish your Android app
- 5:46Install Prerequisites
- 7:22Start Modifying Code
- 15:24Build with Bazel
- 18:58Add Augmented Reality Code
- 29:21Pitfalls of Al + AR: Depth & size of object
related talks
44mNow smile (also in 3D)! Exploring AR, ML and Camera related APIs on Android
42mARCore - Basics and Tools
39mA Real use case of practical AR-AI couple: Size my luggage
42mHello ARCore
39mNow smile (also in 3D)! Exploring AR, ML and Camera related APIs on Android
37mA Real use case of practical AR-AI couple - Size my luggage
44mAdd some reality to your app with ArCore
18mDeep-dive on ARCore | Fundamentals of designing AR apps with machine learning models