Memojis under the hood

on YouTube ↗22m

" summary

ARKit 2 face tracking uses the TrueDepth camera to produce a 3D face mesh with about 1,220 vertices and exposes roughly 52 blend‑shape coefficients plus ARFaceAnchor and ARSCNFaceGeometry APIs. Code shows three lines to enable the mesh, updates vertex positions each frame, and supports occlusion geometry so virtual hats or glasses sit behind and in front of facial features. Demos run on iPhone X, work with occasional tracking glitches, and enable accessibility and identification use cases.

AI-written summary · about

" chapters (7)

1. 0:00Start
2. 3:35Ar Session
3. 6:07Air Face Anchor
4. 6:40Face Geometry
5. 11:37Fluid Blend Shape Movements
6. 14:21Occlusion Geometry
7. 19:53Face Tracking

" related talks

• 48mRendering techniques for augmented reality and a look ahead at AR foundationLukasz Pasek, Jimmy Alamparambil · Unite LA 2018
• 22mDeeper into ARKit with CoreML and Turi CreateSoojin Ro · try! Swift NYC 2019
• 39mAugmented Reality for iOS DevicesSimone Civetta, Julien Datour · XebiCon'17
• 21mSo you want to build an ARKit appNamrata Bandekar · try! Swift Tokyo 2019
• 40mSamsung AR Emoji Studio: Create and Market New AvatarsSergey Litvinenko · Samsung Developer Conference 2018
• 10mNew capabilities in ARCoreJared Finder, Rajat Paharia · Google I/O 2021
• 08mAlasdair Allan interviewed at OSCON 2010Alasdair Allan · OSCON 2010
• 29mHello, AR World!Aleksey Korovyanskiy · GDG DevFest Gorky 2017