This major update contains two big features: Camera Background support, Light Estimation support.
Camera Background
We all love the WYSIWYG approach in our work. With AR Camera Background you'll be able to see video stream from your AR device right in Unity Editor. This feature is implemented on top AR Foundation XRCameraSubsystem so the behavior in Editor is the same as on device.
ARKit: Unity Editor supports native iOS shaders so wring a custom camera shader is a breeze: just copy ARKitBackground.shader and assign custom material to ARCameraBackground and you'll be able to test your shader right in Editor.
ARCore: Unity Editor doesn't support ARCore background shader because of External OES textures API. The plugin uses a custom background shader for displaying camera background in Editor.
Limitations:
Camera video resolution is 400x400 pixels and runs at approximately 15 fps - PlayerConnection API is a bottleneck here. Maybe I'll make a transition to custom network logic in the future.
Is supported only with AR Foundation version <= 4.0.0-preview.3.
ARCameraManager.TryAcquireLatestCpuImage() is not supported. As a low-performance alternative, you can use Graphics.Blit() to read from camera material in Editor.
Light Estimation
To make believable AR scenes, it's important for your scene to match the lighting of the surrounding word. This update brings these Light Estimation features depending on your target platform:
Average Light Intensity, Brightness, and Color Temperature.
Main Light Direction, Color, and Intensity.
Exposure Duration and Offset.
Ambient Spherical Harmonics are not currently supported. I'll implement this feature in the next version.
