Postmortem

Motivation and Idea

This is a small interactive work I made for my old friend Mo’s graduation exhibition, which was used in conjunction with other parts of the exhibition.

The idea was to simulate the experience of seeing floaters in our eyes

The interactivity presented in the simulation of the dynamics of the Floaters being affected by the rotation of the eyeballs, such as the inertia and the mild rotation….

There is also a part of visual effects simulation, such as: the dynamic of blurring degree/the texture of the floaters/the visibility between the center and the periphery of the vision, etc.

Accessibility has always been a major concern for us from the beginning ——we wanted to make it as quick as possible for the audience to access it after scanning the QR-code (so it would have been unacceptable to have to install an additional App).

This is also the main reason why we chose to use the form of WebGL+itch.io.

Fortunately, the goal itself is relatively simple, and does not need to rely on those “more AR” technologies, which can avoid dealing with some unfamiliar tools and platforms, and reduce the cost of learning.

Preparation and Research

During the research phase, I asked a few friends to collect information, thanks to Wenqian and Bladesero, especially thanks to Sijia for providing more detailed personal project experience and clues about AR platforms.

At this period, I also researched whether there are other some works that have adopted similar forms through these keywords: AR/accelerometer/gyro/camera

-https://incal.itch.io/shuffle

-https://farfadet46.itch.io/photobooth

-https://alternatehamlet.itch.io/watch-wizard

-https://danidan.itch.io/scream-pong

-https://hsaka.itch.io/shapewalker

After saw the works above,their accessibility on the phone makes me realize that this form looks generally feasible

Problems and Solutions

However, it was only after the production process began that I finally realized the true meaning of [Unity WebGL doesn’t support mobile devices] as stated on the unity official website.

Because I quickly encountered a series of pitfalls, including but not limited to:

• Input.acceleration is not stable on Android, permission inquiry menu does not pop up in time (need to jump out of the web page and then come back)
• Whether ios or Android, Firefox browser can not get some sensor readings, however, some unfashionable browsers work just fine
• Some parameters on Android and ios get different results (camera serial number/some sensor parameters positive and negative values are opposite)
• Challenges from screen rotation modes …

Some of the things that helped crucially were:

• My nephew, who was visiting for vacation, happened to bring along an old ios phone that I could use for testing purposes（it saves me)
• This post to mentions of the sensor param issues related to the input system:
• A plugin made by MarksAssets that effectively solves the most troublesome [Android+chrome can’t effectively get the sensor parameters] problem.

In terms of horizontal and vertical screen adaptation, I didn’t find out a perfect solution in the end. But this article provides good insights on how to think this problem and resize parameter appropriately

Conclusion

Counting the camera-related interactive work which I made in december last winter, this is the second time I’ve had a similar experience.

Development time: about 7 working days. (excluding research and preparation time) The actual process was relatively compact

Some Bonus:

• Got a rough idea of some unity AR related workflows (ARFoundation ….)
• Found some interesting blur shaders
• The problem of build speed super slow after upgrading unity 2022
• WebGL template can actually support gif effect (in APNG format)
• Remove the trail version watermark in the bottom right corner( but this bug returned recently and no good idea to solve)
• Deeper understanding of the method for smoothing parameter changes

Overall, it was an interesting development process.