06/12/2023
❓ Have you tried implementing 3D effects in Flutter Dev?
At BeTomorrow we are experimenting with integrating more 3D effects in our Flutter apps.
Until the new rendering engine Impeller is production-ready on all platforms, our options are quite limited.
At the moment the only realistic solution we have found is using the ability to apply a fragment shader to a given area of the screen. That means doing everything in a single shader program, in a single pass, no 3D models, no post-processes.
⬇ Below is an experiment in rendering an animated 3D logo using this technique.
This scene being rendered all inside a fragment shader, the displayed logo is not exported from a 3D editor, but instead has to be defined purely in terms of mathematical expressions.
Getting the first version up and running takes a bit of effort, but once done, it becomes possible to simulate how light would bounce and refract through it.
The rendering technique used here is called "raymarching" : for each pixel, a ray is cast forward and advances in steps in the direction of the camera, allowing us to calculate the various ways light ends-up affecting this particular pixel at each step (in front of the model, inside, and behind).
The fluid shape inside the model is not a real fluid simulation, and is also just calculated with simple mathematical expressions, mostly a sum of hand-tweaked sine waves to simulate different aspects of the movement (agitated, centrifugal curvature etc).
To give the fluid a more dynamic look, we’ve added a “lava lamp” like effect, making it looks like there are complex turbulent movements inside it, when in reality it is implemented only with two simple perlin noises.
At last, the fluid surface is highlighted with a lighter gradient, and a very thin noise imitating white foam.
This kind of effect is quite heavy on the device GPU and should of course only be used sporadically, and ideally not on the full screen surface. This is because, unlike usual 2D shader effects, each pixel requires many computations at each step of the raymarching process. But regardless of the rendering techniques available, convincing transparency is always performance-critical, even on desktop GPUs.
Have you tried implementing 3D effects in Flutter?
🎤 What are your thoughts on the upcoming Impeller rendering engine?
