Bump mapping the digital twin
Adding bumps to the digital twin, so light sweeps across the cloud according to its shape instead of lighting each flat pyramid plane all at once.
The pyramid planes all have the same normal, so the light illuminates each plane all at once, evenly. That isn’t like the nice fluffy bumpy clouds we are used to. So, in the digital twin, we added “bumps” as Gaussians, aggregated together to roughly correspond to the bumps on the physical cloud.
So, when light sweeps over the model, rather than washing across it all at once on the flat pyramid planes, it lights it up according to the geometry of the model.
This required a lot of work by hand: placing the Gaussians on the model, sweeping the light across it from different angles, and checking that the lights on the physical cloud seemed to behave properly. It’s not perfect, but it gave a reasonable approximate effect.
Further to just perturbing the light sensor normals, adding translucent bumps allowed for occlusion, so that some light sensors were shaded when the light source moved. Controlling the translucency of the bumps and the pyramid as well allowed some light to bleed through to the bumps and the pyramid in a more cloud-like way.
In the mapping interface, each of the 344 LEDs sits on the virtual cloud surface with a small sensor hemisphere that measures how much light reaches it. Moving the light around the digital twin shades the real cloud in real time.