Teaching a Machine to See Lotus Flowers
The project: Learning to See (Memo Akten, 2017)
Learning to See is an ongoing series by London-based artist Memo Akten. The version that grabbed me is the interactive installation: a camera points down at a table of ordinary objects — crumpled cloth, cables, bits of junk — and as you rearrange them with your hands, a screen shows the scene reinterpreted in real time. Push the cloth around and the display answers with rolling seascapes, clouds, flowers. Akten’s one line says it all: “It can only see what it already knows, just like us.”
What model? Generative Adversarial Networks. Functionally it’s an image-to-image reconstruction — the live webcam frame goes in, and a generator trained on one narrow image set rebuilds the whole frame using only the visual vocabulary it learned. It’s a generative model, not a classifier: it doesn’t label your hand, it re-paints everything.
What data? A deliberately tight dataset per “study.” One instalment draws on tens of thousands of artworks scraped from the Google Arts Project; others are trained purely on oceans, clouds, flowers, or fire. One model = one worldview.
Why that choice? Because the constraint is the artwork. A GAN clamped to a narrow dataset physically cannot represent anything outside it, so whatever you show it gets forced through that learned bias. That’s not a limitation Akten engineered around — it’s the whole point. The piece argues that machine vision isn’t neutral or objective; like human vision, it’s the product of what it was trained on. A classifier would just spit out a word. A GAN lets you watch the misperception happen.
My sketch: Only What It Knows
I took two things from Akten: the feeling of manipulating a scene with your bare hands, and that core idea — it can only see what it already knows. So in my sketch, the machine never draws your hand. It knows exactly one thing: lotus flowers. Whatever you do, that’s all it can give back.
The surprising part is an inversion I’m happy with. Almost every webcam sketch rewards motion — wave your hands, stuff explodes. Mine rewards stillness. Move fast and the machine “loses focus”: nothing forms. Hold your hand quiet and lotuses slowly bloom at your fingertips, sway on an invisible water surface, and linger before fading. It mirrors the installation’s reward for slow, meticulous handling — and it quietly reframes the webcam from a motion sensor into something you have to be calm in front of.
How it works, mechanically:
- HandPose gives me 21 keypoints per hand. I average the wrist and knuckles into a palm center and track its frame-to-frame velocity.
- Velocity drives a smoothed
calmvalue (0 = chaotic, 1 = still). Only whencalmis high does the machine “see” clearly enough to bloom a flower. - The pinch between thumb tip (#4) and index tip (#8) controls how open each lotus is — pinch for a bud, spread for full bloom.
- Each flower is watercolor-style: petals layered three times with tiny jitter so the edges bleed like ink on rice paper.
The design journey (the honest version)
My first pass was the opposite of this. I went full cyber-neon: additive glow, a ghost skeleton hand, glitchy RGB-split text, particles spraying from every fingertip. It looked like effort, but on screen it was a glowing smear — too many light blobs stacked on top of each other, no breathing room. “Cool” turned into “busy.”
So I threw out the maximalism and went the other way: fewer elements, lots of negative space, soft low-saturation color, slow fades. I swapped the generic blooms for lotuses specifically — they suit the watercolor-on-cream look, and there’s something right about a flower associated with stillness being the thing that only appears when you’re still. Last tweak was timing: I slowed the growth, capped how many bloom at once, and let each one linger far longer before dissolving, so the canvas feels composed rather than consumed.
That arc — neon overload → strip it back → stillness — ended up echoing the piece’s own theme more than my first version did. The machine isn’t impressive. It’s limited, and it’s quiet, and that’s the point.
https://editor.p5js.org/jz6294/sketches/DijQvaihW
Leave a comment