A Glimpse of Attention

An interactive piece where the news follows your face and changes every time you blink. Built with p5.js, ml5.js FaceMesh and The New York Times API.

Team: Ruoling Xu & Jayson Ramírez Mafla
Course: Creative Coding, NYU Tandon IDM · Instructor Ofer Shouval
Date: December 2025
Tools: p5.js, ml5.js FaceMesh, p5.sound, NYT Most Popular API

AI-generated concept of the piece as an installation: a person stands in front of a large video wall in a dark room, filled with overlapping New York Times headlines and images. — Conceptual image created with AI.

Try it

Look at the camera: the stories gather around your face. Blink to move to the next one. Without a face in the frame, the news drifts in chaos again.

The demo uses your camera to find your face and eyes. Nothing is recorded or sent anywhere: it all runs in your browser.

If your blinks aren’t detected, try taking off your glasses: the lenses can hide your eyes from the camera. No camera, or blinking isn’t an option? Press Space to change the story while your face is in view. Click the demo to turn on the sound. Open full screen

Finding the idea

Working in a team was both challenging and exciting. Our previous creative coding work had been individual and exploratory, so collaborating required more communication, planning and coordination. We started with mind maps of technical ideas, and realized we both wanted to learn how to use an API and how to build creative interactions with ml5.js FaceMesh.

Knowing what we wanted to try technically wasn’t enough, so we mapped concepts too. We were drawn to concentration and attention, and saw an opportunity in The New York Times APIs: an interactive statement about attention, told through chaos and order. Sketching in FigJam, we agreed on an experience built on the interaction between the viewer’s blinks and the news.

Technical mind map around “Final Project”: computer camera, machine learning library FaceMesh, API, sound, Arduino and controllers. — Mapping what we wanted to learn.

Concept mind map: from Concentration to topics like economy of attention, the New York Times API, chaos and order. — Looking for the concept: attention.

FigJam sketches of chaos and order: scattered sticky notes labelled Chaos next to empty frames labelled Order, in several compositions. — Chaos and order, sketched in FigJam.

Line sketch of a head outline with a cluster of small news cards gathered over the face. — The news gathering around a face.

First iteration

We split the project into three problems: how to use the API, how to present its information, and how to set up FaceMesh to recognize a blink.

The API

Following Daniel Shiffman’s tutorial on the New York Times API, we generated a key, learned how the JSON nests arrays and objects, and loaded it with loadJSON(url, gotData). Seeing the articles print in the console was euphoric. We pulled the title, the abstract and the image of each article, and found that images travel inside an array of quality options.

p5.js code calling loadJSON next to a black canvas with overlapping New York Times headlines and photos. — First data on screen.

p5.js editor with a canvas full of headlines and images piled chaotically on top of each other. — Twenty articles in noise-driven motion.

Chaos

To portray chaos we drew the 20 articles with text() and image() in a loop, and moved them with noise() mapped from 0–1 to the height of the canvas.

A face on screen

With FaceMesh set up, the question was how to react to a face. An if on the number of detected faces changed the noise speed, and a pink ellipse let us check the effect. We then merged the API and FaceMesh sketches, hid the green keypoints, and explored compositions, movements and typography.

p5.js editor with webcam video of Jayson covered in green FaceMesh keypoints and a pink ellipse on his forehead. — Testing face detection with a pink ellipse.

Close-up of a face wearing glasses covered in green FaceMesh points, with red points marking the eye landmarks.

p5.js editor with webcam video of Ruoling and the word BLINK in red over her face.

The blink

We found the FaceMesh keypoints for the top, bottom, left and right of each eye, measured them with dist(), and compared the eye’s height to its width. When that ratio dropped under a threshold, the sketch printed BLINK.

Second iteration

When we presented in class, Ofer and our classmates pointed to two problems: the sketch was extremely chaotic, and it gave poor feedback about the face and the blink. So we reimagined the experience.

The news follows you

We calmed the noise movement and looked for ways to make viewers feel in control with their face. We tracked the nose keypoint and used its position to place the New York Times story, so the news floats and follows the viewer.

Sound and a softer blink

For presence feedback we added sound: once we loaded the right p5.sound library, the music fades in with lerp() when a face enters the frame and fades out when it leaves. Each blink now crossfades to the next story, with a “Blink your eyes” prompt to invite the first one.

p5.js editor with webcam video of Ruoling and the word NOSE drawn in red on the tip of her nose. — Tracking the nose to place the story.

Final version: a single New York Times story, a photo with its headline and abstract in white on a black background. — The final version: one story at a time, following the viewer.

Reflection

We are proud of the result. Next, we’d like to bring the experience into a physical video installation. Blink detection is still imprecise at times, as the camera doesn’t always catch the blink, and that is the first thing we want to solve.

← Back to work