Project · open source · 2026

MediaAttention

I built MediaAttention to answer the first question in advertising: will anyone actually look at this? It predicts where people look in an image or video ad, then scores it for focus, complexity, memorability and attention on the brand, product and call to action, before a single impression is bought. Everything runs locally.

Hugging FaceDocsSource

ad.png · predicted heatmap

Predicted attention heatmap on an example ad

ad.png · fog

Fog map: the parts of the ad viewers are likely to miss stay fogged

ad.png · areas of interest

Detected brand, headline, body and CTA with their share of attention

On this synthetic ad the headline takes over a third of all attention and the logo just 1.3%, so the brand score flags it. Impact score 6.7 / 10: optimise before launch.

What it measures

Heatmap & fog

Where viewers fixate in the first seconds, and what they miss.

Focus

Whether attention lands on a few elements or scatters across the frame.

Cognitive demand

How much there is to process. Moderate wins.

AOI attention

Share of attention on brand, product, headline, body and CTA, plus lift.

Memory

Predicted share of viewers who would recognise the creative.

Impact score

One 1–10 pre-flight number against targets from your own ads.

For marketers that means checking a creative before it goes live: is the brand seen, does the CTA get its share, is the layout too busy, and in video, does the key message land right after a hard cut, where memory dips. Targets come from your own best-performing ads, one category at a time.

How I built it

Openly published models stitched together with transparent maths, so every number can be traced to a formula or a paper. Python and PyTorch underneath, a CLI and a Gradio web UI on top.

  1. 01 · attention

    Attention

    UNISAL fixation maps, picked by an eye-tracking benchmark of six models as the best openly licensed one; 0.04 s per image. Weights published on Hugging Face.

  2. 02 · layout

    Layout

    OCR plus layout rules sort text into headline, body and CTA; buttons are grown to their edges.

  3. 03 · objects

    Objects

    OWLv2 open-vocabulary detection finds the logo and product, each role scored on its own.

  4. 04 · memory

    Memory

    ResMem predicts recognition, per image or per video frame.

  5. 05 · video

    Video

    Cut detection on every frame, one-second attention smoothing, AOIs tracked with optical flow.

  6. 06 · calibration

    Calibration

    bench scores a folder of your ads and sets each target to the top 20%.

ad.mp4 · 2.0 s · AOIs follow the zoom

Video frame with tracked areas of interest during a zoom

ad.mp4 · 3.5 s · new shot, AOIs re-detected

Video frame after a cut with re-detected areas of interest
$ mediaattention ad.mp4 --aoi auto --product "a running shoe"

Research it stands on

  • Droste, Jiao & Noble (2020). Unified image and video saliency modeling (UNISAL). ECCV.
  • Linardos, Kümmerer, Press & Bethge (2021). DeepGaze IIE. ICCV.
  • Minderer, Gritsenko & Houlsby (2023). Scaling open-vocabulary object detection (OWLv2). NeurIPS.
  • Needell & Bainbridge (2022). Embracing new techniques in deep learning for estimating image memorability (ResMem).
  • Khosla, Raju, Torralba & Oliva (2015). Understanding and predicting image memorability at a large scale. ICCV.
  • Pieters & Wedel (2004). Attention capture and transfer in advertising. Journal of Marketing.
  • Pieters, Wedel & Batra (2010). The stopping power of advertising. Journal of Marketing.
  • Teixeira, Wedel & Pieters (2010). Moment-to-moment optimal branding in TV commercials. Marketing Science.
  • Milosavljevic, Navalpakkam, Koch & Rangel (2012). Relative visual saliency differences induce sizable bias in consumer choice.

Full references with links, 25 in all, are on the project site. Code under Apache-2.0; model weights keep their own licences.