Using computer vision to deliver social distancing metrics in the stands.

In the pre-Covid world I would not have written this post.

As an entrepreneur, you just don’t do product development in public.
What if the idea isn’t perfect and people dismiss it/you?
What if it is perfect and people try to copy it?

These are unique times however, so I’ve decided to pull back the curtain a bit and invite the broader sports business community to provide their input while we’re working away.

The goal here is to accelerate the product development process and to help ensure we don’t waste precious time solving the wrong problems.

In a perfect world, I’m hoping for inputs like “could you do X, because that would help solve the issue of Y” or “what about merging that data with this data and then …”

So here we go…

What we have so far

By combining gigapixel photography and computer vision, we’re able to deliver per section social distancing metrics in real time.
In normal speak: We have crazy high-res cameras and wicked smart computers and by making them work together in just the right way, we’re able to tell you:

  1. The percentage of fans wearing masks per section
  2. Occupancy level per section
  3. The average distance between fans per section
  4. Clustering metrics and heat maps for every part of the stands

Potential use cases:

We’re thinking that these data points could be useful on at least four levels:

  1. Game day operations
    Integrate with existing operational platforms to send appropriate alerts to staff members tasked with the safety of fans.
    Example: “Occupancy level in section 103 is above the set threshold, please investigate.”
  2. Flow analysis
    The data will allow teams to analyze the effects of changes made.
    What’s the effect of new sanitization protocols on the time it takes fans to get to their seats? Are we improving from one event to the next?
  3. Fan information
    Appropriate data could be made available to fans to plan their own movement. “I see section 103 is only at 15% occupancy, let’s go to our seats before it becomes too full.”
  4. Audit/research
    Irrespective of your thoughts on the likelihood of second and third waves or even future pandemics like this one (God forbid), the ability to go back to analyze visual data could be invaluable in establishing or confirming safety standards.
    Say we have a stadium that does not implement any social distancing metrics, one chooses to allow 50% capacity, and one allows 20%. Two months into the season there’s an outbreak in the first venue’s community, but not in the 2nd or 3rd. We can therefore deduce that 20% occupancy is a safe number for stadium 1 to hit when they return. (grossly simplified obviously).
    Should any team be accused of not being responsible, the data will also be able to prove that they did in fact implement appropriate safety measures.

A bit about the tech

To help understand the strength and weaknesses of the technologies involved, here’s a quick overview.

  1. Computer Vision
    While technically complex, understanding the application potential of computer vision is actually quite simple.
    Here’s the rule of thumb: everything a human can deduce from a picture, a computer can be ‘taught’ to deduce from a picture.

    There is one important difference though: Computers are fast!
    So while you may be able to accurately guess the gender of one fan, or count the number of team jerseys in one row in a venue, a well trained system can do that for 80,000 fans in a matter of seconds.

    So, when you want to know “Could computer vision do X?”, ask yourself “Could an unlimited amount of interns with an unlimited amount of pizza and coffee do X?”
  2. Gigapixel Photography
    Computers (like interns) need pixels to see.
    Think of it this way: You go to a game, you know your friend, a family member or a celebrity is sitting on the other side of the stadium. You whip out the latest iPhone and take a picture of that part of the venue.
    You intuitively know that zooming in on that picture will not give you a clear image of the person you’re looking to find.
    There simply aren’t enough pixels available.
    That’s where gigapixel photography comes in.
    We can most definitely zoom in and find someone in a crowd, because… pixels.


In summary: both humans and computers need resolution to ‘see.’ In the same way you’re not able to recognize a friend sitting at the other side of a stadium without using a pair of binoculars, computers can’t magically invent pixels to analyze.

This brings us to the unique power and potential of gigapixel photography in the stadium context. You can analyze fans or crowds moving around in the concourse (where they’re close to cameras), but there is no way to analyze the stands without a massive amount of pixels.

Brainstorming time

Fire away with those questions, ideas or possible use cases.

Could this data be useful?
How, where?
Applications we’re not thinking of?
Questions?

(and if you’re too shy to share your thoughts here, please drop me a mail at tinus@fancam.com)

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.