Personalised advertising is fueling the revenue streams of the largest tech companies in the world, including Google, Facebook, and Amazon. While we are all now well used to seeing personalised ads online, ad inventories in live sports are typically still one-size-fits-all, untargeted and often irrelevant to large parts of the viewing audience.
Watching the opening game of Euro 2020, as Italy approached their first of many victories leading up to their second Championship title, my friends and I played a guessing game with the pitch-side ads to see who could figure out the products advertised. Several of the ads were from unfamiliar brands or in unfamiliar languages, drawing our attention and surely the attention of many sports fans, to the diversity of football audiences around the world.
As it is, global brands sponsoring live sports cannot yet tailor their advertising messages in a hyper-targeted way to the diverse groups within their global audience and must therefore settle for non-optimal brand exposure. Computer vision and virtual advertising has the potential to change this by unlocking the full potential of personalised advertising for live sport events, in a similar way to how internet advertising evolved in the past 20 years from static and global to dynamic and highly personalised.
But let us first take a step back and look at how internet advertising began its auspicious journey. In the 90s, increasing usage of the internet gave rise to a new and exciting channel for advertisers. Over the next few years internet advertising exploded in its popularity and advertising became increasingly sophisticated, progressing rapidly from static banner ads to more personalised and dynamic content. Nowadays companies like Facebook and Google analyse different user demographics and behaviour to maximise the relevance and usefulness of the ads shown. Data is at the heart of personalised advertising online and now that television content is making the transition from a broadcast to an internet protocol distribution model, the parallels between the evolution of online advertising and the future of TV consumption are becoming clear.
Live sport events saw a range of novel technical solutions over the past decade, many of them driven by computer vision-based applications which use artificial intelligence to derive meaningful information from digital images, videos and other visual inputs. Popular examples include the Hawk-Eye systems used in various sports, such as cricket and tennis, to visually track the trajectory of the ball and the GoalControl goal-line technology system used in football to determine whether or not the ball crossed the goal line. Recently, an increasing number of sports rights-holders have started using virtual advertising solutions to make better use of their ad inventories. Advances in computer vision are the driving factor. Early virtual advertising solutions relied on a combination of computer vision technology and expensive hardware infrastructure, while recent developments in the market shows that highly scalable, purely software-driven solutions are the way forward.
Virtual advertising solutions enable sports rights holders to maximise revenue from live in-game sponsorship. I feel sure that virtual advertising is embarking on the same journey of personalisation as internet ads. However, virtual is at the beginning of its journey. Today, the broadcast feed is split into multiple localised feeds, each with unique virtual content, allowing brands to tailor their messaging for each regional audience. The next natural step is to increase the number of localised feeds to achieve a finer viewer-specific segmentation and then move to a fully personalised ad experience.
A pre-requisite for personalised virtual advertising is the separation of the processes of analysis and rendering. The analysis process interrogates the incoming camera signals using complex, computer vision algorithms. It derives information from the camera signal to decide where to place the virtual ad content. These computationally intensive algorithms, such as tracking and keying, run on power-hungry and expensive GPU servers. Rendering is the process of “drawing” or augmenting the localised feeds with the targeted advertising messages.
Independent of the number of localised feeds needed (whether just a few for regionalised feeds or billions for personalised ads), the analysis unit needs to perform the same work. It computes a keying mask for every incoming frame denoting which pixel in each frame needs to be replaced with virtual ad content and which not. The analysis unit is usually located where the camera signal is available, i.e., in the broadcast compound for on-site productions and in the cloud for remote setups.
The work of the rendering unit, on the other hand, is content-specific. It needs to “know” the virtual ad content, which is rendered onto the localised feed. Hence, the rendering units needs to run on the end-user device in a personalised advertising model. Compared to the analysis unit, this task is lightweight as it only needs to add the virtual ad content on top of the broadcast feed by using the keying mask received from the analysis unit, hence, it can be done on the end-user device. The device typically knows its user and can request from an ad server personalised ads matched to the user’s profile.
Two independent evolutions will surely introduce a wide range of exciting innovations in the coming years. On the one hand the progress in computer vision tech driven by augmented and virtual reality applications. And on the other hand, the ever-increasing computational capacity available at our fingertips. I strongly believe this will drive and accelerate personalisation in virtual advertising. Exciting times are ahead.