AI-powered Video Analytics Platform

Case Introduction:

The end-to-end product development of an AI-powered video analytics platform was done for a US startup catering to over-the-top (OTT) service providers and media broadcasters. The MVP was executed in 8 months using Agile on Time & Expense basis. The client had following business objectives:

• Process real-time streams for ad viability analysis and verification
• Provide real-time decision-making capability to accept or change the upcoming advertisement to ensure better user engagement.
• Show contextual advertisements that would fetch better rates for ad slots
• Provide top contextual keywords based on AI-powered processing to enable automated-tagging of the video inventory.
• Make the entire media content library searchable.

Solution

The developed solution extracts all the metadata from the video using AI/ML algorithms, removes noise, orchestrates the data to draw further insights that would help the publishers and broadcasters position the content to the right audience and extract additional value using AI and ML.

A multi-tenet system was developed for ensuring uniform updates. Microservices based architecture was used to enable modular access and scaling to different customers of the platform.

Overall platform development was done on the MEAN stack (MongoDB, Express, Angular, and Node). In certain use cases where we were getting a huge amount of data (data velocity was real-time, data volume in GBs), Amazon Athena was used (instead of MongoDB in one of the scenarios) given its performance, ease of use, and cost-benefit.

AI/ML models were developed in python using Tensorflow (ML library maintained by Google). The backend is served using flask framework for python.

• Developed a Natural Language Processing (NLP) layer to derive key contextual inferences from the video input.
• Modular microservices architecture which is easy to scale, easy to maintain, and cost-effective.
• Integration with Google DFP (Ad Manager).
• Powerful visualizations using ECharts to show and experiment with the metadata extractions.
• he web application was developed to ensure that interactions between various components on the application are seamless and fast.
• Real-time analysis of video streams to identify the advertisements

Solution Impact:

• Web page loads of every component boosted by 20x using Amazon Athena from previous load time of 30 to 90 seconds to within 3 – 5 seconds.
• Achieved an enhanced interactivity and decision making capability by processing and detecting the advertisement within 12 seconds of receiving the stream, against the target of 15 seconds. This was achieved through intensive optimization of the core algorithm and parallelization of the processes.
• 10-15% revenue increase for the ad-based OTT players by showing advertisements with an increased focus on context.
• Executed an average of 80 user stories per sprint with sprint cycle of 2 – 3 weeks
• Automated context focussed tagging of the media inventory will help in improving the recommendation engine and thus improve the stickiness of users to the OTT application.
• Better contextual search on the inventory (finding a needle in a haystack).