NOMAD MEDIA GENERATIVE AI
What is Generative AI?
The term generative is used because something new is being generated – for Nomad Media, Generative AI is used to refer to things like metadata that is being generated for customers based on their content libraries and archives. Generative AI uses one or more models to create new types of metadata in a variety of flavours. Nomad Media has been using Generative AI for a number of years for the creation of new metadata for customers’ media.
Oftentimes Nomad Media uses a combination of multiple models to create the whole metadata package, for example:
- Use audio to text for subtitle creation.
- Use image to text for the labels and objects.
- Moderate content to decide if it’s adult or maybe gory content.
Nomad Media Generative AI Use Cases
What Nomad Media Generative AI can do for you and your industry.
Use Cases
- Media & Entertainment Industry
- Entertainment Venues
- News Organizations
- Content Creators
- Video & Audio Production Companies
- Audio Podcasts
- Public Sector/Government
- Public Safety/Law Enforcement Agencies
Business Case
- Search and Discover Hidden Content
- Save Human Labor Investment
- Faster Workflow Turnarounds
- Get Media to Market Faster
- Greater Content Monetization
Real World Use Case: How Can Generative AI Be Applied in Your Business?
Madison Square Garden Case Study
Go HERE to learn more about how Nomad Media leveraged Generative AI to unlock new ways of monetizing content for one of the world’s premiere entertainment arenas.
For More About Models & ML - Read More...
Models and Machine Learning | The Engine and the Pistons of Generative AI
Models are the workhorse of Generative AI - think of the model as the engine behind the scenes that does the work. On one side something is fed into the model and then something is generated out the other side.
The two types of models that Nomad Media has used for many years are:
Audio To Text - The audio track to a video is fed into the model and what pops out the other side is a subtitle that says this is all the things that person was saying. Typically, models are referred to as “something to something." So in this case the model is audio to text because it's taking the audio in and the text is coming out on the other side. Nomad Media has been using audio to text models for many years, and that's how Nomad Media generates subtitles.
Image To Text - Regarding the video content, think of it as a series of of images, and Nomad Media Generative AI takes each image and generates text for that image, creating text labels and text concepts. An example of a concept is something like a concert, gambling, or friendship. A label is an object, for example, a basketball, chair, or car. Other examples include things like taking a license plate out of an image and turning that into text.
The underpinnings of Generative AI is Machine Learning: it's the engine. Machine learning is at the deepest level and it uses the models to generate content. The models used have been trained and validated and tuned and the Generative AI world is making them consistent, repeatable, safe. The term AI is used to represent the overarching services or product– think of AI as like the car and Machine Learning and models are the car engine.
Nomad Media is now able to take advantage of the fast, frantic pace that Machine Learning models are being created. Today there are thousands of research scientists all across the world, figuring out how to make models smarter and faster. They train the models, meaning they literally give it a billion pictures and say, "figure out the similarities and all these," and they use algorithms to figure out how to train the model.
Nomad Media can utilize models from sources like Hugging Face. For example, you can say, "I'm looking for a model that is the best one for doing medical term analysis from audio tracks." That model has been trained to be highly proficient at identifying medical terminology that it was trained on. When those data scientists created that model, they used that type of medical terminology to create the model. They tuned and created it so that it knows those medical words the best from audio. So when it's doing Generative AI, it's creating those words that most people would have no idea how to write it. Generative AI Models today are following two tracks: generic and specific.
Nomad Media Generative AI Features:
Intelligent Search and Discovery
Generative AI provides a new level of human friendly intelligent search and discovery. Better searching experiences with much cleaner, more pertinent results, translates directly into a more highly efficient workflow and saved investment. All customers can benefit from deeper, richer searching.
Customers can leverage human friendly prompts that allow the system to interpret the intended outcome. The power of Generative AI is knowing the associations of all the data points and using Large Language Models (LLM). This removes the restriction of humans needing to know the exact taxonomy of the labels or tags in media. Similar image or video based searching, Generative AI allows for the discovery of similar or next wave videos that are “like” relationships.
Automatic Summarization
Through Generative AI text summarization, you can generate new metadata for review. Automatically create chapters or caption information for a video; Summarize a 2 hour transcript into a paragraph; and Automatically create video segments based on auto-generated chapters and annotations. This capability is critical for large video and audio libraries needing more accurate paragraph descriptions of the content piece. Imagine running a podcast studio and having to pay a team to sit and listen to endless hours of content and then take the time to write out semi-accurate summaries for days on end.
Summarization means an added layer of automated metadata creation. Therein lies the true value for customers. Enhanced metadata translates into far more content monetization opportunities than you knew you had.
Automatic Media Refinement and Modification
Nomad Media Generative AI technologies has refinement capabilities that gives users power they never knew they had: the ability to alter, modify and refine their content. This is done without knowledge whatsoever of image and video modifying tools such as with Adobe Creative Suite. A user without any creative tools experience can change content within the Nomad Media platform.
Nomad Media Generative AI enables rapid modification of media to allow for easy adjustments and additions through human interactive prompts that generate a new version of the asset. Replace the background, replace logos, remove shadows all with just a few key strokes. For example, there is a picture of a girl with purple hair sitting on a beach and there’s a lifeguard tower next to her. If I don’t want the lifeguard tower in my picture I can type the instruction to remove the lifeguard picture from the image and a new picture will be produced with the lifeguard tower removed.
Our Generative AI Customer Process
How Generative AI is applied to a customer’s library or archive is the key to success with a Generative AI project. In some scenarios customers will just turn on all types of Generative AI on all their content, however that could potentially incur significant cost without realizing the expected value. Typically the questions customers have before embarking on a Generative AI project are:
- How do you know that you’re going to get your value from it?
- How do you know what value it’s going to provide you?
Nomad Media answers these customer questions with Generative AI Asset Sampling. This involves sampling specific regions or slivers of customer content as opposed to starting immediately on a massive Generative AI project on the entire library. The Generative AI Asset Sample is inexpensive to get started, to see if customers can find recognized value in the content processed in those parts of the library. Nomad Media works with the customer on a free analysis of a sample of their catalog to be able to see what value can be provided by applying Generative AI. The results of the Generative AI analysis on the sample will produce a report back to the customer and will show what they’ll get as a result. The report could show the subtitles that were able to be generated and the AI captions that were created for the customer’s media. The sampling won’t just use one model, but potentially multiple models on the customer media to see which ones will be useful. It could turn out on the customer samples that they really only care about two of the models that are actually useful for the customer on their media.
Another really important aspect of why it is not advisable to run Generative AI across the entire catalog as a first step, is that it is not known if the correct model is being applied correctly to the appropriate content. For example, if there is an image and there’s no people in that image, then it doesn’t make any sense to run face recognition because its going to incur additional cost only to be told there’s no people in the image. It is already known that there’s no people in the image because the previous object or label detection AI said there’s no people in this image. So Nomad Media is able to save costs for the customer by being able to chain together the model or AI sequences, so only run the AI when it’s needed on a piece of content. This has a substantial impact on cost so that can really bring down the total Generative AI analysis costs, which brings up the value proposition that Nomad Media has — Generative AI sampling of the customer’s catalog shows with tangible samples the value of what Nomad Media can do.
Your Content is Safe and Secure
Does Nomad Media Use Customer Content Or Data To Train Generative AI Models?
Nomad Media does not use customer content or data to train any models. Nomad Media is using the best of breed models presently available that other people and organizations have already trained. So once Nomad Media analyzes an image and gets the results back, for example, the caption, that image is discarded. It’s just there for a brief moment for Nomad Media to be able to analyze it and then it disappears. It doesn’t give Nomad Media any feedback and is not looped back into the model. There is no training. The resulting labels, captions, etc. that come out of this analysis is saved back into the customer’s Nomad Media account, just like all the rest of their metadata and their media. Nomad Media doesn’t retain anything outside after that AI analysis has occurred.
With Face Recognition, there is a misconception that this is also training. This is actually incorrect and what Nomad Media is doing with face recognition is when faces are analysed, faces are given a specific face “thumbprint” without any relation to names or anything like that, so that can detect similar ones. All Nomad Media knows is this face “thumbprint.” In the customer’s account Nomad Media maps those thumbprints to names. Any data that the customer provides about naming faces is stored in their account only and doesn’t ever leave. The only thing that comes back or that comes back out of the account is a thumbprint of somebody’s face that basically has no value other than relationship to another face with another thumbprint.
