Twelve Labs is developing an advanced AI model capable of analyzing videos to perform a wide range of tasks, as reported by TechCrunch. These cutting-edge models are unlocking groundbreaking possibilities in technology — a movement spearheaded by Jae Lee, co-founder of Twelve Labs.
His company specializes in training video-analyzing models to perform tasks like searching specific moments in videos, summarizing clips, or answering questions such as, When did the person enter the shop?
The company’s technology has drawn big backers, including Nvidia, Samsung, Intel, and recently, Databricks and Snowflake.
Traditional keyword searches only look at titles, tags, and descriptions. According to Lee, this approach is inadequate for video. “Finding a specific moment in video is like searching for a needle in a haystack,” he told TechCrunch.
Twelve Labs’ AI maps text to video content, identifying actions, objects, and sounds. Unlike general-purpose models from companies like Google and OpenAI, Twelve Labs focuses solely on video, offering custom models tailored to specific needs.
Applications range from ad placement and content moderation to creating highlight reels. Developers can build apps using the company’s API to search across video footage, images, and audio.
Bias in AI models remains a challenge. Lee acknowledged this and said Twelve Labs conducts bias tests and plans to release ethics benchmarks. The company trains its models on public and licensed data but refrains from using customer data. “We want benchmarks that hold us accountable and set an industry standard,” Lee said.
Twelve Labs’ portfolio now includes tools like Marengo, which performs “any-to-any” search across images, audio, and video, and an Embed API for creating multimodal embeddings.
The startup has secured enterprise clients, including partnerships with Databricks and Snowflake. Both companies have integrated Twelve Labs’ tools into their platforms and invested in its latest $30 million funding round.
Strategic Moves
The company recently hired Yoon Kim, former SK Telecom CTO and an architect of Siri, as president and chief strategy officer. He will help expand Twelve Labs into sectors like automotive, security, and possibly defense.
With $107.1 million raised and plans to grow its 73-person team, Twelve Labs is betting big on AI’s video-first future. “Our goal is to create meaningful, responsible technology that drives change,” Lee said.
