Research breakdown · XiaoHu Explains

Memories.ai ships O-MARC: how to make audio-visual AI faster, cheaper, and more accurate at once

The release also comes with a new benchmark that deletes the audio track and re-tests every candidate question — anything a model can still answer by sight alone gets thrown out.
⚑
Sourced from Memories.ai's research page and its arXiv paper; efficiency and accuracy figures are the authors' own self-reported results. Numbers in this piece have been checked against the paper's tables for their applicable conditions, and any differences are flagged.

On July 8, Memories.ai released a piece of research called O-MARC (the companion paper had already gone up on arXiv on May 26). It tackles a very specific problem: how to make AI that processes video and audio together run faster, cheaper, and more accurately all at once.

The pain point

Today's audio-visual AI is stuck on two problems

1. It's too expensive and too slow — the compute bill doesn't add up

A model doesn't "watch" a video directly. Before it goes in, the video gets chopped into a long string of tokens: each frame is sliced into many chunks, one chunk per token; audio is sliced by time into a run of tokens; then your question gets tacked on. The three pieces are strung together into one long sequence, and the model reads through it from start to finish.

Primary source and reference