Take a manufacturer of commercial refrigeration systems with 480 hours of video sitting on a file server, a composite scenario drawn from the shape of this problem rather than from any one engagement. Installer training sessions, field-service walkthroughs, warranty-inspection recordings, filmed over six years by four different trainers who each had their own way of doing things. Nobody planned the archive. It accumulated.
This spring the warranty claims on one door-gasket assembly went up, and the service director had a specific question. Somewhere in those recordings, a trainer seats a gasket without first fitting the retaining clip, and the technicians who learned from that trainer have been doing it wrong in the field ever since. He wants the clips. Not a summary of the archive, not a topic model, the actual timestamps where it happens, so he can pull those videos and re-shoot them before the fall certification cycle.
That question has two properties that make it hard, and they are the properties most real questions about video turn out to have. It is visual, so a transcript will not answer it. Nobody narrates the step they are skipping. It is also rare, and the moment itself is short. A technician’s hands are in frame for about 4 seconds while the gasket goes in, and in 480 hours of footage there might be 11 such moments that matter.
He had somebody price it. What came back was the reason this archive has never been searched, and the arithmetic is worth walking through slowly, because until you feel it the fix does not land.
Figure 1 - Who Decides What Gets Looked At: The old arrangement has a pipeline choose the sampling rate before anyone asks a question, so coverage is uniform and ignorant. The new arrangement hands the model a reference to the video and the question at the same time, and the model chooses what to fetch while it works. Everything else in this article follows from that one difference.
The archive that nobody can search
Handing a video to a model used to mean handing over all of it. The Gemini API’s documented default samples a video at 1 frame per second [1], which is a sensible neutral choice and also the source of the problem. At 1 frame per second, 480 hours of footage is 1,728,000 frames. Every one of those frames becomes tokens, whether it shows a technician’s hands on a gasket or an empty workbench while somebody looks for a wrench.
Put a number on it, and take the rate from the vendor rather than from somebody’s extrapolation. Google’s video-understanding documentation puts static processing at approximately 100 tokens per second of video at the default media resolution [1]. Those same 480 hours are 1,728,000 seconds, so one complete pass over the archive is roughly 172.8 million tokens.
An independent measurement lands in the same place. In a controlled benchmark published three days after the feature launched, JimmyLiao, publishing through the Google Developer Experts community on DEV, measured a 10 minute 21 second conference talk at 57,610 total tokens under static, fixed-rate processing [2]. The vendor’s rate predicts about 62,100 tokens for a recording of that length, so the two agree within roughly 8%. Token counts still move with resolution and with how much is happening on screen, so treat the archive figure as an order of magnitude rather than a quote. Two independent routes to the same order of magnitude are firmer ground than either one alone.
Google’s pricing page lists Gemini 3.7 Flash input at $0.75 per million tokens through 2026-12-31, a promotional rate with an expiry date rather than a permanent price [3]. One pass over the archive, then, is on the order of $130. That sounds survivable, and it is, right up until you notice what one pass buys. It buys one question. The gasket question is answered, and the next question, about torque sequence on the hinge, starts the meter again from zero, assuming each question has to sweep the whole archive, which is exactly the position a team is in before it knows where anything is. A service team of 10 people asking 3 questions a week is 30 passes a week, roughly $3,900, call it $203,000 a year to ask questions of recordings the company already owns.
The money is only half of it. The larger problem is that 172.8 million tokens does not fit in a context window, so before the model sees a single frame somebody has to cut the archive into pieces, decide how the pieces are ordered, decide how the answers from each piece get reassembled, and maintain all of that. Those decisions get made once, by an engineer, in advance, with no knowledge of what anyone will eventually ask.
So the team does what every team does. It samples less. Drop to 1 frame every 10 seconds and 1,728,000 frames becomes 172,800, and the bill drops by the same factor. That is the point at which the 4-second gasket shot resolves to one frame, or to zero, depending on where the sampling grid happens to land. The search now runs at a price the company will pay and it cannot find the thing the company is looking for.
Figure 2 - The Two Bad Options: Uniform sampling turns 480 hours into roughly 1,728,000 frames and, at Google’s published static rate of about 100 tokens per second of video, around 172.8 million tokens for a single pass. From there the branches are pay that cost again for every question, or cut the sampling rate and accept that short events fall through the grid. Neither branch is a search capability.
This is not a story about video being computationally hard. Nothing in the arithmetic above is about difficulty. It is about coverage being decided in ignorance. The pipeline spent its budget evenly across 480 hours because it had no way to know that 479 of them were irrelevant to the gasket question, and it could not know, because the question had not been asked yet when the sampling rate was chosen.
Figure 3 - Where the Answer Goes: A fixed grid does not know what it is stepping over. At 1 frame every 10 seconds, an event that occupies 4 seconds either lands on a sample by luck or vanishes completely, and there is no signal in the output to tell you which happened.
The change is not the sampling rate. It is who chooses.
On 2026-09-01, Google DeepMind announced agentic video understanding for Gemini, in a post by Rohan Doshi and Mario Lučić [4]. Read quickly, it looks like a sampling improvement. It is not. The sampling rate is a symptom, and the thing that changed is upstream of it.
Under the old arrangement, a pipeline decides what gets looked at. It decides before the work starts, it decides uniformly across the whole file, and it decides without knowing the question. Under the new arrangement, the model is handed a reference to the video rather than the video’s contents, and it decides what to fetch while it is working, with the question in front of it.
That is the entire pivot. A person given the same job behaves the new way without being told to. Asked to find 11 gasket moments in 480 hours, nobody watches 480 hours. They skim transcripts and titles to work out which 12 videos are even about door assemblies, jump to the section of each where the gasket goes in, and then, only there, watch frame by frame with their finger on the arrow key. Coverage is wildly uneven on purpose. The uneven part is the skill.
KEY INSIGHT: Fixed-rate sampling is not a neutral default. It is a decision about coverage made by whoever built the pipeline, before anyone knew what the question would be, and uniform coverage of a mostly irrelevant input is the most expensive way to be wrong.
Figure 4 - The Loop: The model receives a reference to the video and the question together. It requests material, reads what comes back, decides whether it can answer yet, and requests again if it cannot. Coverage is an output of the loop rather than a setting chosen before it starts.
What the loop actually does
Described plainly, the loop is an ordinary think, act, observe cycle pointed at a video file. The model can ask for three broad kinds of material:
- A transcript, of some or all of the recording, which is cheap and tells the model roughly where in the timeline the subject matter lives.
- Frames, over a specific range, at a frames-per-second rate the model picks for that range. It can skim a long stretch at low rate and then re-request one narrow window at high rate when something in the transcript or in the first pass of frames suggests it should look harder there.
- Audio, fetched directly, for the cases where what was said or what was heard carries the answer and the picture does not.
One clarification matters here, because the informal demo material and the official documentation do not agree on it. Google’s own developer video walks through the loop and names three functions while doing so [5]. Those names do not appear in the reference documentation, which describes the loop generically through processing call and processing result steps and does not expose individual tool identifiers [1]. The loop is real and documented. The three function names are demo shorthand, not an API surface, and any code written against them is written against something that does not exist.
Turning the behaviour on is a single field on the video input, set to agentic [1], and the launch post states that it “uses standard Gemini API token pricing with no additional feature fee” [4]. Nothing is billed for the privilege of the loop. The saving, when there is one, is entirely a saving in tokens, because the pricing page also notes that token usage in this mode is variable based on what the model actually loads rather than on the length of the video [3].
Figure 5 - Uniform Coverage Against Directed Coverage: The top strip is what a fixed rate produces, an even spend across the full duration regardless of content. The bottom strip is what the loop produces on the same recording and the same question, most of the timeline touched lightly or not at all, and two narrow windows examined at a much higher rate than the fixed default would ever have used.
Why cost and accuracy move the same way
Almost every cost optimisation in this field is a trade. Quantise the model, run a smaller one, cache more aggressively, cut the retrieved context, and in each case you spend less and accept a slightly worse answer. The interesting claim about this mechanism is that both numbers are reported moving in the good direction at once, and the first reaction to that is usually suspicion.
The reason it happens is in the last section of the arithmetic above, and it is worth stating flatly. Under fixed-rate sampling, the money was not buying accuracy. It was buying frames of an empty workbench. Meanwhile the 4 seconds that contained the answer were sampled at exactly the same rate as the empty workbench, which for a 4-second event at a 10-second grid means not sampled at all. The old method was simultaneously paying for the frames that did not matter and skipping the frames that did. Those are not two separate faults. They are the same fault, seen from the cost side and from the accuracy side.
Once the model chooses, both symptoms resolve together. It stops requesting the workbench, which is where the token reduction comes from, and it re-requests the gasket window at a high rate, which is where the accuracy improvement comes from. You are not buying accuracy with money. You have stopped wasting both.
Figure 6 - One Fault, Two Symptoms: Paying for irrelevant frames and missing the relevant ones are not a tradeoff between cost and quality. They are the same decision, uniform coverage in ignorance of the question, showing up on two different lines of the report. Fixing the decision moves both lines.
KEY INSIGHT: When a cost figure and a quality figure improve together, look for a wasteful default rather than a breakthrough. Something in the old design was spending in the wrong place, and the spend and the quality were coupled through it all along.
What the vendor reports, and what it has not published
Google’s numbers are worth quoting exactly, because every paraphrase of them comes out stronger than the original. The launch post says that “Activating agentic video understanding drops token consumption by up to 88% and boosts accuracy by up to 7% with Gemini 3.7 Flash” [4]. Separately, in an aggregate framing across a set of tests rather than one test, it says that agentic video understanding reduces analysis costs “by up to 66%” while cutting token consumption and improving accuracy by the figures above [4]. Google also places Gemini 3.7 Flash with the feature on “the accuracy-to-cost pareto frontier among tested models for video understanding” [4], which is a comparative statement against other tested models rather than an absolute score.
Every one of those figures carries the words “up to”. They are ceilings. Google states that the gains are most pronounced on long-form video, which by definition means there are shorter cases where they are smaller or absent.
Three limitations belong in the same breath as the numbers, and none of them is hidden. They are simply what happens when you go looking for the supporting table.
The numbers behind the static-versus-agentic comparison have never been published. A chart exists, on the LongVideoBench benchmark, and the launch post points at it [4]. The closest published figure is on the Gemini 3.7 Flash model card, which lists a long-video-understanding score of 85.4% on LVBench [6]. LVBench is not LongVideoBench, so that number is not even measured on the same benchmark as the chart, and it is in any case a single leaderboard score for the model rather than a breakdown of the same model run with and without the loop. It is doubly useless as a substitute for the missing comparison. Without that breakdown, 88% and 66% and 7% are ceilings whose typical case is unknown.
The figures are self-reported by the vendor about its own product, on its own models, with no third-party audit. That is not an accusation. It is the normal state of a launch post, and the correct response is to weight the mechanism, which is sound on first principles, more heavily than the specific percentages.
The cost figure and the accuracy figure are not stated to come from the same run. Google’s own sentence attributes the cost reduction to results “across standard video analysis benchmarks” and illustrates the token and accuracy relationship on LongVideoBench [4]. Nowhere does it say that one execution produced 66% cheaper, 88% fewer tokens, and 7% more accurate simultaneously. Writing it that way would be a cleaner sentence than the vendor was willing to write, which is a good reason not to write it.
Figure 7 - Ceilings, Not Averages: All three vendor figures carry an explicit “up to” qualifier, the cost figure and the accuracy figure are stated across different framings rather than as one measured run, and the numbers behind the static-versus-agentic comparison have not been published. The mechanism is the durable part of this announcement. The percentages are the perishable part.
Four things the mechanism makes possible
Google names four capabilities, and each one traces directly back to the loop rather than to model quality [4]:
- Sub-second moment retrieval. Pinpointing short state changes and cut boundaries that a 1 frame per second default steps over, which is the gasket problem exactly.
- Long-form needle-in-a-haystack search. Answering a narrow question about a multi-hour recording without paying for the hours that have nothing to do with it.
- Anomaly detection. Re-sampling one narrow window at a much higher rate to inspect fast motion or a subtle visual artefact, once the model has worked out where to look.
- Counting action & object. Tracking repeated physical movements over time by rewatching a segment at whatever rate the counting actually requires.
Look at what those four have in common. Every one of them is a question about a small part of a large recording. None of them is a question about the recording as a whole. That is not a coincidence, and it turns out to be the boundary of the entire approach.
Two people outside Google went looking for the failure mode
Within days of the launch, two practitioners ran their own comparisons and published them, and both found that the loop can be worse. This is the most useful evidence in this article, and it should not be read as a caveat attached grudgingly to the end of a positive story. It is the argument’s own prediction coming true. If fetch-on-demand wins when the input is large and mostly irrelevant, it has to lose when the input is short or when the question genuinely needs everything. Two people went looking for that boundary and both of them found it.
They were not the first to name it. Google’s own reference documentation recommends static processing for “Latency-sensitive queries on short clips (under 5 minutes), or cases where frame-level precision across the entire clip is needed”, and warns that the loop’s internal reasoning and tool round trips can raise time to first token on clips under 5 minutes [1]. The launch post mentions none of that. So three routes arrive at the same boundary, the vendor’s own reference docs and two independent testers working from different footage, and what the testers add is the size of the effect.
JimmyLiao ran 24 controlled runs on Gemini 3.7 Flash: two of his own videos, a 2 minute 21 second screen recording and a 10 minute 21 second conference talk, across four workloads, in both modes, three repetitions each, reporting medians of total tokens [2]. He also verified that the agentic path had actually executed by checking the response steps rather than trusting the mode flag, which is the kind of care most informal benchmarks skip. On the long video the loop was not close, cutting tokens by 97.6% on a summary and by 92.2% on a one-off detail question [2]. On the short video it went the other way. Fine UI detail cost 20% more under the loop, a question about a brief motion cost 222% more, and latency moved with the tokens, 32.9 seconds against 10.1 on the worst case [2]. He is careful about that largest figure himself, noting that the three agentic runs on the brief-motion query ranged from 12,300 to 56,487 total tokens, so the median is a shape rather than a constant [2].
His conclusion is the honest framing and it is sharper than the launch post’s: “I do not treat agentic processing as a universally better replacement for static processing. I treat it as a workload-routing decision” [2].
Figure 8 - The Split Runs Along Video Length: Two workloads on a 2 minute 21 second recording cost more under the loop, one of them more than three times as much. Two workloads on a 10 minute 21 second recording cost roughly a fortieth and a thirteenth of the static path. The same feature, the same model, opposite results, sorted by whether the input was large and mostly irrelevant.
The second benchmark, from papereditslab and posted on Google’s own AI developer forum with a disclosed commercial affiliation, took a different approach [7]. Six synthetic 10-minute videos with known short events planted in them, prompts and scoring frozen before any run, five valid matched pairs, Gemini 3.7 Flash only. Agentic inspection recovered more of the planted brief events and scored better on editing decisions, which is the evidence-seeking half of the workload [7]. Static won broad moment retrieval, the kind of question that wants coverage rather than precision, and on this particular set static was also the cheaper path, using 26.42% fewer tokens and costing 23.01% less [7]. One of six agentic outputs failed the required JSON schema [7].
The author’s summary: “The conclusion is deliberately mixed: agentic inspection helped evidence-seeking and editing decisions, while static processing remained better on broad retrieval and efficiency” [7].
Both of these are small. Two videos and six videos, run by individuals, with no peer review and no audit, and neither author is disinterested. One publishes through Google’s own developer-expert community and the other works for a company whose product operates in the same space. Read either one alone and it proves very little.
It is also worth naming what neither of them shows, because the standard applied to the vendor above applies here too. JimmyLiao measured tokens and did not score accuracy at all. The papereditslab run scored accuracy and found the loop more expensive on its own set. Neither outside test produced the cost improvement and the accuracy improvement in the same run, which is precisely the vendor’s claim, and on that specific pairing the vendor remains unreplicated.
What makes them useful is that two unrelated people, using different videos and different scoring, arrived at the same shape of result: the gains concentrate on long recordings and narrow questions, and the losses concentrate on short recordings and broad questions.
Figure 9 - A Deliberately Mixed Result: The second independent benchmark split cleanly. Finding short planted events and making editing decisions went to the agentic path. Broad retrieval and raw efficiency went to static, on 10-minute videos where the whole recording was close to relevant. Six videos is not a verdict, but the direction agrees with the first benchmark for the same structural reason.
The boundary, written as a routing rule
Put the vendor’s claim and both outside benchmarks in the same frame and a usable rule falls out, which is roughly what JimmyLiao meant by workload routing.
Send a job through the loop when the recording is long relative to the answer and the question is narrow. The gasket question qualifies on both counts: 480 hours of input, 11 moments of output, and a clear visual signature to look for once the model is in the right neighbourhood. Every additional hour of irrelevant footage makes the loop look better, because it is the material the loop declines to fetch.
Keep the fixed-rate path when the recording is short, or when the question wants the whole thing. A two-minute screen recording contains almost nothing the model can skip, so the loop’s own reasoning turns and tool round trips become overhead charged on top of an input that would have been cheap to read outright. “Summarise every topic covered in this 12-minute standup” is the same failure in a different form. The question wants coverage, so directed fetching has nothing to decline, and it pays for the machinery of choosing anyway.
The one measurement neither benchmark provides, and the one you actually need, is the ratio in your own archive. That is not a number anybody else can supply. It depends on how much of your footage is a technician’s hands on a gasket and how much is somebody looking for a wrench.
Figure 10 - Route the Workload, Not the Vendor: Two questions decide it. Long input relative to the answer, and a narrow question, send the job through the loop. Short input, or a question that wants full coverage, keep the fixed-rate path and skip the overhead of choosing. Most archives contain both kinds of job, which means the correct answer for most organisations is both paths, not one.
KEY INSIGHT: A capability that improves one workload by 40x and degrades another by 3x is not a better default. It is a routing decision, and the organisation that treats it as a default in either direction will be wrong about half its jobs.
The pattern underneath the announcement
Everything above will age. The model names have already moved: a fourth model shipped with the feature after the launch post was written [1], and any article that anchored its argument to a version number was stale inside a week. The promotional price expires at the end of the year [3]. The feature name will change.
What survives is the shape, and it is stated in one sentence. Fetch-on-demand beats ingest-up-front whenever the input is large and mostly irrelevant to the question being asked.
Video is the clearest instance, because the waste is so easy to see, but it is not the only one. The same shape shows up in a 400-page contract where the question concerns one indemnity clause, in six months of application logs where the question concerns one incident, in a monolithic codebase where the question concerns one function’s callers, and in a data warehouse where the question concerns one customer’s last quarter. In every case the tempting design is to process everything up front into a uniform representation, and in every case that design spends its budget evenly across material whose relevance is wildly uneven.
The test is two questions, and you can run it on your own inputs this afternoon. First, what fraction of this input bears on a typical question? If the answer is most of it, ingest up front and stop reading. Second, how many times will you ask? A single question over a large input favours fetching on demand, because you never pay for the parts you skip. Thousands of questions over the same corpus tilt back toward paying the indexing cost once and amortising it, which is the trade the retrieval stack makes and the reason it exists. We covered how that stack holds up in production, including the freshness problem that ruins it, in The Production RAG Stack Nobody Ships Complete [8].
Worth noting that the diagnosis here is not one vendor’s idea. Two months before Gemini’s launch, James Le of Twelve Labs made the same root critique from an unrelated company, arguing that naive frame-by-frame and transcript-chunk processing of video “collapses structure” and loses why a period of time matters [9]. His answer sits at the opposite end of the investment axis: build a video memory once through an indexing pass, then query it many times, which is much closer to retrieval over video than to a real-time loop with no indexing step at all [9]. Two companies reached the same problem statement independently and answered it from opposite directions, and which direction is right for a given archive comes back to the second question above. How many times will you ask?
Figure 11 - The Axis Behind the Feature: One end pays a large fixed cost once and makes every subsequent question cheap. The other end pays nothing up front and pays per question, but only for the fraction it touches. Two questions place any workload on that axis: how much of the input bears on a typical question, and how many times will it be asked.
What to do with the archive you already have
The practical move is smaller than it sounds, and it is not a procurement decision.
Start by finding out what is actually in the archive. In our experience most organisations do not know. They know the total hours and roughly what the folders are called, and they do not know the ratio that decides everything above, which is how much of any given recording bears on the questions people want answered. Six years of accumulated footage filmed by four different people has wildly variable structure, and some of it is unusable for reasons that have nothing to do with models.
Then write down the questions. Not use cases, actual questions, in the words the person asking would use. “Which clips show a gasket seated without the retaining clip” is a question. “Improve training quality” is not, and it cannot be routed, priced, or evaluated. The list of real questions is what tells you whether your workload is long-input-narrow-question or something else, and that determines whether any of this helps you at all.
Then price it at your volume, not at a benchmark’s. Every figure in this article, the vendor’s and both outside benchmarks’, came from somebody else’s footage answering somebody else’s question. The number that matters is what your archive costs to answer your questions, and the only way to get it is to run a handful of real questions through both paths and read the token counts.
That is the work we would do on a scoped feasibility pass: inventory what is in the archive, turn the vague ambitions into questions that can be evaluated, run a representative sample through both the fixed-rate path and the loop, and hand back the two numbers alongside an honest answer about which of your questions are now in budget and which still are not. Some archives will come back as a clear yes. Some will come back as a no with a specific reason, and that is a cheaper thing to learn in two weeks than in two quarters.
Conclusion
The service director’s gasket question was never blocked by model capability. It was blocked by a coverage decision that had been made years earlier by a pipeline that could not have known what he would eventually ask. Uniform sampling spread the budget evenly across 480 hours, most of which showed nothing relevant, and starved the 4 seconds that mattered. Moving the coverage decision from the pipeline to the model, and moving it from before the work to during it, is the whole of what changed, and it is why the cost line and the accuracy line moved together instead of against each other.
Hold the numbers loosely. The vendor’s figures are ceilings, self-reported, without a published static-versus-agentic table behind them. The two outside benchmarks are small, informally run, and not disinterested. Between them they establish something more useful than a headline percentage anyway, which is where the boundary sits: large mostly-irrelevant inputs with narrow questions on one side, short inputs and coverage questions on the other, and a routing decision rather than a default in either direction.
The part worth keeping when this API is three versions old is the axis, not the feature. Ingesting everything up front is a bet that you will ask many questions of the same corpus. Fetching on demand is a bet that you will ask few questions of a large one. Almost every organisation is making that bet already, usually without noticing, and usually in the direction that was easiest to build rather than the direction the workload deserves. Find out what fraction of your input bears on a typical question, count how often you intend to ask, and let those two numbers pick the architecture. Video is simply where the arithmetic is impossible to ignore.
References
[1] Google, “Video understanding,” Gemini API Documentation, Google AI for Developers, 2026. https://ai.google.dev/gemini-api/docs/video-understanding
[2] JimmyLiao, “Gemini Agentic Video Isn’t Always Cheaper: A 24-Run Benchmark,” DEV Community, September 4, 2026. https://dev.to/gde/gemini-agentic-video-isnt-always-cheaper-a-24-run-benchmark-4ge3
[3] Google, “Gemini Developer API pricing,” Google AI for Developers, 2026. https://ai.google.dev/gemini-api/docs/pricing
[4] R. Doshi and M. Lučić, “Introducing agentic video understanding with Gemini,” Google DeepMind, September 1, 2026. https://deepmind.google/blog/introducing-agentic-video-in-gemini/
[5] Google for Developers, “Agentic video understanding in Gemini,” YouTube, September 1, 2026. https://www.youtube.com/watch?v=ytjgy30Cono
[6] Google DeepMind, “Gemini 3.7 Flash Model Card,” Google DeepMind, 2026. https://deepmind.google/models/model-cards/gemini-3-7-flash/
[7] papereditslab, “Applied benchmark: Gemini 3.7 Flash agentic vs static video inspection,” Google AI Developers Forum, September 3, 2026. https://discuss.ai.google.dev/t/applied-benchmark-gemini-3-7-flash-agentic-vs-static-video-inspection/180611
[8] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “The Production RAG Stack Nobody Ships Complete: Freshness, Evidence, Versioning, and Where It Stops Working,” Dotzlaw Consulting, August 26, 2026. /insights/ai-27-production-rag-stack/
[9] J. Le, “Beyond the Single API Call with Agentic Video Intelligence,” Twelve Labs, presented at Qdrant Vector Search, YouTube, June 24, 2026. https://www.youtube.com/watch?v=i8xZeKP6t6k
Building production AI, or modernizing a legacy system?
That is the kind of work we do at Dotzlaw Consulting. Book a free 20-minute intro call and tell us what you are trying to build, or what is slowing you down.