Can Claude watch video?
Claude can reason over visual evidence, but the practical answer is more specific: if you want Claude to inspect a video reliably, first turn the video into the right evidence. That usually means key frames, a transcript, and a short manifest that explains what the model is looking at.
That is the useful frame behind claude-real-video, a new open-source tool that packages video into material Claude, ChatGPT, Gemini, or another large language model can inspect. It does not make an AI model magically watch a video the way a person sits through it. It extracts the visual and spoken parts that matter, then hands those pieces to the model.
For a business operator, that distinction matters. The job is not to ask whether AI can "watch video" in the abstract. The job is to decide whether video evidence can improve a workflow: reviewing a screen recording, checking a sales call, studying a short-form ad, documenting a bug, or turning an SOP video into written steps.
The honest catch: video analysis can feel authoritative even when the evidence is incomplete. If the tool misses a frame, the transcript is wrong, or the prompt asks the wrong question, the model can still produce a confident summary. Treat the output as inspection support, not as a final audit.
What is claude-real-video?
claude-real-video is a local command-line tool that takes a video file or URL and produces key frames, a transcript, and a manifest for an LLM to read. Its README describes the output as a folder with frames, transcript.txt, and MANIFEST.txt, created from a command like crv "https://www.youtube.com/watch?v=...".
The important design choice is frame selection. Many simple video-to-LLM workflows sample frames at a fixed interval: one frame every second, every two seconds, or every five seconds. That is easy, but it is blunt. A slow screen recording may waste context on nearly identical frames. A fast-cut ad may hide important action between sample points.
claude-real-video says it uses scene-change detection plus deduplication. In plain language: it looks for visual changes, keeps frames that differ, drops near-duplicates, and can pack frames into contact sheets. The project's README gives one example: a 58-second clip where fixed one-frame-per-second sampling produced 58 frames, while the tool kept 26 meaningfully different frames and packed them into three contact sheets.
That is a small implementation detail with a large operational consequence. LLM context is limited. Human attention is limited. A pile of redundant frames makes both worse. A tighter evidence package gives the model less to misunderstand and gives the human reviewer less to verify.
When would a business use Claude to inspect video?
A business should use Claude to inspect video when the video contains work evidence that would otherwise be slow to review manually. Think of recordings that show a process, a customer interaction, a product walkthrough, or an ad structure.
The best early use cases are not cinematic. They are operational:
- A Loom showing how an employee handles a recurring admin task
- A screen recording of a bug or broken checkout flow
- A short sales-call clip where the team wants objections and follow-up items
- A competitor's public landing-page walkthrough or ad, reviewed for structure
- A training video that needs to become a checklist or SOP
- A recorded agent run that needs a human-readable receipt
This connects directly to agentic operations. A useful agent should not only say "done." It should leave evidence. Simon Willison's shot-scraper project, for example, includes the ability to record video demos of web work through Playwright. Pair that kind of run recording with a video-inspection tool and you get a stronger review loop: the agent acts, records what happened, then another model helps summarize the proof for a person.
That does not remove the human. It gives the human a better starting point. Instead of rewatching a ten-minute screen recording from scratch, the operator can review key frames, transcript excerpts, and the model's proposed summary.
How should you use Claude with video without fooling yourself?
Use Claude with video by narrowing the question before you extract evidence. The worst prompt is "summarize this video." It invites a generic answer. A better prompt is tied to the decision you need to make.
For a service business, the question might be:
- "Turn this SOP video into numbered steps and flag any step that depends on employee judgment."
- "Watch this agent run and list every visible action it took in the browser."
- "Review this sales call clip for the customer's actual objections, not generic sales advice."
What are the risks of using AI video analysis?
The main risks are missing evidence, weak transcripts, privacy leakage, and false confidence. None of them make video analysis useless. They mean the workflow needs guardrails.
Missing evidence is the first risk. Scene detection is better than blind sampling for many videos, but it is still a compression step. Compression always chooses. A frame that matters to your business may not look visually dramatic to the extractor. For critical review, keep access to the source video and spot-check the model's answer against it.
Transcript quality is the second risk. Video tools often rely on speech transcription, subtitles, or both. If the audio is noisy, multilingual, fast, or full of product names, the transcript can drift. Any quote that will be sent to a customer, used in training, or treated as a promise should be checked against the recording.
Privacy is the third risk. claude-real-video processes locally, and that is useful. But local processing is not the same as private end-to-end use. The moment a person uploads frames, transcripts, or audio to a cloud model, the business has made a data-sharing decision. Screen recordings often contain customer names, inboxes, internal dashboards, payment screens, or private URLs. Redact before upload, or keep sensitive review inside approved systems.
False confidence is the fourth risk. A model can produce a clean executive summary from messy evidence. That summary may read better than the underlying facts deserve. For low-stakes work, that may be acceptable. For hiring, legal, pricing, medical, financial, or customer-impacting decisions, the model's video review should be a draft, not the decision.
Is claude-real-video worth trying?
claude-real-video is worth trying if your team already uses video as work evidence and wastes time turning it into notes, checklists, or review summaries. It is less useful if your work rarely happens on screen or if your videos contain sensitive material you cannot safely extract and share.
A simple test is enough. Pick one non-sensitive recording under five minutes. Run it through the tool. Ask Claude for three outputs: a factual sequence of visible events, a transcript-based summary, and a list of uncertainties. Then compare all three against the source video.
If the sequence is accurate, you have a workflow candidate. If the summary is helpful but the uncertainties are weak, improve the prompt. If it misses the important moment, adjust extraction settings or keep the use case human-reviewed only.
For BBH's audience, the verdict is measured: this is not a replacement for watching important footage. It is a way to make routine footage inspectable. That is where the leverage sits.
How does account confidence scoring work?
Account confidence scoring usually means ranking a customer, lead, or account by how likely it is to fit a goal: convert, churn, expand, need support, or deserve immediate follow-up. In an AI workflow, the danger is the same as with agent confidence: the score can look authoritative before the evidence is strong. The practical approach is to pair any score with inspectable evidence — source agreement, test results, and human approval lanes — rather than trusting a self-generated number.
Confidence scoring (AI evaluation)
Instead of asking an AI agent to declare its own reliability, build a system that produces receipts. For an AI agent, that means five layers: sources, tests, logs, boundaries, and human approval for irreversible moves. Start with sources: make the agent cite the exact source it used. Then add tests: simple pass/fail checks like valid JSON, source citations, response length limits, avoidance of forbidden claims, and routing high-risk cases to human approval. Keep logs showing what input was received, what tools were called, what source was relied on, what changed, and who approved it. Then set boundaries: restrict which fields the agent may change, keep human approval lanes until the workflow has earned more autonomy. Finally, decide which actions need human approval — this is not anti-AI, it is how useful automation survives contact with reality.
A service business does not need AI to "understand video" as a parlor trick. It needs faster ways to turn lived work into reusable operating knowledge. Screen recordings become SOPs. Agent runs become receipts. Sales clips become follow-up notes. Bug videos become reproducible steps.
That is a practical use of AI: not outsourcing judgment, but reducing the cost of getting the evidence in front of judgment.
If your bottleneck is turning repeated work into managed systems, start with /agentic-systems. If the video is part of lead response, landing-page review, or ad analysis, the same operating discipline applies to /smb-marketing. BBH's venture work sits at /ventures: proof that useful systems are built, tested, and improved in the open.