Prosody

$ pnpm dlx shadcn@latest add @remocn/caption-prosody

Usage

CaptionProsody sizes every word by how long the speaker spent on each of its letters, measured against the rest of the page. A word held twice as long as the page's typical pace grows to maxScale, one said twice as fast shrinks to minScale, and everything in between scales smoothly. Sizes are fixed when a page starts, so the line never reflows: words then appear at the moment they are spoken, and the word being spoken takes activeColor.

The effect comes straight from the transcript timing, so it is only as expressive as the timestamps are precise — word-level timestamps from whisper.cpp or the OpenAI API work best. The component renders only the text block; see Positioning and Get a transcript.

import { AbsoluteFill } from "remotion";
import { CaptionProsody } from "@/components/remocn/caption-prosody";
import captions from "./captions.json";
 
export const MyScene = () => (
  <AbsoluteFill
    style={{
      justifyContent: "flex-end",
      alignItems: "center",
      paddingBottom: 108,
      paddingInline: 192,
    }}
  >
    <CaptionProsody captions={captions} />
  </AbsoluteFill>
);