Direct answer
QR Send streams live media by capturing a camera, microphone, or local media file with MediaRecorder, encoding it to fragmented MP4 (H.264/AAC, WebM fallback) at roughly 200–300 kbps and about 360p, then pushing each fragment over the same fountain-coded QRTP optical stream that carries files. The receiver appends fragments into Media Source Extensions and keeps scanning while it plays.
Key takeaways
- Media is not a separate protocol: each MediaRecorder fragment is wrapped in the same transfer payload a file uses, so the fountain coding that protects a document protects the video.
- Two streams share each frame:
stream_id = 0carries the init chunk (kind: "init"),stream_id = 1carries the live fragments (kind: "media"), and the init chunk is re-sent about every 10 encoder frames so a late receiver can start playback. - The bitrate budget is the design constraint: roughly 200–300 kbps of H.264/AAC, video scaled to about 360p before encoding, and audio-only when a voice link is all you need.
- Every media chunk carries transfer metadata
{ "type": "media", "mime": "...", "kind": ... }; thatmimestring doubles as the MSESourceBuffertype on the receiver. - The receiver runs in continuous mode, appends fragments in arrival order, skips gaps instead of stalling, drops stale fragments, and starts playback at a segment boundary.
How a live stream is built
Capture and encode
The sender opens a MediaStream from a camera and microphone, or plays a local audio/video file and captures its output, and feeds it to MediaRecorder. MediaRecorder emits a container init segment describing the codecs and track layout, then a rolling sequence of media fragments: fragmented MP4 with H.264 and AAC preferred, WebM as the fallback.
Everything after this is deliberate under-encoding, because the frames have to be photographed by a camera rather than delivered over a socket. The media budget is roughly 200–300 kbps, and video is scaled to about 360p before encoding — usable for faces, whiteboards and screen content, not a video call.
How the chunks ride the same QRTP stream
Media gets no second protocol. Each chunk is wrapped in the same transfer payload a file would use, marked with { "type": "media", "mime": "<container mime>", "kind": "init" | "media" }, and pushed down the same optical stream described in how a stream of QR codes carries a whole file.
A media transfer uses two streams:
stream_id = 0, the init stream: the first MediaRecorder chunk (container init plus the first fragment), re-sent about every 10 encoder frames so a receiver that starts scanning late has the codec description before it can decode anything.stream_id = 1, the media stream: one fragment per stream segment. The sender replaces that stream's source message as fragments arrive, and the descriptor'ssegment_ididentifies fragment order.
With both streams present the frame is a full frame with explicit stream headers, not a squeezed frame. The media stream's segmented flag means a descriptor whose segment_id differs from the one being decoded makes the receiver discard the old fountain and start a new one — so playback never waits on a fragment that will never arrive:
frame : qrtp | transfer_id | stream headers... | packets...
stream 0 : stream_type=0 stream_id=0 kind="init" (init + first fragment)
stream 1 : stream_type=0 stream_id=1 kind="media" segment_id = fragment order
packets : 368 B (large code) or 40 B (small code)
block_id: u32 | Wirehair payload: packet_size - 4 bytesThe budget: frame rate and code size
The optical payload rate is frames per second × packets per frame × packet size — 368 bytes for a large code, 40 bytes once the symbol gets small. The media bitrate has to fit inside that with room for fountain overhead and for the init stream taking part of the frame budget. The CLI sender exposes these knobs (--fps, --cols, --rows, --packets-per-qr, --qrs-per-frame, --ecc), covered in the QR Send CLI guide.
| Frame rate | Packets per frame | Packet size | Optical payload | Fits ~200–300 kbps media? |
|---|---|---|---|---|
| 10 fps | 2 | 368 B | ~7.4 kB/s (~59 kbps) | no — audio-only territory |
| 15 fps | 2 | 368 B | ~11 kB/s (~88 kbps) | no — audio-only territory |
| 20 fps | 2 | 368 B | ~14.7 kB/s (~118 kbps) | marginal, room for voice |
| 25 fps | 4 | 368 B | ~36.8 kB/s (~294 kbps) | yes, little headroom |
| 30 fps | 4 | 368 B | ~44.2 kB/s (~353 kbps) | yes, with headroom |
| 30 fps | 4 | 40 B | ~4.8 kB/s (~38 kbps) | no — small codes cannot carry video |
These are raw payload ceilings with every frame scanning. A real camera drops frames, so keep the payload rate comfortably above the 200–300 kbps the recorder is producing. Small codes buy reliability at distance and on slow cameras, not video.
On the receiving side
Four behaviours make playback work on a link that always loses something:
- It scans while it plays. The receiver runs in continuous mode; a live stream spans many transfers, and single-transfer mode would stop accepting new ones after the first.
- It appends fragments in arrival order into Media Source Extensions, using the chunk's
mimeas theSourceBuffertype. - It skips gaps instead of stalling, and drops stale queued fragments so playback stays on the newest received chunk rather than drifting into the past.
- It starts at a segment boundary. Because the init chunk keeps coming, a receiver that joins late gets a codec description, a fragment boundary to start on, and usable output within a few frames instead of a decoder error.
Audio-only is the better voice link
Microphone-only capture follows the same path with far less to carry, so the same frame budget buys more margin: fewer frames per second are needed, fragments are smaller, and there is nothing to scale. If the goal is to carry a voice across an air gap — a radio check, a spoken key — audio-only is the configuration to choose.
What costs you frames
Three things eat the frame rate you configured, because each raises the effective erasure rate: fast motion, fine detail, and distance. Fast motion costs on both ends — a moving subject compresses badly at 200–300 kbps and produces larger fragments competing for the same frames, while a receiver that re-focuses or chases a bright screen through motion misses frames outright. Fine detail is where the low bitrate shows first: dense text, thin lines and busy textures have little room at 360p, and the usual fix is a softer picture rather than a bitrate the optical link cannot carry. Distance costs scan reliability directly, since a small symbol at the same distance needs a bigger code, more error correction, or a slower frame rate to stay readable — and larger codes carry 368-byte packets while small codes fall back to 40 bytes and lose most of the throughput.
Tune a media stream like a radio link, not a web page: pick the lowest resolution and frame rate you can tolerate, keep the sender's screen bright and still, keep the receiver close to square-on, and treat glitches as normal rather than as failures.
FAQ
Can I use this for a live video call?
No. QRTP is a unidirectional optical link and the media budget is roughly 200–300 kbps at about 360p, so the result is a watchable low-bitrate stream rather than an interactive call. When a network exists, the sender also advertises a WebRTC peer connection inside the optical metadata side stream and the transfer can move to network speed.
What do I actually point the phone at?
The sender's screen. The browser sender draws the QR frames and the receiver opens the receive page (or the embedded receiver) and aims its camera at that display; nothing needs to be installed on the receiving device beyond a browser camera.
Why does the video look soft?
The sender scales to about 360p and encodes into roughly 200–300 kbps before the frames are drawn, because a camera has to read every QR symbol. Fine detail is the first thing to go; audio-only or a lower frame rate buys margin back.
What happens when the camera misses a frame?
Nothing fatal. Fountain coding lets the receiver recover what it needs from the packets it actually saw; if a fragment is still missing, the player skips the gap and continues instead of halting.
Can someone join the stream halfway through?
Yes. The sender keeps re-sending the init chunk — container init plus the first fragment — about every 10 encoder frames, and playback starts at the beginning of a segment, so a receiver that starts scanning late still gets usable output.
Related reading
- How a stream of QR codes carries a whole file — A QR code holds at most a few kilobytes. Here is how thousands of them carry an arbitrary file, and why lost frames do not matter.
- Fountain codes in a stream of QR codes — Why QRTP uses Wirehair instead of retransmission, and what that buys when frames are blurred, dropped or read out of order.
- QR Send CLI guide for power users — The qrsend binary renders QR streams straight in your terminal, with flags for frame rate, grid size, error correction, colours and auditing.