FrameInfo schema

The canonical location of this page is https://strawlab.org/89H.

This page describes the format of data saved as Supplemental Enhancement Information (SEI) to contain frame timing information inline within videos recorded from RTP/RTCP cameras. Such cameras send rtp, an RTP timestamp, with each video frame. This is from a nominally 90 kHz clock with an unknown starting point and can be used with other information to determine absolute frame timing. When each frame is received, an additional timestamp recv is recorded which contains the absolute, wall-clock timestamp of the receiving computer.

The data are saved as pure ASCII in JSON format. This avoids the need to insert emulation prevention bytes. The schema is determined by the derive functionality from the rust serde crate using the following definition:

/// Timing information associated with each video frame
#[derive(Debug, Clone, Serialize, Deserialize)]
struct FrameInfo {
    /// Receive timestamp as NTP (Network Time Protocol) timestamp
    recv: u64,
    /// RTP (Real Time Protocol) timestamp as reported by the sender
    rtp: u32,
}

The JSON-encoded values are stored together with UUID strawlab.org/89H as a SEI User Data Unregistered payload.

Data in this format is saved in H264 streams by retina-record as described here.