Hint media
Provide information about data units to stream in hint tracks.
Overview
The QuickTime file format supports streaming of media data over a network as well as local playback. The process of sending protocol data units is time-based, just like the display of time-based data, and is therefore suitably described by a time-based format. A QuickTime file or movie that supports streaming includes information about the data units to stream. This information is included in additional tracks of the movie called hint tracks.
Hint tracks contain instructions for a streaming server which assist in the formation of packets. These instructions may contain immediate data for the server to send (for example, header information) or reference segments of the media data. These instructions are encoded in the QuickTime file in the same way that editing or presentation information is encoded in a QuickTime file for local playback.
Instead of editing or presentation information, information is provided which allows a server to packetize the media data in a manner suitable for streaming, using a specific network transport.
The same media data is used in a QuickTime file which contains hints, whether it is for local playback, or streaming over a number of different transport types. Separate hint tracks for different transport types may be included within the same file and the media will play over all such transport types without making any additional copies of the media itself. In addition, existing media can be easily made streamable by the addition of appropriate hint tracks for specific transports. The media data itself need not be recast or reformatted in any way.
Typically, hinting is performed by media packetizer components. QuickTime selects an appropriate media packetizer for each track and routes each packetizer’s output through an Apple-provided packet builder to create a hint track. One hint track is created for each streamable track in the movie.
Hint tracks are quite small compared with audio or video tracks. A movie that contains hint tracks can be played from a local disk or streamed over HTTP, similar to any other QuickTime movie. Hint tracks are only used when streaming a movie over a real-time media streaming protocol, such as RTP.
Support for streaming in the QuickTime file format is based upon the following considerations:
Media data represented as a set of network-independent standard QuickTime tracks, which may be played or edited, as normal.
A common declaration and base structure for server hint tracks; this common format is protocol independent, but contains the declarations of which protocols are described in the server tracks.
A specific design of the server hint tracks for each protocol which may be transmitted; all these designs use the same basic structure.
The resulting streams, sent by the servers under the direction of hint tracks, do not need to contain any trace of QuickTime information. This approach does not require that QuickTime, or its structures or declaration style, be used either in the data on the wire or in the decoding station. For example, a QuickTime file using H.261 video and DVI audio, streamed under Real-Time Protocol (RTP), results in a packet stream which is fully compliant with the IETF specifications for packing those codings into RTP.
Hint tracks are built and flagged, so that when the movie is viewed directly (not streamed), they are ignored.
The next section describes a generic format for streaming hints to be stored in a QuickTime movie.
Adding hint tracks to a movie
To store packetization hints, one or more hint tracks are added to a movie. Each hint track contains hints for at least one actual media track to be streamed. A streamed media track may have more than one hint track. For example, it might have a separate hint track for the different packet sizes the server supports, or it might have different hint tracks for different protocols. It is not required that all media tracks have corresponding hint tracks in a movie.
The sample time of a hint sample corresponds to the sample time of the media contained in the packets generated by that hint sample. The hint sample may also contain a transmission time for each packet. (The format for the hint sample is specific to the hint track type.)
The hint track may have a different time scale than its referenced media tracks.
The flags field in the track header atom ('tkhd') must be set to 0x000000, indicating that the track is inactive and is not part of the movie, preview, or poster.
The subType field of the handler description atom ('hdlr') contains ’hint', indicating that the media type is packetization hints.
Note that if a QuickTime media track is edited, any previously stored packetization hints may become invalid. Comparing the modification dates of the media track and the hint track is one way to determine this scenario, but it is far from being foolproof. Since the hint track keeps track of which original track media samples and sample descriptions to play at specific times, changes that affect those parts of the original track or media make those hints invalid. Changes to a movie that do not invalidate existing hint tracks include flattening (when there are no edit lists), and adding new tracks. Changes that invalidate hint tracks include:
Flattening (when there are edit lists)
Adding or deleting samples
Changing a track’s time scale
Changing sample descriptions
Packetization hint media header atom
In QuickTime movies, the media information atom ('minf') contains header data specific to the media. For hint tracks, the media header is a base media information atom ('gmhd'). The hint track must contain the base media information atom.
Hint track user data atom
Each hint track may contain track user data atoms that apply to only to the corresponding hint track. There are currently two such atoms defined.
- User data atom type
'hinf' This contains statistics for the hint track. The
'hinf'atom contains child atoms as defined in the table in the Movie info data atom section below. In some cases, there are both 32-bit and 64-bit counters available. Ignore any unknown types.- User data atom type
'hnti' This may contain child atoms. Child atoms that start with
'sdp '(note, again, the space) contain SDP text for this track. Text from these child atoms must be inserted into the proper place in the SDP text for the movie, after any common SDP text. This is analogous to the movie-level'hnti'atom.
Movie hint info atom
A movie may contain an 'hnti' movie user data atom, which may contain one or more child atoms. The child atom contents start with 4 bytes that specify the transport and 4 bytes that specify the type of data contained in the rest of the child atom. Currently, the only defined transport is 'rtp ' (note the space) and the only content data type defined is 'sdp ' (note the space). Skip any child atoms whose transport or type combinations you don’t recognize.
Insert the text from an atom of type 'rtp sdp ' in the proper place in the SDP information generated from this file (for example, by a streaming server) before any SDP information for specific tracks.
The following table describes the type and values of the 'hnti' atom.
Type | Value | Description |
|---|---|---|
| 8 bytes | The total number of bytes that will be sent, including 12-byte RTP headers, but not including any network headers. |
| 4 bytes | 4-byte version of |
| 8 bytes | The total number of network packets that will be sent (if the application knows there is a 28-byte network header, it can multiply 28 by this number and add it to the |
| 4 bytes | 4-byte version of |
| 8 bytes | The total number of bytes that will be sent, not including 12-byte RTP headers. |
| 4 bytes | 4-byte version of |
| 8 bytes | The maximum data rate. This atom contains two numbers: |
| 8 bytes | The number of bytes from the media track to be sent. |
| 8 bytes | The number of bytes of immediate data to be sent. |
| 8 bytes | The number of bytes of repeated data to be sent. |
| 4 bytes | The smallest relative transmission time, in milliseconds. |
| 4 bytes | The largest relative transmission time, in milliseconds. |
| 4 bytes | The largest packet, in bytes; includes 12-byte RTP header. |
| 4 bytes | The largest packet duration, in milliseconds. |
| Variable | The payload type, which includes payload number (32-bits) followed by |