H.264视频的RTPPayload格式详解

需积分: 10 30 浏览量更新于2024-08-01 收藏 1.66MB PDF 举报

本文主要探讨的是RTP(RReal-time Transport Protocol)有效载荷格式在传输H.264视频中的应用。RTP是一种网络协议，主要用于实时数据传输，如音频和视频流。H.264，又称为AVC(Advanced Video Coding)，是一种高效的视频编码标准，广泛应用于网络视频、电视广播、视频会议等领域。 RTP Payload Format是RTP协议中用于封装特定类型媒体数据的结构。对于H.264视频流，它定义了如何将压缩后的H.264 NAL（网络抽象层）单元包装到RTP包中，以便在网络上传输。这个过程涉及到NAL单元的分割、重新排序和可能的错误检测与恢复机制。在RTP Payload Format中，H.264的NAL单元可以被封装成一个或多个RTP包。这通常是因为单个NAL单元可能超过RTP包的最大大小限制，需要分割。同时，为了保证接收端能正确重建视频流，需要对这些分割的NAL单元进行标识和重组。在实际应用中，RTP和H.264的结合使用需要考虑以下几个关键点： 1. **NAL单元的分割与标识**：如果NAL单元太大，必须将其拆分为多个RTP包，每个包头包含NAL单元的标识信息，以便接收端能够识别这些片段并正确重组。 2. **时间戳同步**：RTP包头包含时间戳，用于指示携带的媒体数据的时间信息，这对于保持视频播放的同步至关重要。 3. **序列号**：每个RTP包都有一个序列号，用于接收端确认包的顺序和检测丢失的包。 4. **错误检测与恢复**：RTP可能会使用某些机制来检测传输错误，如通过校验和或者使用前向纠错(FEC)编码来增强数据的可靠性。 5. **RTCP(RTP控制协议)**：与RTP一起使用的RTCP用于提供质量反馈、统计信息以及同步和源认证等控制功能。了解RTP Payload Format对于H.264视频的重要性在于，它直接影响到网络视频服务的质量，包括视频的流畅性、清晰度以及错误恢复能力。在设计和实现基于H.264的实时通信系统时，必须深入理解并正确处理RTP payload格式，以确保最佳的用户体验。此外，上述链接提供的资源涵盖了网络协议、网络安全、无线通信技术、网络协议手册、TCP/IP快速指南、以太网快速指南、数据包分析、磁盘共享、磁盘访问、局域网调查、网络监控工具等多个方面，这些都是构建和支持高效、可靠的网络通信所必需的知识领域。掌握这些知识可以帮助你更好地理解和优化RTP在H.264视频传输中的性能。

Network Dictionary

http://www.javvin.com/networkdiction-

ary.html

Network Protocols Map

http://www.javvin.com/map.html

Network Security Map

http://www.javvin.com/securitymap.

html

Wireless Communications

Technology Map

http://www.javvin.com/wirelessmap.

html

Network Protocols Hand-

book

http://www.javvin.com/model.html

TCP/IP Quick Guide

http://www.javvin.com/tcpipguide.html

Ethernet Quick Guide

http://www.javvin.com/ethernetguide.

html

Packet Analyzer

http://www.javvin.com/packet.html

DiskShare

http://www.javvin.com/diskshare.html

DiskAccess

http://www.javvin.com/diskaccess.html

LANsurveyor

http://www.javvin.com/LANsurveyor.

html

CyberGauge

http://www.javvin.com/CyberGauge.

html

Easy Network Service

Monitor

http://www.javvin.com/easy.html

Business Card Scanner

http://www.javvin.com/businesscard-

scanner.html

Color Cards and Picture

Scanner

http://www.javvin.com/colorcardscan-

ner.html

Portable Document Scan-

ner

http://www.javvin.com/portablescan-

ner.html

www.javvin.com

www.networkdictionary.

com

Network Dictionary

http://www.javvin.com/networkdiction-

ary.html

Network Protocols Map

http://www.javvin.com/map.html

Network Security Map

http://www.javvin.com/securitymap.

html

Wireless Communications

Technology Map

http://www.javvin.com/wirelessmap.

html

Network Protocols Hand-

book

http://www.javvin.com/model.html

TCP/IP Quick Guide

http://www.javvin.com/tcpipguide.html

Ethernet Quick Guide

http://www.javvin.com/ethernetguide.

html

Packet Analyzer

http://www.javvin.com/packet.html

DiskShare

http://www.javvin.com/diskshare.html

DiskAccess

http://www.javvin.com/diskaccess.html

LANsurveyor

http://www.javvin.com/LANsurveyor.

html

CyberGauge

http://www.javvin.com/CyberGauge.

html

Easy Network Service

Monitor

http://www.javvin.com/easy.html

Business Card Scanner

http://www.javvin.com/businesscard-

scanner.html

Color Cards and Picture

Scanner

http://www.javvin.com/colorcardscan-

ner.html

Portable Document Scan-

ner

http://www.javvin.com/portablescan-

ner.html

www.javvin.com

www.networkdictionary.

com

RTP Payload Format for H.264 Video

February 2005

[ Page 9 ]

timestamp is set to the RTP timestamp of the primary coded picture of the access unit in which the NAL

unit is included, according to section 7.4.1.2 of [1].

The setting of the RTP Timestamp for MTAPs is dened in section 5.7.2.

Receivers SHOULD ignore any picture timing SEI messages included in access units that have only

one display timestamp. Instead, receivers SHOULD use the RTP timestamp for synchronizing the dis-

play process.

RTP senders SHOULD NOT transmit picture timing SEI messages for pictures that are not supposed

to be displayed as multiple elds.

If one access unit has more than one display timestamp carried in a picture timing SEI message, then the

information in the SEI message SHOULD be treated as relative to the RTP timestamp, with the earliest

event occurring at the time given by the RTP timestamp, and subsequent events later, as given by the

difference in SEI message picture timing values. Let tSEI1, tSEI2, ..., tSEIn be the display timestamps

carried in the SEI message of an access unit, where tSEI1 is the earliest of all such timestamps. Let

tmadjst() be a function that adjusts the SEI messages time scale to a 90-kHz time scale. Let TS be the

RTP timestamp. Then, the display time for the event associated with tSEI1 is TS. The display time for

the event with tSEIx, where x is [2..n] is TS + tmadjst (tSEIx - tSEI1).

Informative note: Displaying coded frames as elds is needed commonly in an operation known as

3:2 pulldown, in which lm content that consists of coded frames is displayed on a display using

interlaced scanning. The picture timing SEI message enables carriage of multiple timestamps for

the same coded picture, and therefore the 3:2 pulldown process is perfectly controlled. The picture

timing SEI message mechanism is necessary because only one timestamp per coded frame can be

conveyed in the RTP timestamp.

Informative note: Because H.264 allows the decoding order to be different from the display order,

values of RTP timestamps may not be monotonically non-decreasing as a function of RTP sequence

numbers. Furthermore, the value for interarrival jitter reported in the RTCP reports may not be a

trustworthy indication of the network performance, as the calculation rules for interarrival jitter

(section 6.4.1 of RFC 3550) assume that the RTP timestamp of a packet is directly proportional to

its transmission time.

5.2. Common Structure of the RTP Payload Format

The payload format denes three different basic payload structures. A receiver can identify the payload

structure by the rst byte of the RTP payload, which co-serves as the RTP payload header and, in some

cases, as the rst byte of the payload. This byte is always structured as a NAL unit header. The NAL unit

type eld indicates which structure is present. The possible structures are as follows:

Single NAL Unit Packet: Contains only a single NAL unit in the payload. The NAL header type eld will

be equal to the original NAL unit type; i.e., in the range of 1 to 23, inclusive. Specied in section 5.6.

Aggregation packet: Packet type used to aggregate multiple NAL units into a single RTP payload. This

packet exists in four versions, the Single-Time Aggregation Packet type A (STAP-A), the Single-Time Ag-

gregation Packet type B (STAP-B), Multi-Time Aggregation Packet (MTAP) with 16-bit offset (MTAP16),

and Multi-Time Aggregation Packet (MTAP) with 24-bit offset (MTAP24). The NAL unit type numbers

assigned for STAP-A, STAP-B, MTAP16, and MTAP24 are 24, 25, 26, and 27, respectively. Specied in

section 5.7.

Fragmentation unit: Used to fragment a single NAL unit over multiple RTP packets. Exists with two ver-

sions, FU-A and FU-B, identied with the NAL unit type numbers 28 and 29, respectively. Specied in

section 5.8.

Table 1. Summary of NAL unit types and their payload structures

Type Packet Type name Section

------------------------------------------------------------------------------------------

0 undened -

1-23 NAL unit Single NAL unit packet per H.264 5.6

24 STAP-A Single-time aggregation packet 5.7.1

25 STAP-B Single-time aggregation packet 5.7.1

26 MTAP16 Multi-time aggregation packet 5.7.2

27 MTAP24 Multi-time aggregation packet 5.7.2

Network Dictionary

http://www.javvin.com/networkdiction-

ary.html

Network Protocols Map

http://www.javvin.com/map.html

Network Security Map

http://www.javvin.com/securitymap.

html

Wireless Communications

Technology Map

http://www.javvin.com/wirelessmap.

html

Network Protocols Hand-

book

http://www.javvin.com/model.html

TCP/IP Quick Guide

http://www.javvin.com/tcpipguide.html

Ethernet Quick Guide

http://www.javvin.com/ethernetguide.

html

Packet Analyzer

http://www.javvin.com/packet.html

DiskShare

http://www.javvin.com/diskshare.html

DiskAccess

http://www.javvin.com/diskaccess.html

LANsurveyor

http://www.javvin.com/LANsurveyor.

html

CyberGauge

http://www.javvin.com/CyberGauge.

html

Easy Network Service

Monitor

http://www.javvin.com/easy.html

Business Card Scanner

http://www.javvin.com/businesscard-

scanner.html

Color Cards and Picture

Scanner

http://www.javvin.com/colorcardscan-

ner.html

Portable Document Scan-

ner

http://www.javvin.com/portablescan-

ner.html

www.javvin.com

www.networkdictionary.

com

RTP Payload Format for H.264 Video

February 2005

[ Page 10 ]

28 FU-A Fragmentation unit 5.8

29 FU-B Fragmentation unit 5.8

30-31 undened -

Informative note: This specication does not limit the size of NAL units encapsulated in single NAL

unit packets and fragmentation units. The maximum size of a NAL unit encapsulated in any aggrega-

tion packet is 65535 bytes.

5.3. NAL Unit Octet Usage

The structure and semantics of the NAL unit octet were introduced in section 1.3. For convenience, the

format of the NAL unit type octet is reprinted below:

This section species the semantics of F and NRI according to this specication.

F: 1 bit

forbidden_zero_bit. A value of 0 indicates that the NAL unit type octet and payload should not contain

bit errors or other syntax violations. A value of 1 indicates that the NAL unit type octet and payload may

contain bit errors or other syntax violations.

MANEs SHOULD set the F bit to indicate detected bit errors in the NAL unit. The H.264 specication

requires that the F bit is equal to 0. When the F bit is set, the decoder is advised that bit errors or any

other syntax violations may be present in the payload or in the NAL unit type octet. The simplest de-

coder reaction to a NAL unit in which the F bit is equal to 1 is to discard such a NAL unit and to conceal

the lost data in the discarded NAL unit.

NRI: 2 bits

nal_ref_idc. The semantics of value 00 and a non-zero value remain unchanged from the H.264 speci-

cation. In other words, a value of 00 indicates that the content of the NAL unit is not used to reconstruct

reference pictures for inter picture prediction. Such NAL units can be discarded without risking the

integrity of the reference pictures. Values greater than 00 indicate that the decoding of the NAL unit is

required to maintain the integrity of the reference pictures.

In addition to the specication above, according to this RTP payload specication, values of NRI great-

er than 00 indicate the relative transport priority, as determined by the encoder. MANEs can use this

information to protect more important NAL units better than they do less important NAL units. The

highest transport priority is 11, followed by 10, and then by 01; nally, 00 is the lowest.

Informative note: Any non-zero value of NRI is handled identically in H.264 decoders. Therefore,

receivers need not manipulate the value of NRI when passing NAL units to the decoder.

An H.264 encoder MUST set the value of NRI according to the H.264 specication (subclause 7.4.1)

when the value of nal_unit_type is in the range of 1 to 12, inclusive. In particular, the H.264 specica-

tion requires that the value of NRI SHALL be equal to 0 for all NAL units having nal_unit_type equal

to 6, 9, 10, 11, or 12.

For NAL units having nal_unit_type equal to 7 or 8 (indicating a sequence parameter set or a picture

parameter set, respectively), an H.264 encoder SHOULD set the value of NRI to 11 (in binary format).

For coded slice NAL units of a primary coded picture having nal_unit_type equal to 5 (indicating a

coded slice belonging to an IDR picture), an H.264 encoder SHOULD set the value of NRI to 11 (in

binary format).

For a mapping of the remaining nal_unit_types to NRI values, the following example MAY be used and

has been shown to be efcient in a certain environment [13]. Other mappings MAY also be desirable,

depending on the application and the H.264/AVC Annex A prole in use.

Informative note: Data Partitioning is not available in certain proles; e.g., in the Main or Baseline

proles. Consequently, the nal unit types 2, 3, and 4 can occur only if the video bitstream conforms

to a prole in which data partitioning is allowed and not in streams that conform to the Main or

Baseline proles.

剩余46页未读，继续阅读

zihundezhongcheng

粉丝: 1

H.264视频的RTPPayload格式详解

RTP Payload Format for H.264&H.265 Video

RFC-6184 RTP Payload Format for H.264 Video (H.264)

RFC-2032 RTP Payload Format for H.261 Video Streams (FIR)

RFC 3984 - RTP Payload Format for H_264 Video

RFC3984 RTP Payload Format for H264 Video

RTP_Payload_Format_for_H.264_Video

RTP Payload Format for Scalable Video Coding

RFC-7798 RTP Payload Format for High Efficiency Video Coding

RFC-7741 RTP Payload Format for VP8 Video (VP8)

RFC-7587 RTP Payload Format for the Opus Speech and Audio Codec

最新资源