SS之发展史和Technical Overview

 IIS Smooth Streaming Technical Overview

 http://learn.iis.net/page.aspx/626/smooth-streaming-technical-overview/

主要讲了SS的发展史。这篇是文档的记录笔记，很多东西总是看的时候很清楚，过后一段时间不接触就能忘的一干二净。

Smooth Streaming: HTTP-based adaptive streaming extension of IIS 7.0.

1. Multiple Bit Rate Streaming

    Microsoft 最初与2000年引入MBRT这个概念，使用用ASF文件格式，allows storage of multiple video and audio tracks inside a single file,and the WIndows Media streaming protocols support swtiching streams during a broadcast.

2. Intelligent Streaming

    改进了MBR技术，combined bandwidth detection,stream thinning等，但是仍然需要media encode成MBR ASP 文件。

3. From streaming protocols（RTSP,MMS,RTMP） back to plain HTTP download

4. Media delivery methods:

1) Traditional streaming

    RTSP(Real-Time Streaming Protocol) 就是传统流协议的好例子，a stateful protocol. Client 连到streaming server以后，client通过发送像PLAY,PAUSE,TEARDOWN这些命令给server，这样server一直知道client的状态直到断开。client和server之间建立session以后，the server begins sending the media as a steady stream of small packets(The format of these packets is kNown as RTP).一般RTP包的大小是1452bytes。如果视频流的bit rate是1Mbps的话，每个packet负载 (1452*8) / (1024&1024) = 11 (ms)的视频信息。

    而HTTP是一个stateless的协议，Windows Media Services 使用修改过的HTTP，MS-WMSP（Windows Media HTTP Streaming Protocol）来做streaming。MS-WMSP 使用标准HTTP来传输数据和消息，同时维护session状态。Silverlight只支持WMS HTTP-based delivery.

    向RTSP和MS-WMSP这样的传统流协议有2个重要特点。一是，The server sends the data packets to the client as a real-time rate only. Meida endcode成多少比特率，streaming的时候就是多少。二是，The server only sends ahead enough data packets to fill the client buffer. Client buffer 一般是1-10秒（WMP和Silverlight都是5秒），server最多只会多穿1-10秒的数据过来，就算暂停了半个小时，也就缓冲了1-10秒。

2) Progressive download

    Progressive：Most player clients allow the media file to be played back while the downloads is still in progress. 连接建立以后，HTTP web server向client发送数据，其实就是把整个媒体文件都下载到web browser的cache里头。如果用户暂停半个小时，可能视频已经下载完了就可以直接观看了。现在的视频网站比如YouTube，myspace都是使用这种技术。但是如果用户又看了1分钟觉得不好看，那就白白多下载了那么多。IIS7 提供了一个叫做Bit Rate Throttling 的extension来缓解这种浪费。

3) HTTP based Adaptive streaming

    看起来像streaming但其实是HTTP progressive download。In a typical adaptive streaming implementation,the video/audio source is cut into many short segments(chunks) and encoded. Chunks are typically 2-4 seconds long. At the video codec level,this typically means that each chunk is cut along video GOP (Group of Pictures) boundaries (each chunk starts with a key frame) and has no dependencies on past or future chunks/GOPs. 这样每一个chunk都是独立的，client收到以后就有全部的信息进行解码。

    The encoded chunks are hosted on a HTTP Web server. A client requests the chunks from the web server in a linear fashion and downloads them using plain HTTP progressive download. As the chunks are downloaded to the client,the client plays back the sequence of chunks in linear order. Adaptive 的部分体现在，如果video/audio被编码成很多不同的比特率，每种比特率的流都被分成很多2-4秒的chunk，那么client就可以根据网络带宽，cpu处理速度等情况，请求不同比特率的chunk。

    NBC 2008 年北京奥运会网站就使用了这种技术并且获得了成功。整个开发过程分为3大步：

    a）Use encoders to produce multiple WMV files of different bit rates/resolutions for each source.

    b) Split each WMV files into thousands of 2s chunks(actually small files)

    c) Upload the chunks to the CDN'S web server and then build a Silverlight player that would download the chunks and play them in sequence.

    这种技术的缺点在于，每一个chunk最后都成一个小文件，可以想象最后要维护多少个文件！

5. IIS Smooth Streaming

    跟上面的adaptive streaming差不多，但是优化了其缺点，重新设计了content creation 和delivery部分，one-file-per-chunk 演变成a single contiguous file for each encoded bit rate. IIS SS uses the MPEG-4 Part 14 (ISO/IEC 14496-12) file format as its disk(storage) and wire(transport) format. SS spec defines each chunk/GOP as an MPEG-4 Movie Fragment and stores it within a contiguous MP4 file for easy random access. 这样每一个bit rate 流对应一个MP4文件，用户请求特定的source time segment时，server动态的在MP4文件中找合适的Movie Fragment Box （chunk是按需生成的），单独发送出去。而怎么根据用户的网络等情况请求不同bit rate的chunk，这个重任就落到了client player的身上，silverlight会帮我们搞定。server是不管的，只是按url参数办事。

    MP4的几个优点：支持H.264 codec；支持payload fragmentation within the file。MP4 file specification: ISO/IEC 14492-12 ISO Base Media File Format specification.

    SS 有两种格式：Live （wire formate）和VOD (disk file format)。

    MP4文件的基本单元是Box，里面可以有data和Metadata。一般情况下，Metadata会写在data前面，这样client在播放的时候就能先获得关于data的信息，但是不适用于live streaming的情况。MP4 file spec allow MP4 Boxes to be organized in a fragmented manner,where the file can be written "as you go" as a series of short Metadata/data Box pairs,rather than one long Metadata/data pair. 所以Smooth Streaming files 也被称为Fragmented MP4 files or fMp4.

    High-level overview of SS file:

    The file starts with file-level Metadata ('moov') that generically describes the file,but the bulk of the payload is actually contained in the fragment Boxes that also carry more accurate fragment-level Metadata ('moof') and media data ('mdat'). A typical SS file has a fragment for each 2 seconds of video/audio. Closing the file is an 'mfra' index Box that allows easy and accurate seeking within the file.

    MP4 fragment:

6. SS Media Assets

    上一篇SS之创建SS Web Server中下载了Big Buck Bunny的SS资源，解压后的文件列表：

1) *.ismv

如果MP4文件既有视频又有音频，或者只有视频的话，每个比特率的文件被编码成一个ismv文件。

2) *.isma

如果MP4文件只有音频的话，每个比特率的文件被编码成一个isma文件。

3) *.ism

Server manifest file （XML文件）. 只被IIS SS server使用，describes the relationships between the media tracks,bit rates and files on disk. 这个manifest 格式基于SMIL 2.0 XML format specification.

4) *.ismc

Client manifest file （XML文件）. 这是server传给client的第一个文件，描述了server可以提供的流信息，包括codec,bit rates encoded,video resolutions,markers,captions,etc.

    Support VC-1,WMA,H.264 and AAC codecs.

7. SS playback

1) The server must translate URL requests into exact byte range offsets within the MP4 file.

2) The client can request chunks in a more developer-friendly manner.

    client 向 server请求的第一个东西就是ismc manifest文件，client就可以知道视频是用什么样的codec来编码的，有哪些bit rate可选，可选的chunks列表。接着client向server请求SS chunks：http://video.foo.com/NBA.ism/QualityLevels(400000)/Fragments(video=610275114)

这个URL包含的意思是，请给我bit rate是400000，fragment偏移为610275114的chunk。The fragment start offset is expressed in an agreed-upon time unit (usually 100 ns).

    收到这个URL以后，server在 *.ism server manifest 文件里找QualityLevel对应哪个*.ismv/*.isma文件，然后读取这个*.ismv/*.isma文件，根据 tfra index Box，找到URL请求的fragment Box （‘moof’ + 'mdat'）。之后把这个fragment Box 拿出来，以一个单独文件的形式发送给client。The sent fragment/file can Now be automatically cached further down the network,这样另外一个client请求同样的fragment时，可以直接从cache发过去。