摘要：在前两篇Silverlight的文章中跟大家一块学习了Silverlight的基础知识、Silverlight摄像头麦克风的相关操作以及截图、声音录制等，在文章后面也简单的说明了为什么没有视频录制，今天就和大家一块看一下上一节中最后的一个问题：如何使用Silverlight进行视频录制。
主要内容：
1.nesL项目简介
2.使用nesL实现视频录制
3.注意
一、nesL项目简介
在silverlight 中如何录制视频？相信这个问题有不少朋友都搜索过，但是好像目前还没有见到很好的答案，究其原因其实就是视频编码问题。当然也有朋友提到直接进行截图，只要每秒截取足够多的图片，然后依次播放就可以形成视频。但是我看到国外一个朋友使用此方法进行了几十秒的视频录制，其文件大小就达到了百兆级别，而且还进行了优化。因此这种方式要实现视频录制就目前而言还不是很合适。那么到底有没有好的方法呢？答案是有，但有限制，那就是借助于nesL。
Native Extensions for Silverlight（简称nesL）是由微软Silverlight团队进行开发，其目的主要为了增强Silverlight Out-of-browser离线应用的功能。大家都知道虽然Silverlight 4的OOB应用支持信任人权限提升功能，允许Silverlight的OOB应用对COM组件的访问，但对绝大多数Windows API仍旧无法调用，而nesL的出现正是为了解决这个问题。在最新的nesL 2.0中包含了大量有用的功能，而这其中就包括今天要说的视频编码部分。在nesL中有一个类库Microsoft.Silverlight.Windows.LocalEncode.dll主要负责本地视频和音频编码，这里就是用此类库来解决上面提到的视频录制问题。
二、使用nesL实现视频录制
在Microsoft.Silverlight.Windows.LocalEncode.dll中一个核心类就是EncodeSession，它负责音频和视频的编码输出工作。使用EncodeSession进行视频录制大概分为下面两步：
1.准备输入输出信息
在这个过程中需要定义VideInputFormatInfo、AudioInputFormatInfo、VideoOutputFormatInfo、AudioOutputFormatInfo和OutputContainerInfo，然后调用EncodeSession.Prepare()方法。
2.捕获视频输出
当输入输出信息准备好之后接下来就是调用EncodeSession.Start()方法进行视频编码输出。当然为了接收音频和视频数据必须准备两个sink类，分别继承于AudioSink和Videosink，在这两个sink中指定CaptureSource，并且在对应的OnSample中调用EncodeSession的WirteVideoSample()和WirteAudioSample()接收并编码数据（关于AudioSink在前面的文章中已经说过，Videosink与之类似）。
知道了EncodeSession的使用方法后下面就将其操作进行简单封装，LocalCamera.cs是本例中的核心类：
using System; using System.Collections.ObjectModel; using System.IO; using System.Windows; using System.Windows.Threading; using System.Windows.Media; using System.Windows.Controls; using System.Windows.Shapes; using Microsoft.Silverlight.Windows.LocalEncode; namespace Cmj.MyWeb.MySilverlight.SilverlightMeida { /// <summary> /// 编码状态 /// </summary> public enum EncodeSessionState { Start,Pause,Stop } /// <summary> /// 本地视频对象 /// </summary> public class LocalCamera { private string _saveFullPath = ""; private uint _videoWidth = 640; private uint _videoHeight = 480; private VideosinkExtensions _videosink = null; private AudioSinkExtensions _audioSink= null; private EncodeSession _encodeSession = null; private UserControl _page = null; private CaptureSource _cSource = null; public LocalCamera(UserControl page,VideoFormat videoFormat,AudioFormat audioFormat) { //this._saveFullPath = saveFullPath; this._videoWidth = (uint)videoFormat.PixelWidth; this._videoHeight = (uint)videoFormat.PixelHeight; this._page = page; this.SessionState = EncodeSessionState.Stop; //this._encodeSession = new EncodeSession(); _cSource = new CaptureSource(); this.VideoDevice = DefaultVideoDevice; this.VideoDevice.DesiredFormat = videoFormat; this.AudioDevice = DefaultAudioDevice; this.AudioDevice.DesiredFormat = audioFormat; _cSource.VideoCaptureDevice = this.VideoDevice; _cSource.AudioCaptureDevice = this.AudioDevice; audioInputFormatInfo = new AudioInputFormatInfo() { SourceCompressionType = FormatConstants.AudioFormat_PCM }; videoInputFormatInfo = new VideoInputFormatInfo() { SourceCompressionType = FormatConstants.VideoFormat_ARGB32 }; audioOutputFormatInfo = new AudioOutputFormatInfo() { TargetCompressionType = FormatConstants.AudioFormat_AAC }; videoOutputFormatInfo = new VideoOutputFormatInfo() { TargetCompressionType = FormatConstants.VideoFormat_H264 }; outputContainerInfo = new OutputContainerInfo() { ContainerType = FormatConstants.TranscodeContainerType_MPEG4 }; } public LocalCamera(UserControl page,VideoCaptureDevice videoCaptureDevice,AudioCaptureDevice audioCaptureDevice,AudioFormat audioFormat) { //this._saveFullPath = saveFullPath; this._videoWidth = (uint)videoFormat.PixelWidth; this._videoHeight = (uint)videoFormat.PixelHeight; this._page = page; this.SessionState = EncodeSessionState.Stop; //this._encodeSession = new EncodeSession(); _cSource = new CaptureSource(); this.VideoDevice = videoCaptureDevice; this.VideoDevice.DesiredFormat = videoFormat; this.AudioDevice = audioCaptureDevice; this.AudioDevice.DesiredFormat = audioFormat; _cSource.VideoCaptureDevice = this.VideoDevice; _cSource.AudioCaptureDevice = this.AudioDevice; audioInputFormatInfo = new AudioInputFormatInfo() { SourceCompressionType = FormatConstants.AudioFormat_PCM }; videoInputFormatInfo = new VideoInputFormatInfo() { SourceCompressionType = FormatConstants.VideoFormat_ARGB32 }; audioOutputFormatInfo = new AudioOutputFormatInfo() { TargetCompressionType = FormatConstants.AudioFormat_AAC }; videoOutputFormatInfo = new VideoOutputFormatInfo() { TargetCompressionType = FormatConstants.VideoFormat_H264 }; outputContainerInfo = new OutputContainerInfo() { ContainerType = FormatConstants.TranscodeContainerType_MPEG4 }; } public EncodeSessionState SessionState { get; set; } public EncodeSession Session { get { return _encodeSession; } set { _encodeSession = value; } } /// <summary> /// 编码对象所在用户控件对象 /// </summary> public UserControl OwnPage { get { return _page; } set { _page = value; } } /// <summary> /// 捕获源 /// </summary> public CaptureSource Source { get { return _cSource; } } /// <summary> /// 操作音频对象 /// </summary> public AudioSinkExtensions AudioSink { get { return _audioSink; } } public static VideoCaptureDevice DefaultVideoDevice { get { return CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice(); } } public static ReadOnlyCollection<VideoCaptureDevice> AvailableVideoDevice { get { return CaptureDeviceConfiguration.GetAvailableVideoCaptureDevices(); } } public VideoCaptureDevice VideoDevice { get; set; } public static AudioCaptureDevice DefaultAudioDevice { get { return CaptureDeviceConfiguration.GetDefaultAudioCaptureDevice(); } } public static ReadOnlyCollection<AudioCaptureDevice> AvailableAudioDevice { get { return CaptureDeviceConfiguration.GetAvailableAudioCaptureDevices(); } } public AudioCaptureDevice AudioDevice { get; set; } private Object lockObj = new object(); internal VideoInputFormatInfo videoInputFormatInfo; internal AudioInputFormatInfo audioInputFormatInfo; internal VideoOutputFormatInfo videoOutputFormatInfo; internal AudioOutputFormatInfo audioOutputFormatInfo; internal OutputContainerInfo outputContainerInfo; /// <summary> /// 视频录制 /// </summary> public void StartRecord() { lock (lockObj) { if (this.SessionState == EncodeSessionState.Stop) { _videosink = new VideosinkExtensions(this); _audioSink = new AudioSinkExtensions(this); //_audioSink.VolumnChange += new AudioSinkExtensions.VolumnChangeHanlder(_audioSink_VolumnChange); if (_encodeSession == null) { _encodeSession = new EncodeSession(); } PrepareFormatInfo(_cSource.VideoCaptureDevice.DesiredFormat,_cSource.AudioCaptureDevice.DesiredFormat); _encodeSession.Prepare(videoInputFormatInfo,audioInputFormatInfo,videoOutputFormatInfo,audioOutputFormatInfo,outputContainerInfo); _encodeSession.Start(false,200); this.SessionState = EncodeSessionState.Start; } } } /// <summary> /// 音量大小指示 /// </summary> /// <param name="sender"></param> /// <param name="e"></param> //void _audioSink_VolumnChange(object sender,VolumnChangeArgs e) //{ // this.OwnPage.dispatcher.BeginInvoke(new Action(() => // { // ( // this.OwnPage.Tag as ProgressBar).Value = e.Volumn; // })); //} /// <summary> /// 暂停录制 /// </summary> public void PauseRecord() { lock (lockObj) { this.SessionState = EncodeSessionState.Pause; _encodeSession.Pause(); } } /// <summary> /// 停止录制 /// </summary> public void StopRecord() { lock (lockObj) { this.SessionState = EncodeSessionState.Stop; _encodeSession.Shutdown(); _videosink = null; _audioSink = null; } } /// <summary> /// 准备编码信息 /// </summary> /// <param name="videoFormat"></param> /// <param name="audioFormat"></param> private void PrepareFormatInfo(VideoFormat videoFormat,AudioFormat audioFormat) { uint FrameRateratioNumerator = 0; uint FrameRaterationDenominator = 0; FormatConstants.FrameRatetoRatio((float)Math.Round(videoFormat.FramesPerSecond,2),ref FrameRateratioNumerator,ref FrameRaterationDenominator); videoInputFormatInfo.FrameRateratioNumerator = FrameRateratioNumerator; videoInputFormatInfo.FrameRateratioDenominator = FrameRaterationDenominator; videoInputFormatInfo.FrameWidthInPixels = _videoWidth; videoInputFormatInfo.FrameHeightInPixels = _videoHeight ; videoInputFormatInfo.Stride = (int)_videoWidth*-4; videoOutputFormatInfo.FrameRateratioNumerator = FrameRateratioNumerator; videoOutputFormatInfo.FrameRateratioDenominator = FrameRaterationDenominator; videoOutputFormatInfo.FrameWidthInPixels = videoOutputFormatInfo.FrameWidthInPixels == 0 ? (uint)videoFormat.PixelWidth : videoOutputFormatInfo.FrameWidthInPixels; videoOutputFormatInfo.FrameHeightInPixels = videoOutputFormatInfo.FrameHeightInPixels == 0 ? (uint)videoFormat.PixelHeight : videoOutputFormatInfo.FrameHeightInPixels; audioInputFormatInfo.BitsPerSample = (uint)audioFormat.BitsPerSample; audioInputFormatInfo.SamplesPerSecond = (uint)audioFormat.SamplesPerSecond; audioInputFormatInfo.ChannelCount = (uint)audioFormat.Channels; if (outputContainerInfo.FilePath == null || outputContainerInfo.FilePath == string.Empty) { _saveFullPath=System.IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.MyVideos),"cCameraRecordVideo.tmp"); } outputContainerInfo.FilePath = _saveFullPath; //outputContainerInfo.FilePath = _saveFullPath; if (audioOutputFormatInfo.AverageBitrate == 0) audioOutputFormatInfo.AverageBitrate = 24000; if (videoOutputFormatInfo.AverageBitrate == 0) videoOutputFormatInfo.AverageBitrate = 2000000; } /// <summary> /// 开始捕获 /// </summary> public void StartCaptrue() { if (CaptureDeviceConfiguration.AllowedDeviceAccess || CaptureDeviceConfiguration.RequestDeviceAccess()) { _cSource.Start(); } } /// <summary> /// 停止捕获 /// </summary> public void StopCapture() { _videosink = null; _audioSink = null; _cSource.Stop(); } /// <summary> /// 获得视频 /// </summary> /// <returns></returns> public VideoBrush GetVideoBrush() { VideoBrush vBrush = new VideoBrush(); vBrush.SetSource(_cSource); return vBrush; } /// <summary> /// 获得视频 /// </summary> /// <returns></returns> public Rectangle GetVideoRectangle() { Rectangle rctg = new Rectangle(); rctg.Width = this._videoWidth; rctg.Height = this._videoHeight; rctg.Fill = GetVideoBrush(); return rctg; } /// <summary> /// 保存视频 /// </summary> public void SaveRecord() { if (_saveFullPath == string.Empty) { MessageBox.Show("尚未录制视频，无法进行保存！","系统提示",MessageBoxButton.OK); return; } SaveFileDialog sfd = new SaveFileDialog { Filter = "MP4 Files (*.mp4)|*.mp4",DefaultExt = ".mp4",FilterIndex = 1 }; if ((bool)sfd.ShowDialog()) { using (Stream stm=sfd.OpenFile()) { FileStream fs = new FileStream(_saveFullPath,FileMode.Open,FileAccess.Read); try { byte[] buffur = new byte[fs.Length]; fs.Read(buffur,(int)fs.Length); stm.Write(buffur,(int)buffur.Length); fs.Close(); File.Delete(_saveFullPath); } catch (IOException ioe) { MessageBox.Show("文件保存失败！错误信息如下："+Environment.NewLine+ioe.Message,MessageBoxButton.OK); } stm.Close(); } } } } }
当然上面说过必须有两个Sink：
using System; using System.Windows.Media; using System.Windows.Controls; using Microsoft.Silverlight.Windows.LocalEncode; namespace Cmj.MyWeb.MySilverlight.SilverlightMeida { public class VideosinkExtensions:Videosink { //private UserControl _page; //private EncodeSession _session; private LocalCamera _localCamera; public VideosinkExtensions(LocalCamera localCamera) { //this._page = page; this._localCamera = localCamera; //this._session = session; this.CaptureSource = _localCamera.source; } protected override void OnCaptureStarted() { } protected override void OnCaptureStopped() { } protected override void OnFormatChange(VideoFormat videoFormat) { } protected override void OnSample(long sampleTimeInHundrednanoseconds,long frameDurationInHundrednanoseconds,byte[] sampleData) { if (_localCamera.SessionState == EncodeSessionState.Start) { _localCamera.OwnPage.dispatcher.BeginInvoke(new Action<long,long,byte[]>((ts,dur,data) => { _localCamera.Session.WriteVideoSample(data,data.Length,ts,dur); }),sampleTimeInHundrednanoseconds,frameDurationInHundrednanoseconds,sampleData); } } } }
using System; using System.Windows.Media; using System.Windows.Controls; using Microsoft.Silverlight.Windows.LocalEncode; namespace Cmj.MyWeb.MySilverlight.SilverlightMeida { public class AudioSinkExtensions:AudioSink { private LocalCamera _localCamera; public AudioSinkExtensions(LocalCamera localCamera) { this._localCamera = localCamera; this.CaptureSource = _localCamera.source; } protected override void OnCaptureStarted() { } protected override void OnCaptureStopped() { } protected override void OnFormatChange(AudioFormat audioFormat) { } protected override void OnSamples(long sampleTimeInHundrednanoseconds,long sampleDurationInHundrednanoseconds,data) => { _localCamera.Session.WriteAudioSample(data,sampleDurationInHundrednanoseconds,sampleData); //计算音量变化 //for (int index = 0; index < sampleData.Length; index += 1) //{ // short sample = (short)((sampleData[index] << 8) | sampleData[index]); // float sample32 = sample / 32768f; // float maxValue = 0; // float minValue = 0; // maxValue = Math.Max(maxValue,sample32); // minValue = Math.Min(minValue,sample32); // float lastPeak = Math.Max(maxValue,Math.Abs(minValue)); // float micLevel = (100 - (lastPeak * 100)) * 10; // OnVolumnChange(this,new VolumnChangeArgs() { Volumn=micLevel}); //} } } /// <summary> /// 定义一个事件，反馈音量变化 /// </summary> /// <param name="sender"></param> /// <param name="e"></param> //public delegate void VolumnChangeHanlder(object sender,VolumnChangeArgs e); //public event VolumnChangeHanlder VolumnChange; //private void OnVolumnChange(object sender,VolumnChangeArgs e) //{ // if (VolumnChange != null) // { // VolumnChange(sender,e); // } //} } //public class VolumnChangeArgs : EventArgs //{ // public float Volumn // { // get; // internal set; // } //} }
有了这三个类，下面准备一个界面，使用LocalCamera进行视频录制操作。
需要注意的是保存操作，事实上在EncodeSession中视频的保存路径是在视频录制之前就必须指定的（当然这一点并不难理解，因为长时间的视频录制是会形成很大的文件的，保存之前缓存到内存中也不是很现实），在LocalCamera中对保存方法的封装事实上是文件的读取和删除操作。另外在这个例子中用到了前面文章中自定义的OOB控件，不明白的朋友可以查看前面的文章内容。下面是调用代码：
using System; using System.Collections.Generic; using System.Linq; using System.Net; using System.Windows; using System.Windows.Controls; using System.Windows.Documents; using System.Windows.Input; using System.Windows.Media; using System.Windows.Media.Animation; using System.Windows.Shapes; using System.Windows.Threading; using Cmj.MyWeb.MySilverlight.SiverlightOOB; using Cmj.MyWeb.MySilverlight.SilverlightMeida; namespace SilverlightVideoRecord { public partial class MainPage : UserControl { public MainPage() { InitializeComponent(); } OOBInstall install = new OOBInstall(); LocalCamera localCamera = null; dispatcherTimer timer = null; private DateTime startTime = DateTime.Now; private void UserControl_Loaded(object sender,RoutedEventArgs e) { timer = new dispatcherTimer(); timer.Interval = TimeSpan.FromSeconds(1); timer.Tick += new EventHandler(timer_Tick); if (install.IsRunOutOfbrowser) { this.btnInstall.Visibility = Visibility.Collapsed; localCamera = new LocalCamera(this,LocalCamera.AvailableVideoDevice[1].SupportedFormats[0],LocalCamera.DefaultAudioDevice.SupportedFormats[1]); this.bdVideo.Child = localCamera.GetVideoRectangle(); //this.Tag = this.pbVolumn; } else { this.btnInstall.Visibility = Visibility.Visible; this.btnStart.IsEnabled = false; this.btnPause.IsEnabled = false; this.btnStop.IsEnabled = false; this.btnSave.IsEnabled = false; //this.tbTitleBar.IsEnabled = false; //this.rbResizeButton.IsEnabled = false; } } void timer_Tick(object sender,EventArgs e) { TimeSpan tsstart = new TimeSpan(startTime.Ticks); TimeSpan tsEnd = new TimeSpan(DateTime.Now.Ticks); TimeSpan tsTract = tsEnd.Subtract(tsstart); DateTime timeInterval = new DateTime(DateTime.Now.Year,DateTime.Now.Month,DateTime.Now.Day,tsTract.Hours,tsTract.Minutes,tsTract.Seconds); //this.txtblkTimer.Text = string.Format("{0}:{1}:{2}",tsTract.Seconds); this.txtblkTimer.Text = timeInterval.ToLongTimeString(); } private void btnStart_Click(object sender,RoutedEventArgs e) { localCamera.StartCaptrue();//启动视频捕获 } private void btnRecord_Click(object sender,RoutedEventArgs e) { localCamera.StartRecord();//开始录制 this.txtblkTimer.Text = "0:00:00"; this.startTime = DateTime.Now; timer.Start(); } private void btnPause_Click(object sender,RoutedEventArgs e) { localCamera.PauseRecord();//暂停录制 timer.Stop(); } private void btnStop_Click(object sender,RoutedEventArgs e) { localCamera.StopRecord();//停止录制 localCamera.StopCapture();//停止视频捕获 timer.Stop(); } private void btnSave_Click(object sender,RoutedEventArgs e) { localCamera.SaveRecord();//保存视频 } private void btnInstall_Click(object sender,RoutedEventArgs e) { install.Install(); } } }
OK，下面是视频录制的截图：
正在录制
停止录制后保存
播放录制的视频
三、注意：
1.video sink和audio sink都是运行在不同于UI的各自的线程中，你可以使用UI的dispathcher或者SynchronizationContext进行不同线程之间的调用。
2．在video sink和audio sink的OnSample方法中必须进行状态判断，因为sink实例创建之后就会执行OnSample方法，但此时EncodeSession还没有启动因此如果不进行状态判读就会抛出com异常。
3.视频的宽度和高度不能够随意指定，这个在nesL的帮助文档中也是特意说明的，如果任意指定同样会抛出异常。
4.最后再次提醒大家，上面的视频录制是基于nesL的因此必须将应用运行到浏览器外（OOB）。
源代码下载
|
本作品采用知识共享署名 2.5 中国大陆许可协议进行许可，欢迎转载，演绎或用于商业目的。但转载请注明来自崔江涛（KenshinCui），并包含相关链接。 |
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 [email protected] 举报，一经查实，本站将立刻删除。