SharpRTSP 1.11.1

Sharp RTSP

👁 Nuget

The source of the nuget package SharpRTSP is in branch donetcore

A C# library to build RTSP Clients, RTSP Servers and handle RTP data streams. The library has several examples.

• RTSP Client Example - will connect to a RTSP server and receive Video and Audio in H264, H265/HEVC, G711, AAC and AMR formats. UDP, TCP and Multicast are supported. The data received is written to files.
• RTSP Camera Server Example - A YUV Image Generator and a very simple H264 Encoder generate H264 NALs which are then delivered via a RTSP Server to clients with MD5 and SHA256 Digest Options
• RTP Receiver - will receieve RTP and RTCP packets and pass them to a transport handler
• RTSP Server - will accept RTSP connections and talk to clients
• RTP Sender - will send RTP packets to clients
• Transport Handler - Transport handlers for H264, H265/HEVC, G711 and AMR are provided.

:warning: : This library does not handle the decoding of the video or audio (eg converting H264 into a bitmap). SharpRTSP is limited to the transport layer and generates the raw data that you need to feed into a video decoder or audio decoder. Many people use FFMPEG or use Hardware Accelerated Operating System APIs to do the decoding.

Walkthrough of the RTSP Client Example

This is a walkthrough of an old version of the RTSP Client Example which highlights the main way to use the library.

• STEP 1 - Open TCP Socket connection to the RTSP Server

   // Connect to a RTSP Server
   tcp_socket = new Rtsp.RtspTcpTransport(host,port);
  
   if (tcp_socket.Connected == false)
   {
   Console.WriteLine("Error - did not connect");
   return;
   }
  

  This opens a connection for a 'TCP' mode RTSP/RTP session where RTP packets are set in the RTSP socket.

• STEP 2 - Create a RTSP Listener and attach it to the RTSP TCP Socket

   // Connect a RTSP Listener to the TCP Socket to send messages and listen for replies
   rtsp_client = new Rtsp.RtspListener(tcp_socket);
  
   rtsp_client.MessageReceived += Rtsp_client_MessageReceived;
   rtsp_client.DataReceived += Rtsp_client_DataReceived;
  
   rtsp_client.Start(); // start reading messages from the server
  

  The RTSP Listener class lets you SEND messages to the RTSP Server (see below).
  The RTSP Listner class has a worker thread that listens for replies from the RTSP Server.
  When replies are received the MessageReceived Event is fired.
  When RTP packets are received the DataReceived Event is fired.

• STEP 3 - Send Messages to the RTSP Server

  The samples below show how to send messages.

  Send OPTIONS with this code :

   Rtsp.Messages.RtspRequest options_message = new Rtsp. Messages.RtspRequestOptions();
   options_message.RtspUri = new Uri(url);
   rtsp_client.SendMessage(options_message);
  

  Send DESCRIBE with this code :

   // send the Describe
   Rtsp.Messages.RtspRequest describe_message = new Rtsp.Messages.RtspRequestDescribe();
   describe_message.RtspUri = new Uri(url);
   rtsp_client.SendMessage(describe_message);
   // The reply will include the SDP data
  

  Send SETUP with this code :

   // the value of 'control' comes from parsing the SDP for the desired video or audio sub-stream
   Rtsp.Messages.RtspRequest setup_message = new Rtsp.Messages.RtspRequestSetup();
   setup_message.RtspUri = new Uri(url + "/" + control);
   setup_message.AddHeader("Transport: RTP/AVP/TCP;interleaved=0");
   rtsp_client.SendMessage(setup_message);
   // The reply will include the Session
  

  Send PLAY with this code :

   // the value of 'session' comes from the reply of the SETUP command
   Rtsp.Messages.RtspRequest play_message = new Rtsp.Messages.RtspRequestPlay();
   play_message.RtspUri = new Uri(url);
   play_message.Session = session;
   rtsp_client.SendMessage(play_message);
  

• STEP 4 - Handle Replies when the MessageReceived event is fired

  This example assumes the main program sends an OPTIONS Command.
  It looks for a reply from the server for OPTIONS and then sends DESCRIBE.
  It looks for a reply from the server for DESCRIBE and then sends SETUP (for the video stream)
  It looks for a reply from the server for SETUP and then sends PLAY.
  Once PLAY has been sent the video, in the form of RTP packets, will be received.

   private void Rtsp_client_MessageReceived(object sender, Rtsp.RtspChunkEventArgs e)
   {
   Rtsp.Messages.RtspResponse message = e.Message as Rtsp.Messages.RtspResponse;
  
   Console.WriteLine("Received " + message.OriginalRequest.ToString());
  
   if (message.OriginalRequest != null && message.OriginalRequest is Rtsp.Messages.RtspRequestOptions)
   {
   // send the DESCRIBE
   Rtsp.Messages.RtspRequest describe_message = new Rtsp.Messages.RtspRequestDescribe();
   describe_message.RtspUri = new Uri(url);
   rtsp_client.SendMessage(describe_message);
   }
  
   if (message.OriginalRequest != null && message.OriginalRequest is Rtsp.Messages.RtspRequestDescribe)
   {
   // Got a reply for DESCRIBE
   // Examine the SDP
   Console.Write(System.Text.Encoding.UTF8.GetString(message.Data));
  
   Rtsp.Sdp.SdpFile sdp_data;
   using (StreamReader sdp_stream = new StreamReader(new MemoryStream(message.Data)))
   {
   sdp_data = Rtsp.Sdp.SdpFile.Read(sdp_stream);
   }
  
   // Process each 'Media' Attribute in the SDP.
   // If the attribute is for Video, then send a SETUP
   for (int x = 0; x < sdp_data.Medias.Count; x++)
   {
   if (sdp_data.Medias[x].GetMediaType() == Rtsp.Sdp.Media.MediaType.video)
   {
   // seach the atributes for control, fmtp and rtpmap
   String control = ""; // the "track" or "stream id"
   String fmtp = ""; // holds SPS and PPS
   String rtpmap = ""; // holds the Payload format, 96 is often used with H264
   foreach (Rtsp.Sdp.Attribut attrib in sdp_data.Medias[x].Attributs)
   {
   if (attrib.Key.Equals("control")) control = attrib.Value;
   if (attrib.Key.Equals("fmtp")) fmtp = attrib.Value;
   if (attrib.Key.Equals("rtpmap")) rtpmap = attrib.Value;
   }
  
   // Get the Payload format number for the Video Stream
   String[] split_rtpmap = rtpmap.Split(' ');
   video_payload = 0;
   bool result = Int32.TryParse(split_rtpmap[0], out video_payload);
  
   // Send SETUP for the Video Stream
   // using Interleaved mode (RTP frames over the RTSP socket)
   Rtsp.Messages.RtspRequest setup_message = new Rtsp.Messages.RtspRequestSetup();
   setup_message.RtspUri = new Uri(url + "/" + control);
   setup_message.AddHeader("Transport: RTP/AVP/TCP;interleaved=0");
   rtsp_client.SendMessage(setup_message);
   }
   }
   }
  
   if (message.OriginalRequest != null && message.OriginalRequest is Rtsp.Messages.RtspRequestSetup)
   {
   // Got Reply to SETUP
   Console.WriteLine("Got reply from Setup. Session is " + message.Session);
  
   String session = message.Session; // Session value used with Play, Pause, Teardown
  
   // Send PLAY
   Rtsp.Messages.RtspRequest play_message = new Rtsp.Messages.RtspRequestPlay();
   play_message.RtspUri = new Uri(url);
   play_message.Session = session;
   rtsp_client.SendMessage(play_message);
   }
  
   if (message.OriginalRequest != null && message.OriginalRequest is Rtsp.Messages.RtspRequestPlay)
   {
   // Got Reply to PLAY
   Console.WriteLine("Got reply from Play " + message.Command);
   }
   }
  

• STEP 5 - Handle RTP Video

  This code handles each incoming RTP packet, combining RTP packets that are all part of the same frame of video (using the Marker Bit). Once a full frame is received it can be passed to a De-packetiser to get the compressed video data

   List<byte[]> temporary_rtp_payloads = new List<byte[]>();
  
   private void Rtsp_client_DataReceived(object sender, Rtsp.RtspChunkEventArgs e)
   {
   // RTP Packet Header
   // 0 - Version, P, X, CC, M, PT and Sequence Number
   //32 - Timestamp
   //64 - SSRC
   //96 - CSRCs (optional)
   //nn - Extension ID and Length
   //nn - Extension header
  
   int rtp_version = (e.Message.Data[0] >> 6);
   int rtp_padding = (e.Message.Data[0] >> 5) & 0x01;
   int rtp_extension = (e.Message.Data[0] >> 4) & 0x01;
   int rtp_csrc_count = (e.Message.Data[0] >> 0) & 0x0F;
   int rtp_marker = (e.Message.Data[1] >> 7) & 0x01;
   int rtp_payload_type = (e.Message.Data[1] >> 0) & 0x7F;
   uint rtp_sequence_number = ((uint)e.Message.Data[2] << 8) + (uint)(e.Message.Data[3]);
   uint rtp_timestamp = ((uint)e.Message.Data[4] <<24) + (uint)(e.Message.Data[5] << 16) + (uint)(e.Message.Data[6] << 8) + (uint)(e.Message.Data[7]);
   uint rtp_ssrc = ((uint)e.Message.Data[8] << 24) + (uint)(e.Message.Data[9] << 16) + (uint)(e.Message.Data[10] << 8) + (uint)(e.Message.Data[11]);
  
   int rtp_payload_start = 4 // V,P,M,SEQ
   + 4 // time stamp
   + 4 // ssrc
   + (4 * rtp_csrc_count); // zero or more csrcs
  
   uint rtp_extension_id = 0;
   uint rtp_extension_size = 0;
   if (rtp_extension == 1)
   {
   rtp_extension_id = ((uint)e.Message.Data[rtp_payload_start + 0] << 8) + (uint)(e.Message.Data[rtp_payload_start + 1] << 0);
   rtp_extension_size = ((uint)e.Message.Data[rtp_payload_start + 2] << 8) + (uint)(e.Message.Data[rtp_payload_start + 3] << 0);
   rtp_payload_start += 4 + (int)rtp_extension_size; // extension header and extension payload
   }
  
   Console.WriteLine("RTP Data"
   + " V=" + rtp_version
   + " P=" + rtp_padding
   + " X=" + rtp_extension
   + " CC=" + rtp_csrc_count
   + " M=" + rtp_marker
   + " PT=" + rtp_payload_type
   + " Seq=" + rtp_sequence_number
   + " Time=" + rtp_timestamp
   + " SSRC=" + rtp_ssrc
   + " Size=" + e.Message.Data.Length);
  
  
   if (rtp_payload_type != video_payload)
   {
   Console.WriteLine("Ignoring this RTP payload");
   return; // ignore this data
   }
  
  
   // If rtp_marker is '1' then this is the final transmission for this packet.
   // If rtp_marker is '0' we need to accumulate data with the same timestamp
  
   // ToDo - Check Timestamp matches
  
   // Add to the tempoary_rtp List
   byte[] rtp_payload = new byte[e.Message.Data.Length - rtp_payload_start]; // payload with RTP header removed
   System.Array.Copy(e.Message.Data, rtp_payload_start, rtp_payload, 0, rtp_payload.Length); // copy payload
   temporary_rtp_payloads.Add(rtp_payload);
  
   if (rtp_marker == 1)
   {
   // Process the RTP frame
   Process_RTP_Frame(temporary_rtp_payloads);
   temporary_rtp_payloads.Clear();
   }
   }
  

• STEP 6 - Process RTP frame

  An RTP frame consists of 1 or more RTP packets
  H264 video is packed into one or more RTP packets and this sample extracts Normal Packing and Fragmented Unit type A packing (the common two)
  This example writes the video to a .264 file which can be played with FFPLAY

   FileStream fs = null;
   byte[] nal_header = new byte[]{ 0x00, 0x00, 0x00, 0x01 };
   int norm, fu_a, fu_b, stap_a, stap_b, mtap16, mtap24 = 0; // stats counters
  
   public void Process_RTP_Frame(List<byte[]>rtp_payloads)
   {
   Console.WriteLine("RTP Data comprised of " + rtp_payloads.Count + " rtp packets");
  
   if (fs == null)
   {
   // Create the file
   String filename = "rtsp_capture_" + DateTime.Now.ToString("yyyyMMdd_HHmmss") + ".h264";
   fs = new FileStream(filename, FileMode.Create);
  
   // TODO. Get SPS and PPS from the SDP Attributes (the fmtp attribute) and write to the file
   // for IP cameras that only out the SPS and PPS out-of-band
   }
  
   for (int payload_index = 0; payload_index < rtp_payloads.Count; payload_index++) {
   // Examine the first rtp_payload and the first byte (the NAL header)
   int nal_header_f_bit = (rtp_payloads[payload_index][0] >> 7) & 0x01;
   int nal_header_nri = (rtp_payloads[payload_index][0] >> 5) & 0x03;
   int nal_header_type = (rtp_payloads[payload_index][0] >> 0) & 0x1F;
  
   // If the NAL Header Type is in the range 1..23 this is a normal NAL (not fragmented)
   // So write the NAL to the file
   if (nal_header_type >= 1 && nal_header_type <= 23)
   {
   Console.WriteLine("Normal NAL");
   norm++;
   fs.Write(nal_header, 0, nal_header.Length);
   fs.Write(rtp_payloads[payload_index], 0, rtp_payloads[payload_index].Length);
   }
   else if (nal_header_type == 24)
   {
   // There are 4 types of Aggregation Packet (multiple NALs in one RTP packet)
   Console.WriteLine("Agg STAP-A not supported");
   stap_a++;
   }
   else if (nal_header_type == 25)
   {
   // There are 4 types of Aggregation Packet (multiple NALs in one RTP packet)
   Console.WriteLine("Agg STAP-B not supported");
   stap_b++;
   }
   else if (nal_header_type == 26)
   {
   // There are 4 types of Aggregation Packet (multiple NALs in one RTP packet)
   Console.WriteLine("Agg MTAP16 not supported");
   mtap16++;
   }
   else if (nal_header_type == 27)
   {
   // There are 4 types of Aggregation Packet (multiple NALs in one RTP packet)
   Console.WriteLine("Agg MTAP24 not supported");
   mtap24++;
   }
   else if (nal_header_type == 28)
   {
   Console.WriteLine("Fragmented Packet Type FU-A");
   fu_a++;
  
   // Parse Fragmentation Unit Header
   int fu_header_s = (rtp_payloads[payload_index][1] >> 7) & 0x01; // start marker
   int fu_header_e = (rtp_payloads[payload_index][1] >> 6) & 0x01; // end marker
   int fu_header_r = (rtp_payloads[payload_index][1] >> 5) & 0x01; // reserved. should be 0
   int fu_header_type = (rtp_payloads[payload_index][1] >> 0) & 0x1F; // Original NAL unit header
  
   Console.WriteLine("Frag FU-A s="+fu_header_s + "e="+fu_header_e);
  
   // Start Flag set
   if (fu_header_s == 1)
   {
   // Write 00 00 00 01 header
   fs.Write(nal_header, 0, nal_header.Length); // 0x00 0x00 0x00 0x01
  
   // Modify the NAL Header that was at the start of the RTP packet
   // Keep the F and NRI flags but substitute the type field with the fu_header_type
   byte reconstructed_nal_type = (byte)((nal_header_nri << 5) + fu_header_type);
   fs.WriteByte(reconstructed_nal_type); // NAL Unit Type
   fs.Write(rtp_payloads[payload_index], 2, rtp_payloads[payload_index].Length - 2); // start after NAL Unit Type and FU Header byte
   }
  
   if (fu_header_s == 0)
   {
   // append this payload to the output NAL stream
   // Data starts after the NAL Unit Type byte and the FU Header byte
  
   fs.Write(rtp_payloads[payload_index], 2, rtp_payloads[payload_index].Length-2); // start after NAL Unit Type and FU Header byte
   }
   }
  
   else if (nal_header_type == 29)
   {
   Console.WriteLine("Fragmented Packet FU-B not supported");
   fu_b++;
   }
   else
   {
   Console.WriteLine("Unknown NAL header " + nal_header_type);
   }
  
   }
   // ensure video is written to disk
   fs.Flush(true);
  
   // Print totals
   Console.WriteLine("Norm=" + norm + " ST-A=" + stap_a + " ST-B=" + stap_b + " M16=" + mtap16 + " M24=" + mtap24 + " FU-A=" + fu_a + " FU-B=" + fu_b);
   }