Streaming Media from Cloudfront

As with many AWS products, Cloudfront helps to level the playing field a bit – allowing the individual to have access to the same technologies available to large corporations – of course, that doesn’t always make it cost effective, but it is still pretty cool. The focus of this article is how to leverage the streaming capabilities of CloudFront.

Streaming Distributions

CloudFront distributions come in two varieties – download and streaming. Both types of distributions are created in the same with through the AWS Console – the sole difference being in the selection of the Delivery Method (download/streaming). Both will pull data from S3, but can be configured to use a different origin source.

As a brief overview (you must be subscribed to CloudFront to use it):

  • Login to AWS Console (CloudFront)
  • Create Distribution
  • Set the Delivery Type to ‘Streaming’
  • Choose an S3 bucket as your origin source
  • You might want to turn on Logging (logs will be created on the S3 bucket of your choice)
  • Set the Distribution Status to Enabled
  • If you wish to access CloudFront via a CNAME instead of the default address, you may enter it as well
  • Click Create

Newly created (or modified) distributions are in the ‘InProgress’ state, but will shortly change to ‘Deployed’. Do not forget to set the ACL permissions on your S3 files before trying to access them via CloudFront.

Accessing a File on CloudFront (streaming)

CloudFront streaming distributions use Adobe Flash Media Server 3.5 (FMS) to serve files on port 1935 (RTMP and RTMPE protocols) and port 80 (RTMPT and RTMPTE protocols).

Unlike files on distributions of type download , files on streaming distributions are accessed slightly differently. Two key differences exist:

  • Protocol: The protocol is not HTTP, but rather, one of the streaming protocols used by FMS – most commonly, RTMP.
  • Path: Unlike with a download distribution, where the path matches the path to the file in S3, with a streaming distribution, /cfx/st/ must be prepended to the path.

The end result, therefore, resembles: rtmp://abcd.cloudfront.net/cfx/st/filename.ext

Depending on the file type, it may be necessary to prepend the type before the filename. This is usually true of MP4s and MP3s. General syntax is provided below:

  • FLV: rtmp://abcd.cloudfront.net/cfx/st/filename
    (does not require flv extension, but does work with it)
  • MP3: rtmp://abcd.cloudfront.net/cfx/st/mp3:filename
    (should not require mp3 extension but should work with it)
  • MP4: rtmp://abcd.cloudfront.net/cfx/st/mp4:streamname.mp4
    (requires file extension (e.g. .mov, .avi, .mp4, etc.), will not work without it)

At this point, you should have everything setup to begin streaming – if you have an application that can obtain content over the RTMP protocol, then all you need to do is point it at your content.

If you have enabled logging, CloudFront will create a folder on your S3 bucket, and will create gzipped logs within it. Quite a bit of information is included in the logs (IP, bytes transferred, referrer, edge location, etc). An example is included below:

#Fields: date time x-edge-location c-ip x-event sc-bytes x-cf-status x-cf-client-id cs-uri-stem cs-uri-query c-referrer x-page-url c-user-agent x-sname x-sname-query x-file-ext x-sid
2011-01-30     06:58:15     EWR2     xx.xxx.xx.xxx     play     82265     OK      xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx      rtmp://abcd.cloudfront.net/cfx/st      http://example.com/MoviePlayer/MoviePlayer.swf?v=rtmp://abcd.cloudfront.net/cfx/st/Test/%5B%5BDYNAMIC%5D%5D/3     http://example.com/MoviePlayer/MoviePlayer.swf?v=rtmp://abcd.cloudfront.net/cfx/st/Test     WIN%2010,1,102,64     Test     -     flv     1

The following events, at very least, are logged: connect, play, stop, disconnect. A quick look at the logs (and seeking a bit during playback) demonstrates that only the part of the file being requested is downloaded and streaming is in effect.

Playing a Stream

If you do not have an existing RTMP capable player, you can look into JW Player, an open source player that supports a wide variety of formats, including streaming protocols.

On the other hand, if you are interested it what it would take to create your own streaming-capable Flash application, a starting point is provided below.

ActionScript provides a built-in FLVPlayback class, that is capable of playing many formats, if simply pointed to the source. Essentially, all that is required is the following:

  • In Flash, add the FLVPlayback (or updated FLVPlayback 2.5) component to the stage
  • The default parameters are functional, but you will need to set the source (and perhaps skin)
  • If you provide a source starting with rtmp:// the player will receive the content as a stream.

The above solution lacks two fundamental items:

  • The ability to dynamically receive the source path
  • The ability to handle a non-existent source path

The following bit of ActionScript (v3) will create the FLVPlayback object, and receive the source as part of the query string. It will display an error if the source is invalid. No further error handling is built in. It can go on the first frame of a new ActionScript 3 document. The path to the media stream is set by the ‘v’ parameter in the query string:

e.g. MediaPlayer.swf?v=rtmp://abcd.cloudfront.net/cfx/st/filename.ext

import fl.video.FLVPlayback;
import fl.video.VideoEvent;
import fl.video.VideoState;
import flash.text.TextField;
import flash.text.TextFormat;

var MoviePlayer:FLVPlayback = new FLVPlayback();
var queryString = this.loaderInfo.parameters;
MoviePlayer.addEventListener(VideoEvent.STATE_CHANGE, onStateChange);
MoviePlayer.skin = "SkinOverAllNoFullNoCaption.swf"; //copy to folder first!
MoviePlayer.skinBackgroundColor = 0x000000; //black
MoviePlayer.skinAutoHide=true;
MoviePlayer.autoPlay=false;

if (!queryString.v){
	noVid();
}else{
	MoviePlayer.source = queryString.v;
	addChild(MoviePlayer);
}

function onStateChange(e:VideoEvent):void {
	if (VideoState.CONNECTION_ERROR == e.state) {
		noVid();
	}
}

function noVid(){
	var fmtError:TextFormat = new TextFormat();
	fmtError.font="Arial";
	fmtError.size=14;
	fmtError.color = 0xFF0000; //red
	var errLabel:TextField = new TextField();
	errLabel.x = 0;
	errLabel.y = 25;
	errLabel.width = stage.width;
	errLabel.selectable=false;
	errLabel.defaultTextFormat = fmtError;
	errLabel.autoSize = TextFieldAutoSize.CENTER;

	addChild(errLabel);
	errLabel.text = "Error: Unable to load video (Connection Error).";
	if (contains(MoviePlayer)){
		removeChild(MoviePlayer);
	}
}

When published the above generates a SWF file of approximately 115kB. Before using you must copy the skin file to your project folder. The skins are located (adjust for 32-bit operating systems and version of Flash) in:

C:\Program Files (x86)\Adobe\Adobe Flash CS5\Common\Configuration\FLVPlayback Skins\ActionScript 3.0

Of note, this code will begin downloading the media file as soon as the source is set (as this triggers the load() method). From an optimization perspective, it would be advisable to set the source when a button is pressed (i.e. if the user asks for it), instead of loading the media file every time.

A, perhaps more common, method for dynamically passing variable values to Flash is the use of the FlashVars param in HTML (but the query string method allows for direct access). An improvement on this design would be to pass an ID, and have the application retrieve a PHP page, passing the ID – the PHP page, in turn would retrieve the path to the media content from a database and return it to the application (perhaps as an XML file) – however, that is a bit more effort.

Alternatives

As much fun as it is to play with CloudFront and media streaming, the unfortunate fact is that the cost of bandwidth for a site that doesn’t use much is prohibitive ($0.15/GB). The cost of bandwidth does drop considerably with increased usage (down to $0.03/GB over 1000TB/mo), making it potentially viable for larger scale solutions. For individuals with access to virtual (or dedicated) servers, options other than FMS do exist (and many virtual/dedicated servers offer unmetered bandwidth).

Lightweight servers such as lighttpd and nginx have modules that will take an offset in seconds, and add the necessary header so that the file can be played from that point. While not exactly streaming, the end result is often similar.

Nginx:

Lighttpd:

  • FLV: mod_flv_streaming (included but not compiled in by default)
  • FLV: mod_h264_streaming (third-party module)

By cyberx86

Just a random guy who dabbles with assorted technologies yet works in a completely unrelated field.

7 comments

  1. i cannot get a url to the file using the streaming setting in the amazon cloud.. The bucket is not recognized in the s3 firefox addon I am using.. But if I select download , it gives me the distribution url.. Do you know what is going on with that..

    Thanks Scott

    1. The S3 and Cloudfront URLs should be independently accessible – if you cannot access the bucket from S3Fox, then you probably need to check your login information – S3Fox should list all your buckets. You might also consider trying CloudBerry Explorer or CloudBuddy for accessing S3 – both will list your buckets and allow you to browse them (and both are a bit more user friendly than S3Fox). Additionally, you might want to look under the AWS console (the S3 section) – it will let you browse and access your S3 buckets.
      If you have the right bucket, check your ACL settings – if the file is not set to publicly accessible (i.e. read permissions for everyone), then you will not be able to access it via a browser – this will be reflected in your Cloudfront distribution – which will pull an error page (instead of the file), and will cache that page (so even when you do fix the ACL settings, it will take some time for the change to propagate), when testing, I would suggest modifying the filename between changes.
      Amazon should have provided you with a URL for your distribution (whether streaming or download) – if the bucket is not accessible though, there will be problems. For the streaming setup, don’t forget that the path must include ‘cfx/st/’ and that it will not be accessible through http – also, you might need to prepend the file type to the name. Finally, don’t forget that streaming distributions only work with media files.
      Essentially, you need to narrow your problem a bit:

      Do you have a Cloudfront URL? (Check the AWS Console)
      Is your S3 bucket accessible (try to load the file from S3 directly in a browser)
      Using something that works with RTMP (e.g. JW Player), try to play file

      If all else fails, I would suggest giving things a try from scratch
      create a new, empty bucket
      upload a single media file into it
      set the ACL settings for the file
      verify that you can access the file in a browser
      setup a streaming distribution pointing to your new bucket
      construct the correct URL to the file
      pass the URL to your media player (I think that VLC might even accept RTMP URLs)

      I hope it works out for you, good luck.

  2. i’ve been using s3+cloudfront for a while to sream flv. just ran into a problem…
    trying to organize the s3 bucket into folders, i can’t access files through cloudfront that are not at the highest level of the bucket. I’ve tried just about every iteration of distribution url that you can imagine.
    Do you know of any issues with streaming content from subfolders in s3?

    1. I just tried to stream an FLV using S3+Cloudfront, and had no trouble doing so with a subfolder – as such, my only suggestion is to check your ACL settings – the folder and the file need to be set to Public (Read:All Users). Keep in mind that if this was not set before and you now set it, the Cloudfront servers will have a cached copy of the error page returned when trying to access the file. I would therefore suggest creating a new folder if ACL is your problem.
      As a method of diagnostics, I would suggest:
      a) Use the same file (to ensure that the file is playable) – you said you could play it from the root folder, copy that file into a subfolder.
      b) Check your ACL settings – both for the subfolder and the file itself – use Cloudberry, Cloudbuddy, S3Fox, or the AWS console if you don’t already have a method of access.
      c) Ensure you can directly access the file through S3 (i.e. in a browser, download the video from S3) – if you can’t download it, CloudFront won’t be able to access it
      d) Don’t forget the that the protocol is ‘rtmp’ and don’t omit the ‘cfx/st/’ part in the url.
      e) You might want to ensure you have logging enabled on CloudFront so that you can check the logs for anything that might reveal an error.
      Offhand, it does sound like an ACL problem – hopefully it is – good luck.

  3. cyber86, thanks for such a good article, i had few questions and need your help, can i have your email id or a way to communicate with you so that i can explain my issue.

    once again thank you for this article.

    1. Hi, at this time, I am unable to consult on specific projects. You may post an sanitized overview of your situation and I may be able to provide some pointers. I would recommend trying ServerFault if you have a specific problem that you are having difficulty with.

Leave a comment

Your email address will not be published. Required fields are marked *