AWS CloudFront Secure Streaming

This is a followup to an answer I wrote on stackoverflow a few months ago about how to set up signed URLs with AWS CloudFront private streaming.

At the time, the python boto library had limited support for signed URLs and some of the steps were fairly hacky.  Since then, I've submitted some code to boto which will make secure signed URLs much easier. I've rewritten my answer here, making use of the new code. This code requires the 2.1 version of boto. Once version 2.1 of boto is commonly released I will update the stackoverflow answer as well.

To set up secure private CloudFront streaming with signed URLs you need to perform the following steps which I will detail below:

1. Connect, create your s3 bucket, and upload some objects
2. Create a Cloudfront "Origin Access Identity" (basically an AWS account to allow cloudfront to access your s3 bucket)
3. Modify the ACLs on your private objects so that only your Cloudfront Origin Access Identity is allowed to read them (this prevents people from bypassing Cloudfront and going direct to s3)
4. Create a cloudfront distribution that requires signed URLs
5. Test that you can't download private object urls from s3 or the signed cloudfront distribution
6. Create a key pair for signing private URLs
7. Generate some private URLs using Python
8. Test that the signed URLs work

Each step show a code snippet to perform that step.  All the snippets are combined into a single script for reference at the end.
1 - Connect, Create Bucket, and upload object
The easiest way to upload private objects is through the AWS Console but for completeness I'll show how using boto.  Boto code is shown here:

import boto

#credentials stored in environment AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
s3 = boto.connect_s3()
cf = boto.connect_cloudfront()

#bucket name MUST follow dns guidelines
new_bucket_name = "stream.example.com"
bucket = s3.create_bucket(new_bucket_name)

object_name = "video.mp4"
key = bucket.new_key(object_name)
key.set_contents_from_filename(object_name)

2 - Create a Cloudfront "Origin Access Identity"
This identity can be reused for many different distributions and keypairs. It is only used to allow cloudfront to access your private S3 objects without allowing everyone. As of now, this step can only be performed using the API. Boto code is here:

# Create a new Origin Access Identity
oai = cf.create_origin_access_identity(comment='New identity for secure videos')

print("Origin Access Identity ID: %s" % oai.id)
print("Origin Access Identity S3CanonicalUserId: %s" % oai.s3_user_id)

3 - Modify the ACLs on your objects
Now that we've got our special S3 user account (the S3CanonicalUserId we created above) we need to give it access to our private s3 objects. We can do this easily using the AWS Console by opening the object's (not the bucket's!) Permissions tab, click the "Add more permissions" button, and paste the very long S3CanonicalUserId we got above into the "Grantee" field of a new permission.  Make sure you give the new permission "Open/Download" rights.

You can also do this in code using the following boto script:

# Add read permission to our new s3 account
key.add_user_grant("READ", oai.s3_user_id)

4 - Create a cloudfront distribution
Note that custom origins and private distributions are only recently supported in boto version 2.1. To use these instructions you must get the latest release.

There are two important points here:
First, we are specifying an origin with an Origin Access Identifier. This allows CloudFront to access our private S3 objects without making the S3 bucket public. Users must use CloudFront to access the content.

The Second, is that we are specifying a "trusted_signers" parameter of "Self" to the distribution. This is what tells CloudFront that we want to require signed URLs. "Self" means that we will accept signatures from any CloudFront keypair in our own account. You can also give signing rights to other accounts if you want to allow others to create signed URLs for the content.

# Create an Origin object for boto
from boto.cloudfront.origin import S3Origin
origin = S3Origin("%s.s3.amazonaws.com" % new_bucket_name, oai)

# Create the signed distribution
dist = cf.create_distribution(origin=origin, enabled=True,
                              trusted_signers=["Self"],
                              comment="New distribution with signed URLs")

# Or, create a signed streaming distribution
stream_dist = cf.create_streaming_distribution(origin=origin, enabled=True,
                              trusted_signers=["Self"],
                              comment="New streaming distribution with signed URLs")

5 - Test that you can't download unsigned urls from cloudfront or s3
You should now be able to verify:

- stream.example.com.s3.amazonaws.com/video.mp4 - should give AccessDenied
- signed_distribution.cloudfront.net/video.mp4 - should give MissingKey (because the URL is not signed)

6 - Create a keypair for CloudFront
I think the only way to do this is through Amazon's web site. Go into your AWS "Account" page and click on the "Security Credentials" link.  Click on the "Key Pairs" tab then click "Create a New Key Pair". This will generate a new key pair for you and automatically download a private key file (pk-xxxxxxxxx.pem). Keep the key file safe and private. Also note down the "Key Pair ID" from amazon as we will need it in the next step.

7 - Generate some URLs in Python
In order to generate signed CloudFront URLs with boto, you must have the M2Crypto python library installed.  If it is not installed the following commands will raise a NotImplementedError.

For a non-streaming distribution, you must use the full cloudfront URL as the resource, however for streaming we only use the object name of the video file.

#Set parameters for URL
key_pair_id = "APKAIAZCZRKVIO4BQ" #from the AWS accounts page
priv_key_file = "cloudfront-pk.pem" #your private keypair file
expires = int(time.time()) + 300 #5 min

# For a downloading (normal http) use the full name
http_resource = 'http://%s/video.mp4' % dist.domain_name # your resource
# Create the signed URL
http_signed_url = dist.create_signed_url(http_resource, key_pair_id, expires, private_key_file=priv_key_file)

# For a streaming (rtmp) distribution use just the base filename
stream_resource = "video"
# Create the signed URL
stream_signed_url = stream_dist.create_signed_url(stream_resource, key_pair_id, expires, private_key_file=priv_key_file)

# Some flash players don't like query params so we have to escape them
def encode_query_param(resource):
    enc = resource
    enc = enc.replace('?', '%3F')
    enc = enc.replace('=', '%3D')
    enc = enc.replace('&', '%26')
    return enc

stream_signed_url = encode_query_param(stream_signed_url)

print("Download URL: %s" % http_signed_url)
print("Streaming URL: %s" % stream_signed_url)

8 - Try out the URLs

Hopefully your streaming url should look something like this:

video%3FExpires%3D1309979985%26Signature%3DMUNF7pw1689FhMeSN6JzQmWNVxcaIE9mk1x~KOudJky7anTuX0oAgL~1GW-ON6Zh5NFLBoocX3fUhmC9FusAHtJUzWyJVZLzYT9iLyoyfWMsm2ylCDBqpy5IynFbi8CUajd~CjYdxZBWpxTsPO3yIFNJI~R2AFpWx8qp3fs38Yw_%26Key-Pair-Id%3DAPKAIAZRKVIO4BQ

Put this into your js and you sould have something which looks like this:

var so_canned = new SWFObject('http://location.domname.com/~jvngkhow/player.swf','mpl','640','360','9');
so_canned.addParam('allowfullscreen','true');
so_canned.addParam('allowscriptaccess','always');
so_canned.addParam('wmode','opaque');
so_canned.addVariable('file','video%3FExpires%3D1309979985%26Signature%3DMUNF7pw1689FhMeSN6JzQmWNVxcaIE9mk1x~KOudJky7anTuX0oAgL~1GW-ON6Zh5NFLBoocX3fUhmC9FusAHtJUzWyJVZLzYT9iLyoyfWMsm2ylCDBqpy5IynFbi8CUajd~CjYdxZBWpxTsPO3yIFNJI~R2AFpWx8qp3fs38Yw_%26Key-Pair-Id%3DAPKAIAZRKVIO4BQ');
so_canned.addVariable('streamer','rtmp://s3nzpoyjpct.cloudfront.net/cfx/st');
so_canned.write('canned');

Summary
Post a comment if you have any trouble.

Enjoy!