Showing the pts of a frame in the middle of the video

With ffmpeg here is a simple way to clearly show the PTS of the frame playing on the video. Note that the location of the font file will vary from system to system and this is on Mac OS X.

ffmpeg -i oceans.mp4 -vf "drawtext=fontfile=/Library/Fonts/Tahoma.ttf:text='%{pts}': fontcolor=white: fontsize=50: x=(w-text_w)/2:y=(h-text_h)/2: box=1: boxcolor=black@0.2" -s 1920x1080 -c:v libx264 -preset ultrafast oceans-timestamp.mp4 -y

What do Netflix use to create videos?

If you have ever wondered what tools, protocols and encryption Netflix use a quick peak into one of their mp4′s shows that they use a lot of the usual tools in their workflow. Note that this sample was derived from the Chrome browser and there are likely different variations for different devices.

For the first chunk / preview
1. The container is an mp4 for avc1
2. The video is packaged using GPAC aka MP4Box (GPAC0.5.1-DEV-rev4944M)
3. Video is H.264 and audio aac
4. H.264 is encoded using x264, settings
x264 – core 118 r234 d84818a – H.264/MPEG-4 AVC codec – Copyleft 2003-2011 – http://www.videolan.org/x264.html – options: cabac=1 ref=3 deblock=1:0:0 analyse=0×1:0×111 me=umh subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=-2 threads=12 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=2 b_pyramid=0 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=2pass mbtree=1 bitrate=600 ratetol=1.0 qcomp=0.50 qpmin=6 qpmax=51 qpstep=4 cplxblur=20.0 qblur=0.5 ip_ratio=1.40 aq=1:1.00

For other chunks
1. mp4/dash
2. Netflix encryption – NetflixPstrm
3. Created with Netflix Media Library Version 80.0.158
4. AAC-LC audio
mp4a – MP4 Audio Description) at 8 (87 bytes)
| | | | | | | Channels: 2
| | | | | | | Sample Size: 16
| | | | | | | Sample Rate: 24000.0
| | | | | | | (esds – Extended Sample Description) at 28 (51 bytes)
| | | | | | | | Audio Type: 64
| | | | | | | | Buffer Size: 579
| | | | | | | | Bitrate: 65587
| | | | | | | | Max Bitrate: 107720
| | | | | | | | Audio Specifc Config:
| | | | | | | | Audio Object Type: 2 – AAC LC (Low Complexity)
| | | | | | | | Sampling Frequency: 24000
| | | | | | | | Channel Config: 2 channels – Left, Right
| | | | | | | | SBR Present: 0
5. A DASH index / SIDX box
6. AVC video
(avcC – AVC Configuration) at 78 (56 bytes)
| | | | | | | | Version: 1
| | | | | | | | Profile: 77 (Main)
| | | | | | | | Profile Compatibility: 64 (Main)
| | | | | | | | Level: 3.0
| | | | | | | | NAL Unit Length: 4
| | | | | | | | Sequence Set Count: 1
| | | | | | | | NAL unit type: 7
| | | | | | | | Profile: 77 (Main)
| | | | | | | | Level: 3.0
| | | | | | | | SPS ID: 0
| | | | | | | | Max Frame Num: 16
| | | | | | | | Max Ref Frames: 3
| | | | | | | | Chroma Format: 1 (4:2:0)
| | | | | | | | Bit Depth: 8 (luma), 8 (chroma)
| | | | | | | | Macroblock Width: 32 (512 pixels luma, 256 pixels chroma)
| | | | | | | | Map Unit Height: 24 (384 pixels luma, 192 pixels chroma)
| | | | | | | | VUI parameters present: 1
| | | | | | | | Sample Aspect Ratio: 4:3
| | | | | | | | Timing Info: num_units_in_tick=1001, time_scale=48000, fixed_frame_rate_flag=1
| | | | | | | | NAL/HRD present! (
7. Common Encryption for PlayReady, Widevine and a custom system not registered on the DASH list, assume this is Netflix specific encryption
| pssh – Protection System Specific Header at 372 (702 bytes)
| | System ID: 9A04F079-9840-4286-AB92-E65BE0885F95
| | Version: 0
| | Data Size: 670
| | Header: ��16AESCTRAAAAANQDUfcAAAAAAAAAAA==OgsMEY/8lsI=true
| pssh – Protection System Specific Header at 1074 (52 bytes)
| | System ID: EDEF8BA9-79D6-4ACE-A3C8-27DCD51D21ED
| | Version: 0
| | Data Size: 20
| | Header:��Q
| pssh – Protection System Specific Header at 1126 (76 bytes)
| | System ID: 29701FE4-3CC7-4A34-8C5B-AE90C7439A47
| | Version: 0
| | Data Size: 44
| | Header: ��Q�r����ԗ���P\i���
EH%M�<�^

Encryption is using an IV size of 8 which is best for cross platform compatibility
The KeyID looks quite short:

Setting up a simple HLS live recording with DVR player

This is a very simplistic example of how to setup a recording of a live stream that is then saved as HLS for playback and rewind. The aim is that you could use this for rough cut editing of clips off a live stream.

What you will need:

  • Server or desktop with latest build of ffmpeg
  • HTTP server
  • Decent computer and basic HTML skills

1. Record a live stream as HLS

ffmpeg -i http://www.nasa.gov/multimedia/nasatv/NTV-Public-IPS.m3u8 -c copy -f segment -segment_list index.m3u8 -segment_time 10 -segment_format mpeg_ts -segment_list_type m3u8 segment%05d.ts

This sample uses the NASA live stream (no guarantee that will be there) but you should be able to take in just about any live stream. Note that I am just copying the codec out here, if you want to transcode look at some of my other HLS posts.

This also assumes that you are running the command so that it writes directly into a directory on your webserver e.g. in my setup I could access the playlist at:

http://localhost/temp/recordtest/index.m3u8

2. Play the stream back with DVR functions

The best player I found for this was here: http://osmfhls.kutu.ru/

For testing you should just be able to take the stream above and paste it into the test URL, making sure you click on the DVR function first.

 

S3 write performance with yas3fs – 100fps

While S3 is no match for EBS SSD’s it is quite surprising what kind of performance you can get out of it when used as a standard filesystem.

Using yas3fs https://github.com/danilop/yas3fs to join together some video files I am getting a solid 100fps with a 10-15Mbps source file which is quite usable for general encoding workloads.

Note that it is important to make sure you EC2 machine and S3 bucket are in the same region.

 

Setting up a simple proxy config to test geoblocking (or get around geoblocking)

This assumes you have 2 or more devices connected to the same network and that one of the machines is set up to run a proxy server such as Charles Proxy. This also serves as a simple tutorial on how to get devices to use a secure proxy/vpn/tunnel to either test or bypass IP based geo blocking / restriction services.

You will also need an ssh server such as a basic Linux instance in one of the Amazon regions.

1. On your OS X machine setup a ssh tunnel, note you need to replace the path to your .pem and also make sure you have the correct username and server address

ssh -D 8080 -C -N -i /path/to/your/server.pem ec2-user@someec2-server.com

2. Now that you have a port forward running you need to configure your Charles Proxy to also forward it’s traffic via this config, do this using “External Proxy Settings”

Screen Shot 2014-09-03 at 3.02.55 pm

 

3. Your proxy should no be listening on the port you setup e.g. port 8888 by default on Charles. Now configure your iOS or other device to point to the OS X device setup in step 1 as the proxy server. http://www.charlesproxy.com/documentation/faqs/using-charles-from-an-iphone/

libx265 encode test, how low can you go for 720P

testHere is a really quick encoding test showing some of the outputs using libx265, by encoding test means it is certainly not an accurate representation of best quality comparison but what is does show is not too bad playback at 187kbps video (which is very low 720P!)

Some details of the source (some nature footage off youtube):

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Planet Earth - Amazing nature scenery (1080p HD)-6v2L2UGZJAM.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2014-04-25 01:07:55
Duration: 00:13:28.45, start: 0.000000, bitrate: 2292 kb/s

Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 2097 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      creation_time   : 2014-04-25 01:09:08
      handler_name    : IsoMedia File Produced by Google, 5-11-2011

And the result:

Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf56.3.100
  Duration: 00:13:28.50, start: 0.046440, bitrate: 324 kb/s
    Stream #0:0(und): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], 187 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

And to get the idea I have joined together the source and the result here which you can see a quick demo, the one on the left is the source and the right is the result. Now the one on the right is obviously not as good but you are comparing 2000kbps with <200kbps so it is not too bad.

 

 

 

Converting WMA to high quality m4a audio

Here is a simple one liner for converting wma files to high quality m4a for import into iTunes.

Just run this command from the terminal in each folder that has wma files in it. Note that this requires ffmpeg with libfdk_aac compiled which is the highest quality aac encoder.

for f in *.wma; do ffmpeg -y -i "$f" -c:a libfdk_aac -b:a 192k "${f%.wma}.m4a"; done;

Or if you would like to recursively convert all your files for a mass import into iTunes (this won’t delete anything but it is possible some files may error), just run this in the base directory and it will convert everything.

find . -type d | while read -r dir; do  pushd "$dir";for f in *.wma; do ffmpeg -y -i "$f" -c:a libfdk_aac -b:a 192k "${f%.wma}.m4a";done;ls;popd; done;

References for further info:

High quality audio encoding

AAC Encoding guide

ffmpeg OS X compilation guide

Or if you want the easier way to install ffmpeg and brew you can just do

brew install ffmpeg --with-fdk-aac --with-ffplay --with-freetype --with-frei0r --with-libass --with-libvo-aacenc --with-libvorbis --with-libvpx --with-opencore-amr --with-openjpeg --with-opus --with-rtmpdump --with-schroedinger --with-speex --with-theora --with-tools

 

Details here: http://www.renevolution.com/how-to-install-ffmpeg-on-mac-os-x/


												

Debugging Smart TVs and other devices with a transparent proxy

If you are like me and do a lot of work on different devices and need to debug what is going on this little trick can be invaluable. This only takes 3 mins to setup and is invaluable.

Tools you will need:

  1. An OS X based machine (as all good multi device developers should have, could also be done on linux)
  2. A device
  3. A Copy of Charles Proxy (also an essential tools for developers - http://www.charlesproxy.com/
  4. A wifi or fixed network with both devices on the network

What we are going to do is setup the OS X machine to be a router and forward any traffic that is on port 80 and 443 to the Charles Proxy and make sure that Charles has transparent proxy mode enabled. Note that without the SSL certs installed on the devices for the proxy you may need to drop the 443 forwarding.

  1. Enable IP forwarding:
    sudo sysctl -w net.inet.ip.forwarding=1
  2. Place the following two lines in a file called, say, pf.conf:
    rdr on en2 inet proto tcp to any port 80 -> 127.0.0.1 port 8080
    rdr on en2 inet proto tcp to any port 443 -> 127.0.0.1 port 8080

    These rules tell pf to redirect all traffic destined for port 80 or 443 to the local mitmproxy instance running on port 8080. You should replace en2 with the interface on which your test device will appear.

  3. Configure pf with the rules:
    sudo pfctl -f pf.conf
  4. And now enable it:
    sudo pfctl -e

Note I borrowed this form mitmproxy proxy which I will definitely be trying out as sounds like my kind of proxy even though Charles is handy for the formatted JSON/XML views: mitmproxy

Now you need to configure only the default gateway of your device to point at the interface on your OS X machine. Note that on my Macbook using wifi that looks like this:

>ifconfig
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1453
 ether 20:c9:d0:49:98:31
 inet6 fe80::22c9:d0ff:fe49:9831%en0 prefixlen 64 scopeid 0x4
 inet 10.33.195.97 netmask 0xffffffc0 broadcast 10.33.195.127
 nd6 options=1<PERFORMNUD>
 media: autoselect
 status: active

The last bit is you need to set Charles in Transparent Proxy mode:

Screen Shot 2014-08-27 at 2.47.06 pm

And then with some luck like I had it all works first time and I now have an LG TV that doesn’t support proxy settings working through my proxy in <5 mins without any special hardware.