DVB-T2 playback on Android

There are now a number of devices on the market that include a tuner in an Android device, which is not surprising as Android is basically linux however exposing the bitstream up through the Android stack is a bit trickier than in linux as standard libs that people would use such as V4L are not present.

In our testing the Amlogic SoC stack and open linux platform provide a good basis for this and they have also made available a good reference library for implementing the DVB protocols (T2 and S2 for now).

Will be posting more on this topic soon but for now here is a demo of DVB-T2 working off an Android device and captured via my desktop.

Playing HEVC/H.265 on Android with ffmpeg, libx265 and Vitamio

After testing out quite a few other peoples attempts at building and using ffmpeg on Android I settled on the Vitamio build (again as have done the same process before) as it offers the best mix of features and also has a reasonably active community (even if I have to use translate from Chinese!).

First download the latest builds of Vitamio-FfmpegVitamioBundle and OpenSSL-Vitamio (Use git clone for these).

Download and install the latest Android NDK

Note that I run OSX so these tips are for that though should be near identical for linux.

Change into the directory for OpenSSL-Vitamio and run ndk-build

cd OpenSSL-Vitamio
/path/to/ndk/ndk-build

Now we need to make sure that environment variables are available to support our build for the build_android script and then run the build script

export SSL=/path/to/OpenSSL-Vitamio
export ANDROID_NDK=/path/to/ndkinstall
./build_android.sh

For some reason my build was failing for arm7 due to linking of the ssl libs, to fix this I had to remove the –enable-openssl from the ffmpeg setup options in build_android.sh

Stabilising / deshaking GoPro (or other) videos with ffmpeg and libvidstab

If anyone has shot on GoPro, particularly the helmet mounts you would be very familiar with the issue of camera shake. Users of YouTube may have also noticed that they have quite a nice stablisation filter.

Now thanks to Georg Martius you there is a great filter that you can include in your ffmpeg workflow for stablising video and it does quite a decent job http://public.hronopik.de/vid.stab/

1. First download the source

git clone https://github.com/georgmartius/vid.stab
cd vid.stab
cmake .
make
make install

2. configure you ffmpeg with –enable-libvidstab

./configure --prefix=/usr/local --enable-gpl --enable-nonfree --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-libvidstab
make
make install

3. Using libvidstab is a 2 pass process as first you need to detect the stabilisation issues, for example using default settings. Note on the first pass you don’t really need an output file as the real data get’s written to a special file

ffmpeg -i myvideo.mp4 -vf vidstabdetect output.mp4
ffmpeg -i myvideo.mp4 -vf vidstabtransform myvideo_stabilisted.mp4

There are of course a lot more options you can use and see those here: http://ffmpeg.org/ffmpeg-filters.html#vidstabdetect-1

 

 

Building ffmpeg with libx265 for h265/hevc encoding

ffmpeg now has x265 support and while it is still early days for the codec this is great news as there are also a number of players out there now too, not to mention many devices now have the CPU to play back the codec. Note that quite a lot of services now support h265/hevc input and as it is such a small footprint it can make quite a good file for transfer to cloud encoding.

Setup on OS X (likely be similar for linux and will do that at some point)

  1. Make sure you have cmake e.g. >brew install cmake and you will also need yasm
  2. You will also need mercurial to clone x265 >brew install mercurial
  3. Also if you already have ffmpeg installed using something like brew then uninstall that first >brew uninstall ffmpeg

Anyhow here are the simple steps:

1. Make sure you have a current build of ffmpeg checked out of git along with any other libs you are using e.g. libx264

2. Download the libx265 repository and build as per the instructions here: https://bitbucket.org/multicoreware/x265/wiki/Home (note I assume no one still uses Windows for dev!)

hg clone https://bitbucket.org/multicoreware/x265
cd x265/build/linux
./make-Makefiles.bash
make
make install

3. On your ffmpeg configure it with –enable-libx265 (it is disabled by default). Sample from my configure below

./configure --prefix=/usr/local --enable-gpl --enable-nonfree --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-libvidstab --enable-libx265
make
make install

4. You should now be ready to go, e.g.

ffmpeg -i myvideo.MP4 -c:v libx265 encodetest/myvideo.mkv

Update: now with mp4 support

ffmpeg -i anchorman2-trailer-ffmpeg.mp4 -y -s 640x360 -c:v libx265 -c:a libfdk_aac -profile:a aac_he -b:v 200k -b:a 32k anchorman2_640x360_x265.mp4

In some cases you may get an error like:

Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

And you will see above it:

x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [error]: Sample Aspect Ratio width must be greater than 0

There is a bug at present (2014-04-08) that requires the SAR to be set in the header, note that you can fix this by doing a pre-encode and write the SAR / DAR header by forcing the aspect -aspect 16:9 e.g.

ffmpeg -i anchorman2-trailer.mp4 -y -c:v libx264 -c:a copy -aspect 16:9 -crf 0 anchorman2-trailer-ffmpeg.mp4

Some things to note:

  • Moderately slow! It is now much faster than it used to be and I am getting 17fps without tweaking which is pretty good
  • Playback of a 1080P HD clip used 400% on my i7 based laptop, size was
  • Compared to a 2000kbps x264 encode of the same file quality was very good!

 

nginx-rtmp – nice tool for managing rtmp streams

nginx is one of the best web servers out there not only because it is massively scalable but also because it has a nice clean configuration syntax.

The nginx-rtmp module looks like a very impressive add on that provides some really nice additional features for video streaming, including:

  • Ability to re-stream rtmp
  • Push rtmp to other locations
  • Ability to record streams
  • Ability to perform on the fly transcodes of content coming through

https://github.com/arut/nginx-rtmp-module

Using ffmpeg with Akamai HD

Quite often it is useful to put up a test stream or source with a new CDN configuration for testing purposes, this works with authenticated rtmp which is required when connected to Akamai or many Adobe/Flash Media Server servers.

ffmpeg -re -f lavfi -i testsrc=size=1920x1080 -c:v libx264 -b:v 500k -an -s 1920x1080 -x264opts keyint=50 -g 25 -pix_fmt yuv420p -f flv rtmp://<USERNAME>:<PASSWORD>@p.<CPCODE>.i.akamaientrypoint.net/EntryPoint/mystream_1_500@<STREAMID>

You need to get the username, password, entrypoint name and streamid from your Akamai account (Configure -> Live Media)

You can then play it back using the URLs you also see in your Akamai control panel.

Note that for the RTMP to be converted to HLS or HDS you need to make sure you have frequent enough keyframes which is what the keyint directive is for.

Building ffmpeg with librtmp

librtmp is one option for passing additional parameters through to Akamai however you can also just use HTTP style authentication with rtmp://username:password@entrypoint/stream to connect to an Adobe Media Server

It is available in librtmp which can be included in ffmpeg.

git clone https://github.com/mstorsjo/rtmpdump.git

For Mac OSX we set the target to darwin for linux use posix or just leave SYS= out as posix is the default

make SYS-darwin

Output should look as follows:

gcc -dynamiclib -twolevel_namespace -undefined dynamic_lookup -fno-common -headerpad_max_install_names -install_name /usr/local/lib/librtmp.0.dylib -o librtmp.0.dylib rtmp.o log.o amf.o hashswf.o parseurl.o -lssl -lcrypto -lz 
ln -sf librtmp.0.dylib librtmp.dylib
gcc -Wall -DRTMPDUMP_VERSION=\"v2.4\" -O2 -c -o rtmpdump.o rtmpdump.c
gcc -Wall -o rtmpdump rtmpdump.o -Llibrtmp -lrtmp -lssl -lcrypto -lz 
gcc -Wall -DRTMPDUMP_VERSION=\"v2.4\" -O2 -c -o rtmpgw.o rtmpgw.c
gcc -Wall -DRTMPDUMP_VERSION=\"v2.4\" -O2 -c -o thread.o thread.c
gcc -Wall -o rtmpgw rtmpgw.o thread.o -lpthread -Llibrtmp -lrtmp -lssl -lcrypto -lz 
gcc -Wall -DRTMPDUMP_VERSION=\"v2.4\" -O2 -c -o rtmpsrv.o rtmpsrv.c
gcc -Wall -o rtmpsrv rtmpsrv.o thread.o -lpthread -Llibrtmp -lrtmp -lssl -lcrypto -lz 
gcc -Wall -DRTMPDUMP_VERSION=\"v2.4\" -O2 -c -o rtmpsuck.o rtmpsuck.c
gcc -Wall -o rtmpsuck rtmpsuck.o thread.o -lpthread -Llibrtmp -lrtmp -lssl -lcrypto -lz

Then install (note for OSX you need to specify darwin)

sudo make SYS=darwin install
mkdir -p /usr/local/bin /usr/local/sbin /usr/local/man/man1 /usr/local/man/man8
cp rtmpdump /usr/local/bin
cp rtmpgw rtmpsrv rtmpsuck /usr/local/sbin
cp rtmpdump.1 /usr/local/man/man1
cp rtmpgw.8 /usr/local/man/man8
sed -e "s;@prefix@;/usr/local;" -e "s;@libdir@;/usr/local/lib;" \
 -e "s;@VERSION@;v2.4;" \
 -e "s;@CRYPTO_REQ@;libssl,libcrypto;" \
 -e "s;@PRIVATE_LIBS@;;" librtmp.pc.in > librtmp.pc
mkdir -p /usr/local/include/librtmp /usr/local/lib/pkgconfig /usr/local/man/man3 /usr/local/lib
cp amf.h http.h log.h rtmp.h /usr/local/include/librtmp
cp librtmp.a /usr/local/lib
cp librtmp.pc /usr/local/lib/pkgconfig
cp librtmp.3 /usr/local/man/man3
cp librtmp.0.dylib /usr/local/lib
cd /usr/local/lib; ln -sf librtmp.0.dylib librtmp.dylib

On trying a test build of ffmpeg with the new library (./configure –enable-librtmp) it appeared my PKG_CONFIG_PATH was correct so I updated this:

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

then searched for the package and all was OK

pkg-config --libs librtmp

Now you can build ffmpeg with the new libraries, I tend to just run my local ./ffmpeg to get the other configure settings I last used, this now is:

./configure --enable-gpl --enable-version3 --enable-nonfree --enable-postproc --enable-libaacplus --enable-libcelt --enable-libfaac --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-openssl --enable-libopus --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-libvidstab --prefix=/usr/local --enable-librtmp

Now for some reason my ffmpeg build started failing on linking x264 which it hadn’t done before so I had to add –cc=clang to change the compiler. Maybe one of the recent xcode updates so will check.

LD ffmpeg_g
Undefined symbols for architecture x86_64:
 "_x264_encoder_open_129", referenced from:
 _X264_init in libavcodec.a(libx264.o)
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
make: *** [ffmpeg_g] Error 1

With clang in place this built but I still get an error on connecting to Akamai:

ffmpeg -i udp://233.xxx.xxx.xxx:5266 -s 512x288 -aspect 16:9 -profile baseline -b 500k -vcodec libx264 -acodec libmp3lame -ar 44100 -ab 64k -ac 2 -deinterlace -coder 0 -f flv 'rtmp://p.ep12345.i.akamaientrypoint.net/EntryPoint flashver=FMLE/3.0\20(compatible;\20FMSc/1.0) live=true pubUser='User' pubPasswd='Password' playpath=live_chan1_999@12345'

 

Troubleshooting

Anatomy of a successful connection (using Flash Live Media Encoder)

Connect

connect.?……….app..aEntryPoint?authmod=adobe&user=214743&challenge=L1PcQA==&response=I1mUYIbUXlFLDVmNHU9Taw==&opaque=..t.cUrl…rtmp://p.ep111222.i.akamaientrypoint.net/EntryPoint?authmod=adobe&user=user&challenge=L1PcQA==&response=I1mUYIbUXlFLDVm.NHU9Taw==&opaque=..type..
nonprivate..flashVer…FMLE/3.0 (compatible; FMSc/1.0)..swfUrl…rtmp://p.ep111222.i.akamaientrypoint..net/EntryPoint?authmod=adobe&user=user&challenge=L1PcQA==&response=I1mUYIbUXlFLDVmNHU9Taw==&opaque

Response

_result.?……….fmsVer…FMS/4,5,5,4013..capabilities.@o……..mode.?………….level…status..code…NetConnection.Connect.Success..description…Connection succeeded…objectEncoding………..data…….version..
4,5,5,4013………………..

Sent

 

Recording a live HLS stream to a file for a specific time period

This is an example of how to record a live HLS stream to a file for a specific time period. Useful for DVR like functions of live adapative streams (noting that at the time of writing multi-rate adapative for ffmpeg isn’t great). I have selected ts as the output as this is the minimum overheard though you could easily go to MP4 of equivalent.

This records the live stream to a file for 30 seconds:

ffmpeg -re -i http://yourserver.com/live/stream1/playlist.m3u8 -ss 00:00:00.0 -t 00:00:30.0 -c:v copy -c:a copy test_record.ts -y

Live encoding with ffmpeg

ffmpeg is without doubt one of the (if not the!) best file based encoders out there. However getting it to run as a 24×7 live encoder can be somewhat tricky as one of the main issue with ffmpeg is there are few options to handle and retry any failure conditions, which is perfectly acceptable for a file based encoder.

To accommodate this scenario I have been working on some simple scripts that can wrap ffmpeg to produce 24×7 live streams.

This is a work in progress and my wrapper code is available here:

https://github.com/sinkers/ffmpeg_live

Impact of adding additional keyframes on video size

The following is a quick test on looking at the impact on filesize of adding additional keyframes to an encode for the purpose of making it more suitable for segmentation in adaptive bitrate delivery.

The following shows that on a very hard to encode source file (reflections on water) that adding a keyframe every 50 frames (2 seconds on 25 fps source) that the overall increase in size was only 96k (approx 1%).

Note that the video is 1153 frames long so we would expect to have at least 23 key frames given a requested keyframe interval of 50. The source also had very few traditional scene changes which is reflected by the x264 generated number of keyframes of only 6 when left to only adding keyframes on scenecut.

Sample command to show GOP / I frame structure using ffprobe:

ffprobe -select_streams v:0 -show_frames goptest.mov |grep key_frame|less

File with x264 selected key frames

Encode settings

/usr/local/bin/ffmpeg -y -i goptest.mov -codec:v libx264 -b:v 6000K -s 1920x1080 -preset slower -tune film -me_range 24 -bufsize 50000K -maxrate 50000K -refs 4 -profile:v high -level 4.1 -threads 0 -sn 'goptest_1080p.mp4'

ffmpeg encode results

video:34160kB audio:538kB subtitle:0 global headers:0kB muxing overhead 0.077031%
[libx264 @ 0x7fc873846800] frame I:6     Avg QP:16.50  size: 25800
[libx264 @ 0x7fc873846800] frame P:1131  Avg QP:23.22  size: 30785
[libx264 @ 0x7fc873846800] frame B:16    Avg QP: 6.54  size:   354
[libx264 @ 0x7fc873846800] consecutive B-frames: 98.0%  0.3%  0.3%  1.4%
[libx264 @ 0x7fc873846800] mb I  I16..4: 66.5% 32.0%  1.5%
[libx264 @ 0x7fc873846800] mb P  I16..4: 49.4% 40.7%  1.3%  P16..4:  6.4%  0.4%  0.1%  0.0%  0.0%    skip: 1.6%
[libx264 @ 0x7fc873846800] mb B  I16..4:  0.2%  0.1%  0.0%  B16..8:  9.5%  0.0%  0.0%  direct: 0.1%  skip:90.2%  L0:68.0% L1:31.9% BI: 0.1%
[libx264 @ 0x7fc873846800] final ratefactor: 22.37
[libx264 @ 0x7fc873846800] 8x8 transform intra:44.5% inter:93.6%
[libx264 @ 0x7fc873846800] direct mvs  spatial:62.5% temporal:37.5%
[libx264 @ 0x7fc873846800] coded y,uvDC,uvAC intra: 21.3% 67.9% 13.0% inter: 17.0% 39.8% 0.6%
[libx264 @ 0x7fc873846800] i16 v,h,dc,p: 35% 25% 18% 22%
[libx264 @ 0x7fc873846800] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 19% 36%  3%  5%  5% 10%  3%  5%
[libx264 @ 0x7fc873846800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 19% 44%  1%  6%  4%  6%  1%  2%
[libx264 @ 0x7fc873846800] i8c dc,h,v,p: 43% 33% 19%  5%
[libx264 @ 0x7fc873846800] Weighted P-Frames: Y:0.3% UV:0.2%
[libx264 @ 0x7fc873846800] ref P L0: 62.8%  3.6% 17.0%  9.3%  7.3%
[libx264 @ 0x7fc873846800] ref B L0: 99.4%  0.6%
[libx264 @ 0x7fc873846800] ref B L1: 99.7%  0.3%
[libx264 @ 0x7fc873846800] kb/s:6067.44

File with keyint=50

Encode settings

Note that just using keyint on it’s own doesn’t necessarily mean that keyframes will be forced every 50 frames, see below for alternative option by switching x264 scenecut off

/usr/local/bin/ffmpeg -y -i goptest.mov -codec:v libx264 -b:v 6000K -s 1920x1080 -preset slower -tune film -me_range 24 -bufsize 50000K -maxrate 50000K -refs 4 -profile:v high -level 4.1 -x264opts keyint=50 -threads 0 -sn 'goptest_1080p_keyint50.mp4'

ffmpeg encode results

video:34256kB audio:538kB subtitle:0 global headers:0kB muxing overhead 0.077043%
[libx264 @ 0x7fe631846800] frame I:26    Avg QP:19.07  size: 41326
[libx264 @ 0x7fe631846800] frame P:1112  Avg QP:23.33  size: 30574
[libx264 @ 0x7fe631846800] frame B:15    Avg QP: 6.27  size:   300
[libx264 @ 0x7fe631846800] consecutive B-frames: 98.1%  0.3%  0.5%  1.0%
[libx264 @ 0x7fe631846800] mb I  I16..4: 52.5% 43.3%  4.2%
[libx264 @ 0x7fe631846800] mb P  I16..4: 49.8% 40.4%  1.3%  P16..4:  6.3%  0.4%  0.1%  0.0%  0.0%    skip: 1.8%
[libx264 @ 0x7fe631846800] mb B  I16..4:  0.2%  0.1%  0.0%  B16..8:  2.8%  0.0%  0.0%  direct: 0.0%  skip:96.8%  L0:12.8% L1:87.0% BI: 0.2%
[libx264 @ 0x7fe631846800] final ratefactor: 22.35
[libx264 @ 0x7fe631846800] 8x8 transform intra:44.1% inter:94.0%
[libx264 @ 0x7fe631846800] direct mvs  spatial:60.0% temporal:40.0%
[libx264 @ 0x7fe631846800] coded y,uvDC,uvAC intra: 21.3% 67.2% 13.0% inter: 16.4% 38.5% 0.6%
[libx264 @ 0x7fe631846800] i16 v,h,dc,p: 35% 25% 18% 22%
[libx264 @ 0x7fe631846800] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 19% 37%  3%  5%  5% 10%  3%  5%
[libx264 @ 0x7fe631846800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 20% 43%  1%  5%  4%  6%  1%  2%
[libx264 @ 0x7fe631846800] i8c dc,h,v,p: 43% 33% 19%  5%
[libx264 @ 0x7fe631846800] Weighted P-Frames: Y:0.3% UV:0.2%
[libx264 @ 0x7fe631846800] ref P L0: 63.5%  3.6% 16.9%  9.1%  7.0%
[libx264 @ 0x7fe631846800] ref B L0: 89.3% 10.7%
[libx264 @ 0x7fe631846800] kb/s:6084.54

Whith scene cut disabled

Note that with scenecut disable there are actually less key frames than with scenecut enabled, which makes sense as opposed to just dropping a key frame in every 50 frames it would also add them when a scene change is detetecd

/usr/local/bin/ffmpeg -y -i goptest.mov -codec:v libx264 -b:v 6000K -s 1920x1080 -preset slower -tune film -me_range 24 -bufsize 50000K -maxrate 50000K -refs 4 -profile:v high -level 4.1 -x264opts keyint=50:no-scenecut -threads 0 -sn 'goptest_1080p_keyint50_noscenecut.mp4'
frame= 1153 fps=1.1 q=-1.0 Lsize= 34821kB time=00:00:46.04 bitrate=6195.7kbits/s dup=15 drop=0 
video:34255kB audio:538kB subtitle:0 global headers:0kB muxing overhead 0.077056%
[libx264 @ 0x7f9b9b846800] frame I:24 Avg QP:19.70 size: 45538
[libx264 @ 0x7f9b9b846800] frame P:1112 Avg QP:23.33 size: 30551
[libx264 @ 0x7f9b9b846800] frame B:17 Avg QP: 7.33 size: 643
[libx264 @ 0x7f9b9b846800] consecutive B-frames: 97.9% 0.2% 0.5% 1.4%
[libx264 @ 0x7f9b9b846800] mb I I16..4: 51.0% 45.6% 3.4%
[libx264 @ 0x7f9b9b846800] mb P I16..4: 49.8% 40.4% 1.3% P16..4: 6.3% 0.4% 0.1% 0.0% 0.0% skip: 1.7%
[libx264 @ 0x7f9b9b846800] mb B I16..4: 0.2% 0.1% 0.0% B16..8: 8.3% 0.0% 0.0% direct: 0.5% skip:90.8% L0:73.7% L1:26.2% BI: 0.0%
[libx264 @ 0x7f9b9b846800] final ratefactor: 22.35
[libx264 @ 0x7f9b9b846800] 8x8 transform intra:44.2% inter:94.3%
[libx264 @ 0x7f9b9b846800] direct mvs spatial:64.7% temporal:35.3%
[libx264 @ 0x7f9b9b846800] coded y,uvDC,uvAC intra: 21.2% 67.2% 12.9% inter: 16.5% 38.6% 0.5%
[libx264 @ 0x7f9b9b846800] i16 v,h,dc,p: 35% 25% 18% 22%
[libx264 @ 0x7f9b9b846800] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 19% 37% 3% 5% 5% 10% 3% 5%
[libx264 @ 0x7f9b9b846800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 20% 43% 1% 5% 4% 6% 1% 2%
[libx264 @ 0x7f9b9b846800] i8c dc,h,v,p: 43% 33% 19% 5%
[libx264 @ 0x7f9b9b846800] Weighted P-Frames: Y:0.3% UV:0.2%
[libx264 @ 0x7f9b9b846800] ref P L0: 63.3% 3.6% 16.9% 9.2% 7.0%
[libx264 @ 0x7f9b9b846800] ref B L0: 98.5% 1.5% 0.0%
[libx264 @ 0x7f9b9b846800] ref B L1: 99.6% 0.4%
[libx264 @ 0x7f9b9b846800] kb/s:6084.43