Custom implementation of ffmpeg on an Ubuntu 8.10 server
Update (20 october 2009) - This is now two Ubuntu releases out of date, so please check some more recent sources before trying this.
I've been working for a while on a problem at the NFB where we are moving a Drupal site to a new server running Ubuntu Intrepid. The site depends on transcoding handled by the outdated Drupal video module. Because the video module did only a single-pass encode, NFB staff created a shell script to fool the video module and replace the single-pass encode with a double-pass encode that normalized the film audio.
The result was a generally better quality product. Now the problem is that this shell script uses ffmpeg for multiplexing at the end of the two-pass encoding process, and that the Ubuntu Intrepid ffmpeg package crashed gloriously on every multiplex.
So, I decided to compile a new ffmpeg package from source. But my problems with multiplexing didn't end there. Many of the blog posts I found made mention of source patches for the ubuntu x264 package. Anyway, I finally found my answer in the ubuntu forums:
http://ubuntuforums.org/showthread.php?t=786095
I followed these steps in a local virtual machine I created just for this purpose. After doing this, I had two deb packages ready (or so I thought) to be installed on our transcoding server. I fired off an e-mail to the supervisor of our IT department and transferred the files over to my home account on the server.
My supervisor, having lost custom packages to ubuntu updates in the past, insisted on a more comprehensive install, and sent me this link:
http://linux.togaware.com/survivor/Local_Debian.html
We followed these instructions to create a local repository. Then we popped into aptitude and tried to install our packages. BANG! Dependency errors.
After a couple of hours of combined troubleshooting we hit on a recipe that worked, so with no further ado, I present you with a complete recipe for building Debian packages of ffmpeg and x264 from source, transferring them to another machine, creating a local repository, and successfully installing them to your machine.
Compiling the custom packages
First we purged the existing packages:
apt-get purge apt-get purge ffmpeg x264 libx264-dev
Then we installed the following:
apt-get install build-essential subversion git-core checkinstall yasm texi2html libfaac-dev libfaad-dev libmp3lame-dev libtheora-dev libxvidcore4-dev
The build-essential package contains compilation tools, subversion and git-core are to fetch the required sources, the libs are for configuring ffmpeg, and checkinstall is the program we will use to install the programs and generate .debs.
Next we fetched, configured and compiled x264:
git clone git://git.videolan.org/x264.git
cd x264
./configure --enable-shared
make
checkinstall --fstrans=no --install=yes --pkgname=x264 --pkgversion "1:0.svn`date +%Y%m%d`-0.0ubuntu1"
ldconfig
And we did the same for ffmpeg:
svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg
cd ffmpeg
./configure --enable-gpl --enable-postproc --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libtheora --enable-libx264 --enable-libxvid
make
checkinstall --fstrans=no --install=yes --pkgname=ffmpeg --pkgversion "3:0.svn`date +%Y%m%d`-12ubuntu3"
Modifying the control files on the transcoding server
Checkinstall does not add dependancies to the packages, so we did that manually.
First we uncompressed the .deb packages and their control file:
dpkg-deb -x x264_1:0.svn20090206-0.0ubuntu1-1_i386.deb x264
dpkg-deb -x ffmpeg_3:0.svn20090206-12ubuntu3-1_i386.deb ffmpeg
dpkg-deb --control x264_1:0.svn20090206-0.0ubuntu1-1_i386.deb x264/DEBIAN
dpkg-deb --control ffmpeg_3:0.svn20090206-12ubuntu3-1_i386.deb ffmpeg/DEBIAN
Then we edited the control files:
vim x264/DEBIAN/control
vim ffmpeg/DEBIAN/control
We copied and pasted the dependancies from
/var/lib/apt/lists/ubuntu.nfb.ca_ubuntu_dists_intrepid_main_binary-i386_Packages
for ffmpeg, and
/var/lib/apt/lists/ubuntu.nfb.ca_ubuntu_dists_intrepid_multiverse_binary-i386_Packages
for x264. We found these files by grepping for "Package: ffmpeg" or "Package: x264", and used those search terms again within the files to locate the package information.
Here's what our control files looked like, more or less, after this process:
Package: ffmpeg
Version: 3:0.svn20090206-12ubuntu3-1
Architecture: i386
Maintainer: root@localhost
Installed-Size: 48192
Depends: libavcodec51 (>= 3:0.svn20080206-8) | libavcodec-unstripped-51 (>= 3:0.svn20080206-8), libavdevice52 (>= 3:0.svn20080206-8) | libavdevice-unstripped-52 (>= 3:0.svn20080206-8), libavformat52 (>= 3:0.svn20080206-8) | libavformat-unstripped-52 (>= 3:0.svn20080206-8), libavutil49 (>= 3:0.svn20080206-8) | libavutil-unstripped-49 (>= 3:0.svn20080206-8), libc6 (>= 2.7), libfreetype6 (>= 2.3.5), libimlib2, libsdl1.2debian (>= 1.2.10-1), libswscale0 (>= 3:0.svn20080206-8) | libswscale-unstripped-0 (>= 3:0.svn20080206-8)
Provides: ffmpeg
Filename: dists/local/local/binary-i386/ffmpeg_3:0.svn20090206-12ubuntu3-1_i386.deb
Size: 19122026
MD5sum: 06847640d947b33b697c03182e2c593f
Section: checkinstall
Priority: extra
Description: Package created with checkinstall 1.6.1
Package: x264
Version: 1:0.svn20090206-0.0ubuntu1-1
Architecture: i386
Maintainer: root@localhost
Installed-Size: 2508
Depends: libatk1.0-0 (>= 1.20.0), libc6 (>= 2.7), libcairo2 (>= 1.6.0), libglib2.0-0 (>= 2.12.0), libgtk2.0-0 (>= 2.12.0), libpango1.0-0 (>= 1.20.2), libx11-6, libx264-59 (>= 1:0.svn20080408)
Provides: x264
Filename: dists/local/local/binary-i386/x264_1:0.svn20090206-0.0ubuntu1-1_i386.deb
Size: 892740
MD5sum: bfb45ec75936b3acdc4fc0d6be7fe5b6
Section: checkinstall
Priority: extra
Description: Package created with checkinstall 1.6.1
While we were in the control files, we also took the time to modify the maintainer and description strings. Once we were satisfied, we rebuilt our packages:
dpkg -b ffmpeg ffmpeg_3:0.svn20090206-12ubuntu3-1_i386.deb
dpkg -b x264 x264_1:0.svn20090206-0.0ubuntu1-1_i386.deb
You will be overwriting your old packages - if you want to use a different name for the rebuilt packages, then make sure to modify the "Version:" line in your control files so that the new package name and version string while match up.
Creating the local repository
We chose to create our local repository in opt:
mkdir /opt/my_repository/dists/local/local/binary-i386
And set permissions.
chmod -R g+w /opt/my_repository
chown -R root:admin /opt/my_repository
We copied both packages to inside /opt/my_repository/dists/local/local/binary-i386 and ran a command to create package information:
dpkg-scanpackages dists/local/local/binary-i386 /dev/null > dists/local/local/binary-i386/Packages
This command MUST be run from inside /opt/my_repository. Otherwise when you use apt or one of its friends to try to install the packages, they won't be able to find them.
We added these lines to /etc/apt/sources.list:
#Repository of local packages deb file:/opt/my_repository local local
At this point we were able to start aptitude, update ("u") the package lists, find our custom packages and add them ("+") and finally install them ("g").
The final step was to create a symbolic link to ffmpeg in /usr/bin, because our shell script and drupal were looking for /usr/bin/ffmpeg.
ln -s /usr/local/bin/ffmpeg /usr/bin/ffmpeg
