Fixing Email Addresses in Git Repos after migration from Mercurial using Fast Export

Migrating repos from Mercurial to Git can be achieved by a variety of methods. The best method I’ve found is to use fast-export (not HgGit), however regardless of the method  they all borked the importing of my email address on commits. In this post I’ll detail how to fix this.

First I performed the conversion as detailed here.

After this all my commits where shown in gitk as devnull@localhost although this only came to my attention when I tried to push to github and got an invalid-email-address error.

This can be easily fixed using the git filter-branch command:

#!/bin/bash

git filter-branch -f --env-filter '

an="$GIT_AUTHOR_NAME"
am="$GIT_AUTHOR_EMAIL"
cn="$GIT_COMMITTER_NAME"
cm="$GIT_COMMITTER_EMAIL"

# Repeat this for each user / email which needs fixing
if [ "$GIT_AUTHOR_NAME" = "<Name used on commit>" ]
then
    cn="<Name used on commit>"
    cm="<New email address>"
    an="<Name used on commit>"
    am="<New email address>"
fi

export GIT_AUTHOR_NAME="$an"
export GIT_AUTHOR_EMAIL="$am"
export GIT_COMMITTER_NAME="$cn"
export GIT_COMMITTER_EMAIL="$cm"
' -- --all

Obviously the placeholders need to be replaced with your values.

This code is based on a stackoverflow answer but that only works for the current branch, mine applies to all branches.

Cropping videos using ffmpeg / libav / avconv

Explanatory note:

Ubuntu (my distro of choice) and others are transitioning from ffmpeg to libav, libav is a fork of ffmpeg and most tools are drop in compatible, the method described in this post should work with recent versions of either, the command line tools ffmpeg and avconv are interchangeable.

Old Method

Historically ffmpeg had -croptop, -cropleft etc. parameters used for cropping videos, these have now been replaced by the -vf or video filter option which is a little more complex.

Current Method

The -vf option can be used to specify a section of the source video to use in the output by specifying the size and position of a rectangle to crop to:

The -vf option takes the argument crop=out_w:out_h:x:y - to create a new video file output.mpeg cropped to 720px x 600px and aligned 240px from the top:

avconv -i input.webm -vf crop=720:600:0:240 output.mpeg

In the example I’m also converting a webm video to mpeg along with cropping it, to convert webm to mpeg at the same dimensions just remove the cropping options.

Atomic Counters using Mongodb’s findAndModify with PHP

A common problem when developing web applications is the ability to generate unique sequential numbers, my recent use case was an API which generated an order number on receiving an order but before the order had been stored in a database, normally I would use MySQL auto incrementing keys but I needed to send the order number back to the client long before the order was stored, as Mongodb was already in the application stack it seemed the natural place to generate a persistent source of sequential numbers.

MongoDB provides a useful function findAndModify which, while MongoDB doesn’t support transactions, allows a value to be retrieved and updated atomically, making it ideal for this situation where I wanted the order number to increment by one every time it was used.

For those who’re in the dark about why you’d need to retrieve and update a value in one operation, the advantage of using this over separate find and updates is that it avoids the race condition where the same value could be retrieved twice if another client attempted to retrieve the counter before the update has taken place.

Read more ▼

Styling the HTML5 Meter Tag Using the Shadow DOM

Almost three years ago I read an article on html5doctor describing the meter tag, during the course of the article an example is given of the http://www.justgiving.com/  gauge of how near you are to your target:

The article goes on to suggest this would be an ideal use case for the <meter> tag, however at that time it was impossible to style the meter tag to look like this. This afternoon after discovering that shadow DOM support had made it into Chrome stable I attempted to recreate the Just Giving meter using no images and without a mess of divs.

This is the result, one I’m pretty happy with, I haven’t really made much attempt to style the surrounding areas, I just wanted to concentrate on the (frankly rather cool) meter:

Or see a live demo.

Read more ▼

Inspecting the Shadow DOM using Google Chrome

Over the past year or so there has been much hype surrounding the shadow DOM and the new options it allows for styling standard elements such as <input> or <progress>, it can be used with any element which consists of markup generated by the browser but normally hidden from view. Developers can also create shadow DOM elements too.

Until recently even where there is support for styling shadow DOM elements it has been difficult to determine what is available, however the webkit inspector in Google Chrome now has support for viewing and rendering the shadow DOM however I’ve found documentation on this very thin on the ground. I’ve tested this in the current stable branch (v19.0) on Linux, I believe the same functionality should be present across other platforms too.

UPDATE

Now Chrome has matured this feature has matured and it is no longer an experiment, step 1 and 2 can be ignored on recent versions and the option to show the Shadow DOM can be found in the general inspector options.

To enable support and to inspect shadow DOM elements :

  1. Enable the Shadow DOM and Developer Tools experiments in chrome://flags
  2. Enable the Shadow DOM in the Inspector settings, this panel can be accessed by the cog on the bottom right of the inspector.
  3. Check the ‘show shadow DOM’ checkbox.
  4. Restart your browser

Elements which have shadow DOM elements will now be displayed in the elements tab:

This functionality now makes it possible to explore the different style options without having to take wild stabs in the dark or examine the webkit source code.

Using PHP – Resque with Silex and the Symfony2 Classloader

Resque is a popular Redis-backed Ruby library for creating and processing background jobs, this is all well and good if you’re building applications in Ruby but fortunately for those, like me, who prefer PHP there is also a PHP port - PHP-Resque. This post will describe how to use php-resque in conjunction with the Silex micro framework to queue jobs, initially this was not immediately obvious to me as php-resque is not namespaced so wouldn’t work using the default autoloading configuration but fortunately the class naming seems to follow the PEAR convention where the path to the class is defined by the classname (i.e. Resque_Event would be found in Resque/Event.php). The method described here should work for any similar libraries.

One point to note, there is a Symfony2 bundle that could be easily integrated with Silex which aims to integrate php-resque but it’s a lot more complicated than I require and in most cases is just not necessary.

The key to using php-resque is to use the registerPrefix method of the symfony classloader (included with Silex) which uses PEAR naming conventions load libraries.

To configure php-resque first clone a copy of the library from github into the vendors folder (or wherever you like to store your external libraries):

git clone https://github.com/chrisboulton/php-resque.git vendors/php-resque

Now we have the library in place where ever you configure your application, in my case /src/bootstrap.php which is included in the main application file point the Synfony class loader to the place for classes beginning with the prefix Resque, all the relevant php-resque files are in the lib subdirectory:

$app['autoloader']->registerPrefix('Resque', __DIR__ . '/../vendor/php-resque/lib');
$app['autoloader']->register();

It does need to be before the $app['autoloader']->register();

The library can then be used in your application at will without requiring or manually including it:

Resque::setBackend('localhost:6379');

$args = array();

Resque::enqueue('default', 'My_Job', $args);

Once you get your head around the different options the synfony2 classloader (and the rest of the components) are wonderful.

HTTP Basic Auth in Silex

Silex is a great platform for building small web applications and APIs, recently I’ve been using it to build an API with only a couple of routes. As this API will only be used by a couple of users it made sense to use use HTTP basic auth (over SSL of course). HTTP auth could be left to apache/nginx etc. but that wouldn’t give me the control I’d like over the output and authentication so I implemented it in Silex, I hope someone finds this useful:

HTTP basic authentication is very simple and just passes a username and password in the headers, PHP has built in functionality to extract these values which can be used in the Silex before hook to ensure it happens before every request is fulfilled, my example is for an API which returns JSON but it would work equally well for a conventional website:

$app->before(function() use ($app)
{
    if (!isset($_SERVER['PHP_AUTH_USER']))
    {
        header('WWW-Authenticate: Basic realm="<website name>"');
        return $app->json(array('Message' => 'Not Authorised'), 401);
    }
    else
    {
        //once the user has provided some details, check them
        $users = array(
            'workflow' => 'password'
            );

        if($users[$_SERVER['PHP_AUTH_USER']] !== $_SERVER['PHP_AUTH_PW'])
        {
            //If the password for this user is not correct then resond as such
            return $app->json(array('Message' => 'Forbidden'), 403);
        }

        //If everything is fine then the application will carry on as normal
    }
});

Full details of implementing HTTP auth in PHP can be found in the PHP manual, this includes how to implement HTTP digest auth.

JSON Error / Exception Messages in Codeigniter 2.0+

The Problem

One of my recent projects required me to build a quick JSON only API to abstract interaction with multiple databases for multiple web applications, as I’d already got some of the logic in Codeigniter I just added Phil Sturgeon’s Codeigniter REST Library. However while this handles all method not found errors when URL routing gets as far as the controller all other errors still appear as HTML, when using curl and attempting to parse as JSON this isn’t helpful.

The Solution

My solution was to extend the core Exceptions class which normally deals with these errors. This is done by creating MY_Exceptions.php in application/core and using something similar to the code below:
Read more ▼

Streaming audio from Ubuntu Linux to a DLNA player (Blu Ray or PS3) using Rygel

This project started out of researching how to play sound from spotify or rhythmbox from my laptop running ubuntu 11.10 through my hifi. Initially I set out to see if an airport express would work using raop and pulseaudio but it seems that support for the new 802.11 version is flakey so I didn’t wish to invest £80 in a device that might not work. During my research I found that DLNA supported streaming, DLNA is a protocol commonly used for sharing media files with devices such as networked dvd players, internet tvs and consoles like the ps3 so I explored further.

DLNA is supported in Ubuntu (and other modern linux distros) by Rygel, part of the Gnome project. Rygel provides a DLNA server which also has the capability to capture a pulseaudio sink (an input or output stream) and stream it to a DLNA enabled device.

Below are the steps I took to enable me to stream audio from my computer to my Sony BDP-S370, they should be applicable to any similar device:
Read more ▼