Fixing Email Addresses in Git Repos after migration from Mercurial using Fast Export

Migrating repos from Mercurial to Git can be achieved by a variety of methods. The best method I’ve found is to use fast-export (not HgGit), however regardless of the method  they all borked the importing of my email address on commits. In this post I’ll detail how to fix this.

First I performed the conversion as detailed here.

After this all my commits where shown in gitk as devnull@localhost although this only came to my attention when I tried to push to github and got an invalid-email-address error.

This can be easily fixed using the git filter-branch command:

#!/bin/bash

git filter-branch -f --env-filter '

an="$GIT_AUTHOR_NAME"
am="$GIT_AUTHOR_EMAIL"
cn="$GIT_COMMITTER_NAME"
cm="$GIT_COMMITTER_EMAIL"

# Repeat this for each user / email which needs fixing
if [ "$GIT_AUTHOR_NAME" = "<Name used on commit>" ]
then
    cn="<Name used on commit>"
    cm="<New email address>"
    an="<Name used on commit>"
    am="<New email address>"
fi

export GIT_AUTHOR_NAME="$an"
export GIT_AUTHOR_EMAIL="$am"
export GIT_COMMITTER_NAME="$cn"
export GIT_COMMITTER_EMAIL="$cm"
' -- --all

Obviously the placeholders need to be replaced with your values.

This code is based on a stackoverflow answer but that only works for the current branch, mine applies to all branches.

Atomic Counters using Mongodb’s findAndModify with PHP

A common problem when developing web applications is the ability to generate unique sequential numbers, my recent use case was an API which generated an order number on receiving an order but before the order had been stored in a database, normally I would use MySQL auto incrementing keys but I needed to send the order number back to the client long before the order was stored, as Mongodb was already in the application stack it seemed the natural place to generate a persistent source of sequential numbers.

MongoDB provides a useful function findAndModify which, while MongoDB doesn’t support transactions, allows a value to be retrieved and updated atomically, making it ideal for this situation where I wanted the order number to increment by one every time it was used.

For those who’re in the dark about why you’d need to retrieve and update a value in one operation, the advantage of using this over separate find and updates is that it avoids the race condition where the same value could be retrieved twice if another client attempted to retrieve the counter before the update has taken place.

Read more ▼

Styling the HTML5 Meter Tag Using the Shadow DOM

Almost three years ago I read an article on html5doctor describing the meter tag, during the course of the article an example is given of the http://www.justgiving.com/  gauge of how near you are to your target:

The article goes on to suggest this would be an ideal use case for the <meter> tag, however at that time it was impossible to style the meter tag to look like this. This afternoon after discovering that shadow DOM support had made it into Chrome stable I attempted to recreate the Just Giving meter using no images and without a mess of divs.

This is the result, one I’m pretty happy with, I haven’t really made much attempt to style the surrounding areas, I just wanted to concentrate on the (frankly rather cool) meter:

Or see a live demo.

Read more ▼

Inspecting the Shadow DOM using Google Chrome

Over the past year or so there has been much hype surrounding the shadow DOM and the new options it allows for styling standard elements such as <input> or <progress>, it can be used with any element which consists of markup generated by the browser but normally hidden from view. Developers can also create shadow DOM elements too.

Until recently even where there is support for styling shadow DOM elements it has been difficult to determine what is available, however the webkit inspector in Google Chrome now has support for viewing and rendering the shadow DOM however I’ve found documentation on this very thin on the ground. I’ve tested this in the current stable branch (v19.0) on Linux, I believe the same functionality should be present across other platforms too.

UPDATE

Now Chrome has matured this feature has matured and it is no longer an experiment, step 1 and 2 can be ignored on recent versions and the option to show the Shadow DOM can be found in the general inspector options.

To enable support and to inspect shadow DOM elements :

  1. Enable the Shadow DOM and Developer Tools experiments in chrome://flags
  2. Enable the Shadow DOM in the Inspector settings, this panel can be accessed by the cog on the bottom right of the inspector.
  3. Check the ‘show shadow DOM’ checkbox.
  4. Restart your browser

Elements which have shadow DOM elements will now be displayed in the elements tab:

This functionality now makes it possible to explore the different style options without having to take wild stabs in the dark or examine the webkit source code.

Using PHP – Resque with Silex and the Symfony2 Classloader

Resque is a popular Redis-backed Ruby library for creating and processing background jobs, this is all well and good if you’re building applications in Ruby but fortunately for those, like me, who prefer PHP there is also a PHP port - PHP-Resque. This post will describe how to use php-resque in conjunction with the Silex micro framework to queue jobs, initially this was not immediately obvious to me as php-resque is not namespaced so wouldn’t work using the default autoloading configuration but fortunately the class naming seems to follow the PEAR convention where the path to the class is defined by the classname (i.e. Resque_Event would be found in Resque/Event.php). The method described here should work for any similar libraries.

One point to note, there is a Symfony2 bundle that could be easily integrated with Silex which aims to integrate php-resque but it’s a lot more complicated than I require and in most cases is just not necessary.

The key to using php-resque is to use the registerPrefix method of the symfony classloader (included with Silex) which uses PEAR naming conventions load libraries.

To configure php-resque first clone a copy of the library from github into the vendors folder (or wherever you like to store your external libraries):

git clone https://github.com/chrisboulton/php-resque.git vendors/php-resque

Now we have the library in place where ever you configure your application, in my case /src/bootstrap.php which is included in the main application file point the Synfony class loader to the place for classes beginning with the prefix Resque, all the relevant php-resque files are in the lib subdirectory:

$app['autoloader']->registerPrefix('Resque', __DIR__ . '/../vendor/php-resque/lib');
$app['autoloader']->register();

It does need to be before the $app['autoloader']->register();

The library can then be used in your application at will without requiring or manually including it:

Resque::setBackend('localhost:6379');

$args = array();

Resque::enqueue('default', 'My_Job', $args);

Once you get your head around the different options the synfony2 classloader (and the rest of the components) are wonderful.

HTTP Basic Auth in Silex

Silex is a great platform for building small web applications and APIs, recently I’ve been using it to build an API with only a couple of routes. As this API will only be used by a couple of users it made sense to use use HTTP basic auth (over SSL of course). HTTP auth could be left to apache/nginx etc. but that wouldn’t give me the control I’d like over the output and authentication so I implemented it in Silex, I hope someone finds this useful:

HTTP basic authentication is very simple and just passes a username and password in the headers, PHP has built in functionality to extract these values which can be used in the Silex before hook to ensure it happens before every request is fulfilled, my example is for an API which returns JSON but it would work equally well for a conventional website:

$app->before(function() use ($app)
{
    if (!isset($_SERVER['PHP_AUTH_USER']))
    {
        header('WWW-Authenticate: Basic realm="<website name>"');
        return $app->json(array('Message' => 'Not Authorised'), 401);
    }
    else
    {
        //once the user has provided some details, check them
        $users = array(
            'workflow' => 'password'
            );

        if($users[$_SERVER['PHP_AUTH_USER']] !== $_SERVER['PHP_AUTH_PW'])
        {
            //If the password for this user is not correct then resond as such
            return $app->json(array('Message' => 'Forbidden'), 403);
        }

        //If everything is fine then the application will carry on as normal
    }
});

Full details of implementing HTTP auth in PHP can be found in the PHP manual, this includes how to implement HTTP digest auth.

JSON Error / Exception Messages in Codeigniter 2.0+

The Problem

One of my recent projects required me to build a quick JSON only API to abstract interaction with multiple databases for multiple web applications, as I’d already got some of the logic in Codeigniter I just added Phil Sturgeon’s Codeigniter REST Library. However while this handles all method not found errors when URL routing gets as far as the controller all other errors still appear as HTML, when using curl and attempting to parse as JSON this isn’t helpful.

The Solution

My solution was to extend the core Exceptions class which normally deals with these errors. This is done by creating MY_Exceptions.php in application/core and using something similar to the code below:
Read more ▼

PHP 5.4 File Upload Progress and HTML5 Progress Bars

This posts was originally written for using pre release versions of php5.4, now PHP5.4 has been release everything should work as before I haven’t tested it. There is probably more choice in ways to install it now.

There was a live demo of this but I couldn’t justify the cost of a whole ec2 instance just for this!

The way I have implemented this may be problematic in Chrome/Safari due to Webkit Bug 23933,  ”XMLHttpRequest doesn’t work while submitting a form (useful for progress tracking)” I tested this in firefox (the latest version at the time) and it worked fine. Chrome may or may not work. Let me know how you get on.

Full details of the upload progress implementation can be found the official RFC: https://wiki.php.net/rfc/session_upload_progress

Developers who work with PHP applications that upload files commonly struggle with providing user feedback on the upload progress, usually using flash and javascript solutions like uploadify. In PHP 5.4 there is now integrated functionality to allow file upload progress to be passed back to the browser.

In this post I’ll describe the basic operation of this feature and describe a quick example of its use.

How it Works

The upload progress functionality stores the current progress in a session variable which can then be queried as required to give the current progress, it requires the use of PHP native sessions. The $_SESSION key is set by the form name and a prefix defined in php.ini

Read more ▼

Installing PHP 5.4 in Ubuntu

I’ve been excitedly awaiting some of the new features found in PHP 5.4, in particular array notation and file upload progress monitoring so I decided to try and install PHP 5.4 on Ubuntu. Fortunately there is a repository of prebuilt packages for Ubuntu.

I recommend using a VM so you don’t mess up any stable PHP install.

Update

Since writing this post php 5.4 stable has been released and a more up to date package is available in this ppa: https://launchpad.net/~ondrej/+archive/php5 I haven’t tested this personally but should work as described below.

Installation

The ppa can be simply added to ubuntu:

sudo add-apt-repository ppa:ondrej/php5

Then installing as normal

sudo apt-get update
sudo apt-get install php5 libapache2-mod-php5

Codeigniter Unit Testing with Simpletest and PHP 5.3

I’m a regular user of codeigniter-simpletest developed by Eric Barnes however I ran into a number of issues when attempting to run some features under PHP5.3, this was due to using an old version of Simpletest. I have now updated this and had my pull request accepted and merged into the main repo so that it now uses the most recent versions (1.1alpha3).

There is however one important change to note which will stop old tests working, that is the way that test classes define a label, these are optional so this only applies if you use them in your project. The name of the test is now passed through the simpletest constructor otherwise the default is to use the classname of the test.

Before

class test_users_model extends CodeIgniterUnitTestCase
{
	public function __construct()
	{
		parent::__construct();

		$this->UnitTestCase('Users Model');

		$this->load->model('users/users_model');
	}

etc...

After

class test_users_model extends CodeIgniterUnitTestCase
{
	public function __construct()
	{
		parent::__construct('Users Model');

		$this->load->model('users/users_model');
	}
etc...

I’m also working on integration with CI systems, Jenkins in particular based on this rather cool presentation: Setting up a Continuous Integration Server for Ubuntu with Codeigniter and Github