chemicaloliver » Internet

Installing PHP 5.4 in Ubuntu

chemicaloliver — Sat, 03 Dec 2011 23:46:02 +0000

I’ve been excitedly awaiting some of the new features found in PHP 5.4, in particular array notation and file upload progress monitoring so I decided to try and install PHP 5.4 on Ubuntu. Fortunately there is a repository of prebuilt packages for Ubuntu 11.04 (Natty Narwhal) at http://apt.damz.org/

I recommend using a VM so you don’t mess up any stable PHP install – I only tested this on Ubuntu 11.04 as that’s what was recommended and I already had a VM with it installed.

Installation

Installation is a very simple matter of adding the gpg key:

curl http://apt.damz.org/key.gpg | sudo apt-key add -

Adding the repository to /etc/apt/sources.list:

deb http://apt.damz.org/ubuntu natty php54

Then installing as normal

sudo apt-get install php5 libapache2-mod-php5

Codeigniter Unit Testing with Simpletest and PHP 5.3

chemicaloliver — Thu, 03 Nov 2011 22:03:04 +0000

I’m a regular user of codeigniter-simpletest developed by Eric Barnes however I ran into a number of issues when attempting to run some features under PHP5.3, this was due to using an old version of Simpletest. I have now updated this and had my pull request accepted and merged into the main repo so that it now uses the most recent versions (1.1alpha3).

There is however one important change to note which will stop old tests working, that is the way that test classes define a label, these are optional so this only applies if you use them in your project. The name of the test is now passed through the simpletest constructor otherwise the default is to use the classname of the test.

Before

class test_users_model extends CodeIgniterUnitTestCase
{
	public function __construct()
	{
		parent::__construct();

		$this->UnitTestCase('Users Model');

		$this->load->model('users/users_model');
	}

etc...

After

class test_users_model extends CodeIgniterUnitTestCase
{
	public function __construct()
	{
		parent::__construct('Users Model');

		$this->load->model('users/users_model');
	}
etc...

I’m also working on integration with CI systems, Jenkins in particular based on this rather cool presentation: Setting up a Continuous Integration Server for Ubuntu with Codeigniter and Github

Enabling web sockets on Opera Mobile

chemicaloliver — Mon, 05 Sep 2011 20:57:21 +0000

A little project I’m working on requires the use or websockets in Opera Mobile, I’d read they were supported but when I tried it just threw an exception, then I discovered you have to enable them in the config:

Browse to opera:config in opera mobile
Open the User Prefs section
Check the enable websockets box
Scroll right to the bottom of the page and select save
Enjoy much websockety goodness

getUserMedia(), the device API and the state of the webcam in browser using javascript and html5 video

chemicaloliver — Sat, 27 Aug 2011 19:03:42 +0000

Since first considering web camera applications during my MSc thesis I’ve be fascinated by different uses and application of webcams on the web. I believe outside of skype they are a very under utilised tool for both communication and interaction and despite their long existence I believe there is still a very long way to go in what they can do.

In this post I intend to summarise recent developments surrounding using a webcam within web applications and go on to discuss current implementations available for using a webcam with javascript.

Buzz about html5

Last year at Full Frontal 2010 Raul Rouget gave a talk entitled ‘Batshit crazy stuff you’ll be able to do in browsers’, he basically stole the show with a series of demos of webgl, audio api and file api developments in firefox, however the parts I was most intrigued by involved the use of the now defunct device API – using a webcam or other similar input devices directly in the browser.

Later that evening I sat in may hotel room, after a good amount of beer, and attempted to compile a patched version firefox from source. After downloading the whole lot over a tethered mobile connection and merging in some patches attached to a bug report I got a binary which crashed on startup. I tried again several times later and still didn’t get the desired result, having seen no other blog posts about it I assume very few other people got it to work!

getUserMedia()

Now, 8 months on, the former HTML5 API has been depreciated in favour of a new API based around getUserMedia() this is part of a larger API for peer to peer, in browser video communication. It allows html5 elements to have their source defined as a stream from a device:



Simple web camera display demo

Code sample taken from http://my.opera.com/core/blog/2011/03/23/webcam-orientation-preview and commented by me

That code produces this output when viewed in an opera mobile development build:

Implementations

At present implementions of this API are very thin on the ground:

Support is available in webkit on 32 bit ubuntu if a custom version of libwebkitgtk is used.
There are experimental builds of opera mobile which support some aspects of the API

Mobile

Opera Mobile

Opera have released a build of opera mobile which has support for use of a mobile phone webcam using the new API on android phones. I have successfully tested this on my HTC Desire Z (see screenshots) the support for the full API for peer to peer communication is still in development but this has support for the basics of video capture. This implementation appears to follow the published specification fairly closely

Desktop

Webkit (Ericsson)

An even more experimental implementation of the API has been written by Ericsson which uses a modified version of the libwebkitgtk library which allows use of the new API with compatible desktop browsers (supported with Epiphany). The installation is simple as they’ve provided a PPA:

#Install the Ericsson Labs public GPG key used to verify the package signatures
wget -O- --quiet https://labs.ericsson.com/files/gpg/public.key | sudo apt-key add -

#Add the Ericsson Labs PPA
sudo add-apt-repository http://files.labs.ericsson.net/ubuntu
sudo apt-get update

# Upgrade to the Ericsson Labs modified libwebkitgtk packages
sudo apt-get -y install libwebkitgtk-1.0-0

# (optional) Install Epiphany from the default Ubuntu repository
sudo apt-get install epiphany-browser

Once this installation is complete epiphany (and other browsers configured to use the modified library will have support for the API. It does however support a great deal more functionality compared with the Opera implementation including peer to peer functionality to enable serverless webcam chat type applications. Full documentation is available here

Sadly this implementation of the API differs from the standard specification and uses a prefixed version using webkitGetUserMedia() along with various other annoying differences. A comparable example to the simple webcam view above could be written thus:




Simple web camera display demo

Annoyances

This is not an API ready for use in any kind of real work application as there is no implementation in any mainstream browser and the specification is still in a state of flux. There are also functions which need to be worked out to handle multiple input sources and choosing the correct one, selecting the input size and many other bits and pieces. It could be many months or years before there is a reliable implementation across browsers.

The Future

This API could be the basis for many cool web applications and potentially replace applications like skype but there is a long way to go before we see standard implementations. However many parts of the API may be able to be replicated in flash and, for the simple parts at least, this may be a realistic step on the ladder for those wishing to get going with the API. A polyfill for the simple features is on my list of things to try…get hold your breath though!

My Plans

Recently I’ve been playing with the API on Opera Mobile in conjunction with jquery mobile, the results can be seen on github. At present it contains a simple camera app which can post a snapshot of the video stream to the sever as a base64 encoded image via the canvas.

Backing up my world – Online back ups for work and play

chemicaloliver — Fri, 24 Jun 2011 19:17:02 +0000

For a good while I’ve been meaning to get around to backing up my data properly. I’ve always had an external hard disk on which I backed up my important data periodically but that only covers the most basic of situations. If my house burnt down, or more likely somebody broke into my house,even just a power surge, I could conceivably loose both my real copy and the backup. From this I decided the only sensible way was to explore an offsite backup solution to use in addition to local backups. I’ve also been spurred on after buying an SSD and being unconvinced about their reliability. Indeed since starting to write this post my new SSD was recalled by corsair…

In this post I’ll describe what I backup and the online services and software I used.

My Data

I consider my requirements to be fairly standard for a web developer, I have a few types of data which need to be considered separately:

Code both open and closed source – close source much remain confidential as it’s mostly for work but open source code can and should be shared. ~50MB
Documents (Including some confidential data) – Letters, CVs, other random documents - none of which I wish to be available publicly at any cost. ~150MB
Photos – My archive of photos I’ve taken over the years, this is one of my most valuable assets, I’m not really concerned that they’re private. ~30GB
Music – A big problem…. ~150GB

Solutions

Code

All my code is stored under version control using either git or mercurial, this makes backing up very easy.

Closed Source

All my closed source code is work related and is backed up on a private Linode VPS using mercurial, this acts as the central repository and is also backed up automatically to a local machine at work using rsync.

Open Source

All the code I write personally is open source so I combine sharing with backup using github this does lead to me having one or two repositories with odd selections of code in but somebody may benefit from them one day!

Photos & Documents

Traditionally I had always survived with my photos stored in my desktop computer and a usb hard disk sitting next to the computer, now I’ve decided this isn’t sufficient so I’ve started to use Amazon S3 as an extra level of security. I use the command line too s3cmd to sync files in an rsync-like way. There is no encryption in this. I use a cron job to automate the synchronization periodically. The only negative point I found was the large amount of time taken to upload 30GB of photos on a domestic ADSL line.

For my documents containing confidential data I simply add the extra step of using the gpg encryption built into s3cmd. This means I can still get to my files even without the s3cmd tool.

Music

As I have around 150GB of music stored in mp3 format I fell this is an unrealistic amount to upload to amazon S3 as it would take several weeks. At present I use spotify for a large amount of my music listening and am becoming less and less dependent on mp3s so at the moment there is no other sensible option apart from mirroring on a usb drive, often this is kept at my parents house so it’s just as good as any online system.

Services & Tools

When choosing tools and services I was looking for known names which are likely to stick around, not the latest great start-up which might be gone next week.

Github & Mercurial

These standard version control systems require little explanation, there is little to choose between them and they work fast and reliably to record, version and push code around.

Amazon S3

There are several cloud storage providers available currently, I choose Amazon without looking round much as to me it seems the least likely to go away, it’s used by many large websites and based on the Amazon .com infrastructure. Out of the box it’s not a user friendly solution with only a primitive web interface, but when it’s comprehensive API is combined with one of the many third party tools it becomes a very powerful storage platform. Cost wise it’s around $0.14 per GB per month + $0.10 per GB of data transferred in or out, this to me is a negligible cost.

There are many tools available which provide different functionality and differing operating system.

S3cmd

S3cmd is a command line tool to interface with amazon s3, once you’ve given it your api details you can access your buckets on S3 much like any other filesystem using s3cmd to put/get/sync files or folders between your local filesystem and S3, the interface is very similar to rsync. For example to syncronise the current folder with a bucket called photos:

s3cmd sync ./ s3://photos

Familiar commands such as ls are also available:

s3cmd ls

would list all the buckets you have created. Similar commands are available to create and delete buckets.

On the fly gpg encryption is also available but sadly only with the put and get commands, not sync.

One slight gotcha is that unless you specifically state in the config file the location, all buckets are created in the US Amazon datacenter, for speed I’d prefer the one in Ireland.

I use this program for all my regular contact with S3 as it’s easily scriptable for use in a cron job.

S3cmd is also available in common linux repositories.

Duplicity

Duplicity is a general purpose command line backup tool with creates tar volumes of files or folders and moves them to external media, it then creates incremental backups as requested until another full backup is created. All files are compressed and optionally encrypted on demand so it’s very simple to the user. I have used duplicty in the past but I felt it was a little overly complex, s3cmd is much easier to setup and use, for most of my files I also felt that I’d prefer just to keep a copy rather than incremental backups. If you need more sophistication than s3cmd, or indeed would like to backup to something that’s not s3 then Duplicity may be for you.

Deja Dup

Deja dup is a nice GUI for duplicity which is increasingly being included with Gnome based linux distributions, in short you specify some folders, a destination and a backup frequency and then it takes care of the rest. For me however it was all too simple and vague, there is no possibility to set when the backups take place, no possibility to decide if an incremental backup or a full one is made. It may be nice for a non geeky user who just wants their data backed up sometime regularly, but I want to specify and know and see a log.

…

All in all I’m very happy with a mix on encrypted and unencrypted files hosted on an external drive locally and on S3, as long as they are maintained this should provide me with the ability to recover from most eventualities. ….Famous last words…

Websocket URL routing (specifying MQTT subscription topic by URL)

chemicaloliver — Sun, 30 Jan 2011 16:10:09 +0000

My previous example Node.js, MQTT and Websockets showed the use of a websocket to broadcast messages from a subscribed MQTT topic, however the topic was hard coded and the messages broadcast to all who connected. The example below uses the same client side script to connect but allows the MQTT topic to be specified in the URL and only broadcasts to the individual client. Now to subscribe to a topic “test” the websocket address would be ws:///test.

/* Include required libraries */
var util   = require('util'),
    sys = require("sys"),
    url = require("url"),
    ws = require("websocket-server"),
    spawn = require('child_process').spawn,

    /* Create websocket server */
    server = ws.createServer({debug: true});

/*
 * Add listener for web socket connections
 */
server.addListener("connection", function(conn){

/*
 * Ceate call to mosquitto_sub cli client
 * to subscribe on topic specificed by the url
 *
 * substr(1) removes intial / from path
 */
mosq = spawn('mosquitto_sub',['-t',conn._req.url.substr(1)]);

/*
 * Bind an event to stdout to get output from mosquitto_sub
 * and publish it to the websocket
 */
mosq.stdout.on('data', function (data) {
    /*
     * Brodcast the MQTT message straight back
     * out on the websocket
     */
    server.send(conn.id, data)

    /* Log the message to the console */
    console.log('' + data);
    });

    /*
     * Bind an event to stderr so we can see any
     * errors that cause mosquitto_sub to crash
     */
    mosq.stderr.on('data', function (data) {
    console.log('error: ' + data);
    });

});

/* Start the websocket server listening on port 8000 */
server.listen(8000);

Node.js, MQTT and Websockets

chemicaloliver — Sat, 29 Jan 2011 14:04:25 +0000

For a while I’ve been looking at how to bridge the MQTT protocol and websockets to make it easier to build web applications using data broadcast in MQTT streams. In the past I used python and mod_pywebsocket along with mosquitto python libraries however this was cumbersome and difficult to install. Here I present a simple solution using node.js to interact with mosquitto MQTT clients. Node.js lends itself to working well with messaging systems like MQTT and websockets due to its event driven nature. I’m also in love with node.js at the moment!

This is much simpler than previous attempts and I put the initial test together in less than ten minutes.

Prerequisites

Obviously you’ll need to have node.js and mosquitto installed and also the node library node-websocket-server (which can easily be installed using npm). All my work was tested under ubuntu but there is no reason why it wouldn’t work on OSX or even cygwin. To test you’ll need a websocket compatible browser such as recent versions of chrome.

System Structure

mosquitto_pub and mosquitto_sub are command line MQTT clients supplied with the mosquitto MQTT broker, here mosquitto_sub will be called using node.js as child processes, events are generated on output from the process. The data is then captured and broadcast over a websocket. In this case a simple jquery page is used to display the broadcast messages but in a real application there would be more client side processing to make a useful application.

Simple Server Side Subscriber Code

This example subscribes to the topic “test” and broadcasts all messages on this topic over the websocket on port 8000.

/* Include required libraries */
var util   = require('util'),
sys = require("sys"),
ws = require("websocket-server"),
spawn = require('child_process').spawn,

/* Create websocket server */
server = ws.createServer({debug: true}),

/*
* Ceate call to mosquitto_sub cli client
* to subscribe on topic "test"
*/
mosq = spawn('mosquitto_sub',['-t','test']);

/*
* Bind an event to stdout which occurs when
* mosquitto_sub outputs anything
*/
mosq.stdout.on('data', function (data) {
/*
* Brodcast the MQTT message straight back
* out on the websocket
*/
server.broadcast(data);
/* Log the message to the console */
console.log('' + data);
});

/*
* Bind an event to stderr so we can see any
* errors that cause mosquitto_sub to crash
*/
mosq.stderr.on('data', function (data) {
console.log('error: ' + data);
});

/* Start the websocket server listening on port 8000 */
server.listen(8000);

Demonstration Client

This is a bare bones page which prints out messages received over the websocket.

Further Development

This example is of little practical use on its own however it could easily be the basis of a flashy home monitoring system. Publishing MQTT messages could be easily achieved by calling mosquitto_pub in the child process.

Chromium OS – another look 1 year on

chemicaloliver — Sun, 16 Jan 2011 12:21:03 +0000

Just over a year when the chromiumOS code was first released I spent a while compiling it and experimenting and posted my opinions

I made a few basic points:

Little wifi hardware support
Requires a DHCP network
Quick boot time (~10s)
It’s just like Google chrome and nothing else

A few weeks ago @popey mentioned on twitter he was compiling it and this inspired me to have another go with it. In this post I intend to see if things have moved on.

Wifi & Networking

Chromium OS now seems to have decent wifi support (at least it works on the two laptops I own) how ever Google still have not addressed the lack of a network control panel which is able to define a static IP address, this basically means unless you use DHCP then you cannot use the operating system. On the first boot you are required to enter details of a Google account, obviously this requires internet access, however subsequent logins can proceed even when offline. However 99% of functionality is useless without an internet connection:

Boot & Login

Previously when I tested chromiumOS the boot was a plain blue screen and login box, the graphics of this have been considerably improved and details of previously used accounts are displayed on the login screen, as well as allowing guest logins. I didn’t take any measurements but I feel the boot time has slightly increased, probably due to the increased complexity of the OS. Sadly it would not let me take a screenshot of the login screen.

Filesystem Access

I was pleasantly surprised to find that there is some access to local files, to take the screenshots for this review I logged into dropbox and uploaded them from chromiumOS, on trying to upload a file it brings up a nice styled upload dialogue. There is some strange behaviour, the filesystem you see in the upload browser dialogue does not reflect the actual filesystem the folder ‘Screenshots’ is nowhere to be found when browsing using the terminal, this is not really a problem and is maybe a nice user friendly feature.

Initially when I tried to upload files to dropbox it tried to load a flash uploader, while flash is supported it is very flakey and promptly crashed! However before it crashed it did show evidence of the raw filesystem, an upload dialogue triggered from flash shows a more conventional file browser with the linux root filesystem visible…possibly an indication there is a good way to go before chromiumOS is polished enough for release.

New Stuff

Since last time there has obviously been work done but I feel it’s more as a result of the development of the browser rather the OS, for example it nicely syncs your bookmarks, apps and settings between all other versions of chrome which is nice but not ground breaking anymore.

Build & Compilation

Since last time I tried the build environment has changed from just a basic compilation to building a gentoo chroot and then building the OS inside that, however the process is well documented and managed by build scripts and presented no issues aside from the several hours taken to complete the download and compilation, even on quite a powerful machine.

Overall Impression

Functionality wise it’s still just google chrome and nothing else, just a newer version! I can see it working well for non technical people who use just webmail and browse the web but at present even something like android provides a great deal more overall functionality. To me chromiumOS is little more that an interesting experiment into what the world would be like with only web applications…and I’m not looking forward to that if it ever happens!

Building a Low Powered and Silent Atom Based Server (Sheevaplug alternative)

chemicaloliver — Wed, 15 Sep 2010 18:08:12 +0000

After I had problems with my sheevaplug power supply I had lost faith in the device and didn’t trust it to survive being on 24/7 without exploding/setting on fire etc… so after NewIT kindly fixed it I sold it. This left me with no server to mess with so I decided to try and build a new one – the two primary requirements being that it used as little power as possible (ideally around 10W – the same as my sheevaplug with hd and usb hub) and was totally silent. In this post I’ll describe what I came up with.

Component Choice

My requirements for quietness and low power stipulated that a fanless system would be best so as a balance between low power and performance I choose a 1.6GHz N270 atom based motherboard (the Intel D945GSEJT) which consists of a mixture of laptop and desktop components while providing ample speed for my requirements in silence. The motherboard is also nice because it has an internal DC-DC power supply meaning that only an external DC power supply is required, no extra faffing as with a lot of ITX boards on the market. I added 2GB of RAM to this, the maximum allowed by the motherboard.

I decided I could cope with the slight noise generated by a hard disk so went for the best value 2.5 inch disk I could find (A Hitachi 250GB), an SSD was beyond the performance required and budget of the project!

When choosing the case I went for the quite large Lian Li PC-Q07, it has space for a full atx power supply but I had in mind that I wanted a system that I could grow and modify if I thought of a new use for it so currently I have a motherboard and small hard disk in a case able to accommodate a full ATX PSU, 5.25″ optical drive and 3.5″ Hard Disk…I mentioned space was consideration!

The motherboard has headers for two serial ports and a parallel port so I added a serial PCI header adaptor into the only PCI slot. The large metal panel visible below is custom blanking plate as I haven’t fitted an internal PSU – it’s just a piece of 0.5mm aluminium from ebay cut to the correct size.

Software

My current Linux distribution of choice is ubuntu so I chose ubuntu server for the machine as it’s what I’m with familiar with and does everything I need. I usually run apache, mysql, php, python, mosquitto (mqtt broker) and irssi (IRC) on the machine 24/7 and so far it has performed flawlessly. I used powertop for power optimisations.

Power Consumption

When I first started out using the sheevaplug one of my main motivations was the low power usage, as my needs grew the sheevaplug + SD card became a sheevaplug + powered USB Hub + USB HD all in all consuming around 10.5W of power, therefore this was my target for this machine. In addition to using low power components I also used PowerTOP to optimise the power usage of the operating system. PowerTOP accomplishes this by ensuring cpu scaling is working correctly and powering down systems which are not required.

Before carrying out optimisation with powertop my mains power consumption was ~ 14W idle and after nearer the target of 10.5W, now using ~11-12W:

The low power consumption I achieved surprised me as the sheevaplug has been developed with this as its main aim and sacrifices processing power to achieve it. My atom system could rings around the sheevaplug in terms of processing power and comes within around 1-2W in terms of power consumption…so much as I’d like to love the sheevaplug the bad design and service provided by Globalscale force me to take the less cool but more functional route.

WebSocket-Arduino Forked – Updated arduino websocket library now in development

chemicaloliver — Sat, 11 Sep 2010 13:20:43 +0000

After consultation with the original developer of the only existing Arduino (@razorbeans) websockets library I have forked the project to allow me to take over maintenance. Over the coming weeks and months I intend to:

Migrate the library to use the new bundled string library rather than the WString library (mostly done)
Tidy up the code
Add functions for onconnect, onmessage and allow sending of strings at a time determined by the Arduino, not just in response to a message from the client.
Update the library to allow recent browsers to connect. Currently the library supports the version 75 handshake protocol but most browsers have now moved to the more recent 76 protocol..

The project is available to download on Github currently this will only work with versions of Google chrome before 6.0.414.0 but this will hopefully change soon as the handshake is updated.