We are a small development team, at Harvest Digital, handling multiple tasks throughout the day, across different environments and projects. It makes for an exciting challenge. Our roles/ skills are clearly defined and our small size means we can react quickly to change, with little margin for communication errors and so on (see Getting Real and Less is more: Jumpstarting Productivity with Small Teams). But, when it comes to deploying to a staging or production server, I am ashamed to admit that we had often introduced a huge potential for human error by doing much of the work manually. Each project is different and so there is no consistent deployment process across the board.

Deployment is often the last step in the process to get any attention and it is one that can be unnecessarily tedious and prone to error. With more agile development projects, where deployments may be required several times a day, we obviously cannot afford the time nor the potential mistakes that may be made by manually deploying a project.

At its simplest level, deployment may just involve making sure that the target server (i.e. staging or production) has the latest revision of files from the project’s source control repository (we use SVN). But factor in tests and database migration and you can see how easily things can go wrong. Especially if you are deploying several times a day. You need to automate as many steps in the process as possible, thereby eliminating any manual tasks that introduce potential error and duplication.

Whenever we reach a point in a project where the code can be considered stable, we should tag it in SVN as a release. This gives you the ability to deploy a particular release based on its tag, instead of just the latest revision in the repository. By convention you never create revisions in the tag folder – tags are simply named snapshots of the source repository. We typically use version numbers as tags with some scheme based on feature-set or milestone and release date that can be ordered alpha-numerically e.g. “rel_belfast_2.04″. This allows us to say with certainty what is deployed at any one time, without having to scan logs etc. Tags give context to a release name (you can easily associate the release with project milestones) that revision numbers and timestamps do not. Tags are usually kept in a ‘tags’ subfolder, in the top level of the repository.

For a while now at Harvest, we’ve been working with symfony – an excellent PHP framework – that includes a few tools to help simplify the deployment process (such as Pake). These tools allow for remote syncronisation of files with rsync (the benefits of synchronisation over standard FTP are much touted elsewhere) via SSH (with the shell command: e.g. symfony sync staging) and semi-automate the build process (including database schema changes, data dumps etc: e.g. symfony propel-build-all myApp) by using Pake on the target machine.

But there is still a lot of room for improvement, particularly when it comes to database migrations and SVN integration. Plus, symfony is not our only set-up. We need an automated deployment process that can be applied across all our projects. This is where I’ve previously seen shell scripts being used but Phing can do much of this heavy lifting work for us, as it has many standard deployment tasks built-in to it. According to the Phing blurb:

Features include file transformations (e.g. token replacement, XSLT transformation, Smarty template transformations), file system operations, interactive build support, SQL execution, CVS operations, tools for creating PEAR packages, and much more.

Phing (PHing Is Not GNU make) is a project build system based on Apache Ant and is available through the PEAR installer. If you use --alldeps on Phing, it’ll grab, funnily enough, all the dependencies. But most are good packages (e.g. PHPUnit, PHPdoc and VersionControl_SVN). It can be used in a multitude of ways, including unit testing, creating docs, running SQL – which allows us to keep database schema changes in version control. The beauty of Phing is everything can be configured easily through the build.xml file (and then further with properties files) – so that the project’s deployment configuration can be kept in version control too.

In short, we love this thing called Phing!

For more detail on advanced tasks such as database migration take a look at these posts from Federico Cargnelutti and Dave Marshall.

Jet Set Willy (1984: ZX Spectrum), Sonic the Hedgehog (1991: Sega Megadrive), Monkey Island (1990: PC), Doom (1993: PC), Worms (1995: PC), Abe’s Oddysee (1997: PS1), Grand Theft Auto (1997: PC), Rainbow Six (1998: PC) and many more; they all chart a certain personal view of the evolution of console and computer gaming. And it’s incredible to think how things have changed.

I can mark eras of my life in the same way you might signpost your autobiographical memory with where you were living. Sad but true. It tends to be just the one game as I don’t devote masses of time to gaming – when I find a game that I like, I stick with it.

Recently, I persuaded my wife that a PS3 would be a great addition to our family because “a PS3 is so much more than just a games console – you can use it to view all those digital photos and videos of our son”. And I’m glad I did as I think I’ve found the game to mark the next era - LittleBigPlanet. This game is incredible. It has brought the traditional platform game into the future with a fun, creative and collaborative online world that is constantly changing and ever evolving. Irrespective of what it represents in terms of amazing media and technological innovation, it also represents a return to pure and simple platform based game-play, with a few twists. And, possibly most importantly, it is impossible not to feel happy playing this game. It looks like we’re going to have some fun with this one…

Exciting times indeed but, please, let’s make sure we don’t lose any of the media attention and international focus that is needed on the urgent situation in the Democratic Rebublic of Congo.

Ituri: Waiting for change. (Photo by Fred)

Polka Tables, originally uploaded by markrocky.

From our photography excursion (with Harvest Digital) on the South Bank using “Lomolitos” (mini Lomos), on Wednesday evening…

1) Open source paves the way for the odd utopian dream:

Sir Thomas More’s description of Utopia as a place where “nobody owns anything, but everyone is rich” is not a bad way to describe the open source movement in which people around the world collaborate with each other to produce services that anyone can use – or improve on – for nothing.

2) Open source is recession proof:

One of the interesting things about the collaborative movement is that it is probably recession-proof, though you won’t see it in economic statistics because it mostly does not involve cash transactions.

3) Open source is robust and well-trusted by large successful organisations:

Big corporations, such as IBM, Google and Amazon, are devourers of open source software because they find it cheap, efficient, low-maintenance and reliable. But UK government departments, including health and the foreign office, have proved risk-averse with hardly any open source in their infrastructure.

4) Open source is socialism (well, social collaboration at least):

…open source combines the cooperative spirit that was at the heart of the Labour party in the past with the entrepreneurial skills needed today.

And with nothing more to add to that summary, I’m off to join the revolution (well, okay, I’m off down the pub to carry on ranting to anyone who’s too polite to stop me).

Whatever you say about Twitter, I was enjoying the ability it gave me to receive updates via SMS. In fact, it was this feature that won me over to using it and helped demonstrate what it was all about to mates. I figured that the initial free SMS alerts that appeared in my Twitter account would run out one day and then I’d have to pay for more credits – easy business model. I’d happily have paid too. So why pull it? Maybe its all part of some ploy by Twitter to gather a crowd reaction – so that when they announce payment plans for SMS alerts, they already have the crowds’ support. Yeah – okay, that may be a tad cynical but I do hope they listen to their UK users (and, in fact, anyone who used the UK based SMS alerts). The reaction is already gathering pace – with comments and tweets a plenty – and now a Facebook group. SMS is huge here and it seems to be a glaringly obvious way to actually turn Twitter into a revenue making business, in the UK at the very least.

iR2P is an initiative that aims to connect, support and motivate individuals to further the public understanding and political support for implementation of the United Nation’s “Responsibility To Protect” (R2P) doctrine, and to promote the importance of individual responsibility in this context.

The essential principle being that an individual can make a difference – no matter how small – to any issue. The issue that we draw your attention to is genocide and crimes against humanity. The United Nations and its member states undertook a ‘responsibility to protect’ the victims of these crimes yet Darfur continues, the Burmese government continue to block aid workers, and Mugabe remains in power and able to cause suffering at will. We fear that similar atrocities will continue into the future. We also fear that the international community will not live up to its obligations.

If you share our concern, but also our optimism that governments are accountable, that we can make a difference, that individual actions matter, then join our movement, sign our pledge, unite: www.ir2p.org/pledge

We will alert you to developments in the international arena, advise you on actions that you can take, and encourage you to get involved in a range of activities. We also want to hear from you with ideas, suggestions and critiques. We are building up our own capabilities at the moment but are excited by your interest and the possibility of what we can achieve together.

And with this promise of an enhanced search presence will come the marketing need for publishers to create content that capitalises on this. Just as SEO emerged as an industry all of its own, I expect Semantic Web Optimisation to emerge as an industry all of its own in the very near future.

[Before submitting this post, I quickly searched on the topic (yes - I appreciate the irony of having used Google!) and came across this article - essentially saying the same thing as me above. But please can we not fall into the trap of applying version numbers after "Web" for every evolution in web technology that occurs!]

Scary or exciting?

Vote Ken, originally uploaded by markrocky.

I thought there was something strange about these stencils I saw this morning.

Rumour has it they are part of a campaign against Banksy’s backing of Ken. Subtle. Most people will simply think Ken has gone all street and down with the kids.

[Edit: cycled past this morning and they've now been boarded over]

• Dynamic webserving
• Persistent storage
• Automatic scaling and load balancing
• APIs for authenticating users and sending email
• Fully featured local development environment

Python is the only language supported at the moment but they say more languages will be supported in the future. With the weight of Google behind it, this service looks set to provide an excellent all-in-one route to rapid web-app development. So long as you are happy using Python and all of Google’s Web services.

• Gmail CustomTime – never miss a deadline again
• TrustBanners – incredible mind-altering online advertising.
• Google buys Facebook – a bit close to the truth?
• Microsoft finally acquires Yahoo! – even closer?
• Virgin founder Richard Branson and Google co-founders Larry Page and Sergey Brin launch Project Virgle, the first permanent human colony on Mars

Yet another critical web app has a brief spell offline…

Again – while annoying – I do find a slight comfort in the fact that even the big boys have bad days.

37signals System Status, originally uploaded by markrocky.

One reason why you shouldn’t rely on all this fancy web 2.0 stuff for your business…

Although I do get a certain reassurance from seeing that even people like 37signals can have bad days (sorry 37signals)!

Lately there’s been a spate of negative blog entries and articles slating Facebook – and the furore surrounding Mr Scoble’s recent quarrel with Facebook pushed things even further. [Although some may argue that this simply served as an excellent PR pre-cursor to last week's announcement that Facebook (along with Google) were to join the Data Portability initiative.]

Is this the beginning of Facebook fatigue? In 2008, will Facebook go the way that Second Life went in 2007? Or is this simply the natural media/ public reaction to “hype”, as described in Gartner’s Hype Cycle, and we’re now in the “Trough of Disillusionment“? I have to admit that the tedious slew of unanswered invites in my inbox from zombies, pirates and vampires has sent me sliding down my own trough of Facebook disillusionment. Although, rather lazily, I do find it handy to have friends’ details in one place on the web – I use Facebook to arrange meeting after work or even to message someone, for example, as it is easier than digging around for their contact details. Will I still be doing this in 12 months? I certainly wasn’t a year ago, so who knows? Maybe Gartner does.

dew drop bye, originally uploaded by markrocky.

Dew drops on barbed wire, one fine Winter’s morning in rural Somerset.

Chelsea chimneys, originally uploaded by markrocky.

Looking west across London from Peter Jones in Chelsea, on a bright and Christmassy winter afternoon.

Please note that this is your final notification of the withdrawal of the Zimki service, which will occur on the 24/12/2007.

All user applications and data remaining on the Zimki service will be deleted and the servers decommissioned shortly after this date.

It certainly highlights the importance of building your fancy-pants web 2.0 application on solid and reputable services.

Tim Berners-Lee stated in a post yesterday that

…we have the technology — it is Semantic Web technology…

In other words, the connections and relationships made possible by the semantic web (or social graph – use interchangeably from hereon). He goes on:

Now, people are making another mental move. There is realization now, “It’s not the documents, it is the things they are about which are important”. Obvious, really.

and on opening up this data:

It is about getting excited about connections, rather than nervous.

So, yes – I am excited but how are we going to do this? Think data feeds, microformats and openID – things that would tend to be met with blank stares if I were to suggest them to clients. But show a client how you could remove barriers (such as log-in/ sign-up) to that all important “conversion” (with openID or microformats – here’s an excellent microformat implementation doing just that) and show how this person would be able to instantly tell all their friends about it (via their social graph) and then they’ll be interested.