Sunday, February 26, 2006 Crash report

Last night happened what I've always been afraid of. It started when I tried to update PostgreSQL to the latest version, something that I'm not very familiar with (so to be honest - it was completely my own fault). I also have to admit that I'm not a Linux guru - I like Linux and I can work with it quite comfortably, but when it comes to problems that I didn't expect, I sometimes end up trying and messing things up instead of really keeping everything under control.

First I tried to update the server with the .tar.gz file and followed the instructions from the PostgreSQL documentation. The installation process seemed to work fine, but when I thought, the installation was done, I didn't manage to start the server with the new version.

Next I downloaded the .rpm file from the SuSE 10.1 development repository. That's where the real problems started to arise - there were dependencies over and over and I tried to solve them one by one until I found out, that more things were broken than repaired. Finally I gave up upgrading PostgreSQL - MySQL still worked and was completely unaffected by my actions. But the disaster was finished when I realized that I couldn't connect via SSH anymore.

So I had no other choice than to start the re-installation from the customer interface of my server provider. It took several hours until the installation was finished with a completely fresh server (unfortunately, still a SuSE 9.0). Nevertheless, the rest was easy - MySQL was quickly set up again (the 5.0 server through an .rpm binary, the 5.1 server through the .tar.gz file). I was clever enough to backup the user's data shortly before the server crashed, so I could easily recover both MySQL servers with hardly any data loss (how great would it be if I had a second server with a replication system ;-)). So in the end, not much damage was done, except a few hours of downtime and for the moment there's no more PostgreSQL server.

My server provider informed me that very soon (probably next week) they will provide a new SuSE 10.0 image. So probably, I'll soon make an update, but in contrast to the situation last night, I'll be prepared well and everything should run quite quickly and smoothly. And maybe, I'll also get the PostgreSQL server up running again with at least an 8.0 version.


Frank said...

Seems like we both had kinda similar weekend working with crashes (of different kinds).


Markus Popp said...

That sounds really bad - our both's scenarios show pretty well that backups can be worth gold. Even better would be replication systems, but for little projects, the costs for a second server are most often too much.

Just good that we both ended up doing fine ;-).

Frank said...

Yeah man. From time to time we are reminded of the importance of backups.

It's amazing cause people like us are typically known to be paranoid and preaching backups.

Glad it didn't take more than a weekend.