Upgrading from Fedora 5 to Fedora 6 over the net

So far whenever I try to update my linux systems I always do a fresh install. I have my home and data partitions seperate from the root partition. So except for few configuration files which I backup I don't have anything to preserve and a fresh install is the least troublesome way to upgrade.

But I'm deeply bothered about writing few CD's and throw them away once the installation is complete. Hence this time I tried to the upgrade without reintall and without any CD's.

First I made the system clean and up to date.

yum update
yum clean all

Then I forced the update of fedora release packages.

rpm -Uhv http://download.fedora.redhat.com/pub/fedora/linux/core/6/i386/os/Fedora/RPMS/fedora-release-6-4.noarch.rpm
http://download.fedora.redhat.com/pub/fedora/linux/core/6/i386/os/Fedora/RPMS/fedora-release-notes-6-3.noarch.rpm

Since I modified the repo files the rpm installation deposited the repo files as .rpmnew. So I backed up the old files and moved the rpmnew files to the appropriate names. I had livna repo also. So next I moved that repo from release 5 to 6.

yum remove livna-release-5
rpm -ivh http://rpm.livna.org/livna-release-6.rpm

At this point I crossed my finger and ran yum update.

yum update

It found tons of packages to upgrade and a few to uninstall and few more to fresh install. But it also found few dependency issues mostly with gtls package depency tree. So I had to uninstall tons of packages including kde and gnome ( yeah I have both on my machine ;) ). I created a text file of those uninstalled packages with entries like this(cut and paste from yum output).

file /tmp/uninstalled.yum.lst :

GraphicsMagick x86_64 1.1.7-7.fc5 installed 8.2 M
ImageMagick x86_64 6.2.5.4-4.2.1.fc5.7 installed 11 M
ImageMagick-c++ x86_64 6.2.5.4-4.2.1.fc5.7 installed 470 k
NetworkManager-gnome x86_64 0.6.4-1.fc5 installed 403 k
control-center x86_64 1:2.14.2-1 installed 7.7 M
cups x86_64 1:1.2.8-1.fc5 installed 8.7 M
cups-libs x86_64 1:1.2.8-1.fc5 installed 311 k
dvdauthor x86_64 0.6.11-4.lvn5 installed 309 k
eel2 x86_64 2.14.3-1.fc5 installed 1.2 M
eel2-devel x86_64 2.14.3-1.fc5 installed 225 k
ekiga x86_64 2.0.1-5 installed 11 M
eog x86_64 2.14.3-1.fc5 installed 2.2 M
evince x86_64 0.5.1-4 installed 2.2 M
evolution-data-server x86_64 1.6.3-2.fc5 installed 11 M
........

Now "yum update" went fine with out any complains. Unfortunately for some strange reason it also installed i386 packages. Mine is AMD64 and I prefer only x86_64 packages. Again I created a list like the one above and gave it to yum to uninstall.

yum list | grep i386 | grep installed | awk '{print $1}' | xargs > /tmp/i386.lst
yum erase `/tmp/i386.lst`

Now I'm back to the list of packages I unistalled. Again I piped the list to yum this time I asked it to do the installation instead.

cat /tmp/uninstalled.yum.lst | awk '{print $1}' > /tmp/install.txt
yum `cat /tmp/install.txt`

Now I should have the functional system. Wait the infamous nvidia driver. I installed that too...and then rebooted the machine. Everything came up without any problem. I had to fix few inode issues so rebooted again this time logged in a single user mode ( 1 or softlevel=single in grub kernel config line) and ran fsck -y.

So there you go I made a diskless over the internet upgrade of fedora 5 to fedora 6 and I'm still alive to tell the story. My system is working fine so far with out any problems.

Power Saving - turnoff hard disks after an idle time

I have 1 IDE and 4 SATA disks on my server. Each SATA drive mostly serve one function. One drive for photos, another one for music etc. Though this is a file server, the disks are not used when we are away from home which is a good portion of the day. So I searched ways to reduce power and general hard disk usage. "hdparm" came to rescue in the form of -S option. Here is what man page has to say about -S option.

-S
Set the standby (spindown) timeout for the drive. This value is used by the drive to determine how long to wait (with no disk activity) before turning off the spindle motor to save power. Under such circumstances, the drive may take as long as 30 seconds to respond to a subsequent disk access, though most drives are much quicker. The encoding of the timeout value is somewhat peculiar. A value of zero means "timeouts are disabled": the device will not automatically enter standby mode. Values from 1 to 240 specify multiples of 5 seconds, yielding timeouts from 5 seconds to 20 minutes. Values from 241 to 251 specify from 1 to 11 units of 30 minutes, yielding timeouts from 30 minutes to 5.5 hours. A value of 252 signifies a timeout of 21 minutes. A value of 253 sets a vendor-defined timeout period between 8 and 12 hours, and the value 254 is reserved. 255 is interpreted as 21 minutes plus 15 seconds. Note that some older drives may have very different interpretations of these values.

So I executed the following commands.

hdparm -S60 /dev/hda
hdparm -S60 /dev/sda
hdparm -S60 /dev/sdb
hdparm -S60 /dev/sdc
hdparm -S60 /dev/sdd

And had to make few configuration changes to smartd ( to make it perform the checks less frequently )

/dev/sda -d ata -H -m root -s (S/../.././02|L/../../7/04)
/dev/sdb -d ata -H -m root -s (S/../.././02|L/../../7/04)
/dev/sdc -d ata -H -m root -s (S/../.././02|L/../../7/04)
/dev/sdd -d ata -H -m root -s (S/../.././02|L/../../7/04)

Yes there is a small delay when first navigate to any of the disks. But I'm willing to pay the penalty for the power saving and far quieter machine.

boost::tokenizer and streams

I had to parse a huge file and build some lookup tables based on that. Each line had a comma seperated fields with white spaces. Initially some "performance" minded person wrote aperl script which writes out c++ function which statically populate the lookup tables. The compiler took literally an hour to compile that file (whose size was ~ 1MB). So I set out to write a small parser and populate the lookup fields. So I read each line and used boost tokenizer to split the comma seperated value. It looked something like this...

while( f.getline(str) )
{
typedef boost::tokenizer<boost::char_separator<char>>
tokenizer;
boost::char_separator<char> sep("\t, ", "\n");
tokenizer tokens(str, sep);
for (tokenizer::iterator tok_iter = tokens.begin();
tok_iter != tokens.end(); ++tok_iter)
std::cout << "<" << *tok_iter << "> ";
}

The initial version worked. But seeing the constructor for boost::tokenizer I got curious and thought what if I passed the the stream iterator to tokenizer? That would make my code much prettier and it is obviously a better way of doing it. So I did this.

{
std::ifstream ifile(filename.c_str());
std::istream_iterator<char> file_iter(ifile);
std::istream_iterator<char> end_of_stream;

typedef boost::tokenizer<boost::char_separator<char>,
std::istream_iterator<char> >
tokenizer;
boost::char_separator<char> sep("\t, ", "\n");

tokenizer tokens(file_iter,end_of_stream, sep);

for (tokenizer::iterator tok_iter = tokens.begin();
tok_iter != tokens.end(); ++tok_iter)
std::cout << "<" << *tok_iter << "> ";
}

Soon I hit a snag. For some reason I'm not seeing the "newline" characters which are supposed to be printed since I specifically instruct the tokenizer to keep the "newline" delimeters. That could mean only one thing...The stream iterator is eating up the "\n"s. Ofcourse it is..duh..! Forgot the locales? So I changed to istreambuf_iterator which won't do any parental controls over the stream and show me everything...

{
std::ifstream ifile(filename.c_str());
std::istreambuf_iterator<char> file_iter(ifile);
std::istreambuf_iterator<char> end_of_stream;

typedef boost::tokenizer<boost::char_separator<char>,
std::istreambuf_iterator<char> >
tokenizer;

boost::char_separator<char> sep("\t, ", "\n");

tokenizer tokens(file_iter,end_of_stream, sep);

for (tokenizer::iterator tok_iter = tokens.begin();
tok_iter != tokens.end(); ++tok_iter)
std::cout << "<" << *tok_iter << "> ";
}

I got what I wanted. A token parser which could parse a stream. Now the stream could be any stream and it will work. And yeah...it takes only few seconds to compile this program and building the lookup table is actually faster than the statically populated version ( because of all the temporary storage the compiler has to allocate and deallocate in the static version ) .