I had to calculate sum of a particular column from a huge data file. So I wrote a small bc script like this
----sum.bc-----
sum = 0
while(1)
{
c = read()
if ( c == q )
{
break
}
sum += c
}
sum
quit
----sum.bc-----
and invoked it like this
cat data.dat | awk -F '"' '{print $24}END{print "q"}' | bc -q -l sum.bc
This worked perfectly on my home system which is a linux. But when I did the same in Solaris which had a older bc that dude didn't understand my modern lingo. So I had to abandon the bc script.
So I thought I will just produce the entire sum string with numbers and "+" signs and pump it to bc, like this.
cat data.dat | awk -F '"' '{print $24}' | awk '{printf "%s+",$1}END{print "0"}' | bc -l
Again this worked with my small sample file but bc ran out of internal space when I fed a huge data file. Finally I had to abandon the specialized tool for calculation and fall back on my usual file processor awk to take over the job of the calculator.
cat data.dat | awk -F '"' '{print $24}' | awk '{s += $1}END{print s}'
Well..well ...I fed more than a Gig of data to this small wonder in that old dinky Solaris machine and it worked like a charm.
Thursday, July 20, 2006
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment