Monday, May 26, 2008

More memory does not necessarily mean more speed

If the processor waits for i/o most of the time then adding more Primary Memory (RAM) usually helps. But before that decision is made, there are a few important things to look at:

1) What is the total data size that can be accessed?
2) How much data is accessed multiple times (at least more than once)? And how much Memory does this data require?

If the total data size is only a couple of gigs then you can probably think of having equivalent amount of Memory. But if the data size is tens or hundreds of gigabytes then adding that much Memory will certainly be very expensive.

Lets consider a case where the a database instance stores 200 GB of data. Out of this, only 4 GB is accessed multiple times in day and the rest might be accessed only once. In such a scenario, anything above 6 GB (the extra memory for kernel and other processes, if any) should be of no help because every time that extra data (other than the 4 GB) is accessed, the CPU has to wait for the disk i/o to complete.

Saturday, May 24, 2008

Parsing apache logs to identify IP with max requests

To find out the IP Addresses which generated the maximum number of requests, following Linux command can be used:

gawk {'print $1'} access_log | sort -n | uniq -d -c | sort -n

Note: This assumes that you're logging the IP Address in the first column