Apr 21, 2018

How to Detect User Idle Time or Inactivity in Acess logs

How to Detect User Idle Time or Inactivity in Acess logs
Requirement:

  • Read access log
  • Find the top most idle time(s) between the requests
Script


import itertools
import datetime
import logging

fo = open("access_log_time", "r+")
print "Name of the file: ", fo.name

lst = fo.readlines()
print len(lst)

def diff_date(x, y):
diff=0
try:
x = x.strip()
y = y.strip()
d1 = datetime.datetime.strptime(x, '%d/%b/%Y:%H:%M:%S')
d2 = datetime.datetime.strptime(y, '%d/%b/%Y:%H:%M:%S')
diff = (d2 - d1).total_seconds()
print '-------'
print diff
print x
print y
except Exception, e:
logging.error(e)
return int(diff)

#zip Vs izip
#zip computes all the list at once, izip computes the elements only when requested.
#One important difference is that 'zip' returns an actual list, 'izip' returns an 'izip #object', which is not a list and does not support list-specific features

res= [diff_date(x,y) for x, y in itertools.izip (lst, lst[1:])]
print sorted(res, reverse=True)
#print res




View number of requests by time from Apache access log

View number of requests by time from Apache access log
  • Overall requests in an hour
    • grep "18/Apr/2018:11" /var/log/httpd/access_log | wc -l
  • Overall requests in a minute
    • grep "18/Apr/2018:11:05" /var/log/httpd/access_log | wc -l
  • Overall requests in a minute
    • grep "18/Apr/2018:11:05:10" /var/log/httpd/access_log | wc -l
  • Overall requests by sec in an hour (group by sec)
    • grep "18/Apr/2018:11" /var/log/httpd/access_log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c | awk '{ if ($1 > 10) print $0}'

Find Sum of RSS memory in ps command in linux/unix

Find Sum of RSS memory in ps command in linux/unix
  • In KB
    • ps aux | awk 'BEGIN {sum=0} {sum +=$6} END {print sum}'
  • In MB
    • ps aux | awk 'BEGIN {sum=0} {sum +=$6} END {print sum/1024}'
  • In GB
    • ps aux | awk 'BEGIN {sum=0} {sum +=$6} END {print sum/1024/1024}'

How to find free memory available
  • cat /proc/meminfo
  • using free command
    • In KB
      • free
    • In MB
      • free -m
    • In GB
      • free -g 
I observed
  • Sum of RSS memory in ps less than memory actually used
  • Total used memory a lot higher than sum of RSS
  • Reason
    • The Linux kernel will use available memory for disk caching