User Tools

Site Tools

Trace:
apache_sitedown

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
apache_sitedown [2015/08/24 16:08] – luke7858apache_sitedown [2024/05/23 07:26] (current) – external edit 127.0.0.1
Line 1: Line 1:
 +Note: I have written a python script to check a system and report back on any potential Out-Of-Memory issues:
 +<sxh bash>
 +curl -s https://raw.githubusercontent.com/LukeShirnia/out-of-memory-investigation.py/master/oom.py | python
 +</sxh>
 +\\
 +\\
 == Step 1 ==  == Step 1 == 
 did apache hit max clients? did apache hit max clients?
Line 12: Line 18:
 == Step 2 == == Step 2 ==
 Did the server run out of memory and kill processes that killed a vital service? Did the server run out of memory and kill processes that killed a vital service?
 +CentOS/RHL
 <sxh bash> <sxh bash>
 +egrep -i 'killed process|invoked oom' /var/log/messages
 </sxh> </sxh>
 +Ubuntu/Debian 
 +<sxh bash>
 +egrep -i 'killed process|invoked oom' /var/log/syslog
 +</sxh>
 +\\
 +Summarise date and times the server ran out of memory (CentOS/RHEL):
 +<sxh bash>
 +zgrep -i 'killed process\|invoked oom' /var/log/messages* | awk '/Killed process/ {print $1, $2, $3}' | awk -F: '{print $2}' | uniq -c | sort -k2,3r
 +</sxh>
 +<sxh bash>
 +zgrep -i 'killed process\|invoked oom' /var/log/messages* | awk '/Killed process/ {print $1, $2, $11}' | awk -F: '{print $2}' | uniq -c | sort -k2,3
 +</sxh>
 +Example output:
 +<sxh bash>
 +      2 Apr 25 (driveclient)
 +      1 Apr 27 (driveclient)
 +      2 Apr 29 (driveclient)
 +      1 May 12 (driveclient)
 +      1 May 2 (driveclient)
 +      1 May 22 (driveclient)
 +      1 May 5 (driveclient)
 +      1 May 9 (driveclient)
 +</sxh>
 +\\
 +Debian/Ubuntu
 +<sxh bash>
 +zgrep -i 'killed process\|invoked oom' /var/log/syslog* | awk '/Killed process/ {print $1, $2, $3}' | awk -F: '{print $2}' | uniq -c
 +</sxh>
 +\\
 \\ \\
 ==Step 3 == ==Step 3 ==
 Check to see if the website received traffic during the period of 'down time' Check to see if the website received traffic during the period of 'down time'
 Investigating a 'site down' issue. Investigating a 'site down' issue.
 +Best command to see check all log is:
 +
 +<sxh bash>
 +cat /var/log/httpd/*access*log* | grep "27/Aug/2015:04:[3..5][0..9]" | awk {' print $1 '} | sort | uniq -c | sort -rn
 +</sxh>
 +
 +<sxh bash>
 +cat /var/log/httpd/*access*log* | grep "22/Aug/2015:04:[3-5]" | awk {' print $1 '} | sort | uniq -c | sort -n
 +</sxh>
 +
 +Checking old log files (compresses) \\
 +<sxh bash>
 +zcat /var/log/httpd/*access*log*.gz | grep "22/Aug/2015:04:[3..5][0..9]" | awk {' print $1 '} | sort | uniq -c | sort -n
 +</sxh>
 +\\
 <sxh bash> <sxh bash>
 grep 24/Apr/2015:02:[3-6][0-9]:* /var/log/httpd/access_log | awk '{print "IPaddress", $1,"Time",$4, $9}' grep 24/Apr/2015:02:[3-6][0-9]:* /var/log/httpd/access_log | awk '{print "IPaddress", $1,"Time",$4, $9}'
 </sxh> </sxh>
 +
 This will grep for: This will grep for:
 date 24/April/2015 02:30 - 02:60 date 24/April/2015 02:30 - 02:60
apache_sitedown.1440432494.txt.gz · Last modified: 2024/05/23 07:26 (external edit)
Except where otherwise noted, content on this wiki is licensed under the following license: Public Domain
Public Domain Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki