Introduction
One of the first things that I would usually do in case I notice high CPU usage on some of my Linux servers would be to check the process list with either top or htop and in case I notice a lot of Apache or Nginx processes I would quickly check my access logs to determine what has caused or is causing the CPU spike on my server or to figure out if anything malicious is going on.
Sometimes reading the logs could be quite intimidating as the log might be huge and going through it manually could take a lot of time. Also, the raw log format could be confusing for people with less experience.
Here's a cool short bash script that would summarize the whole access log for you without the need of installing any additional software.
About the script
This BASH script will quickly summarize your access logs and provide you with very useful information like:
* The 20 top pages with the most POST requests
* The 20 top pages with the most GET requests
* Top 20 IP addresses and their geo-location
You can run it on any Linux server without root or sudo access. All you need is the access log itself and a terminal.
The link to the repository of the script:
https://github.com/apotitech/quick_access_logs_summary-master
If you notice any issues with the script, make sure to report an issue or submit a pull request!
Downloading the script
To download the script, you can either clone the repository with the following command:
git clone https://github.com/bobbyiliev/quick_access_logs_summary.git
Or run the following command which would download the script in your current directory:
wget https://raw.githubusercontent.com/bobbyiliev/quick_access_logs_summary/master/spike_check
The script does not make any changes to your system, it only reads the content of your access log and summarizes it for you, however, once you've downloaded the file, make sure to review the content yourself.
Running the script
All that you have to do once the script has been downloaded is to make it executable and run it.
To do that run the following command to make the script executable:
chmod +x spike_check
Then run the script:
./spike_check /path/to/your/access_log
Make sure to change the path to the file with the actual path to your access log. For example, if you are using Apache on an Ubuntu server, the exact command would look like this:
./spike_check /var/log/apache2/access.log
If you are using Nginx the exact command would be almost the same, but with the path to the Nginx access log:
./spike_check /var/log/nginx/access.logTop 20 IP logs and their geo location
Understanding the output
Once you run the script, it might take a while depending on the size of the log.
The output that you would see should look something like this:
Essentially what we can tell in this case is that we've received 16 POST requests to our xmlrpc.php file which is often used by attackers to try and exploit WordPress websites by using various username and password combinations.
In this specific case, this was not a huge brute-force attack, but it gives us an early indication that we can take action to prevent a larger attack in the future.
We can also see that there were a couple of Russian IP addresses accessing our site, so in case you do not expect any traffic from Russia, you might want to block those IP addresses as well.
Conclusion
This simple BASH script allows you to quickly summarize your access logs and determine if anything malicious is going on. Of course, you might want to also manually go through the logs as well.
If you notice any issues please give me a poke so I could try and patch them!
For all of my demos, I use DigitalOcean, you can use my referral link to get a free $100 credit that you could use to deploy your virtual machines and test the guide yourself on a few DigitalOcean servers:
Hope that this helps!