How find the top 10 IP addresses accessing your IIS web server with a Bash script and WSL

The number of insights that are offered by any modern analytics platform is really invaluable and easy to use, but we tend to forget that even our server logs are an incredible source of information that with little or no effort we can query with a really simple script.

The idea of this article was from a request of analysing the IIS logs in real-time, but without changing the web application code or adding any javascript to google analytics (or similar platform), so my choice was using a bash script to analyse the logs.

Installing Splunk, SpectX, or other tools for this case would probably obtain a more elegant solution and with a lot of options, but for the specific request was not implemented but just mentioned.

Goal

I wanted to generate a top 10 list of IPs for each IIS log file.

Internet Information Services (IIS) Logs

It’s important to start from the way that IIS implements the W3C standard, more in detail from the official doc pages.

This is an IIS log file

Looking at the first five lines of  the sample log file we will notice a few things:

  • This log is generated by Internet Information Services (version number)
  • Date time is in UTC format so for me in living down under in Australia the Time Zone is AEST (Australia Est Standard Time, UTC +10) will be midnight.
  • The 4th line is describing the fields
  • The 5th line is my first entry.

This is a table with number-field :

# Field
1 date
2 time
3 s-ip
4 cs-method
5 cs-uri-stem
6 cs-uri-query
7 s-port
8 cs-username
9 c-ip
10 cs(User-Agent)
11 cs(Referer)
12 sc-status
13 sc-substatus
14 sc-win32-status
15 time-taken

Choosing a BASH script

I think that my choice was obvious because the combination of cut | sort | uniq | head is the answer to this problem. But the nice part of this is that Windows Subsystem For Linux (WSL) is bringing bash to Windows and can be called in-line.

At line 12 the combination of cut, sort, uniq, head and how they can work together is maybe not obvious to everybody, but if you read the man page of these commands and experiment by inspecting the output adding one command after the other using the pipe symbol.

Line 16 is simply calling the find_top_ip function for each log file in the current directory.

I’ve copied this script on the IIS log folder, opened a WSL terminal on the log folder, then I’ve run the script and review the result.

Other ideas

  • c-ip is field #9, but this script could be edited quickly to get the top 10 client username or browser fingerprints by looking at the right field.
  • changing the ‘-f 9’ at line 12 to ‘-f 10’ will generate the top 10 browser agents for instance

Log Parser

It worth mentioning that there is also an interesting tool called Log Parser that you can use to run more advanced queries against your iis logs.

  • Link: https://www.iis.net/downloads/community/2010/04/log-parser-22
  • Doc:  https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-xp/bb878032(v=technet.10)

Wrap-Up

I’ve chosen to create a script once again to get the job done, even if there are a lot of other alternatives or better solutions available. I would recommend alternative solutions for most advanced scenarios, but for this request,  I think the simplicity of the script wins hands-down. As usual, this script is available on my GitHub repository.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.