Access Logs

What is the location of access logs.

All access-logs belonging to a specific application are found in the domain folder and divided per domain. ~/domains/<domain>/var/log/<domain_or_pointer>.nginx.access.log.

The logs are rotated everyday or renewed when they reach the maximum size limit of 100MB.

Old logs are compressed and renamed to <domain_or_pointer>.nginx.access.log.X.gz

Tail / live log output

To watch the log files live, use tail. By using wildcards you will be able to view your log files live in one command:

tail -qf ~/domains/<domain>/var/log/*.nginx.access.log

This will generate a large bulk of data, to view specific data filtering is recommended: To filter use grep.

For example using a regex to filter media en static files:

tail -qf ~/domains/<domain>/var/log/*.nginx.access.log | grep -vP '(static|media)'

Or just URLs of trousers:

tail -qf ~/domains/<domain>/var/log/*.nginx.access.log | grep trousers

Search / specific times

Often you want to know what happened in a certain period. In this case instead of using tail you could print your files to screen and use grep to search the results

For example: all hits on 17 Jan 2020 at 13:50 without media and static:

cat  *.nginx.access.log | grep '17/Jan/2020:13:50' | grep -vP '(static|media)'

Or on 17 Jan 2020 between 13:50 and 13:55:

cat  *.nginx.access.log | grep -P '17/Jan/2020:13:5[1-5]'

Sometimes you need to view you logs created yesterday or even the day before yesterday. These logs are compressed which makes this slightly more difficult. Just combine the preivous commands to do this with gzip.

For example all requests on the rest endpoints of the past days.

cat  *.nginx.access.log.*.gz | gunzip | grep 'rest'

Log analysis

Often it is useful to know where requests came from or how often certain requests are made in a certain period of time etc. To fulfill these requests use goaccess. After analyzing it will show the following statistics:

• Unique visitors per day
• Requested URLs
• Required static content
• 404 pages
• IP addresses of visitors
• System information such as OS and Browser
• Times on the day of visit
• Response status codes
• Referring sites

An analysis of 17 Jan 2020 between 13:00 and 12:00 without media or static.

cat *.nginx.access.log | grep -P '17/Jan/2020:1[23]:' | grep -vP '(static|media)' | goaccess --log-format=COMBINED

Depending on the quantity of your requested data it may take goaccess some time to print the results to your screen.