Setting up traffic monitoring using GoAccess

Table of Contents

1 Nginx setup
2 GoAccess configuration
3 Systemd setup
4 Serving the static file
5 Serving live statistics
6 GeoIP database

GoAccess (goaccess.io, github) is a tool that analyses server logs and gives real-time statistics on network traffic. It took me some time to figure out exactly how to get the real-time websocket server working through Nginx, so I’m just sharing my configuration here.

Install via your package manager, e.g. sudo pacman -S goaccess on Arch.

The result will look like this.

Figure 1: (Click it to open in new tab.)

1 Nginx setup Link to heading

First we make sure that nginx logs all requests. I have a bunch of nginx server blocks, so I’m logging the curiouscoding.nl logs to their own file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# /etc/nginx/sites-available/curiouscoding.nl
server {
        server_name curiouscoding.nl;
        root /srv/nginx/...;
        listen [::]:80;
        listen [::]:443 ssl http2;
        include /etc/nginx/ssl_settings.conf;

        # Write logs here
        access_log "/var/log/nginx/access-curiouscoding.log";

        ...
}

Code Snippet 1: Nginx access log setup.

2 GoAccess configuration Link to heading

Now we’ll do some configuration to parse nginx logs. I’m not quite sure all of these are strictly needed, but this is what I currently have. See the full file on your system for docs for each option.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# /etc/goaccess/goaccess.conf
# nginx uses the combined format by default. Important!
log-format COMBINED
# The nginx logs can be found here.
log-file /var/log/nginx/access-curiouscoding.log

# default time & date formats
time-format %H:%M:%S
date-format %d/%b/%Y
datetime-format %d/%b/%Y:%H:%M:%S %z

# Some UI tweaks (most are defaults)
config-dialog false
hl-header true
json-pretty-print false
agent-list false
http-method yes
http-protocol no
no-query-string false
no-term-resolver false
444-as-404 false
4xx-to-unique-count false
all-static-files true
browsers-file /etc/goaccess/browsers.list
# Show the initial 'visitors per day' by hour.
date-spec hr
double-decode false
# Eanble some additional panels
enable-panel REFERRERS
enable-panel GEO_LOCATION
# Show the 'time distribution' graph per 10-minutes rather than hours.
hour-spec min
ignore-crawlers true
crawlers-only false
unknowns-as-crawlers false
ignore-statics panels
real-os true

Code Snippet 2: GoAccess configuration file.

You should now be able to run goaccess --output /tmp/index.html on your server. You can either cp it somewhere into the public/ directory of your site or scp it to your local machine to view it in a browser.

3 Systemd setup Link to heading

We’d like to have goaccess running in the background.

First, create the goaccess user:

1
sudo useradd goaccess --system --no-create-home

Then, create /srv/nginx/goaccess, owned by the goaccess user, which is where we’ll host the static index.html page.

1
2
sudo mkdir /srv/nginx/goaccess
sudo chown goaccess:goaccess /srv/nginx/goaccess

Now, we can crate a systemd service:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# /etc/systemd/system/goaccess.service
[Unit]
Description=GoAccess

[Service]
Type=simple
ExecStart=/usr/bin/goaccess --real-time-html --output /srv/nginx/goaccess/index.html
ExecStop=/bin/kill ${MAINPID}
PrivateTmp=false
RestartSec=100
User=goaccess
Group=goaccess
Restart=always

[Install]
WantedBy=multi-user.target

Code Snippet 3: Systemd service.

Now start and enable the service.

1
2
sudo systemctl daemon-reload
sudo systemctl enable --now goaccess

This should now write /srv/nginx/goaccess/index.html.

4 Serving the static file Link to heading

Now we want to view /srv/nginx/goaccess/index.html. Add the following server block:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# /etc/nginx/sites-available/goaccess
server {
        # (I'm actually hosting it somewhere else.)
        server_name goaccess.curiouscoding.nl;
        root /srv/nginx/goaccess/;
        listen [::]:80;
        listen [::]:443 ssl http2;
        include /etc/nginx/ssl_settings_wildcard.conf;

        # Add http authentication to it.
        auth_basic "Authentication required";
        auth_basic_user_file /etc/nginx/.htpasswd;
}

Now sudo systemctl restart nginx and go to goaccess.curiouscoding.nl to see the generated report.

5 Serving live statistics Link to heading

By default, we just see the generated index.html file, and we have to restart goaccess.service to regenerate it. But GoAccess also supports a websocket server that can show real-time statistics. This was slightly more tricky to get working, but ends up being very nice!

Heads-up: while the static page shows up to 366 table rows per panel, the live view only shows up to 50 to save data. See this issue for possible workarounds if you want to see the full data anyway. (To work around this, I created a second goaccess-static one-shot systemd service that drops the --realtime-html flag and the Restart and RestartSec lines, that writes to .../index-static.html.)

First, make sure that you add the --real-time-html flag to the systemd service, as I already did in Code Snippet 3.

Then, update the goaccess configuration with:

1
2
3
4
5
6
7
# /etc/goaccess/goaccess.conf
# The goaccess server listens on this port.
port 7890
# (Not actually sure we need this.)
pid-file /var/run/goaccess.pid
# The browser can find the websocket server here.
ws-url wss://goaccess.curiouscoding.nl:443/ws

Code Snippet 4: Updated goaccess configuration for websocket server.

Also update the nginx configuration for goaccess.curiouscoding.nl like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# /etc/nginx/sites-available/goaccess
server {
        server_name goaccess.curiouscoding.nl;
        ...

        # Redirect requests to /ws to local port 7890, where goaccess is listening.
        # Note: No trailing slash after /ws!
        location /ws {
                proxy_connect_timeout 7d;
                proxy_send_timeout 7d;
                proxy_read_timeout 7d;
                proxy_pass http://localhost:7890;
                proxy_set_header Connection "upgrade";
                proxy_set_header Connection "keep-alive";
                proxy_http_version 1.1;
                # Drop the leading /ws from the passed-through url.
                rewrite ^/ws(.*) /$1 break;
        }
}

Now sudo systemctl restart nginx, and go to goaccess.curiouscoding.nl. You should see a green dot in the top left indicating the websocket server is working.

6 GeoIP database Link to heading

If you want to see where your users are coming from, you’ll need a database for it. As linked in goaccess.conf, I went to https://db-ip.com/db/download/ip-to-city-lite and downloaded the ‘IP to city lite MMDB’. Copy that to /usr/local/share/GeoIP, and then add the following to the goaccess configuration:

1
2
3
# /etc/goaccess/goaccess.conf
# Make sure to update the date to your version.
geoip-database /usr/local/share/GeoIP/dbip-city-lite-2024-12.mmdb

Now sudo systemctl restart goaccess and refresh goaccess.curiouscoding.nl.