Using AWStats To Report On Domino Web Logs

Lotus Notes/Domino doesn't have very good built-in web log reporting functionality (and by that I'm referring to logs of website traffic, not blogs). As a web server, Domino can send traffic information to text files or to a DomLog database, but there's no real reporting built-in, so you're stuck with reams of information and no good way to look at it.

Many people make modifications to the DomLog.nsf database design, which can be useful, but logging stats to a database can cause performance problems on busy servers, and a large DomLog database is pretty slow and unwieldy in the middle of the day.

Fortunately, if you send your web log stats to a text file, it gets written in a standard logging format which can be parsed and sliced and diced by a number of popular reporting applications. One of the nicer log reporting tools available is the free and open-source AWStats application, which not only has really nice default reporting options, but is also very customizable and has a number of plug-ins. It can also be used on Windows, Unix, or Linux servers, because it's just a series or Perl scripts (very complex ones, but that's what they are). As an example for how nice the reports look, you can look at the stats for the AWStats site.

Here are some pretty basic step-by-step directions on how to set up your Lotus Domino server to use the AWStats tool (note that when I make reference to "%DOMINO_DATA_DIR%" below, I mean the Domino data directory on your server -- usually something like "d:\domino\data"). I encourage you to also refer to both the Domino Administrator Help and the AWStats documentation for more information and explanation.

Instructions for setting up AWStats on Domino

1. Set up your Domino server to log to text files. Open the server document in your Public Name and Address Book/Domino Directory and go to the "Internet Protocols - HTTP" tab, and set the following fields:

Save and close the server document, and restart HTTP on the server.

2. Download AWStats from

3. Extract to a directory of your choice.

4. Create a directory called "awstats" under the %DOMINO_DATA_DIR%\domino\html directory on your server (for example, "d:\domino\data\domino\html\awstats").

5. Copy the "wwwroot\icon" directory from the awstats files to %DOMINO_DATA_DIR%\domino\html\awstats (for example: "d:\domino\data\domino\html\awstats\icon")

6. Copy the contents of the "wwwroot\cgi-bin" directory from the awstats files (and all subdirectories beneath it) to your %DOMINO_DATA_DIR%\domino\cgi-bin directory on the Notes server (for example, "d:\domino\data\domino\cgi-bin"). Also copy the contents of the "tools" directory from awstats to %DOMINO_DATA_DIR%\domino\cgi-bin -- in other words, take all of the .pl files from awstats and put them in your "cgi-bin" directory.

7. From the Domino "cgi-bin" directory, make a copy of the awstats.model.conf file. Call it "awstats.domino.conf". If the file is marked as "Read-Only", you'll want to go to the file properties and uncheck that flag so you can edit it. Open the file in a text editor and make the following changes (please note that Windows users will not be able to read the file very well using Notepad because it uses Unix-style linefeeds, so you'll want to open it in Wordpad or Crimson Editor or something like that):

8. If you don't have Perl installed on your server, download and install it. For Windows, you can use the free ActivePerl distribution at The install for ActivePerl doesn't require a reboot, so you can actually install it on a live Domino server if you're feeling brave.

9. Run your initial update by opening a command prompt, navigating to the "cgi-bin" directory from step 6, and entering the following command:

perl -config=domino -update

Where "domino" in the "-config=domino" parameter refers to the middle name in the "awstats.domino.conf" filename from step 7. This will parse your log files (as defined in the "LogFile" parameter in your .conf file, from step 7) and create an index file that's used to build the reports. The first time you run the update process against a large set of data, it may take a few minutes.

If you get errors when you try to run this command, one of the following things probably happened:

10. Once you've run the update command and you have an index file, you can manually create a report as an HTML page by running one of the following commands from the command prompt in the "cgi-bin" directory:

perl -config=domino -output -staticlinks > ../html/awstats/awstats.html

perl -config=domino -output -staticlinks -month=01 
-year=2004 > ../html/awstats/awstats200401.html

perl -config=domino -dir=../html/awstats/

perl -config=domino -dir=../html/awstats/ -buildpdf

Note that if you're running and viewing these files locally (not on your Domino web server), and you don't see any graphs or charts, you may need to adjust the "DirIcons" setting from step 7 and re-run the report.

11. To schedule your static reports to be built on a regular basis, use your favorite scheduling program and set or to run with whatever options you want from step 10, and include the "-update" switch.

12. Normally, you will want the "-update" process from step 9 to run every night, after midnight. The update process is the processor-intensive activity in this case -- it actually parses all the log files and writes the compiled information to an index file. Creating a report based on the index file is actually pretty quick.

While you could schedule this process through a Server Program document in Notes, you should probably use a scheduling program that runs separately from Notes (like "AT" on Windows, or "cron" on *nix), so you don't run into problems with using the Notes server memory space to run the updates.

13. Because it's pretty quick to run a report, you can also pull up the reports directly from a browser by navigating to this URL:

where "" is, obviously, the DNS name for your Domino server. This is the way that most web servers that use AWStats for reporting seem to work (including the AWStats site itself). One nice thing about this is that you can quickly and easily view prior month reports by adding parameters to the query string. For example:

will bring up the report for January 2004. In fact, the current version of AWStats allows you to choose a month and year from a drop-down menu when you run the report dynamically like that. Because you're pulling the data from an index file that's already been built, the report generation is fairly fast.

14. OPTIONAL: create a File Protection document to restrict access to the AWStats files on the server (see the Domino Administrator Help topic "Protecting files on a server from Web client access" for more information). You should definitely protect the "cgi-bin" directory, and if you're outputting any static reports to a directory, you can protect those as well.


Additional Reports For Domino

Here are some example additions to the AWStats .conf file for Notes-specific file tracking (you may have to delete the index and re-run -update for them to take effect):

# report on the .nsf files that have been accessed
ExtraSectionName1="List of most active databases (top 50)"
ExtraSectionCodeFilter1="200 304"
ExtraSectionFirstColumnTitle1="Database Filename"

# report on the .gif, .jpg, and .jpeg files that have been accessed
ExtraSectionName2="List of most active images (top 20)"
ExtraSectionCodeFilter2="200 304"
ExtraSectionFirstColumnTitle2="Image Filename"

# report on the top file downloads, assuming the URL had /$FILE/ in it
ExtraSectionName3="List of most active $FILES (top 20)"
ExtraSectionCodeFilter3="200 304"
ExtraSectionFirstColumnTitle3="Image Filename"
# this returns the whole line
# this returns everything up to and including $FILE
# this returns everything before /$FILE
# this returns everything after /$FILE