Missing Static Files from Apache Access Log

July 29th, 2024

This is a story about investigating an issue.

I was working on legacy PHP code inside a Docker container. I needed to log all files accessed via Apache, especially the static ones, such as css, js, gif, etc. On the container for a PHP 5 app, things were logged correctly, but on the container for a PHP 4 app, only PHP files were in the log.

I didn't create any of these Docker containers, so I wasn't sure where the issue was coming from. I looked at the vhost line responsible for the log, and it looked fine:

CustomLog ${APACHE_LOG_DIR}/access.log combined

I hadn't worked with Apache configuration in ages, so I looked up what combined meant and whether CustomLog takes additional arguments that might help me. Turns out that combined is only a format, so it wouldn't exclude any files from being logged. CustomLog takes an optional third argument, which does filter logs, but since none was provided, I could only assume that all files should be logged. Just in case, I combed through the Apache documentation as well as several articles on the subject. All agreed that the absence of the third argument meant that all files were to be logged. Again, just in case and because my container used the older Apache 2.2, I experimented by explicitly including only the file extension I cared about, as per the docs:

SetEnvIf Request_URI \.gif$ gif-image
CustomLog ${APACHE_LOG_DIR}/access.log combined env=gif-image

After reloading Apache and clearing my log, no new entries were appended. I allowed only gif images, and those were already excluded by some other mechanism. I had spent nearly an hour researching at this point.

I went back to the docs and my vhost config, and one line caught my attention:

LogLevel warn

I wondered whether I needed to lower this setting to info or something. I checked the docs, and LogLevel wasn't even on the page for mod_log_config. I looked for it on the entire site, and found this quote: "LogLevel adjusts the verbosity of the messages recorded in the error logs." That part of the docs mentioned ErrorLog and referred specifically to errors. Not a single hint as to its impact on CustomLog, so I went to look elsewhere.

I combed the entire /etc/apache2 directory for a hint. I even read all the code for the base Docker image, in case I'd find some overrides that would give me a hint, since all articles seem to imply that CustomLog would log everything by default unless you change something. And so I spent another hour or so combing through forums, where everyone wanted to accomplish the exact reverse of what I wanted. Nobody else has a Docker container that overrides some config to make this an issue.

Eventually, I decided to try something, and changed LogLevel to info. Ta-da! All my static files started appearing in the log. I was glad, but also frustrated, because it took me so long. I went back to the docs and indeed, there was no way to suspect a relationship between LogLevel and omissions in the access log. Why is the PHP file being logged at LogLevel warn, despite a status code of 200, but not a GIF image? This was neither documented nor made sense.

My takeaways from this experience:

1. Do not assume that documentation is complete or accurate.
2. Software isn't always logical, and likely has many legacy decisions that cannot be easily rectified.
3. If I see something that might work and doesn't cost much to try, I should just try it.

Previous: Legacy Tales: Arrays Initialized Using Strings Next: Zero Arguments for Gatekeeping