Problematic Asterisk Logs

Reporting an Issue VitalPBX 4
1

Hello,

The Issue:

We ran into an issue where we had to review Asterisk logs from two days ago. But the Asterisk logs were gone. The only Asterisk log, was the full from the current day.

After reviewing the contabs, we see the following:

Problematic Script 1

Under /etc/cron.daily/vpbx_clean_old_logs we see that there is the following:

path="/var/log/vitalpbx/"
find $path -name "log_*.log" -type f -mtime +7 | xargs rm >/dev/null 2>&1
/usr/share/vitalpbx/scripts/clean_logs >/dev/null 2>&

First, as far as I am aware there are no files named log_, there are _log, for example, queue_log.log. So the first line seems like it is useless.

Then, it calls the clean_logs script. That script, among other files that it deletes, it does the below:

rm -f /var/log/asterisk/full.*
rm -f /var/log/asterisk/full-*

Meaning, this will delete any Asterisk full log file that is older than today.

Problematic Script 2

Additionally, under /etc/cron.d/vpbx there is the following rule:

0 0 */2 * * root /usr/share/vitalpbx/scripts/clean_server >/dev/null 2>&1

This script, runs on every second day of the month. Among other stuff, that script does the below

rm -f /var/log/asterisk/full.*
rm -f /var/log/asterisk/full-*

Meaning, every second day, if the above script didn’t nuke the Asterisk logs, this one will.

Solution:

What did, is we stopped both of these scripts and modified /etc/logrotate.d/asterisk-pbx to look like this:

var/log/asterisk/messages
/var/log/asterisk/event_log
/var/log/asterisk/full
/var/log/asterisk/dtmf
/var/log/asterisk/fail2ban {
        daily
        dateext
        dateformat -%Y%m%d
        rotate 7
        missingok
        notifempty
        sharedscripts
        create 0640 asterisk asterisk
        su asterisk asterisk
        postrotate
                /usr/sbin/asterisk -rx 'logger reload' > /dev/null 2> /dev/null
        endscript
}

Now, after more than a week being like this, we see that it works fine:

root@core1:~# ls -la /var/log/asterisk/ | grep full
-rw-r-----  1 asterisk asterisk 12283171 Jul 23 11:42 full
-rw-r-----  1 asterisk asterisk 41153722 Jul 17 00:00 full-20240717
-rw-r-----  1 asterisk asterisk 38352665 Jul 18 00:00 full-20240718
-rw-r-----  1 asterisk asterisk 32241982 Jul 19 00:00 full-20240719
-rw-r-----  1 asterisk asterisk 32663989 Jul 20 00:00 full-20240720
-rw-r-----  1 asterisk asterisk  3597405 Jul 21 00:00 full-20240721
-rw-r-----  1 asterisk asterisk 23478627 Jul 22 00:00 full-20240722
-rw-r-----  1 asterisk asterisk 49269564 Jul 23 00:00 full-20240723

With that being said, you can likely set dateext and dateformat -%Y%m%d under /etc/logrotate.conf

Suggestion:

To fix this problem, we suggest doing the following:

  1. Use log rotate to rotate and delete files. Of course, with the correct config.
  2. Use a single script that runs nightly to cleans up anything that log rotate missed. For example, there may be full.1 or queue_log.2 that log rotate could not delete. Or for example full files older than 8 days and queue_log files older than 30 days.

Thank you

5 Likes
2

Hello,

Per the VitalPBX Phone System Change Log you can see that in v4.1.0 R7, the following change has been made.

UPDATED: The script to clean old logs now runs every 15 days.

First, it seems like after doing apt update && apt upgrade -y the version doesn’t actually update.
You have to either do the update from the GUI or apt reinstall vitalpbx to get the actual update.

Second.

The /usr/share/vitalpbx/scripts/clean_logs is now updated:

find /var/log/asterisk/ -name "full.*" -type f -mtime +10 -delete
find /var/log/asterisk/ -name "full-*" -type f -mtime +10 -delete
find /var/log/asterisk/ -name "fail2ban.*" -type f -mtime +2 -delete

THANK YOU!

However, the actual /etc/logrotate.d/asterisk-pbx is still broken:

var/log/asterisk/messages
/var/log/asterisk/event_log
/var/log/asterisk/full
/var/log/asterisk/dtmf
/var/log/asterisk/fail2ban {
        daily
        rotate 5
        size 5G
        missingok
        notifempty
        sharedscripts
        create 0640 asterisk asterisk
        su asterisk asterisk
        postrotate
                /usr/sbin/asterisk -rx 'logger reload' > /dev/null 2> /dev/null
        endscript
}
  1. This will only rotate if the file has reached 5GB, otherwise it will not be rotated.
  2. The files are only kept for 5 days instead of 7.
  3. The filename isn’t set.

Finally, we see that you changed under /etc/cron.d/vpbx that the clean server script will run on the 15th day of the month

0 0 */15 * * root /usr/share/vitalpbx/scripts/clean_server >/dev/null 2>&1

The problem is, that the content of that script is still bad.

root@vitalpbx:~# cat /usr/share/vitalpbx/scripts/clean_server
#!/bin/bash

#Delete Logs
...
rm -f /var/log/asterisk/full.*
rm -f /var/log/asterisk/full-*
...

That means, that ANY asterisk log older than today, will be deleted. So on the 15th day of the month, your logs from a week long will be gone.

Summary

  1. /etc/logrotate.d/asterisk-pbx Needs to be fixed.
  2. /usr/share/vitalpbx/scripts/clean_server Needs to be fixed.
3

Yes, the package vitalpbx is kept back with a regular upgrade. This seems to be caused because the new version of the vitalpbx package has new dependencies. It will remove systemd-timesyncd in favor of the ntp package. Wonder why the change was done.

apt install vitalpbx
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  ntp
Suggested packages:
  ntp-doc
The following packages will be REMOVED:
  systemd-timesyncd
The following NEW packages will be installed:
  ntp
The following packages will be upgraded:
  vitalpbx
1 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
Need to get 14.7 MB of archives.
After this operation, 1,998 kB of additional disk space will be used.
4

Hello VitalPBX team, any update on this please?

1 Like
5

We will take note of this! However, we don’t have an ETA for the next update!

1 Like