[Erlang] Mnesia Overload Events

Normally when running an Erlang application (like Ejabberd) using Mnesia internal database in a heavy loaded environment you run into the following log message:

=ERROR REPORT==== 23-Jun-2010::00:59:19 ===
Mnesia(node@server): ** WARNING ** Mnesia is overloaded: {dump_log, write_threshold}

This warning event is usually shown when using mnesia disc_copies tables and doing a lot of writes to disk all at once (considering you are using asynchronous writes), and of course it can get really annoying when they start happening every second. We can drastically reduce the occurrence of this error changing 2 configuration parameters, the dc_dump_limit and dump_log_write_threshold.

dc_dump_limit
This parameter controls how often disc_copies tables are dumped from memory, the default value is 4 that means if the log size is greater than the table size divided per 4 (table size / 4) it starts the dump. You can make dumps happen more often increasing this value.

dump_log_write_threshold
This parameter defines the maximum number of writes to the transaction log before a new dump is performed, using the default value 100 a new transaction log dump is performed after every 100 writes. You can increase this value giving more time to the write process to finish, but be sure that you have enough RAM on your server to maintain the transactions running.

And now, what does it mean?
Mnesia activity is recorded in transaction logs, that gets dumped to table logs, which in turn are dumped to table files on disk. By increasing the dump_log_write_threshold, transaction logs are dumped much less often, giving more time to the last dump be completed before the next dump is triggered. By the way, increasing the dc_dump_limit helps ensure that the disk table is dumped form the logs more often.

You can configure the parameters using the following syntax:

{mnesia, [{dc_dump_limit, 40}, {dump_log_write_threshold, 50000}]}
* Remember to change the values according to your needs!