Penguin, small TECH.BARWICK.DE

Recent posts





Powered By


Tuesday, September 24, 2019  11:12 PM

Yum and "Thread died in Berkeley DB library" errors

I started getting intermittent sets of error messages like this:

error: rpmdb: BDB0113 Thread/process 26154/140393252489024 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 -  (-30973)
error: cannot open Packages database in /var/lib/rpm

when deploying changes to a bunch of AWS EC2 instances.

The error messages are misleading as (in this case at least) the RPM database is not corrupted; the underlying issue was this:

[67897.740241] Out of memory: Kill process 28759 (yum) score 330 or sacrifice child
[67905.749492] yum invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0

i.e. simply a lack of memory (the instances were just too small).

Posted in Linux | add a comment

The Prometheus PostgreSQL storage adapter does not seem amenable to being executed directly from a systemd service file.

As a workaround I created a wrapper script like this (adjust parameters as required):


# Wrapper to launch prometheus-postgresql-adapter, as calling
# it directly from the systemd service file doesn't seem to work.
# Disclaimer: there is probably a better way of doing this.

nohup /usr/local/bin/prometheus-postgresql-adapter \
-pg-host=... \
-pg-port=... \
-pg-database="..." \
-pg-user="..." \
>> /var/log/prometheus-postgresql-storage-adapter/prometheus-pg-adapter.log 2>&1

and a service file like this:

Description=Prometheus PostgreSQL Storage Adapter



which works fine (YMMV of course).

There may of course be a more elegant way of solving this issue, if so feel free to share.

Posted in Linux | add a comment