Commits (2)
  • Matt Smiley's avatar
    Reduce sensitivity of alert for dead tuples in Postgres. · 6edabeca
    Matt Smiley authored
    The "project_mirror_data" table regularly exceeds the old threshold of 10000 dead tuples
    and also has few enough live rows that it exceeds 5% ratio of dead / dead_or_alive tuples.
    Because this is a regular pattern, this alert was being ignored.
    Autovacuum is doing its job for this table, so we really can safely ignore that churn rate.
    In general, Postgres performance will tolerate having many more dead tuples than just 10000.
    Increasing the threshold to 1 million dead tuples and keeping the ratio threshold at 5% means
    we will only alert on large tables (at least 20 million rows), which is appropriate.
    Tiny tables do not require intervention.  Large tables with a high ratio of dead tuples may
    start to suffer from performance degradation if left unchecked, but they too do not require
    immediate action by the on-call engineer.
  • Matt Smiley's avatar
    Merge branch 'reduce_sensitivity_of_dead_tuples_alert' into 'master' · 9fdf7fa5
    Matt Smiley authored
    Reduce sensitivity of alert for dead tuples in Postgres.
    See merge request gitlab-com/runbooks!1590
......@@ -224,9 +224,9 @@ groups:
- alert: PostgreSQL_TooManyDeadTuples
expr: >
(pg_stat_user_tables_n_dead_tup > 10000)
(pg_stat_user_tables_n_dead_tup > 1000000)
(pg_stat_user_tables_n_live_tup + pg_stat_user_tables_n_dead_tup > 1000)
(pg_stat_user_tables_n_live_tup + pg_stat_user_tables_n_dead_tup)
) >= 0.05
unless ON(instance) (pg_replication_is_replica == 1)
for: 30m