Probes

A probe is a scheduled task that retrieves information about the database objects that are being monitored by the PEM agent. PEM uses the collected information to build the graphs displayed on each homepage. The Manage Probes tab (accessed via the Management menu) allows you to modify the data collection schedule and the length of time that PEM will retain information returned by a specific probe.

Unless otherwise noted, Postgres Enterprise Manager™ enables the following probes at the server, database, schema, extension (starting with version 8.4), or agent levels:

Probe Name

Information Monitored by Probe

Probe Configuration Level

Background Writer Statistics

This probe monitors information about the background writer. The information includes:

  • The number of timed checkpoints

  • The number of requested checkpoints

  • The number of buffers written (by checkpoint)

  • The number of buffers written (by background writer)

  • The number of background writer cycles

  • The number of background buffers written

  • The number of buffers allocated

Server

Barman Configuration

This probe returns information about the Barman tool global configuration.

Agent

Barman Information

This probe returns information about the Barman tool.

Agent

Barman Server

This probe returns information about the respective database server configuration monitored by Barman.

Agent

Barman Server Status

This probe returns information about the respective database server status monitored by Barman.

Agent

Barman Server Backup

This probe returns information about the backups of the respective database servers.

Agent

Barman Server WAL Status

This probe returns information about the Barman server WAL files.

Agent

Blocked Session Information

This probe returns information about the blocked sessions.

Server

CPU Usage

This probe monitors CPU Usage information.

Agent

Data and Log File Analysis

This probe monitors information about log files. The information includes:

  • The name of the log file

  • The directory in which the log file resides

Server

Database Frozen XID

This probe monitors the frozen XID of each database.

Server

Database Size

This probe monitors information about the size of the monitored databases. The information includes:

  • The time the information was gathered

  • The database name

  • The database size (in MB’s)

Server

Database Statistics

This probe monitors database statistics. The information includes:

  • The number of backends

  • The number of transactions committed

  • The number of transactions rolled back

  • The number of blocks read

  • The number of blocks hit

  • The number of rows returned

  • The number of rows fetched

  • The number of rows inserted

  • The number of rows updated

  • The number of rows deleted

Server

Disk Busy Info

This probe monitors information about disk activity.

  • Note: This probe is not supported on Mac OS X, Solaris or HP-UX

Agent

Disk Space

This probe monitors information about disk space usage. The information includes:

  • The amount of disk space used

  • The amount of disk space available

Agent

EDB Audit Configuration

This probe monitors the audit logging configuration of Postgres Plus Advanced Servers.

Server

Failover Manager Cluster Info

This probe monitors a Failover Manager cluster, returning information about the cluster. This probe is disabled unless a cluster name and path of the Failover Manager binary is provided on the Server Properties dialog.

Server

Failover Manager Node Status

This probe monitors a Failover Manager cluster, returning detailed about each node within the cluster. This probe is disabled unless a cluster name and path of the Failover Manager binary is provided on the Server Properties dialog.

Server

Function Statistics

This probe monitors a database, retrieving information about functions. The information includes:

  • Function names

  • Argument types

  • Return values

Database

Index Size

This probe monitors a database, retrieving information about indexes. The information includes:

  • The name of the index

  • The time the data was gathered

  • The size of the index (in MB’s)

Database

Index Statistics

This probe monitors index statistics. The information includes:

  • The number of index scans

  • The number of rows read

  • The number of rows fetched

  • The number of blocks read

  • The number of blocks hit

Database

Installed Packages

This probe monitors the packages that are currently installed. The information gathered includes:

  • The name of the installed package

  • The version of the installed package

  • The date and time that the probe executed

Agent

IO Analysis

This probe monitors disk I/O information in. The information includes:

  • The number of blocks read

  • The number of blocks written

  • The date and time that the probe executed

  • Note: This probe is not supported on Mac OS X

Agent

Load Average

This probe monitors CPU load averages. The information includes:

  • The 1-minute load average

  • The 5-minute load average

  • The 15-minute load average

  • Note: This probe is not supported on Windows

Agent

Lock Information

This probe monitors lock information. The information includes:

  • The database name

  • The lock type

  • The lock mode

  • The process holding the lock

Server

Memory Usage

This probe monitors information about system memory usage. The information includes:

  • Total RAM in MB

  • Free RAM in MB

  • Total swap memory in MB

  • Free swap memory in MB

  • Shared system memory in MB (It is used by tuning wizard to tune the memory parameters for the database server)

    • On non-windows system, it is shmmax value and read from /proc/sys/kernel/shmmax

    • On windows, it is same as total memory.

Agent

Network Statistics

This probe monitors network statistics. The information includes:

  • The interface IP address

  • The number of packets sent

  • The number of packets received

  • The number of bytes sent

  • The number of bytes received

  • The link speed (in MB/second)

Agent

Number of Prepared Transactions

This probe stores the number of prepared transactions.

Server

Number of WAL Files

This probe monitors the number of WAL files.

Server

Object Catalog: Database

This probe monitors a list of databases and their properties The information includes:

  • The database name

  • The database encoding type

  • If the database allows user connections or system connections

Server

Object Catalog: Foreign Key

This probe monitors a list of foreign keys and their properties. The information includes:

  • The name of the table that contains the foreign key

  • The name of the table that the foreign key references

  • The name of the database in which the table resides

  • The name of the schema in which the table resides

Schema

Object Catalog: Function

This probe monitors a list of functions and their properties. The information includes:

  • The name of the function

  • The name of the schema in which the function resides

  • The name of the database in which the function resides

Schema

Object Catalog: Index

This probe monitors a list of indexes and their properties. The information includes:

  • The name of the index

  • The name of the table that the index is associated with

  • The name of the database in which the indexed table resides

Schema

Object Catalog: Schema

This probe monitors a list of schemas and their associated databases and servers.

Database

Object Catalog: Sequence

This probe monitors a list of sequences and their properties.

Schema

Object Catalog: Table

This probe monitors a list of table information. The information includes:

  • The table name

  • The name of the schema in which the table resides

  • The name of the database in which the schema resides

  • A Boolean indicator that indicates if the table has a primary key

Schema

Object Catalog: Tablespace

This probe monitors a list of tablespaces.

Server

Operating System Information

This probe monitors the operating system details and boot time.

Agent

Package Catalog

This probe monitors the packages that are currently available for installation. The information gathered includes:

  • The package name

  • The package version

Agent

PG HBA Conf

This probe monitors authentication configuration information from the pg_hba.conf file.

Server

Server Information

This probe monitors information about servers.

Server

Session Information

This probe monitors session information. The information includes:

  • The name of the session user

  • The date and time that the session connected to the server

  • The status of the session at the time that the information was gathered (idle, waiting, etc)

  • The client address and port number

Server

Settings

This probe monitors the values currently assigned to GUC variables.

Server

SQL Protect

This probe monitors a server, retrieving information about SQL injection attacks.

Server

Slony Replication

This probe monitors lag data for clusters replicated using Slony.

Database

Streaming Replication

This probe monitors a cluster that is using streaming replication, retrieving information about:

  • The sent Xlog location (in bytes)

  • The write Xlog location (in bytes)

  • The flush Xlog location (in bytes)

  • The replay Xlog location (in bytes)

  • The Xlog lag (in segments)

  • The Xlog lag (in pages)

Server

Streaming Replication Lag Time

This probe monitors a cluster that is using streaming replication, retrieving lag information about:

  • Replication lag time (in seconds)

  • Current status of replication (running/paused)

Server

Streaming Replication Database Conflicts

This probe monitors a database that is using streaming replication, retrieving information about any conflicts that arise. This includes information about queries that have been canceled due to:

  • The # of drop tablespace conflicts

  • The # of lock timeout conflicts

  • The # of old snapshot conflicts

  • The # of pinned buffer conflicts

  • The # of deadlock conflicts

Server

Table Bloat

This probe monitors information about the current table bloat. The information includes:

  • The name of the table

  • The name of the schema in which the table resides

  • The estimated number of pages

  • The estimated number of wasted pages

  • The estimated number of bytes per row

Database

Table Frozen XID

This probe monitors the frozen XID of each table.

Schema

Table Size

This probe monitors information about table size. The information includes:

  • Table size (in MB’s)

  • Total index size (in MB’s)

  • Total table size, with indexes and TOAST (in MB’s)

Database

Table Statistics

This probe monitors table statistics. The information includes:

  • The number of sequential scans

  • The number of sequential scan rows

  • The number of index scans

  • The number of index scan rows

  • The number of rows inserted

  • The number of rows updated

  • The number of rows deleted

  • The number of live rows

  • The number of dead rows

  • The last VACUUM

  • The last auto-vacuum

  • The last ANALYZE

  • The last auto-analyze

  • The number of pages estimated by ANALYZE

  • The number of rows estimated by ANALYZE

Database

Tablespace Size

This probe monitors a list of tablespaces and their sizes.

Server

User Information

This probe monitors a list of the current users. The stored information includes:

  • The user name

  • The user type (superuser vs. non-superuser)

  • The server to which the user is connected

Server

WAL Archive Status

This probe monitors the status of the WAL archive. The stored information includes:

  • The # of WAL archives done

  • The # of WAL archives pending

  • The last archive time

  • The # of WAL archives failed

  • The time of the last failure

Server

xDB Replication

This probe monitors lag data for clusters replicated using xDB replication.

Database

PGD Probes

To monitor the PGD Group via PGD dashboards the following probes must be enabled. All these probes are configured at the extension level.

Note

Prior to version 8.4, all these probes are available at the server level.

PGD Probes

The user with bdr_superuser will be able to view information from all the following probes.

All the following probes works with PGD Enterprise Edition.

Probe Name

Information Monitored by Probe

bdr_monitor role required

Works with PGD SE?

PGD Conflict History Summary

This probe returns information about row conflicts per conflict type. The stored information inlcudes:

  • The local time of the conflict

  • The type of the conflict

Yes

Yes

PGD Global Locks

This probe returns information about global locks in a PGD Group. The stored information includes:

  • The name of the node where the global lock has originated

  • The PID of the process holding the lock

  • The type of lock (DDL or DML)

  • The name of the locked relation(for DML Locks) or Keys(for advisory locks)

  • The internal state of the lock acquistion process

  • The list of backends waiting for the same global lock

  • The time when the global lock acquire was initiated by origin node

  • The time when the local node started trying to acquire the local lock

  • The time acquire_stage last changed

Yes

Yes

PGD Group Camo Details

This probe returns information about Camo’s in PGD Group. The stored information includes:

  • The name of the node

  • The node name for whom this node is partner

  • The node name for whom this node is origin

  • The connection status

  • The readiness status

  • The number of pending or unresolved camo transactions

  • The lsn of last applied wal log

  • The lsn of last received wal log

No

No

PGD Group Replication Slot Details

This probe returns information about replication slots in PGD Group. The stored information includes:

  • The name of the PGD Group

  • The name of the origin node

  • The name of the target node

  • The slot name on the origin node used by this subscription

  • The active status

  • The state of the replication (catchup,streaming,disconnected,..)

  • The approximate lag time for reported write

  • The approximate lag time for reported flush

  • The approximate lag time for reported replay

  • The bytes difference between sent_lsn and current WAL write position

  • The bytes difference between write_lsn and current WAL write position

  • The bytes difference between flush_lsn and current WAL write position

  • The bytes difference between replay_lsn and current WAL write position

No

Yes

PGD Group Subscription Summary

This probe returns information about the summary of Subscriptions in the PGD Group. The stored information includes:

  • The name of the origin of the subscription

  • The name of the target of the subscription

  • The timestamp of the last replayed transaction

  • The lag between now and time of last replayed transaction

No

Yes

PGD Monitor Group Raft

This probe returns the status and message of a cluster-wide raft check.

Yes

Yes

PGD Group Raft Details

This probe returns the information about raft consensus status from all the nodes in PGD Group. The stored information includes:

  • The name of the node

  • The raft worker state on the node

  • The node id of the RAFT_LEADER

  • The raft election internal id

  • The raft snapshot internal id

No

Yes

PGD Monitor Group Versions

This prove returns the status and message of cluster-wide version check.

Yes

Yes

PGD Group Versions Details

This probe returns the information about version details of the installed postgres, pglogical, pgd, and pgd edition for each node in the PGD Group. The stored information includes:

  • The name of the node

  • The installed postgres version on the node

  • The installed pglogical version on the node

  • The version of the PGD on the node

  • The PGD edition (Standard/Enterprise)

No

Yes

PGD Node Replication Rates

This probe returns information about outgoing replication activity from a given node. The stored information includes:

  • The name of the target peer node

  • The latest sent position

  • The latest position reported as replayed

  • The approximate lag time for reported replay

  • The bytes difference between replay_lsn and current WAL write position on origin

  • The human readable bytes difference between replay_lsn and current WAL write position

  • Approximate time required for the peer node to catchup to all the changes that are yet to be applied

Yes

No

PGD Node Slots

This probe returns information about the mapping of local PGD nodes to replication slots, their status, and replication progress. The stored information includes:

  • The name of the slot

  • The name of the target node

  • The name of the PGD Group

  • The database name on the target node

  • The PID of the process attached to the slot

  • The catalog XID needed by the slot

  • The ip address of the client connection

  • The latest sent position

  • The latest position reported as replayed

  • The approximate lag time for reported replay

  • The Bytes difference between replay_lsn and current WAL write position

  • The Human-readable bytes difference between replay_lsn and current WAL write position

Yes

Yes

PGD Node Summary

This probe returns information about all the nodes in the PGD Group. The stored information includes:

  • The name of the Node

  • The name of the PGD group the node is part of

  • The consistent state of the node in human readable form

  • The state which the node is trying to reach (during join or promotion)

  • The name of subscribed repsets

Yes

Yes

PGD Workers

This probe returns information about workers in PGD node. The stored information includes:

  • The PID of the worker process

  • The worker query start timestamp

  • The worker state change timestamp

  • The worker wait event type

  • The worker wait event

  • The worker state

  • The worker role name

  • The worker commit timestamp

  • The worker local timestamp

  • The name of the origin node

  • The receive LSN

  • The receive commit LSN

  • The last exact replay LSN

  • The last exact flush LSN

  • The last exact replay timestamp

  • The worker query

Yes

Yes

PGD Worker Errors

This probe returns information about the worker errors in PGD node. The stored information includes:

  • The process id of the worker causing the error

  • The name of the PGD Group the node is part of

  • The name of the origin node

  • The name of the source node

  • The name of the target node

  • The name of the subscription

  • The internal identifier of the role of this worker

  • The name of the role of this worker

  • The date and time of the error

  • The age of the error

  • The description of the error

  • The context in which the error happened

  • The remote relation id

  • The subscription writer id

  • The subscription writer name

Yes

Yes

PGD Stat Relation

This probe returns statistics for each relation. The stored information includes:

  • The name of the relation’s schema

  • The name of the relation

  • The oid of the relation

  • Total time spent processing replication for the relation

  • Number of inserts replicated for the relation

  • Number of updates replicated for the relation

  • Number of deletes replicated for the relation

  • Number of truncates replicated for the relation

  • Total number of shared block cache hits for the relation

  • Total number of shared blocks read for the relation

  • Total number of shared blocks dirtied for the relation

  • Total number of shared blocks written for the relation

  • Total time spent reading blocks for the relation, in milliseconds (if track_io_timing is enabled, otherwise zero)

  • Total time spent writing blocks for the relation, in milliseconds (if track_io_timing is enabled, otherwise zero)

  • Total time spent acquiring locks on the relation, in milliseconds (if pglogical.track_apply_lock_timing is enabled,otherwise zero)

Yes

Yes

PGD Stat Subscription

This probe returns statistics for each subscription. The stored information includes:

  • The name of the subscription

  • The oid of the subscription

  • Number of times this subscription has connected upstream

  • Number of commits this subscription did

  • Number of aborts writer did for this subscription

  • Number of errors writer has hit for this subscription

  • Number of transactions skipped by writer for this subscription

  • Number of inserts this subscription did

  • Number of updates this subscription did

  • Number of deletes this subscription did

  • Number of truncates this subscription did

  • Number of DDL operations this subscription has executed

  • Number of errors that were caused by deadlocks

  • Number of retries the writer did

  • Total number of shared block cache hits by the subscription

  • Total number of shared blocks read by the subscription

  • Total number of shared blocks dirtied by the subscription

  • Total number of shared blocks written by the subscription

  • Total time the subscription spent reading blocks, in milliseconds (if track_io_timing is enabled, otherwise zero)

  • Total time the subscription spent writing blocks, in milliseconds (if track_io_timing is enabled, otherwise zero)

  • Time when the current upstream connection was established, NULL if not connected

  • Time when the last upstream connection was dropped

  • LSN from which this subscription requested to start replication from the upstream

  • Number of attempts the subscription was restarted from the same LSN value

  • Number of commits this subscription did after the current connection was established

Yes

Yes