Request a Demo
6.7. Integration with Postgres Pro Enterprise Manager
Prev UpChapter 6. Configuration and SetupHome Next

6.7. Integration with Postgres Pro Enterprise Manager #

6.7.1. Setting up Metrics
6.7.2. Setting up Logs
6.7.3. Configuration Example

Postgres Pro Enterprise Manager (PPEM) supports integration with pgpro-otel-collector and can receive metrics and logs.

6.7.1. Setting up Metrics #

To export metrics to PPEM, follow the steps below:

  1. Enable and configure the required metric receivers (as per usual).

  2. Enable and configure the otlphttp exporter.

  3. Specify the PPEM connection URL.

  4. In the header X-Ppem-Source-Agent-Name, specify the name of the agent that controls the database instance (see ppem-agent.yml).

  5. In the header X-Ppem-Source-Instance-Port, specify the port number of the required DBMS instance (see postgresql.conf).

  6. Set up a pipeline (as per usual). 

receivers:
  hostmetrics:
    # ...
  postgrespro:
    # ...
exporters:
  otlphttp/ppem_metrics:
    compression: ''
    endpoint: "https://ppem.example.org"
    headers:
      X-Ppem-Source-Agent-Name: "dbserver01"
      X-Ppem-Source-Instance-Port: "5432"
service:
  extensions: []
  pipelines:
    metrics:
      receivers:
        - postgrespro
        - hostmetrics
      processors:
      exporters:
        - otlphttp/ppem_metrics

Make sure you have specified the correct values in the HTTP headers — these headers are used to connect metrics with instances in PPEM.

6.7.2. Setting up Logs #

To export logs to PPEM, do the following:

  1. Enable and configure the filelog receivers according to the database instance logging format (csvlog or jsonlog).

  2. Enable and configure the attributes/convert processor according to the database instance logging format (csvlog or jsonlog).

  3. Enable and configure the otlphttp exporter.

  4. Specify the PPEM connection URL.

  5. Specify the agent name that controls the required database instance in the header X-Ppem-Source-Agent-Name (see ppem-agent.yml).

  6. Specify the port number of the required DBMS instance in the header X-Ppem-Source-Instance-Port (see postgresql.conf).

  7. Set up a pipeline (as per usual). 

receivers:
  filelog:
    # ...
processors:
  attributes/convert:
    # ...
exporters:
  otlphttp/ppem_logs:
    compression: ''
    endpoint: "https://ppem.example.org"
    headers:
      X-Ppem-Source-Agent-Name: "dbserver01"
      X-Ppem-Source-Instance-Port: '5432'
service:
  extensions: []
  pipelines:
    logs:
      receivers:
        - filelog
      processors:
        - attributes/convert
      exporters:
        - otlphttp/ppem_logs

6.7.3. Configuration Example #

The example below shows how to set up the collector configuration for exporting metrics and logs (CSV) to PPEM. 

receivers:
  filelog:
    include:
      - /var/log/postgresql/*.csv
    multiline:
      line_start_pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}
    operators:
      - header: timestamp,user,dbname,pid,connection_from,session_id,line_num,ps,session_start,vxid,txid,error_severity,state_code,message,detail,hint,internal_query,internal_position,context,statement,cursor_position,func_name,application_name,backend_type,leader_pid,query_id
        timestamp:
          layout: '%Y-%m-%d %H:%M:%S.%L %Z'
          layout_type: strptime
          parse_from: attributes.timestamp
        type: csv_parser
      - field: attributes.timestamp
        type: remove
    retry_on_failure:
      enabled: true
      initial_interval: 1s
      max_elapsed_time: 5m
      max_interval: 30s
    start_at: end
  hostmetrics:
    collection_interval: 60s
    initial_delay: 1s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      disk: null
      load: null
      memory: null
      network: null
  postgrespro:
    max_threads: 3
    initial_delay: 1s
    collection_interval: 60s
    transport: tcp
    endpoint: localhost:5432
    database: postgres
    username: postgres
    password: ${env:POSTGRESQL_PASSWORD}
    metrics: null
    plugins:
      activity:
        enabled: true
      bgwriter:
        enabled: true
      locks:
        enabled: true
      versions:
        enabled: true
      wal:
        enabled: true
      cache:
        enabled: true
processors:
  attributes/convert:
    actions:
      - action: convert
        converted_type: int
        key: pid
      - action: convert
        converted_type: int
        key: line_num
      - action: convert
        converted_type: int
        key: txid
      - action: convert
        converted_type: int
        key: remote_port
      - action: convert
        converted_type: int
        key: cursor_position
      - action: convert
        converted_type: int
        key: internal_position
      - action: convert
        converted_type: int
        key: leader_pid
  batch:
    send_batch_size: 8192
    timeout: 10s
  resource:
    attributes:
      - action: upsert
        key: service.name
        value: postgresql
      - action: upsert
        key: service.instance.id
        value: dbserver-01.example.org:5432
exporters:
  otlphttp/ppem_logs:
    endpoint: 'https://ppem.example.org'
    headers:
      X-Ppem-Source-Agent-Name: 'dbserver-01.example.org'
      X-Ppem-Source-Instance-Port: '5432'
  otlphttp/ppem_metrics:
    endpoint: 'https://ppem.example.org'
    headers:
      X-Ppem-Source-Agent-Name: 'dbserver-01.example.org'
      X-Ppem-Source-Instance-Port: '5432'
service:
  extensions: []
  pipelines:
    logs:
      receivers:
        - filelog
      processors:
        - resource
        - attributes/convert
        - batch
      exporters:
        - otlphttp/ppem_logs
    metrics:
      receivers:
        - postgrespro
        - hostmetrics
      processors:
        - batch
      exporters:
        - otlphttp/ppem_metrics
Prev Up Next
6.6. Security Setup Home Chapter 7. pgpro-otel-collector Components
epub pdf