Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Source configuration for Ping Identity PingFederate

Integrating with Ping Identity PingFederate

Ping Identity PingFederate integration uses Amazon S3 and Amazon SQS to ingest audit log data into CloudWatch pipelines. PingFederate is deployed on-premises and does not expose a REST API for audit log retrieval. Fluent Bit tails the JSON-formatted audit log files on the PingFederate host and delivers them to an Amazon S3 bucket. Amazon SQS notifications alert the pipeline when new log objects arrive.

Prerequisites

Log forwarding setup

PingFederate writes audit logs to local files using Log4j2. You configure JSON-formatted output by adding JsonTemplateLayout appenders to the log4j2.xml configuration file. Fluent Bit tails these JSON log files and delivers them to Amazon S3. For detailed configuration instructions, see Step 5 in the following section.

Instructions to setup Amazon S3 and Amazon SQS

Complete the following steps to configure the Amazon S3 and Amazon SQS infrastructure for Ping Identity PingFederate log ingestion.

Step 1: Create Amazon S3 bucket

Create an Amazon S3 bucket to store PingFederate audit logs. The bucket must reside in the same Amazon Region where you plan to create the CloudWatch pipeline.

Step 2: Create Amazon SQS queue

Create an Amazon SQS queue in the same Amazon Region as your Amazon S3 bucket. This queue receives notifications when new log files are added to the bucket.

Step 3: Connect Amazon S3 to Amazon SQS

Configure the Amazon S3 bucket to send event notifications for s3:ObjectCreated:* events to the Amazon SQS queue.

Step 4: Configure Amazon SQS queue policy

Configure the Amazon SQS queue policy to allow the Amazon S3 bucket to send messages to the queue. Apply the following policy to your Amazon SQS queue:

{
  "Version": "2012-10-17",
  "Id": "AllowS3ToSQS",
  "Statement": [
    {
      "Sid": "AllowS3BucketNotification",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "sqs:SendMessage",
      "Resource": "arn:aws:sqs:<region>:<account-id>:<queue-name>",
      "Condition": {
        "ArnEquals": {
          "aws:SourceArn": "arn:aws:s3:::<YOUR-BUCKET>"
        },
        "StringEquals": {
          "aws:SourceAccount": "<account-id>"
        }
      }
    }
  ]
}

Step 5: Configure Ping Identity PingFederate log export and Fluent Bit

Enable JSON logging on PingFederate by editing <PF_HOME>/server/default/conf/log4j2.xml. Add the following RollingFile appenders:

<RollingFile name="AdminAudit-JsonTemplateLayout"
             fileName="${sys:pf.log.dir}/admin_json.log"
             filePattern="${sys:pf.log.dir}/admin_json.%d{yyyy-MM-dd}.log"
             ignoreExceptions="false">
    <JsonTemplateLayout eventTemplateUri="${sys:pf.log4j.json.templates.uri}/admin-audit-log.json"/>
    <Policies>
        <TimeBasedTriggeringPolicy maxRandomDelay="45"/>
    </Policies>
</RollingFile>

<RollingFile name="SecurityAudit-JsonTemplateLayout"
             fileName="${sys:pf.log.dir}/audit_json.log"
             filePattern="${sys:pf.log.dir}/audit_json.%d{yyyy-MM-dd}.log"
             ignoreExceptions="false">
    <JsonTemplateLayout eventTemplateUri="${sys:pf.log4j.json.templates.uri}/runtime-audit-log.json"/>
    <Policies>
        <TimeBasedTriggeringPolicy maxRandomDelay="45"/>
    </Policies>
</RollingFile>

Update the Loggers section to reference the new appenders for the multiple audit loggers:

<Logger name="org.sourceid.websso.profiles.sp.SpAuditLogger" level="INFO" additivity="false">
    <AppenderRef ref="SecurityAudit-JsonTemplateLayout"/>
</Logger>
<Logger name="org.sourceid.websso.profiles.idp.IdpAuditLogger" level="INFO" additivity="false">
    <AppenderRef ref="SecurityAudit-JsonTemplateLayout"/>
</Logger>
<Logger name="org.sourceid.websso.profiles.idp.AsAuditLogger" level="INFO" additivity="false">
    <AppenderRef ref="SecurityAudit-JsonTemplateLayout"/>
</Logger>
<Logger name="org.sourceid.websso.profiles.idp.ClientRegistrationAuditLogger" level="INFO" additivity="false">
    <AppenderRef ref="SecurityAudit-JsonTemplateLayout"/>
</Logger>
<Logger name="org.sourceid.wstrust.log.STSAuditLogger" level="INFO" additivity="false">
    <AppenderRef ref="SecurityAudit-JsonTemplateLayout"/>
</Logger>
<Logger name="com.pingidentity.sdk.logging.LoggingUtil" level="INFO" additivity="false">
    <AppenderRef ref="SecurityAudit-JsonTemplateLayout"/>
</Logger>
<Logger name="AuditLogger" level="INFO" additivity="false">
    <AppenderRef ref="AdminAudit-JsonTemplateLayout"/>
</Logger>

Configure Fluent Bit to collect both audit log files and deliver them to Amazon S3:

########################################
# SERVICE SECTION
########################################
[SERVICE]
    Flush        5
    Daemon       Off
    Log_Level    info
    storage.path /etc/fluent-bit/db/storage
    storage.sync normal
    storage.checksum off
    storage.backlog.mem_limit 10M

########################################
# INPUT SECTION - Audit Log
########################################
[INPUT]
    Name             tail
    Path             /opt/pingfederate/log/audit_json.log
    Tag              pingfederate.audit
    Read_from_Head   True
    DB               /etc/fluent-bit/db/audit_checkpoint.db
    DB.Sync          Normal
    storage.type     filesystem
    Rotate_Wait      30
    Refresh_Interval 60

########################################
# INPUT SECTION - Admin Log
########################################
[INPUT]
    Name             tail
    Path             /opt/pingfederate/log/admin_json.log
    Tag              pingfederate.admin
    Read_from_Head   True
    DB               /etc/fluent-bit/db/admin_checkpoint.db
    DB.Sync          Normal
    storage.type     filesystem
    Rotate_Wait      30
    Refresh_Interval 60

########################################
# OUTPUT SECTION
########################################
[OUTPUT]
    Name              s3
    Match             pingfederate.*
    Bucket            <YOUR-BUCKET>
    Region            <region>
    S3_Key_Format     /pingfederate/%Y/%m/%d/%H/%M/%S.log.gz
    compression       gzip
    use_put_object    On
    store_dir         /etc/fluent-bit/db/s3
    total_file_size   1M
    log_key           log
    upload_timeout    60

Step 6: IAM permissions

Create an IAM policy with the following permissions for the Fluent Bit host to write objects to the Amazon S3 bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "FluentBitS3Write",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetBucketLocation",
        "s3:ListBucket",
        "s3:ListMultipartUploadParts",
        "s3:AbortMultipartUpload",
        "s3:CreateMultipartUpload",
        "s3:CompleteMultipartUpload"
      ],
      "Resource": [
        "arn:aws:s3:::<YOUR-BUCKET>",
        "arn:aws:s3:::<YOUR-BUCKET>/pingfederate/*"
      ]
    }
  ]
}

Step 7: Verify

Verify the setup by confirming that log files appear in the Amazon S3 bucket and that Amazon SQS notifications are being generated. Check the Amazon SQS queue for messages indicating new object creation events.

Configuring the CloudWatch pipeline

Supported Open Cybersecurity Schema Framework Event Classes

This integration supports OCSF schema version v1.5.0. The following table shows the mapping between PingFederate audit log types and OCSF event classes.