Scribd
Upload
203 views5 pages

Using AWS Athena & Document For ALB-ELB Access Logs Analysis

This document provides steps for using Amazon Athena to analyze Application Load Balancer (ALB) and Elastic Load Balancer (ELB) access logs stored in S3. It describes creating an Athena database and tables for the two log types, including example queries. The results of Athena queries can be found in the Athena console or downloaded from the S3 location specified in the Athena settings.

Uploaded by

Arpit Awasthi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
203 views5 pages

Using AWS Athena & Document For ALB-ELB Access Logs Analysis

This document provides steps for using Amazon Athena to analyze Application Load Balancer (ALB) and Elastic Load Balancer (ELB) access logs stored in S3. It describes creating an Athena database and tables for the two log types, including example queries. The results of Athena queries can be found in the Athena console or downloaded from the S3 location specified in the Athena settings.

Uploaded by

Arpit Awasthi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Using AWS Athena & Document for ALB-ELB Access Logs Analysis

Athena Introduction:

Amazon Athena is an interactive query service that makes it easy to analyze data directly in
Amazon Simple Storage Service (Amazon S3) using standard SQL. With a few actions in the AWS
Management Console, you can point Athena at your data stored in Amazon S3 and begin using
standard SQL to run ad-hoc queries and get results in seconds.

Prerequisites
• If you have not already done so, sign up for an account in Setting Up.
• Using the same AWS Region (for example, US West (Oregon)) and account that you are using
for Athena, Create a bucket in Amazon S3 to hold your query results from Athena.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-
overview.html
You can use above url to know how to create S3 Bucket in AWS.

Steps to Getting Started with Athena:

Step 1: Open the AWS Athena console


Step 2: Click The Settings In the Settings dialog box, enter the path to the bucket that you
created in Amazon S3 for your query results. Prefix the path with s3:// and add a forward
slash to the end of the path

Step 3: Create an Athena "database".

First you will need to create a database that Athena uses to access your data. It's still
a database but data is stored in text files in S3.

Syntax: CREATE DATABASE database_name ;


.

Step 4: Create A Table.


Here we are creating Tables for Two Types of Logs.

1.alb_logs Table

2.elb_logs table

1.Creating the Table for ALB Logs.


An Application Load Balancer is a load balancing option for Elastic Load Balancing
that enables traffic distribution in a microservices deployment using containers. Querying
Application Load Balancer logs allows you to see the source of traffic, latency, and bytes
transferred to and from Elastic Load Balancing instances and backend applications. For more
information, see Access logs for your Application Load Balancer in the User Guide for
Application Load Balancers.
Prerequisites:
• Enable access logging so that Application Load Balancer logs can be saved to your
Amazon S3 bucket. To Enable Access Logs Go through below link.
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-
balancer-access-logs.html#enable-access-logging
The following CREATE TABLE statement into the Athena console. Replace the values in
LOCATION 's3://your-alb-logs-
directory/AWSLogs/<ACCOUNT_ID>/elasticloadbalancing/<REGION>/' with those
corresponding to your Amazon S3 bucket Location.

Run the query in the Athena console. After the query completes, Athena registers the
alb_logs table, making the data in it ready for you to issue queries.

Example Queries for ALB Logs:

1. View the First 100 access log entries in chronological order.

select * from “database_name”.”alb_logs” limit 100;

2. List all client IP addresses that accessed the Application Load Balancer, and how many
times they accessed the Application Load Balancer.

select distinct client_ip, count() as count from alb_log GROUP by client_ip ORDER by
count() DESC;

3. List the times that a client sent a request to the Application Load Balancer and then
terminated the connection to the Application Load Balancer before the idle timeout elapsed
(HTTP 460 error) for 24 Hours.
select * from "database_name"."alb_logs" where elb_status_code like '4%%' and
time >='2021-03-09T23:55:15' and time <= '2021-03-10T23:55:28';
Kindly You can Search In the below link for more Examples:
https://aws.amazon.com/premiumsupport/knowledge-center/athena-analyze-access-logs/
Where can I find the results of my Amazon Athena queries?
After execute each query in the AWS Athena Console You can see the results at
Result Section in the Athena console or Whatever that Location given in AWS Athena Settings
Meanwhile you can Download the query results files using the Athena console or Download the
query results files from the Amazon Simple Storage Service (Amazon S3) that you specified for the
query location.

2.Creating the Table for ELB Logs.


Use Classic Load Balancer logs to Analyze and understand traffic patterns to and
from Elastic Load Balancing instances and backend applications. You can see the source of
traffic, latency, and bytes that have been transferred.
Before you Analyze the Elastic Load Balancing logs, configure them for saving in the
destination Amazon S3 bucket. For more information, see
https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-access-logs.html
The following DDL statement into the Athena console. Check the syntax of the Elastic Load
Balancing log records. You may need to update the following query to include the columns
and the Regex syntax for latest version of the record.

Modify the LOCATION Amazon S3 bucket to specify the destination of your Elastic Load
Balancing logs.
Run the query in the Athena console. After the query completes, Athena registers
the elb_logs table, making the data in it ready for queries.
Examples Queries for ELB Logs:
1.This Query will show the number of the same IP requesting your backend with HTTP
response code 200.
Select request_ip, COUNT(*) as count from “elb_logs” WHERE
elb_response_code=200 GROUP BY reques_ip ORDER BY COUNT DESC LIMIT 10;
2. This one will show you which company behind a proxy are using on default browser
“Firefox”
SELECT request_ip, COUNT(*) as client_ip FROM “elb_logs” WHERE user_agent LIKE
'%Firefox%' GROUP BY request_ip ORDER BY client_ip DESC LIMIT 10;
For More Queries Follow the below link.
https://docs.aws.amazon.com/athena/latest/ug/elasticloadbalancer-classic-
logs.html#query-elb-classic-example

Where can I find the results of my Amazon Athena queries?


After execute each query in the AWS Athena Console You can see the results at
Result Section in the Athena console or Whatever that Location given in AWS Athena Settings
Meanwhile you can Download the query results files using the Athena console or Download the
query results files from the Amazon Simple Storage Service (Amazon S3) that you specified for the
query location.

You might also like