Intermediate AWK Guide

when follow this thing ; give me the essential elements included in this syntaxe given to get a
intermediate level in awk command :
`bash
awk 'pattern { action }' input_file`

To reach an intermediate level in `awk`, you need to understand the essential elements in this syntax:

bash

`awk 'pattern { action }' input_file

`

Essential Elements:

1. `pattern` (Optional)
Defines the condition that selects which lines are processed.
Can be a regular expression, comparison, or special pattern (e.g., `BEGIN`, `END`).
If omitted, `{ action }` applies to all lines.
2. `{ action }` (Optional)
Specifies what to do with matching lines.
Can include printing, modifying fields, performing calculations, etc.
If omitted, `awk` prints the whole line by default.
3. `input_file`
The file that `awk` processes.
If omitted, `awk` reads from standard input.

Intermediate-Level Concepts

To go beyond the basics, you should learn:

✅ Field Handling:
`$1`, `$2`, `$0` (full line), `NF` (number of fields), `FS` (field separator).

bash

`awk '{ print $1, $3 }' file.txt

`

✅ Built-in Variables:
Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 1/27
 `NR` (line number), `FNR` (per file line number), `OFS` (output field separator).

bash

`awk '{ print NR, $0 }' file.txt

`

✅ Pattern Matching:
Use regex `/pattern/` or conditions `if ($1 == "value")`

bash

`awk '/error/ { print $0 }' log.txt

`

✅ BEGIN & END Blocks:

`BEGIN` (executed before processing), `END` (executed after processing).

bash

`awk 'BEGIN { print "Start Processing" } { print $1 } END { print "Done" }' file.txt
`

✅ Mathematical Operations:
bash

`awk '{ sum += $3 } END { print "Total:", sum }' file.txt

`

✅ String Manipulation:
`length()`, `toupper()`, `tolower()`, `substr()`

bash

`awk '{ print toupper($1) }' file.txt

`

✅ Conditional Statements ( `if`, `else`, `ternary`):

bash

`awk '{ if ($3 > 50) print $1, "Passed"; else print $1, "Failed" }' file.txt
`

✅ Looping ( `for`, `while`):

bash

`awk '{ for (i=1; i<=NF; i++) print $i }' file.txt

`

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 2/27
✅ Processing Multiple Files:
bash

`awk 'FNR==1 { print "Processing", FILENAME } { print $1 }' file1.txt file2.txt

`

Mastering these elements will give you solid intermediate skills in `awk`. 🚀 Let me know if you need
specific examples!

so could you list me the essential { action } and pattern available ?

Sure! Here’s a structured list of essential `{ action }` (commands you can use inside the action block)
and patterns (conditions that determine which lines get processed) in `awk`.

1️⃣ Essential `{ action }` Commands

These are the most commonly used actions inside `{}`.

1.1 Print Output

Print entire line:

bash

`awk '{ print }' file.txt

`

Print specific fields:

bash

`awk '{ print $1, $3 }' file.txt

`

Print with custom separator:

bash

`awk '{ print $1, $3, "->", $NF }' file.txt

`

1.2 Variable Assignment & Arithmetic

Assign a variable:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 3/27
 bash

`awk '{ sum = $1 + $2; print sum }' file.txt

`

Accumulate values:

bash

`awk '{ total += $3 } END { print "Total:", total }' file.txt

`

1.3 String Manipulation

Convert to uppercase/lowercase:

bash

`awk '{ print toupper($1), tolower($2) }' file.txt

`

Get substring:

bash

`awk '{ print substr($2, 2, 4) }' file.txt

`

Get length:

bash

`awk '{ print length($1) }' file.txt

`

1.4 Conditional Execution

Using `if-else`:

bash

`awk '{ if ($3 > 50) print $1, "Pass"; else print $1, "Fail" }' file.txt
`

Using ternary operator:

bash

`awk '{ print $1, ($3 > 50 ? "Pass" : "Fail") }' file.txt
`

1.5 Looping

`for` loop (iterate over fields):

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 4/27
 bash

`awk '{ for (i=1; i<=NF; i++) print $i }' file.txt

`

`while` loop:

bash

`awk '{ i=1; while (i<=NF) { print $i; i++ } }' file.txt
`

1.6 Built-in Functions

Math functions:

bash

`awk '{ print sqrt($1), int($2), rand() }' file.txt

`

Formatting output:

bash

`awk '{ printf "User: %s, Score: %d\n", $1, $2 }' file.txt
`

2️⃣ Essential `pattern` Conditions

Patterns define which lines are selected for processing.

2.1 Regular Expressions

Match lines containing "error":

bash

`awk '/error/ { print }' log.txt

`

Match lines starting with "Warning":

bash

`awk '/^Warning/ { print }' log.txt

`

Match lines ending with "done":

bash

`awk '/done$/ { print }' log.txt

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 5/27
 `

2.2 Field-Based Conditions

Match lines where first field is "admin":

bash

`awk '$1 == "admin" { print }' users.txt

`

Match lines where third field is greater than 50:

bash

`awk '$3 > 50 { print }' scores.txt

`

2.3 Built-in Variables as Patterns

Variable Description Example

`NR` Current line number `NR == 5 { print }` (5th line)

`FNR` Line number per file `FNR == 1 { print FILENAME }` (first line of each file)

`NF` Number of fields `$NF == "ERROR" { print }` (last field is "ERROR")

`FS` Field separator `BEGIN { FS="," } { print $2 }` (CSV files)

2.4 Special Patterns

Pattern Description Example

Executes before processing

`BEGIN` `BEGIN { print "Header" }`
any line

Executes after all lines are

`END` `END { print "Processing Done" }`
processed

`BEGIN` + `BEGIN { sum=0 } { sum+=$3 } END { print

Used for summaries
`END` "Total:", sum }`

2.5 Logical Operators in Patterns

AND (`&&`): Match lines where column 2 > 50 AND column 3 < 100.

bash

`awk '$2 > 50 && $3 < 100 { print }' file.txt

`

OR (`||`): Match lines where column 1 is "admin" OR column 3 > 75.

bash

`awk '$1 == "admin" || $3 > 75 { print }' file.txt

`

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 6/27
 NOT (`!`): Match lines where column 2 is NOT "guest".

bash

`awk '$2 != "guest" { print }' file.txt

`

💡 Summary
To become intermediate in `awk`, focus on: ✔ Mastering `{ action }`: `print`, variables, conditions,
loops, string functions.
✔ Understanding `pattern`: regex, field conditions, built-in variables, logical operators.
✔ Using and
`BEGIN` `END` for setup and summaries.

Let me know if you need more examples or real-world use cases! 🚀

make an example for each expression given

Sure! Below are practical `awk` examples for each action and pattern discussed.

1️⃣ Examples for `{ action }` Commands

1.1 Print Output

Print entire line:

bash

`awk '{ print }' file.txt

`

Outputs each line as it is.

Print specific fields:

bash

`awk '{ print $1, $3 }' file.txt

`

Prints the first and third column.

Print with custom separator:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 7/27
 bash

`awk '{ print $1, $3, "->", $NF }' file.txt

`

Prints first and third column, adds "->", and then the last column.

1.2 Variable Assignment & Arithmetic

Add two columns:

bash

`awk '{ sum = $1 + $2; print sum }' file.txt

`

Adds the first two fields and prints the result.

Accumulate column values:

bash

`awk '{ total += $3 } END { print "Total:", total }' file.txt

`

Calculates sum of column 3.

1.3 String Manipulation

Convert to uppercase/lowercase:

bash

`awk '{ print toupper($1), tolower($2) }' file.txt

`

Changes case of column 1 and 2.

Get substring:

bash

`awk '{ print substr($2, 2, 4) }' file.txt

`

Extracts 4 characters from column 2, starting at position 2.

Get string length:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 8/27
 bash

`awk '{ print length($1) }' file.txt

`

Prints the length of the first column.

1.4 Conditional Execution

Using `if-else`:

bash

`awk '{ if ($3 > 50) print $1, "Pass"; else print $1, "Fail" }' file.txt
`

Checks if column 3 is greater than 50.

Using ternary operator:

bash

`awk '{ print $1, ($3 > 50 ? "Pass" : "Fail") }' file.txt
`

Shorter version of `if-else`.

1.5 Looping

Iterate over fields:

bash

`awk '{ for (i=1; i<=NF; i++) print $i }' file.txt

`

Prints each field separately.

Using `while` loop:

bash

`awk '{ i=1; while (i<=NF) { print $i; i++ } }' file.txt
`

Loops over all fields using `while`.

1.6 Built-in Functions

Math functions:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 9/27
 bash

`awk '{ print sqrt($1), int($2), rand() }' file.txt

`

Applies square root, integer conversion, and prints a random number.

Formatted output:

bash

`awk '{ printf "User: %s, Score: %d\n", $1, $2 }' file.txt
`

Prints fields in a structured format.

2️⃣ Examples for `pattern` Conditions

2.1 Regular Expressions

Match lines containing "error":

bash

`awk '/error/ { print }' log.txt

`

Prints lines with "error" in them.

Match lines starting with "Warning":

bash

`awk '/^Warning/ { print }' log.txt

`

Prints lines starting with "Warning".

Match lines ending with "done":

bash

`awk '/done$/ { print }' log.txt

`

Prints lines ending with "done".

2.2 Field-Based Conditions

Match lines where first field is "admin":

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 10/27
 bash

`awk '$1 == "admin" { print }' users.txt

`

Prints lines where the first column is "admin".

Match lines where third field > 50:

bash

`awk '$3 > 50 { print }' scores.txt

`

Prints lines where the third column is greater than 50.

2.3 Built-in Variables as Patterns

Variable Example Description

`NR` `awk 'NR == 5 { print }' file.txt` Prints the 5th line

`awk 'FNR == 1 { print FILENAME }' file1.txt

`FNR` Prints filename at first line of each file
file2.txt`

Prints lines where last column is

`NF` `awk '$NF == "ERROR" { print }' file.txt`
"ERROR"

Sets field separator to comma for CSV

`FS` `awk 'BEGIN { FS="," } { print $2 }' file.csv`
files

2.4 Special Patterns

Using `BEGIN`:

bash

`awk 'BEGIN { print "Processing Started" } { print $0 }' file.txt

`

Prints a message before processing begins.

Using `END`:

bash

`awk '{ sum += $3 } END { print "Total:", sum }' file.txt

`

Prints the total after processing all lines.

Using `BEGIN` and `END`:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 11/27
 bash

`awk 'BEGIN { sum=0 } { sum+=$3 } END { print "Total:", sum }' file.txt
`

Initializes `sum`, processes each line, then prints the total.

2.5 Logical Operators in Patterns

AND (`&&`):

bash

`awk '$2 > 50 && $3 < 100 { print }' file.txt

`

Prints lines where column 2 > 50 AND column 3 < 100.

OR (`||`):

bash

`awk '$1 == "admin" || $3 > 75 { print }' file.txt

`

Prints lines where column 1 is "admin" OR column 3 > 75.

NOT (`!`):

bash

`awk '$2 != "guest" { print }' file.txt

`

Prints lines where column 2 is NOT "guest."

💡 Final Takeaways
✔ Use for printing, calculations, loops, and formatting.
✔ Use patterns for filtering lines based on regex, conditions, and built-in variables.
`{ action }`

✔ Combine and for setup and final calculations.

✔ Use logical operators to refine pattern matching.
`BEGIN` `END`

Would you like me to create a real-world scenario using ?🚀

`awk`

so give me a data like example and bind it with some exercice to handy

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 12/27
Great idea! Below is a sample dataset along with some practical exercises to help you get hands-on
with `awk`.

📌 Sample Data: `employees.txt`

This file represents an employee database with fields:

ID | Name | Department | Salary | Status

txt

`101 John Sales 5000 Active

102 Alice HR 6000 Active
103 Bob IT 5500 Inactive
104 Eve Sales 7000 Active
105 Charlie IT 6500 Active
106 Dave HR 6200 Inactive
107 Carol Sales 7200 Active
108 Frank IT 5800 Active
109 Grace HR 6800 Active
110 Mallory Sales 5300 Inactive
`

📝 Exercises with `awk` Commands

1️⃣ Basic Printing
Q1: Print the entire file.

bash

`awk '{ print }' employees.txt

`

Expected Output: (Same as input file)

Q2: Print only names and salaries.

bash

`awk '{ print $2, $4 }' employees.txt

`

Expected Output:

txt

`John 5000
Alice 6000
Bob 5500
Eve 7000
Charlie 6500
Dave 6200
Carol 7200
Frank 5800
Grace 6800

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 13/27
 Mallory 5300
`

2️⃣ Filtering Data (Patterns)

Q3: Print employees from the Sales department.

bash

`awk '$3 == "Sales" { print }' employees.txt

`

Expected Output:

txt

`101 John Sales 5000 Active

104 Eve Sales 7000 Active
107 Carol Sales 7200 Active
110 Mallory Sales 5300 Inactive
`

Q4: Print employees who earn more than 6000.

bash

`awk '$4 > 6000 { print $2, $4 }' employees.txt

`

Expected Output:

txt

`Eve 7000
Charlie 6500
Carol 7200
Grace 6800
`

3️⃣ Using `BEGIN` and `END`

Q5: Add a header and footer when displaying names and salaries.

bash

`awk 'BEGIN { print "Employee Salary Report"; print "----------------------" } { print $2, $4 } END
`

Expected Output:

txt

`Employee Salary Report

----------------------
John 5000

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 14/27
 Alice 6000
Bob 5500
Eve 7000
Charlie 6500
Dave 6200
Carol 7200
Frank 5800
Grace 6800
Mallory 5300
----------------------
Report Complete
`

4️⃣ Using Built-in Variables ( `NR`, `NF`)

Q6: Print only the first 5 employees.

bash

`awk 'NR <= 5 { print }' employees.txt

`

Expected Output:

txt

`101 John Sales 5000 Active

102 Alice HR 6000 Active
103 Bob IT 5500 Inactive
104 Eve Sales 7000 Active
105 Charlie IT 6500 Active
`

Q7: Print employees where the last field (Status) is "Inactive".

bash

`awk '$NF == "Inactive" { print $2, "is not currently working." }' employees.txt
`

Expected Output:

txt

`Bob is not currently working.

Dave is not currently working.
Mallory is not currently working.
`

5️⃣ String Manipulation

Q8: Convert all employee names to uppercase.

bash

`awk '{ print toupper($2) }' employees.txt

`

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 15/27
Expected Output:

txt

`JOHN
ALICE
BOB
EVE
CHARLIE
DAVE
CAROL
FRANK
GRACE
MALLORY
`

Q9: Extract the first 3 letters of each employee's name.

bash

`awk '{ print substr($2, 1, 3) }' employees.txt

`

Expected Output:

txt

`Joh
Ali
Bob
Eve
Cha
Dav
Car
Fra
Gra
Mal
`

6️⃣ Conditional Execution ( `if-else`)

Q10: Label employees based on salary.

bash

`awk '{ if ($4 >= 6000) print $2, "is HIGH SALARY"; else print $2, "is LOW SALARY"; }' employees.txt
`

Expected Output:

txt

`John is LOW SALARY

Alice is HIGH SALARY
Bob is LOW SALARY
Eve is HIGH SALARY
Charlie is HIGH SALARY
Dave is HIGH SALARY
Carol is HIGH SALARY
Frank is LOW SALARY
Grace is HIGH SALARY

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 16/27
 Mallory is LOW SALARY
`

7️⃣ Using Logical Operators ( `&&`, `||`, `!`)

Q11: Print employees who are Active AND earn more than 6000.

bash

`awk '$4 > 6000 && $5 == "Active" { print $2, "->", $4 }' employees.txt
`

Expected Output:

txt

`Eve -> 7000

Charlie -> 6500
Carol -> 7200
Grace -> 6800
`

Q12: Print employees from IT OR HR departments.

bash

`awk '$3 == "IT" || $3 == "HR" { print $2, "-", $3 }' employees.txt

`

Expected Output:

txt

`Alice - HR
Bob - IT
Charlie - IT
Dave - HR
Frank - IT
Grace - HR
`

🚀 Bonus Challenge
Try to write an `awk` command that:

Counts the total number of employees.

Calculates the total salary for all employees.

✅ Solution:
bash

`awk 'BEGIN { count=0; total_salary=0 } { count++; total_salary+=$4 } END { print "Total Employees:"
`

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 17/27
Expected Output:

txt

`Total Employees: 10
Total Salary: 61300
`

💡 Recap: What You Practiced

✔ Printing fields and entire lines.
✔ Filtering data using patterns.
✔ Using and for formatted output.
✔ Applying built-in variables like , .
`BEGIN` `END`

✔ String manipulations like

`NR` `NF`
`toupper()` `substr()`.
,
✔ Conditional execution ( ).
✔ Using logical operators ( , , ).
`if-else`

✔ Aggregation and calculations.

`&&` `||` `!`

Would you like more advanced challenges? 🚀🔥

yes i would want advanced fun chanllenges

Awesome! Here are some fun and advanced `awk` challenges that will test your skills. 🚀
🔹 Challenge 1: Find the Employee with the Highest Salary
❓ Task:
Find the employee with the highest salary and display:

txt

`Highest Paid Employee: [Name] - [Salary]

`

💡 Hint: Use a variable to track the maximum salary.

✅ Try solving it first!
💡 Solution:
bash

`awk 'BEGIN { max_salary=0 } { if ($4 > max_salary) { max_salary=$4; name=$2 } } END { print "Highes
`

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 18/27
🔹 Challenge 2: Count Active and Inactive Employees
❓ Task:
Calculate and print:

txt

`Active Employees: X
Inactive Employees: Y
`

💡 Hint: Use counters for "Active" and "Inactive" employees.

✅ Solution:
bash

`awk 'BEGIN { active=0; inactive=0 } { if ($5 == "Active") active++; else inactive++ } END { print "
`

🔹 Challenge 3: Find Average Salary by Department

❓ Task:
Calculate the average salary per department and print:

txt

`HR: X
IT: Y
Sales: Z
`

💡 Hint: Use associative arrays (hash tables) to track department-wise totals.

✅ Solution:
bash

`awk '{ dept[$3] += $4; count[$3]++ } END { for (d in dept) print d ":", dept[d]/count[d] }' employe
`

🔹 Challenge 4: Sort Employees by Salary (Descending)

❓ Task:
Sort the employees by salary in descending order.
💡 Hint: Since alone doesn't sort, use
`awk` `sort -nr` for numeric sorting.

✅ Solution:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 19/27
 bash

`awk '{ print $4, $2 }' employees.txt | sort -nr

`

🔹 Expected Output: (Sorted Salaries)

txt

`7200 Carol
7000 Eve
6800 Grace
6500 Charlie
6200 Dave
6000 Alice
5800 Frank
5500 Bob
5300 Mallory
5000 John
`

🔹 Challenge 5: Mask Employee IDs for Privacy

❓ Task:
Replace employee IDs with `XXXX` while keeping all other data.
✅ Solution:
bash

`awk '{ $1="XXXX"; print }' employees.txt

`

🔹 Expected Output: (IDs hidden)

txt

`XXXX John Sales 5000 Active

XXXX Alice HR 6000 Active
XXXX Bob IT 5500 Inactive
XXXX Eve Sales 7000 Active
XXXX Charlie IT 6500 Active
XXXX Dave HR 6200 Inactive
XXXX Carol Sales 7200 Active
XXXX Frank IT 5800 Active
XXXX Grace HR 6800 Active
XXXX Mallory Sales 5300 Inactive
`

🔹 Challenge 6: Add a Bonus of 10% to Salaries

❓ Task:
Increase each employee's salary by 10% and print the new salary.
✅ Solution:
Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 20/27
 bash

`awk '{ new_salary = $4 * 1.1; print $2, "New Salary:", new_salary }' employees.txt
`

🔹 Challenge 7: Generate an Employee Email List

❓ Task:
Generate emails using a simple format:

txt

`[email protected]
[email protected]
[email protected]
`

💡 Hint: Convert names to lowercase and append `@company.com`.

✅ Solution:
bash

`awk '{ print tolower($2) "@company.com" }' employees.txt

`

🔹 Challenge 8: Create a Salary Histogram

❓ Task:
Create a visual salary distribution using stars ( ).
`*`

💡 Hint: Divide salary by 1000 to get the number of stars.

✅ Solution:
bash

`awk '{ print $2, ":", sprintf("%s", substr("**********", 1, $4/1000)) }' employees.txt
`

🔹 Expected Output:
txt

`John : *****
Alice : ******
Bob : *****
Eve : *******
Charlie : ******
Dave : ******
Carol : ********
Frank : *****
Grace : *******
Mallory : *****
`

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 21/27
🔹 Challenge 9: Identify Duplicate Departments
❓ Task:
Find departments that have more than one employee.
✅ Solution:
bash

`awk '{ dept[$3]++ } END { for (d in dept) if (dept[d] > 1) print d, "has", dept[d], "employees" }'
`

🔹 Expected Output:
txt

`Sales has 4 employees

HR has 3 employees
IT has 3 employees
`

🔹 Challenge 10: Convert Data into CSV Format

❓ Task:
Convert the file into comma-separated values (CSV format).
✅ Solution:
bash

`awk '{ print $1 "," $2 "," $3 "," $4 "," $5 }' employees.txt

`

🔹 Expected Output:
txt

`101,John,Sales,5000,Active
102,Alice,HR,6000,Active
103,Bob,IT,5500,Inactive
104,Eve,Sales,7000,Active
105,Charlie,IT,6500,Active
106,Dave,HR,6200,Inactive
107,Carol,Sales,7200,Active
108,Frank,IT,5800,Active
109,Grace,HR,6800,Active
110,Mallory,Sales,5300,Inactive
`

🔥 Bonus Challenge: Predict Salary Increase for Next 5 Years

❓ Task:
Estimate salary for the next 5 years, assuming a 5% yearly increase.

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 22/27
✅ Solution:
bash

`awk '{ print $2, "Yearly Increase:"; for (i=1; i<=5; i++) { $4 *= 1.05; print "Year", i, "->", int
`

🔹 Expected Output (for one employee):

txt

`John Yearly Increase:

Year 1 -> 5250
Year 2 -> 5512
Year 3 -> 5787
Year 4 -> 6076
Year 5 -> 6379
`

🚀 What’s Next?
🎯
These advanced challenges will push your `awk` skills to the next level!
Would you like more real-world projects using `awk`? Maybe integrating with bash scripts? 😃

yes

Awesome! Here are real-world projects where `awk` integrates with Bash scripting to solve practical
problems. 🚀
📌 Project 1: Log File Analyzer
💡 Scenario: You're a sysadmin, and you need to extract insights from an Apache web server log file
( ).
`access.log`

🔹 Example Log File ( ):

`access.log`

txt

`192.168.1.10 - - [02/Feb/2025:12:34:56 +0000] "GET /index.html HTTP/1.1" 200 1024

192.168.1.11 - - [02/Feb/2025:12:35:00 +0000] "POST /login.php HTTP/1.1" 401 512
192.168.1.12 - - [02/Feb/2025:12:35:05 +0000] "GET /dashboard HTTP/1.1" 200 2048
`

🔹 Tasks:
✅ Count the number of requests per IP address.
✅ Identify the most requested page.
✅ Find the number of failed login attempts (HTTP 401).
Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 23/27
🔹 Bash Script ( `log_analyzer.sh`) using `awk`:

bash

`#!/bin/bash

LOG_FILE="access.log"

echo " 📌Total Requests per IP:"

awk '{ip[$1]++} END {for (i in ip) print i, "-", ip[i], "requests"}' $LOG_FILE | sort -nr -k2

echo -e "\n📌 Most Requested Page:"

awk '{page[$7]++} END {for (p in page) print page[p], p}' $LOG_FILE | sort -nr | head -1

echo -e "\n📌 Failed Login Attempts (401):"

awk '$9 == 401 {count++} END {print "Total failed logins:", count}' $LOG_FILE
`

🔹 Output Example:
txt

` 📌
Total Requests per IP:
192.168.1.10 - 1 requests
192.168.1.11 - 1 requests
192.168.1.12 - 1 requests

📌 Most Requested Page:

1 /dashboard

📌 Failed Login Attempts (401):

Total failed logins: 1
`

📌 Project 2: Employee Payroll Processing

💡 Scenario: You have an employee salary file ( `salaries.csv`) and need to calculate net salaries after
tax deduction.
🔹 Example Salary File ( ):
`salaries.csv`

txt

`ID,Name,Department,Salary
101,John,Sales,5000
102,Alice,HR,6000
103,Bob,IT,5500
104,Eve,Sales,7000
105,Charlie,IT,6500
`

🔹 Tax Rules:
Salary ≤ 5500 → 10% tax
Salary > 5500 → 20% tax

🔹 Bash Script ( `payroll.sh`) using `awk`:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 24/27
 bash

`#!/bin/bash

CSV_FILE="salaries.csv"

echo " 📌 Employee Net Salary Report"

awk -F, 'NR==1 {next} { tax = ($4 <= 5500) ? $4 * 0.10 : $4 * 0.20; net_salary = $4 - tax; print $2
`

🔹 Output Example:
txt

`📌 Employee Net Salary Report

John - Net Salary: 4500
Alice - Net Salary: 4800
Bob - Net Salary: 4950
Eve - Net Salary: 5600
Charlie - Net Salary: 5200
`

📌 Project 3: Disk Space Monitoring

💡 Scenario: You need a script to check disk usage and alert when usage exceeds 80%.
🔹 Bash Script ( ) using
`disk_monitor.sh` +
`df` :
`awk`

bash

`#!/bin/bash

echo " 📌 Checking Disk Usage..."

df -h | awk 'NR>1 { if ($5+0 > 80) print "⚠️Warning:", $1, "is", $5, "full!" }'
`

🔹 Output Example (if usage is high):

txt

`📌 Checking Disk Usage...

⚠️Warning: /dev/sda1 is 85% full!
`

📌 Project 4: CSV Data Cleanup & Normalization

💡 Scenario: You have a messy CSV file ( `products.csv`) with extra spaces and inconsistent
capitalization.
🔹 Example Raw File ( ):
`products.csv`

txt

`ID, Name , Category ,Price

1, iPhone , Electronics, 1200
2, macbook, Electronics , 2500
3, T-shirt , Clothing , 25

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 25/27
 4, sneakers,Clothing , 80
`

🔹 Bash Script ( `normalize_csv.sh`) using `awk`:

bash

`#!/bin/bash

CSV_FILE="products.csv"

echo " 📌
Normalized Product Data"
awk -F, 'NR==1 { print; next } { gsub(/ /, "", $0); print toupper($2) "," $3 "," $4 }' $CSV_FILE
`

🔹 Output Example:
txt

`📌 Normalized Product Data

ID,Name,Category,Price
IPHONE,Electronics,1200
MACBOOK,Electronics,2500
T-SHIRT,Clothing,25
SNEAKERS,Clothing,80
`

📌 Project 5: Extract Top 5 Processes by Memory Usage

💡 Scenario: You need to monitor high-memory processes running on the system.
🔹 Bash Script ( ) using
`top_memory.sh` + :
`ps` `awk`

bash

`#!/bin/bash

echo " 📌
Top 5 Memory-Consuming Processes"
ps aux --sort=-%mem | awk 'NR==1 || NR<=6 {print $1, $2, $3, $4, $11}'
`

🔹 Output Example:
txt

`📌
USER
Top 5 Memory-Consuming Processes
PID %CPU %MEM COMMAND
root 1234 20.5 8.3 /usr/bin/python3
root 5678 10.2 6.7 /usr/bin/java
john 9101 5.1 4.3 /usr/bin/firefox
root 3141 3.7 3.2 /usr/bin/nginx
alice 4159 2.8 2.9 /usr/bin/mysql
`

📌 Project 6: Network Traffic Monitor

💡 Scenario: Monitor network usage and alert if a device exceeds 1000 packets sent/received.
Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 26/27
🔹 Bash Script ( `network_monitor.sh`) using `awk`:

bash

`#!/bin/bash

echo " 📌 Checking Network Usage..."

netstat -i | awk 'NR>2 && $4 > 1000 { print "⚠️Alert:", $1, "has high traffic:", $4, "packets sent"
`

🔹 Output Example:
txt

`📌 Checking Network Usage...

⚠️Alert: eth0 has high traffic: 1200 packets sent
`

🚀 What’s Next?
These projects blend `awk` with Bash to solve real-world problems.
Would you like a large-scale automation project combining `awk`, `sed`, and `bash scripting`? 😃

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 27/27