 Editing Files using Substrings  

Editing Files using Substrings

1 hour 30 minutes Free Rate Lab

Introduction

In this lab, you'll change the username of your coworker Jane Doe from "jane" to
"jdoe" in compliance with company's naming policy. The username change has
already been done. However, some les that were named with Jane's previous
username "jane" haven't been updated yet. To help with this, you'll write a bash script
and a Python script that will take care of the necessary rename operations.

What you'll do

Practice using the cat, grep, and cut commands for le operations
Use > and >> commands to redirect I/O stream
Replace a substring using Python
Run bash commands in Python

You'll have 90 minutes to complete this lab.

Start the lab

You'll need to start the lab before you can access the materials in the virtual machine
OS. To do this, click the green “Start Lab” button at the top of the screen.
 Editing Files using Substrings 
Note: For this lab you are going to access the Linux VM through your local SSH

Client, and not use the Google Console (Open GCP Console button is not available
for this lab).

After you click the “Start Lab” button, you will see all the SSH connection details on
the left-hand side of your screen. You should have a screen that looks like this:

Accessing the vi ual machine

Please nd one of the three relevant options below based on your device's operating
system.

Note: Working with Qwiklabs may be similar to the work you'd perform as an IT
Support Specialist; you'll be interfacing with a cutting-edge technology that requires
multiple steps to access, and perhaps healthy doses of patience and persistence(!).
You'll also be using SSH to enter the labs -- a critical skill in IT Support that you’ll be
able to practice through the labs.

Option 1: Windows Users: Connecting to your VM

 Editing Files using Substrings 
In this section, you will use the PuTTY Secure Shell (SSH) client and your VM’s
External IP address to connect. 

Download your PPK key le

You can download the VM’s private key le in the PuTTY-compatible PPK format
from the Qwiklabs Start Lab page. Click on Download PPK.

Connect to your VM using SSH and PuTTY

1. You can download Putty from here

2. In the Host Name (or IP address) box, enter username@external_ip_address.

Note: Replace username and external_ip_address with values provided in the lab.
 Editing Files using Substrings  

3. In the Category list, expand SSH.

4. Click Auth (don’t expand it).

5. In the Private key le for authentication box, browse to the PPK le that you
downloaded and double-click it.

6. Click on the Open button.

Note: PPK file is to be imported into PuTTY tool using the Browse option available in
it. It should not be opened directly but only to be used in PuTTY.
 Editing Files using Substrings  

7. Click Yes when prompted to allow a rst connection to this remote SSH server.
Because you are using a key pair for authentication, you will not be prompted for a
password.

Common issues

If PuTTY fails to connect to your Linux VM, verify that:

You entered <username>@<external ip address> in PuTTY.

You downloaded the fresh new PPK le for this lab from Qwiklabs.

You are using the downloaded PPK le in PuTTY.

Option 2: OSX and Linux users: Connecting to your VM via SSH

Download your VM’s private key le.

You can download the private key le in PEM format from the Qwiklabs Start Lab
page. Click on Download PEM.
 Editing Files using Substrings  

Connect to the VM using the local Terminal application

A terminal is a program which provides a text-based interface for typing commands.

Here you will use your terminal as an SSH client to connect with lab provided Linux
VM.

1. Open the Terminal application.

To open the terminal in Linux use the shortcut key Ctrl+Alt+t.

To open terminal in Mac (OSX) enter cmd + space and search for terminal.

2. Enter the following commands.

Note: Substitute the path/filename for the PEM file you downloaded, username and
External IP Address.

You will most likely nd the PEM le in Downloads. If you have not changed the
download settings of your system, then the path of the PEM key will be
~/Downloads/qwikLABS-XXXXX.pem

chmod 600 ~/Downloads/qwikLABS-XXXXX.pem

ssh -i ~/Downloads/qwikLABS-XXXXX.pem username@External Ip Address

 
Option 3: Chrome OS users: Connecting to your VM via SSH
Editing Files using Substrings 

Note: Make sure you are not in Incognito/Private mode while launching the
application.

Download your VM’s private key le.

You can download the private key le in PEM format from the Qwiklabs Start Lab
page. Click on Download PEM.

Connect to your VM

1. Add Secure Shell from here to your Chrome browser.

2. Open the Secure Shell app and click on [New Connection].

3. In the username section, enter the username given in the Connection Details Panel of
the lab. And for the hostname section, enter the external IP of your VM instance that
is mentioned in the Connection Details Panel of the lab.
 Editing Files using Substrings  

4. In the Identity section, import the downloaded PEM key by clicking on the Import…
button beside the eld. Choose your PEM key and click on the OPEN button.

Note: If the key is still not available after importing it, refresh the application, and
select it from the Identity drop-down menu.

5. Once your key is uploaded, click on the [ENTER] Connect button below.

6. For any prompts, type yes to continue.

7. You have now successfully connected to your Linux VM.

You're now ready to continue with the lab!

 Prerequisites
Editing Files using Substrings  
For this lab, you should have a sound knowledge of these Linux commands:

cat
grep
cut

cat:

The cat command allows us to create single or multiple les, view the contents of a
le, concatenate les, and redirect output in terminal or other les.

Syntax:

cat [file]

grep:

The grep command, which stands for "global regular expression print", processes
text line-by-line and prints any lines that match a speci ed pattern.

Syntax:

grep [pattern] [file-directory]

Here, [file-directory] is the path to the directory/folder where you want to

perform a search operation. The grep command is also used to search text and
match a string or pattern within a le.

Syntax:

grep [pattern] [file-location]

cut:

The cut command extracts a given number of characters or columns from a le. A
delimiter is a character or set of characters that separate text strings.
 Editing Files using Substrings 
Syntax:

cut [options] [file]

For delimiter separated elds, the - d option is used. The -f option speci es the eld,
a set of elds, or a range of elds to be extracted.

Syntax:

cut -d [delimiter] -f [field number]

Linux I/O Redirection

Redirection is de ned as switching standard streams of data from either a user-

speci ed source or user-speci ed destination. Here are the following streams used
in I/O redirection:

Redirection into a le using >

Append using >>

Redirection into a le

Each stream uses redirection commands. A single greater than sign (>) or a double
greater than sign (>>) can be used to redirect standard output. If the target le
doesn't exist, a new le with the same name will be created.

Commands with a single greater than sign (>) overwrite existing le content.

cat > [file]

Commands with a double greater than sign (>>) do not overwrite the existing le
content, but it will append to it.

cat >> [file]

So, rather than creating a le, the >> command is used to append a word or string to
the existing le.
 Exercise
Editing Files using Substrings  
The Scenario

Your coworker Jane Doe currently has the username "jane" but she needs to it to
"jdoe" to comply with your company's naming policy. This username change has
already been done. However, some les that were named with Jane's previous
username "jane" haven't been updated. For example, "jane_pro le_07272018.doc"
needs to be updated to "jdoe_pro le_07272018.doc".

Navigate to data directory by using the following command:

cd data

You can list the contents of the directory using the ls command. This directory
contains a le named list.txt. You will also nd some other les within this directory.

To view the contents of the le, use the following command:

cat list.txt

Output:

This le contains three columns: line number, username, and full path to the le.

You could also view the complete /data directory using the ls command.

ls
 Editing Files using Substrings 
Let's try out the commands we learned in the previous section to catch all the "jane"
lines. 

grep 'jane' ../data/list.txt

This returns all the les with the pattern "jane". It also matches the le that has string
"janez" within it.

Now, we'll list only the les containing the string "jane" and not include "janez".

grep ' jane ' ../data/list.txt

This now returns only les containing the string "jane".

Next, we'll use the cut command with grep command. For cut command, we'll use
the whitespace character (‘ ‘) as a delimiter (denoted by -d ) since the text strings
are separated by spaces within the list.txt le. We'll also fetch results by specifying
the elds using -f option.

Let's fetch the different elds (columns) using -f ag :

grep " jane " ../data/list.txt | cut -d ' ' -f 1

Output:
 Editing Files using Substrings
grep " jane " ../data/list.txt | cut -d ' ' -f 2
 
Output:

grep " jane " ../data/list.txt | cut -d ' ' -f 3

Output:

You can also return a range of elds together by using:

grep " jane " ../data/list.txt | cut -d ' ' -f 1-3

To return a set of elds together:

grep " jane " ../data/list.txt | cut -d ' ' -f 1,3

Test command

We'll now use the test command to test for the presence of a le. The command test
is a command-line utility on Unix-like operating systems that evaluates conditional
expressions.

The syntax for this command is:

test EXPRESSION
 Editing Files using Substrings 
We'll use this command to check if a particular le is present in the le system. We
 as a parameter and returns
do this by using the -e ag. This ag takes a lename
True if the le exists.

We'll check the existence of a le named jane_pro le_07272018.doc using the

following command:

if test -e ~/data/jane_profile_07272018.doc; then echo "File exists";

else echo "File doesn't exist"; fi

Output:

Create a le using a Redirection operator

We'll now use the redirection operator (>) to create an empty le simply by specifying
the le name. The syntax for this is:

> [file-name]

Let's create a le named test.txt using the redirection operator.

> test.txt

Output:

To append any string to the test.txt le, you can use another redirection operator
(>>).

echo "I am appending text to this test file" >> test.txt

 Editing Files using Substrings 
You can view the contents of the le at any time by using the cat command.

cat test.txt

Output:

Iteration

Another important aspect of a scripting language is iteration. Iteration, in simple

terms, is the repetition of a speci c set of instructions. It's when a set of instructions
is repeated a number of times or until a condition is met. And for this process, bash
script allows three different iterative statements:

For: A for loop repeats the execution of a group of statements over a set of items.
While: A while loop executes a set of instructions as long as the control condition
remains true.
Until: An until loop executes a set of instructions as long as the control condition
remains false.

Let's now iterate over a set of items and print those items.

for i in 1 2 3; do echo $i; done

Output:

Find les using bash script

 Editing Files using Substrings 
In this section, you are going to write a script named ndJane.sh within the scripts
directory. 

This script should catch all "jane" lines and store them in another text le called
oldFiles.txt. You will complete the script using the command we practiced in earlier
sections. Don't worry, we'll guide you throughout the whole process.

Navigate to /scripts directory and create a new le named ndJane.sh.

cd ~/scripts

nano findJane.sh

Now, add the shebang line.

#!/bin/bash

Create the text le oldFiles.txt and make sure it's empty. This oldFiles.txt le should
save les with username "jane".

> oldFiles.txt

Now, search for all lines that contain the name " jane " and save the le names into a
variable. Let's call this variable les, we will refer to it with that name later in the lab.

Since none of the les present in the le list.txt are available in the le system, check
if le names present in les variable are actually present in the le system. To do
this, we'll use the test command that we practiced in the previous section.

Now, iterate over the les variable and add a test expression within the loop. If the
item within the les variable passes the test, add/append it to the le oldFiles.txt.

Once you have completed writing the bash script, save the le by clicking Ctrl-o, Enter
key, and Ctrl-x.

Make the le executable using the following command:

chmod +x findJane.sh
 Editing Files using Substrings 
Run the bash script ndJane.sh.

./findJane.sh

This will generate a new le named oldFiles.txt, which consists of all the les
containing the name "jane".

Use the cat command followed by the le name to view the contents of the newly
generated le.

cat oldFiles.txt

Output:

Click Check my progress to verify the objective.

Find les using bash script

Check my progress

Rename les using Python script

In this section, you are going to write a Python script, changeJane.py, that takes
oldFiles.txt as a command line argument and then renames les with the new
username "jdoe". You will be completing the script, but we will guide throughout the
section.

Create a Python script changeJane.py under /scripts directory using nano editor.

nano changeJane.py
 Editing Files using Substrings 
Add the shebang line.

#!/usr/bin/env python3

Now, import the necessary Python module to use in the Python script.

import sys
import subprocess

The sys (System-speci c parameters and functions) module provides access to

some variables used or maintained by the interpreter and to functions that interact
with the interpreter. The subprocess module allows you to spawn new processes,
connect to their input/output/error pipes, and get their return codes.

Continue writing the script to achieve the goal!

Since oldFiles.txt is passed as a command line argument, it's stored in the variable
sys.argv[1]. Open the le from the rst argument to read its contents using open()
method. You can either assign it to a variable or use a with block. Hint: traverse each
line in the le using readlines() method. Use line.strip() to remove any whitespaces or
newlines and fetch the old name.

Once you have the old name, use replace() function to replace "jane" with "jdoe". This
method replaces occurrences of any older substring with the new substring. The old
and new substrings are passed as parameters to the function. Therefore, it returns a
string where all occurrences of the old substring is replaced with the new substring.

Syntax:

string.replace(old_substring, new_substring)

Now, invoke a subprocess by calling run() function. This function takes arguments
used to launch the process. These arguments may be a list or a string.

In this case, you should pass a list consisting of the command to be executed,
followed by arguments to the command.

Use the mv command to rename the les in the le system. This command moves a
le or directory. It takes in source le/directory and destination le/directory as
 Editing Files using Substrings 
parameters. We'll move the le with old name to the same directory but with a new
name. 

Syntax:

mv source destination

Now it must be clear. You should pass a list consisting of the mv command, followed
by the variable storing the old name and new name respectively to the run() function
within the subprocess module.

Close the le that was opened at the beginning.

f.close()

Make the le executable using the following command:

chmod +x changeJane.py

Run the script and pass the le oldFiles.txt as a command line argument.

./changeJane.py oldFiles.txt

Navigate to the /data directory and use the ls command to view renamed les.

cd ~/data

ls

Click Check my progress to verify the objective.

Rename les using Python script

 Check my progress
Editing Files using Substrings  

Congratulations!

Congrats! You've successfully renamed les containing named "jane" with "jdoe" and
we bet you're feeling much more comfortable performing operations using cat, grep,
cut, and I/O stream commands. On top of that, you now know how to use conditional
statements and iteration through bash scripts. Working with bash and Python
scripting simultaneously will be super helpful throughout your IT specialist career.
Great job completing this lab!

End your lab

When you have completed your lab, click End Lab. Qwiklabs removes the resources
you’ve used and cleans the account for you.

You will be given an opportunity to rate the lab experience. Select the applicable
number of stars, type a comment, and then click Submit.

The number of stars indicates the following:

1 star = Very dissatis ed

2 stars = Dissatis ed
3 stars = Neutral
4 stars = Satis ed
5 stars = Very satis ed

You can close the dialog box if you don't want to provide feedback.

For feedback, suggestions, or corrections, please use the Support tab.