Shell Scripting for DevOps|Shell Scripting Zero 2 Hero|Shell

Scripting Interview Questions| #devops

Hello everyone. My name is Abhishek and welcome back to my channel. So today I'm here with
an exciting content that is Shell scripting tutorial.

Okay, so I have posted this a couple of weeks back or a month back on my channel that I'm
going to do a complete shell scripting course where I'll talk. It's kind of shell scripting zero to
hero, right? And today I'm here where I'll talk about the basics of shell script. We'll try to
understand what is shell script.

But we'll try to look at the shell scripting in the perspective of a DevOps engineer, right? So why
shell scripting is used by a DevOps engineer? What is the role of shell scripting in a DevOps
engineer day-to-day activities? And further we'll also try to look at some advanced as well as
intermediate level shell scripting. What are the different components that are involved? What
makes a shell scripting complicated? Or I'll give an example of a complicated shell script. So this
is what we're going to learn today.

And I'll also try to put timestamps of each and everything that we're trying to discuss. Not each
and everything but most of the things so that if you already know some of these concepts you
can look at the timestamp and you can directly jump over to that specific section of the video.
Okay, so without wasting any time, now let's start with the basics and try to understand what is
shell scripting at all.

Okay, so what is automation in your thought process? Okay, so automation is a process where
you will try to reduce your manual activities, right? Let's say you are given a manual activity. So
now to reduce this manual activity, you choose to automate this one. This is common in any
field, right? This has nothing to do with DevOps or software.

So even in your day-to-day activities, if you're doing something constantly and something that
is very tedious or something that requires a lot of your effort you will try to do automation. So
similarly, if you're doing any such thing on your Linux hosted machine or on your Linux
compute, whether it can be a Linux machine on your AWS, right? Or a Linux machine that you
install on your laptop. I mean, if you take a laptop and if you install Linux software that
becomes a Linux virtual machine.

You can use a virtualization software or you can use Oracle VirtualBox and you can create a
virtual VM. Now, what are the kind of things that we can automate on a Linux machine? So let's
take a very simple example. Somebody asked me, okay, says Abhishek, write numbers from 1 to
10.

So what I'll simply do, I'll use the echo command. I'll print the numbers from 1 to 10. Now you
might say that this is okay.

This is very simple and I've done it without any automation or without requirement of any shell
scripting. Now, what if this number increases from 10 to 1000? Still okay. So I can spend more
time and I can increase this numbers from 10 to 1000.

I can keep writing. Now, what if I increase the zeros? So I keep on adding more zeros. So now it
is technically impossible.

So in such cases or let's say somebody asked you to create 100 files on your Linux machine.
What are files? Files are very similar to your Windows. So on Windows you create a notepad or
on Windows you create a mp3 file or mp4 file.

Similarly, in Linux also you can create those files. You can create zip files. You can create tar
files.

So somebody came to me and said that Abhishek create 100 files. So I use one of the Linux
commands probably called as touch. Let's for example, so touch is a command which creates
files which we'll look into it.

So what I'll do is I'll use the touch command to create 100 files. So somebody says that okay,
not 100, 2000. So I'll take more time and further if they keep increasing this count.

So this is why you need automation or this is why you need shell scripting. So this is a very basic
or very simple example for you to understand what is shell scripting in Linux. So shell scripting
in Linux is a process of automating your day to day activities or regular activities on your Linux
computer.

And like I mentioned this can be anywhere. So irrespective of your AWS hosted Linux virtual
machine or your laptop Linux virtual machine as far as you are using the same shell. So in your
shell scripting you can define which shell to execute this script which I'll show you.

So this can be executed anywhere. Okay, so this is what we are going to learn today. So this is
the reason or this is the one of the how do I say like why do you require shell scripting is an
example that I just stated now.

So what I'll what we'll try to do is we will try to learn how to write a shell script. Okay. Now that
we understood what is shell scripting now, we need to start writing a shell script.

First of all, so once you start writing a shell script or what are the basic requirements. So like
any programming language whether you're learning Java whether you're using whether you're
learning C. So whenever you submit a set of actions, okay, so let's quickly jump onto the
terminal and let's start looking from here. So this is my Linux machine.

So I'm currently using a Mac Mac laptop. So I don't have to install anything. But if you're on
Windows, so what you need to do before this video, you need to create an account with AWS or
any cloud platform and or you can also use Oracle Virtual Box and install a virtual machine.

But to make it easy, you can go to AWS create a virtual machine and you can start following this
demo. So first of all to write a shell script. What is the basic thing that requires is you need to
have a file.

So inside your file you write a script. So how do you create file in on a Linux machine? So one of
the basic commands that works on any Linux machine is touch. So you can say touch and
probably let's say this is okay first shell script.

Okay, because I'm writing a shell script. The extension would be dot assets similar to if you're
writing a python file, it would be dot p y if you're writing Java file. So I mean, depending upon
the program language, you use the extension.

So this is how you create a file on your Linux machine. Now, as soon as I do it, now, how do I list
the files? So list the files is nothing but if you're using a Windows machine, you can simply go to
any specific folder like you have C drive, you have D drive. What you will do, you will use your
cursor, you will use your mouse, and you will go to the C drive and you will click on the button
on the C drive and it shows all the folders that are present or all the files that are present inside
your drive.

Similarly, in Linux, you cannot do all of those things if you don't have your, you know, prompt or
if you don't have, I mean, if you're simply using the command line, the command that used
here is LS. So as you do LS, so this is the file that I just created. First, shell-script.sh. So this
shows all the files.

Now, if I want to look the files with timestamp, okay, so which file is created first and which file
is created next, I can simply say LS minus LTR and it shows the files that is created, who created
the file, when did they create, what are the permissions, which group they belong to. Now, I'll
explain about each and everything in detail, but for now, you understood what is LS and what is
LS minus LTR. Now, you can ask me that I just remember the command called LS, which is used
to list the files, but I don't know the extension that you use that is minus LTR.

Okay, or you might say that, okay, I know the LS command, but I totally forgot what and why
this command is used. So simply Linux provides you an option which is very good called MAN.
MAN is nothing but manual.

So just suffix any command with MAN and simply type the command. So it gives you the details
of the command like it can be any command. Previously, we use the touch command so I can
suffix touch with MAN and once I enter, press enter, it provides the details of this command.

So what does it say? Touch, change file access and modification times. So description, touch
utility sets the modification and access times of the files. If any file does not exist, it is created
with default permissions.
Okay, so what does it say if you're using the touch command and if the file does not exist, it is
created with the default permissions. That's what it describes. So similarly, if I use the LS and if I
want to look at all the options that are provided by the LS command, like I just used minus LTR.

So what is minus T stands for if I simply type minus T here. Okay, so here it describes about the
option for minus T. Similarly, you can look at all the options that are available for this
command. Okay, for any command.

So what is the thing that we use here? We simply suffix it with the MAN command. So till now,
we just learned about three commands. Okay, so that's how your Linux is.

So within a matter of one minute, we learned three commands already. What are the
commands that we learned? First command that we learned is LS. Okay, what is LS used for? LS
is used for listing the files or folders, right files or folders or simply if you want to list a directory,
then you simply use the LS command.

And I also showed you how to create a file that is using the touch command. And we also
learned about the another new command that is MAN command. What is MAN command? MAN
command is used as a manual for any command that you want to reference.

So how do you use the MAN command? Simply suffix is use MAN in front of the command that
you want to use. Simply like MAN LS or MAN touch or any other commands that we are going to
learn. So if you are forgetting something or if you just know the command but you are not sure
what this command is used for, what is your go-to option? The manual command that is
present on your Linux machine.

So any Linux machines will have this basic commands. Okay, so these are installed by default,
right? So I have not installed any of these things. By default whenever you have a Linux
machine which comes with shell, so they have all of these binaries or all of these commands
pre-installed on the Linux machine.

Okay, so perfect. Now we just learned about the three commands. Now let me open the file.

Okay, so what I have done, I have used the touch command to create the file. I have used the LS
command to verify that the file is created. But what I want actually is to write a shell script in
this file.

So to write a shell script in this file, I have to open the file, right? So similarly, if you are on
Windows and somebody asks you to write a notepad, probably you create a notepad to list
some of the items that you want to reference in the future. So what you will do, you will right
click and you will create a file using the new file option and then you open the file in a notepad
and you start writing. Similarly, you can also use the vim command.

So vim or vi. So vim is not available by default. You have to install it.
But vi is directly available on any platform. So if you're using any Linux machine, vi is by default
available. So you can directly use vi.

So now I'll use the vi and I'll open the file that I just created. So now you might say that okay,
there are hundreds of files here. So what you will do simply use LS command.

If you don't remember the file name that you created, double click which will save the file
name. So Linux is kind of different. So you don't have to right click and use the copy option that
you use on Windows.

If you simply double click on any of the content, it gets automatically copied and you can
directly use command V or control V depending upon your operating system and that gets
pasted on your terminal. Okay. So if people are not aware how I'm using this terminal, what is
this terminal? So it's very easy.

You have multiple terminal options. So all that you need to do is go to your browser and let's
say you're on Windows. Come here and say download putty.

So putty is one of the terminal which provides you the graphical user interface. Okay. So using
putty you can basically SSH to your virtual machine and you can start using the terminal that
I'm kind of using.

I'm basically using a tool called terminal. The name is also terminal or item that is I term that is
provided by Mac. Okay.

So no without wasting time. Now let me open this file. What is the command to open the file VI?
And if you want a good user friendly interface, then I can use VIM.

So VIM is also it's not difficult to download. I'll show you how to download VIM as well. But if
you don't have VIM, don't worry, you can directly use VI.

So now the file is open. So what is the command to open the file? You can simply use VIM. Now
let's say what happens if I don't use the touch command and if I directly use the VIM command.

Okay. So let me call VIM second shell script dot sh. So any guesses what happens here? Okay.

As soon as I press enter. So the file is automatically created. It says this is a new file and you can
start writing on this file.

Okay. So even you can basically create a file using VIM command. But what is the point of using
the touch command? So you might ask me, okay, if touch and VIM are using the same thing and
VIM has advantage over touch or VA has advantage over touch.

That is you can create the file and you can write the file at the same time. Then why should I use
the touch command? So you might get this question. But touch command is basically used in
your automations.
So it is very important to learn that touch command and if it is very important to remember the
touch command because whenever you are doing automations, you cannot use the VIM
command because VIM is basically used to write inside a file. But whenever you whenever
you're doing some automations like you might have you have you might have a requirement to
create thousand files. Okay.

For some random reason. Let's forget about the reason but somebody comes to you and say
Abhishek create thousand files. Now you cannot create.

Now you cannot use VIM inside your script because VIM opens the file. Okay, it creates and also
opens the file. What happens if the file is open? Let's say you have a Windows laptop and I'll
play thousand movies at once.

What happens? Your machine gets crashed. Okay. Similarly even in Linux you cannot open a lot
of files and if a lot of files are open then it's a problem or what happens if a movie is opened
and left like that and if you try to close that process Windows says that okay this process is used
by somebody else first close that file and then only you can close this process.

Let's say you are watching a movie and you want to close the mp3 mp3 or mp4 player that you
are using. So what when you try to close the mp4 player what your Windows says okay you
cannot close the mp4 player because somebody is using the mp4 player. Similarly you cannot
keep a lot of files open.

Okay, so that is why we also use touch command. So now let me open this one. So I'll just
double click and using the VIM or VI I'll open this file and now to start writing the shell script the
very first thing that you do is use this specific indentation or use the specific syntax slash okay
hashtag followed by exclamatory followed by slash so this is called shebang.

So what is this called? This is called shebang. So what is the use case of shebang? I'll explain
and then followed by you will say slash bin slash bash or slash sh or slash ksh. What are these
things? Okay.

So now this is something very important to learn. So before I start writing a shell script, first of
all I need to explain you what is this first line that everybody writes in a shell script. So now you
go randomly to any shell script.

Okay, so let's say I will go to a GitHub. Okay, GitHub shell script examples. Okay, and I'll
randomly open any GitHub repository.

So I just opened a GitHub repository here and let me increase the font for you and let me open
this script folder. And there is a simple shell script here called addition dot sh and what is the
first line that this person wrote? Shebang user bin env bash. So everybody uses the shebang
and writes this specific syntax or effect dot sh randomly you open any of these files.

Okay. So the first line would be using the shebang and the shell that they are using now. What
is the requirement? Why you have to do that? Okay, so let me explain you this with a detailed.

I mean, what is the history of writing these things? So let me explain. Okay, so you're seeing my
board right now. Let me erase all this content here.

My band. What is happening? Yeah. So why would somebody write a shebang inside every shell
script? Okay, so what is the purpose of writing? Shebang followed by slash bin followed by any
specific thing here.

So somebody is using bash. Somebody is using dash. Somebody is using sh.

Somebody is using ksh. What are these things? So basically, if you follow the history of Linux. So
these are the different executables of your shell script.

Okay. So a Linux machine depending upon kind of the Linux machine that you're using. These
are the different executables.

So whenever you are writing a shell script and you are executing the shell script, somebody has
to take your action, right? So let's say you're running a Java Java file or you're running a Java
program. Who is executing this Java program? Who is executing this python program? Similarly,
there has to be an executable who is executing the shell script as well. So these are the
executables.

What are there? If I have to repeat one more time. There are there is bash. There is ksh.

There is sh. And similarly, there is also something called as tash. So these are the different
options.

There are other options as well. If you're using ax, there is different things. So I'm not going
into details of it, but these are some of the most used things.

So these are the different options that are available. So you have to specify. You have to inform
your Linux or your kernel that okay.

So this is the executable that I want to use for executing my shell script. So what is the
difference between each of them? So each of them have their own syntax differences. So there
is not much difference.

So there is not a drastic difference between bash and ksh and dsh. So more or less the kind of
shell script that you're writing or the kind of commands that you're writing is similar, but they
vary slightly in terms of syntax. So be very careful on what you want to use.

So now you might ask, okay Abhishek, don't confuse. What is the thing that I have to use? So
one of the popularly used or one of the most widely used is bash. So bash is something that is
most widely used.
So instead of learning all of them, because there is kind of syntax difference. For example, the
way you write a shell for loop in bash and the way you write a for loop in ksh is totally different.
So bash provides you more easy way of writing a for loop.

Whereas with ksh, it is slightly different and because it is obsolete, you don't have to learn it.
What is obsolete mean? Like nobody is using it or very less people are using ksh. So what is the
most commonly used is bash.

Now one more thing that you find here is if you look at your program, like if you look at the
shell script that are written in your organization or if you randomly look at shell script that are
used, that are provided in the GitHub. So you widely see opinion between these two things. So
most of the people are writing shebang followed by slash bin slash sh and some other people
are using shebang slash bin slash bash.

So what is the difference between them? What is the difference between sh and bash? Now
previously I explained to you what is the difference between ksh, bash and also the other things
like dsh or dash. But what is the reason why people are using slash sh also for using the bash
scripting. Even though people are using bash scripting they sometimes use sh.

So previously, if you again go back to the history of Linux so slash bin slash sh is something that
redirects your like even though they are providing slash bin slash sh so there is a concept in
Linux called linking. So there is soft link and hard link which we will not cover in this video but
there is an option called linking. So using linking even if you are providing this one this
previously it was redirecting to slash bin slash bash that means so even though you are
providing this the request is taken by sh but it is forwarded to bash.

So it is using the concept of linking. So don't worry if you see sh or bsh in your previous scripts
that were written previously in your organization. So if you see slash bin slash sh and if you
think that it is executing bash that is because of the linking concept that is provided.

But these days like over couple of years I can say or like just previously in a year what happened
is some of the operating systems like ubuntu they have started using I mean they have started
linking dash slash bin slash sh to slash bin slash dash so instead of bash they started using
dash as the default now why they are using it is out of scope of this video but you have to
understand that now you cannot use this syntax so you have to be very careful if you are
writing a bash scripting always use slash bin slash bash. So do not complicate the script or if
you like you know if you start writing this one your script might not work in some cases like in a
ubuntu machine like you share your shell script you write a bash scripting and you share your
shell scripting to one of your colleagues and inside their machine slash bin slash sh if by default
it has set a link to slash bin slash dash your script will not work. So that's why you have to be
very careful.

So always the linking for slash bin slash sh is not pointed to slash bin slash bash. So previously
it was happening but now some operating systems have decided that they want to use dash by
default and not bash by default. So be careful whenever you are writing a new shell script
always use the proper syntax that is slash bin slash bash.

So this is your interview question as well. So what is the difference between slash bin slash bash
and what is sorry slash bin slash sh and slash bin slash bash. So if somebody is asking what is
the difference you have to clearly explain that okay previously both of them were same because
slash bin slash sh was redirecting using the linking concept to slash bin slash bash but now it is
not the same because some of the operating systems have decided to use dash as default.

So your script might not execute if you are writing in bash scripting on a machine where dash is
default. Okay now we have discussed a lot about the difference and we totally understood what
is the first line here that is slash bin slash bash. Because we are learning bash scripting we will
always use the same first line.

Now I will try to keep this script very simple and the purpose of this script is to just print my
name. Now how do I print my name? So the requirement is you have to write a shell script.
What that shell script has to do is whenever somebody is executing the shell script it has to
print my name is Abhishek.

So every time people can't write write every time like you say 100 people have to write this
thing. So you are given a automation that write a small shell script that will enter I mean
whenever you execute that shell script it has to print my name is Abhishek. So to print
something if you are on Windows what you will do is you will simply go to your keyboard and
you will start writing saying that my name is Abhishek but on Linux it is different.

Whenever you are using the shell scripting there is a command called echo. Using echo or echo
what you will do is you will write my name is Abhishek. So if you are using Java you might be
using print statement.

Similarly in shell scripting we use the echo statement. That's it. Now let me save this file.

Now how do I save this file? So this is one of the challenges that a new Linux users might face.
So if you are very new to Linux first of all you have to get used to the Linux environment. So
how do you open a file? How do you close a file? Like I told you to open a file I just used vim
command but let's say I just opened this file.

Let me go back and I use the vim command and I just opened this file. By default if I enter my
keyboard so the first thing that I have to do is come to the insert mode. Okay if you see here I
am in the insert mode.

What is this thing? Why is it so complicated? So okay. So if you are using a Linux machine or a
Linux environment so there are set of things that you have to follow that once you open this file
the Linux has to know that what is the purpose of opening this file? Are you trying to read this
file? Are you trying to write inside this file? Or you are trying to do like just copy some content
from this file and paste it somewhere. So for that purpose you have to inform that I just want to
open this file and write something.

So for that you have to use the escape button and press the I. Okay once you press I it says that
okay now you are in the insert mode. So what I have to do so whenever I am opening a file I
just have to open the file press escape and click on I. So once you click on I so it goes to the
insert mode. How do you know if you are in insert mode? So to your left bottom you will see
something called insert.

So you are now in the insert mode. Now once I am in insert mode I can start writing like the one
that I just wrote slash bin slash bash. Okay so I now wrote I just wrote something into this shell
script and now I want to save this file.

To save this file again you have to press escape. Okay followed by colon you just have to give
the colon and if you see here carefully notice what I am doing here then press WQ WQ is for
saving the file and followed by the exclamatory mark and then press enter. So now your file is
saved with this content.

What happens if I don't do all of those things your whatever you wrote onto the file will not be
saved. Okay so let's say now I also have one more file here right. So second so I just open this
one let's go to the escape and press insert mode and now I'll start writing something here slash
bin slash bash and I'll say echo else just say hi okay instead of escape colon WQ what else just
say is just Q okay and I'll press the exclamatory mark what happens here is if I reopen this file
there will be nothing inside this file now why there is nothing because I did not use WQ but I
used Q Q is just like quit so you are not writing anything inside the file and you are just quitting
the file so if you want to save the file what you have to do you have to use colon WQ
exclamatory mark so this is the thing so this is how you save a file so we learnt how to open a
file we learnt how to write a file and we also learnt how to save a file so these are the different
things now you wrote this file what is inside this file let me again see so every time I can't use
the whim command and always close it right so there has to be easy option so every time I
open this file I have to press escape colon Q close I don't want to do all of these things so
simply I just want to copy this file name and I want to look at the contents of the file so for that
we can use the cat command what is cat command? cat command is used to print the content
of the file did you see? so now I don't have to open the file and close the file I can simply
without opening and closing the file print the contents of the file so that is the purpose of using
cat command so we also learnt about the cat command now how do I execute this script? ok
now I spoke a lot of things like open the file close the file without opening the file print the file
but I want to see how to execute the file itself so for that what I have to do is either I can say sh
you can simply suffix with sh and enter your file name ok I can also do this or I can simply use
dot slash so two options

This file is longer than 30 minutes.

Go Unlimited at TurboScribe.ai to transcribe files up to 10 hours long.