DevOps-Notes Documentation

Release 2.0.2

Vinay Hegde

Jul 06, 2022

 Basic-Concepts

1 Overview 1
1.1 Tech Stack: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 How-To-Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Please Note: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

i
ii
 CHAPTER 1

Overview

Welcome to my notes on various topics related to the DevOps / SRE terminology such as -
• Linux Operating Systems
• Networking
• Web / Email / DNS / Database Servers
• GIT
• Configuration Management Tools (Chef, Ansible, Puppet)
• Programming (Python, Ruby, Shell Scripting)..and many more

1.1 Tech Stack:

• reStructuredText (RST) - To write .rst files for the formatting. You can start learning via this
• Sphinx - The engine generating content in HTML, LaTeX, ePub. A beginner’s guide can be found on their
page
• GitHub - For maintaining the source code in version control.
• ReadTheDocs - Hosting my documentation. Refer this excellent document to help you get up and running in
no time.

1.2 How-To-Use

• Please click any link on the Left Hand side of this webpage to read more such as Utilities, CPU, Monitoring
among others.
• Then you will find subsections (varies upon topics) expandable by clicking on the + icon to read them.

1
DevOps-Notes Documentation, Release 2.0.2

• You can find keywords in the Search Box present in the top Left Corner of this webpage to return relevant
results.

1.3 Please Note:

• Due to dynamic content for any topic in here, this will forever be a work in progress. Feedback, suggestions and
queries are always appreciated!
• For contributions, please read the guidelines on how to share Contributing to this project.
• At all times, please refer to the Code-of-Conduct guidelines on interacting with other community members in a
respectful and cordial manner.

1.3.1 Boot Process

Some useful links to explain the concepts of the Boot Process

Concepts

Overview of the Linux Boot process

• https://www.ibm.com/developerworks/library/l-linuxboot/
• https://0xax.gitbooks.io/linux-insides/content/Booting/linux-bootstrap-1.html
• http://www.slashroot.in/linux-booting-process-step-step-tutorial-understanding-linux-boot-sequence
• http://www.golinuxhub.com/2014/03/step-by-step-linux-boot-process.html?
• http://www.tecmint.com/linux-boot-process/
• https://www.digitalocean.com/community/tutorials/the-upstart-event-system-what-it-is-and-how-to-use-it
• http://linoxide.com/booting/boot-process-of-linux-in-detail/
• http://bencane.com/2013/09/16/understanding-a-little-more-about-etcprofile-and-etcbashrc/

Difference between commands for bringing down a linux server

• http://unix.stackexchange.com/questions/195898/what-is-the-difference-between-these-commands-for-bringing-down-a-linux-se

Difference between GRUB / LILO

• https://www.ibm.com/developerworks/library/l-bootload/index.html

Is a separate boot partition necessary in Linux OS?

• http://superuser.com/questions/522971/is-a-boot-partition-always-necessary

2 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

1.3. Please Note: 3

DevOps-Notes Documentation, Release 2.0.2

Commands

Configuration

• http://www.if-not-true-then-false.com/2012/delete-remove-old-kernels-on-fedora-centos-red-hat-rhel/
• https://linuxconfig.org/how-to-remove-unused-kernel-images-from-centos-linux-system
• https://markmcb.com/2013/02/04/cleanup-unused-linux-kernels-in-ubuntu/
• http://www.ostechnix.com/how-to-create-bootable-usb-drive-using-dd-command/
• https://blog.packagecloud.io/eng/2016/03/08/how-to-extract-and-disassmble-a-linux-kernel-image-vmlinuz/

Troubleshooting & Log Parsing

1.3.2 CPU

Some useful links to explain the concepts of CPU processing

Concepts

• https://www.slashroot.in/difference-between-process-and-thread-linux
• http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages

4 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

1.3. Please Note: 5

DevOps-Notes Documentation, Release 2.0.2

Types of Processes

• http://www.gmarik.info/blog/2012/orphan-vs-zombie-vs-daemon-processes/

Output of ps command - Explained

• http://superuser.com/questions/117913/ps-aux-output-meaning

• http://thelinuxstuff.blogspot.in/2012/08/process-state-codes-in-ps-output.html

6 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Commands

NoHUP &

• https://stackoverflow.com/questions/15595374/whats-the-difference-between-nohup-and-ampersand
• https://unix.stackexchange.com/questions/3886/difference-between-nohup-disown-and

1.3. Please Note: 7

DevOps-Notes Documentation, Release 2.0.2

8 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

top

• http://www.thegeekstuff.com/2010/01/15-practical-unix-linux-top-command-examples/
• http://www.slashroot.in/linux-cpu-performance-monitoring-tutorial
• http://www.tecmint.com/command-line-tools-to-monitor-linux-performance/
• http://tecadmin.net/understanding-linux-top-command-results-uses/
• http://linuxaria.com/howto/understanding-the-top-command-on-linux
• http://www.tecmint.com/12-top-command-examples-in-linux/
• http://linoxide.com/linux-command/linux-top-command-examples-screenshots/
• http://www.dowdandassociates.com/blog/content/howto-troubleshoot-with-linux-top-command/
• http://www.linuxjournal.com/magazine/hack-and-linux-troubleshooting-part-i-high-load
• http://www.crucialp.com/resources/tutorials/server-administration/server-loads-explained-linux-unix/
• https://haydenjames.io/use-atop-linux-server-performance-analysis/
• https://alvinalexander.com/linux/unix-linux-process-memory-sort-ps-command-cpu
• https://math-linux.com/linux/tip-of-the-day/article/find-out-biggest-cpu-memory-consuming-processes-with-ps-command

SAR

• http://www.thegeekstuff.com/2011/03/sar-examples/
• http://linoxide.com/linux-command/linux-system-performance-monitoring-using-sar-command/
• http://www.slashroot.in/examples-using-sar-command-system-monitoring-linux
• http://www.blackmoreops.com/2014/06/18/sysstat-sar-examples-usage/
• http://linoxide.com/linux-command/linux-pidstat-monitor-statistics-procesess/
• http://www.thegeekstuff.com/2014/11/pidstat-examples/
• http://www.yourownlinux.com/2014/08/sar-command-in-linux-tutorial-with-examples.html
• https://crybit.com/sar-command/
• http://www.cyberciti.biz/open-source/command-line-hacks/linux-monitor-process-using-pidstat/

Nice / Renice / IOnice

• https://www.faqforge.com/linux/reduce-load-of-backup-scripts-with-nice-and-ionice/
• http://www.slashroot.in/nice-and-renice-command-usage-examples-process-priority-linux

KILL

• http://www.thegeekstuff.com/2012/03/linux-signals-fundamentals/
• https://www.quora.com/What-is-the-difference-between-Kill-and-Kill-9-command-in-Unix
• http://bencane.com/2014/04/01/understanding-the-kill-command-and-how-to-terminate-processes-in-linux/
• https://rtcamp.com/tutorials/linux/kill-all-processes/

1.3. Please Note: 9

DevOps-Notes Documentation, Release 2.0.2

• http://www.tecmint.com/how-to-kill-a-process-in-linux/
• https://www.digitalocean.com/community/tutorials/how-to-use-ps-kill-and-nice-to-manage-processes-in-linux
• http://www.cyberciti.biz/faq/unix-linux-killall-command-examples-usage-syntax/
• https://unix.stackexchange.com/questions/252349/what-is-the-difference-between-kill-pkill-and-killall

HTop

Visual representation of all HTop parameters

• https://codeahoy.com/2017/01/20/hhtop-explained-visually/

10 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

HTop command - Explained

• https://peteris.rocks/blog/htop/
• http://www.servercraftmen.com/performance-tuning-htop-usage/
• http://www.webhostingbuzz.com/wiki/install-htop-linux/
• https://www.tecmint.com/install-htop-linux-process-monitoring-for-rhel-centos-fedora/
• https://delightlylinux.wordpress.com/2014/03/24/htop-a-better-process-viewer-then-top/
• http://www.thegeekstuff.com/2011/09/linux-htop-examples

SysDig

• https://sysdig.com/blog/sysdig-vs-dtrace-vs-strace-a-technical-discussion/
• https://www.sysdig.org/wiki/sysdig-user-guide/
• https://github.com/draios/sysdig/wiki/Sysdig-Examples
• https://sysdig.com/blog/linux-troubleshooting-cheatsheet/

1.3. Please Note: 11

 DevOps-Notes Documentation, Release 2.0.2

• http://bencane.com/2014/04/18/using-sysdig-to-troubleshoot-like-a-boss/
• http://xmodulo.com/monitor-troubleshoot-linux-server-sysdig.html

Configuration

Troubleshooting & Log Parsing

• Find the most number of processes running on the system

1 sudo ps -AL --no-headers | awk -F: '{print $3}' | cut -d' ' -f2 | sort | uniq -c |
˓→sort -n | tail -10

• Check for zombie processes with PPID

1 sudo ps axo stat,ppid,pid,comm | grep -w defunct

• View Column Headers in ps output

1 sudo ps aux | head -1 && sudo ps aux | grep <process-name> | grep -v grep

1.3.3 Memory

Some useful links to explain the concepts of Memory

Concepts

• http://blog.ioflood.com/2015/03/27/making-sense-of-linux-memory-usage-part-1-how-to-read-top-in-centos/
• https://www.reddit.com/r/linux/comments/1hk5ow/free_buffer_swap_dirty_procmeminfo_explained/

Commands

Free Command - Explained

• http://corlewsolutions.com/articles/article-6-understanding-the-free-command-in-ubuntu-and-linux
• http://www.linuxatemyram.com/
• http://www.cloudibee.com/know-about-procmeminfo/
• http://www.linuxnix.com/find-ram-details-in-linuxunix/

Other Commands

• http://www.thegeekstuff.com/2010/08/ipcs-command-examples/

12 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• http://www.tecmint.com/dstat-monitor-linux-server-performance-process-memory-network/

Configuration

• http://github.com/pixelb/scripts/commits/master/scripts/ps_mem.py

Create a RAM Disk

• https://www.jamescoyle.net/how-to/943-create-a-ram-disk-in-linux

Troubleshooting & Log Parsing

1.3.4 Swap

Some useful links to explain the concepts of Swap

Concepts

• https://www.linux.com/news/all-about-linux-swap-space
• http://www.linux-tutorial.info/modules.php?name=MContent&pageid=311
• http://aarvik.dk/what-is-swap-memory-and-how-to-use-it/
• http://blog.scoutapp.com/articles/2015/04/10/understanding-page-faults-and-memory-swap-in-outs-when-should-you-worry

Commands

• Check total swap space used & sort it descending

1 for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}'
˓→$file; done | sort -k 2 -nr | head -10
2 for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}'
˓→$file; done | awk '{print $1 " " $2/1024 " MB" }'|sort -k 2 -n -r | head -10

• Alternatively - run below command

1.3. Please Note: 13

 DevOps-Notes Documentation, Release 2.0.2

1 sudo nice top (Press Shift+o → p (To sort processes by swap usage)

Configuration

• http://www.cyberciti.biz/faq/linux-add-a-swap-file-howto/
• http://bencane.com/2016/05/18/creating-a-swap-file-for-tiny-cloud-servers/
• https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-centos-7

Troubleshooting & Log Parsing

• http://northernmost.org/blog/find-out-what-is-using-your-swap/
• http://www.digitalinternals.com/unix/linux-swap-usage-per-process/379/
• http://www.cyberciti.biz/faq/linux-which-process-is-using-swap/
• https://unix.stackexchange.com/questions/294600/i-cant-enable-swap-space-on-centos-7

1.3.5 Storage

Some useful links to explain the concepts of Storage, I/O

Concepts

• http://linuxbsdos.com/2011/09/18/guide-to-disks-and-disk-partitions-in-linux/
• http://www.howtogeek.com/howto/35676/how-to-choose-a-partition-scheme-for-your-linux-pc/
• http://www.slashroot.in/linux-system-io-monitoring
• http://www.cyberciti.biz/tips/linux-disk-performance-monitoring-howto.html

14 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• http://xmodulo.com/how-to-monitor-disk-io-in-linux.html
• http://www.cyberciti.biz/tips/freebsd-why-command-df-and-du-reports-different-output.html
• http://yoonusp.blogspot.in/2012/06/du-and-ls-output-show-different-sizes.html
• http://www.howtogeek.com/howto/38125/htg-explains-what-is-the-linux-fstab-and-how-does-it-work/
• http://linoxide.com/file-system/understanding-each-entry-of-linux-fstab-etcfstab-file/
• http://geek-university.com/linux/etc-fstab-file/

GRUB

• https://linuxnorth.wordpress.com/2011/03/09/grub2-revisited/
• http://askubuntu.com/questions/2793/how-do-i-remove-or-hide-old-kernel-versions-to-clean-up-the-boot-menu
• http://askubuntu.com/questions/218286/how-can-i-safely-remove-extra-grub-installs
• http://docs.fedoraproject.org/en-US/Fedora/20/html/Installation_Guide/sn-medialess-editing-grub-conf.html
• https://ask.fedoraproject.org/en/question/32771/multiple-fedora-entries-in-grub/

LVM

• http://www.tecmint.com/create-lvm-storage-in-linux/
• http://www.howtogeek.com/howto/40702/how-to-manage-and-use-lvm-logical-volume-management-in-ubuntu/
• http://www.cyberciti.biz/tips/fdisk-unable-to-create-partition-greater-2tb.html
• https://www.atlantic.net/community/howto/configure-2tb-partitioning-parted-mdadm/
• http://www.tecmint.com/parted-command-to-create-resize-rescue-linux-disk-partitions/
• http://www.thegeekstuff.com/2012/08/2tb-gtp-parted/
• http://www.thegeekstuff.com/2011/09/parted-command-examples/
• http://magicmonster.com/kb/os/linux/large_hdd.html
• http://www.cyberciti.biz/tips/fdisk-unable-to-create-partition-greater-2tb.html

RAID

• http://www.slashroot.in/raid-levels-raid0-raid1-raid10-raid5-raid6-complete-tutorial
• http://www.tecmint.com/understanding-raid-setup-in-linux/
• http://www.thegeekstuff.com/2010/08/raid-levels-tutorial/
• http://www.thegeekstuff.com/2011/11/raid2-raid3-raid4-raid6/
• http://www.thegeekstuff.com/2011/10/raid10-vs-raid01/
• https://www.pugetsystems.com/labs/articles/RAID-Explained-24/
• http://www.tldp.org/HOWTO/Software-RAID-HOWTO-11.html
• http://www.gagme.com/greg/linux/raid-lvm.php
• https://wiki.archlinux.org/index.php/Software_RAID_and_LVM
• https://www.cyberciti.biz/tips/raid-hardware-vs-raid-software.html

1.3. Please Note: 15

DevOps-Notes Documentation, Release 2.0.2

• http://www.olearycomputers.com/ll/linux_mirrors.html
• https://raid.wiki.kernel.org/index.php/Linux_Raid
• http://cavepopo.hd.free.fr/wordpress/linux/how-to-create-a-raid-array-using-omconfig-omreport-cli/

GlusterFS

• https://www.interserver.net/tips/kb/what-is-glusterfs/
• http://www.slashroot.in/gfs-gluster-file-system-complete-tutorial-guide-for-an-administrator
• http://www.tecmint.com/introduction-to-glusterfs-file-system-and-installation-on-rhelcentos-and-fedora/
• https://www.digitalocean.com/community/tutorials/how-to-create-a-redundant-storage-pool-using-glusterfs-on-ubuntu-servers
• https://gluster.readthedocs.io/en/latest/Quick-Start-Guide/Quickstart/

NFS

• http://cavepopo.hd.free.fr/wordpress/linux/about-nfs/
• http://cavepopo.hd.free.fr/wordpress/network/nfs-the-network-file-system-practical-examples/
• http://www.cyberciti.biz/faq/mhddfs-linux-combines-a-several-mount-points-into-single-one/
• https://romanrm.net/mhddfs
• http://www.tecmint.com/how-to-setup-nfs-server-in-linux/

SCP

• http://www.tecmint.com/scp-commands-examples/
• http://www.binarytides.com/linux-scp-command/
• http://www.garron.me/en/linux/scp-linux-mac-command-windows-copy-files-over-ssh.html

Rsync

• http://www.server-world.info/en/note?os=CentOS_6&p=rsync
• https://www.linode.com/docs/tools-reference/tools/introduction-to-rsync
• http://www.evbackup.com/support-commonly-used-rsync-arguments/
• http://www.tecmint.com/rsync-local-remote-file-synchronization-commands/
• http://www.thegeekstuff.com/2010/09/rsync-command-examples/
• http://www.howtogeek.com/175008/the-non-beginners-guide-to-syncing-data-with-rsync/
• http://www.techonthenet.com/linux/commands/rsync.php
• https://unix.stackexchange.com/questions/4712/rsync-two-file-types-in-one-command/4713#4713
• http://www.crucialp.com/resources/tutorials/server-administration/how-to-copy-files-across-a-network-internet-in-unix-linux-red
• https://stackoverflow.com/questions/20244585/how-does-scp-differ-from-rsync/20257021#20257021

16 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• https://unix.stackexchange.com/questions/4712/rsync-two-file-types-in-one-command/4713#4713

LsyncD

• http://www.nginxtips.com/lsyncd-live-file-syncronization-linux/
• http://www.rackspace.com/knowledge_center/article/install-and-configure-lsyncd
• http://crosstek.net/2013/06/28/two-way-sync-with-lsyncd-in-a-clustered-wordpress-installation/

Explanation of lsyncd.conf parameters

• https://www.lucasrolff.com/ha/replication-using-lsyncd/
• https://github.com/axkibe/lsyncd/wiki/Manual%20to%20Lsyncd%202.1.x

1.3. Please Note: 17

DevOps-Notes Documentation, Release 2.0.2

Max User Watches Error of Lsyncd

• https://xopo3o.blogspot.in/2015/11/how-to-lsyncd-error-consider-increasing.html
• https://unix.stackexchange.com/questions/13751/kernel-inotify-watch-limit-reached

18 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

1.3. Please Note: 19

DevOps-Notes Documentation, Release 2.0.2

Configuration

• http://askubuntu.com/questions/109856/how-to-mount-a-windows-folder-in-linux
• https://access.redhat.com/solutions/448263
• http://www.tecmint.com/fdisk-commands-to-manage-linux-disk-partitions/
• http://www.thegeekstuff.com/2010/09/linux-fdisk/
• https://wiki.ubuntu.com/MountWindowsSharesPermanently
• http://www.cyberciti.biz/faq/linux-finding-using-uuids-to-update-fstab/
• https://codesilence.wordpress.com/2013/03/14/live-resizing-of-an-ext4-filesytem-on-linux/
• https://www.tecmint.com/parted-command-to-create-resize-rescue-linux-disk-partitions/

Creating & mounting a 2nd Hard drive in linux

• http://geektnt.com/how-to-format-and-mount-second-hard-drive-on-linux.html

Good explanation fstab parameters

• http://ubuntuforums.org/showthread.php?t=1604251

20 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Troubleshooting & Log Parsing

• http://www.tecmint.com/linux-performance-monitoring-with-vmstat-and-iostat-commands/
• http://linoxide.com/linux-command/linux-iostat-command/
• http://www.thegeekstuff.com/2011/07/iostat-vmstat-mpstat-examples/
• https://www.linode.com/docs/uptime/monitoring/use-vmstat-to-monitor-system-performance

1.3. Please Note: 21

DevOps-Notes Documentation, Release 2.0.2

• https://www.thomas-krenn.com/en/wiki/Linux_Performance_Measurements_using_vmstat
• https://www.thomas-krenn.com/en/wiki/SMART_tests_with_smartctl
• https://www.cyberciti.biz/hardware/linux-iotop-simple-top-like-io-monitor/
• http://articles.slicehost.com/2010/11/12/using-iotop-to-check-i-o-and-swap

Online Visual RAID Calculator

• https://www.synology.com/en-global/support/RAID_calculator
• http://grijpink.eu/tools/raid/index.php

1.3.6 File Systems

Some useful links to explain the concepts of File Systems

Concepts

• https://www.nixtutor.com/linux/understanding-the-linux-directory-layout/
• https://www.slashroot.in/understanding-file-system-superblock-linux
• https://unix.stackexchange.com/questions/4402/what-is-a-superblock-inode-dentry-and-a-file
• http://www.cyberciti.biz/faq/understanding-unix-linux-bsd-device-files/
• http://askubuntu.com/questions/397493/what-does-the-first-character-of-unix-mode-string-indicate

ext(2|3|4)

• http://www.thegeekstuff.com/2011/05/ext2-ext3-ext4/
• https://adarsh5388.wordpress.com/2013/11/29/difference-between-ext2-ext3-and-ext4-filesystem-in-linux/
• https://kerneltalks.com/disk-management/difference-between-ext2-ext3-and-ext4/amp/
• https://linoxide.com/how-tos/explained-in-detail-linux-ext2-ext3-and-ext4-filesystem/amp/

File Descriptors

• https://linuxmeerkat.wordpress.com/2011/12/02/file-descriptors-explained/amp/
• https://www.bottomupcs.com/file_descriptors.xhtml
• http://www.cyberciti.biz/tips/linux-procfs-file-descriptors.html

Proc File Systems

• http://www.slashroot.in/proc-file-system-linux-explained
• http://www.thegeekstuff.com/2010/11/linux-proc-file-system/
• https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/3/html/Reference_Guide/
s1-proc-directories.html

22 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

1.3. Please Note: 23

DevOps-Notes Documentation, Release 2.0.2

ULimit

• https://easyengine.io/tutorials/linux/increase-open-files-limit/
• http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/
• http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
• http://askubuntu.com/questions/181215/too-many-open-files-how-to-find-the-culprit

Directory Structures

• http://askubuntu.com/questions/34880/use-of-opt-and-usr-local-directories-in-the-context-of-a-pc
• http://www.ghacks.net/2009/04/04/get-to-know-linux-the-etcinitd-directory/
• http://askubuntu.com/questions/19320/how-to-enable-or-disable-services

Inodes

• http://www.slashroot.in/inode-and-its-structure-linux
• https://tecadmin.net/what-is-inode-number-in-linux/
• http://www.grymoire.com/Unix/Inodes.html
• http://teaching.idallen.com/dat2330/04f/notes/links_and_inodes.html

Commands

S-Trace command

• https://blog.packagecloud.io/eng/2016/02/29/how-does-strace-work/
• http://linoxide.com/linux-command/linux-strace-command-examples/
• http://hokstad.com/5-simple-ways-to-troubleshoot-using-strace
• http://www.cyberciti.biz/tips/linux-strace-command-examples.html
• https://blog.packagecloud.io/eng/2015/11/15/strace-cheat-sheet/
• https://www.ibm.com/developerworks/aix/library/au-unix-strace.html

1.3.7 Provisioning

Some useful links to explain the concepts of Cobbler / Kickstart

• https://www.ibm.com/developerworks/library/l-cobbler/
• http://cobbler.github.io/manuals/quickstart/
• https://cobbler.readthedocs.io/en/latest/installation-guide.html
• http://cavepopo.hd.free.fr/wordpress/linux/how-to-basic-pxe-infrastructure-3-tftp-server-installation-and-setup/
• http://www.networkworld.com/article/2224075/opensource-subnet/set-up-a-local-linux-installation-and-update-server-with-kick
html

24 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• https://marclop.svbtle.com/creating-an-automated-centos-7-install-via-kickstart-file
• https://dark.ca/2009/08/03/complex-partitioning-in-kickstart/
• https://help.ubuntu.com/community/Cobbler/Preseed
• https://thornelabs.net/2014/12/19/problems-provisioning-ubuntu-with-cobbler-and-kickstart-profiles.html
• https://awaseconfigurations.wordpress.com/2011/09/29/cobbler-set-up-network-installation-of-ubuntu-11-04/

1.3.8 Package-Management

Some useful links to explain the concepts of Package-Management

Concepts

• https://www.digitalocean.com/community/tutorials/package-management-basics-apt-yum-dnf-pkg
• https://blog.packagecloud.io/eng/2015/10/26/use-alien-to-convert-deb-to-rpm-and-rpm-to-deb/

Difference between yum update / yum upgrade

RPM

• https://rpmbuildtut.wordpress.com/development-packages/
• http://www.tldp.org/HOWTO/RPM-HOWTO/build.html

1.3. Please Note: 25

DevOps-Notes Documentation, Release 2.0.2

• https://fedoraproject.org/wiki/How_to_create_an_RPM_package
• http://www.tecmint.com/20-practical-examples-of-rpm-commands-in-linux/

YUM

• http://www.slashroot.in/yum-repository-and-package-management-complete-tutorial
• http://www.tecmint.com/20-linux-yum-yellowdog-updater-modified-commands-for-package-mangement/
• https://blog.packagecloud.io/eng/2015/04/05/yum-cheat-sheet/
• https://www.digitalocean.com/community/tutorials/how-to-set-up-and-use-yum-repositories-on-a-centos-6-vps
• http://www.rpm.org/max-rpm-snapshot/

DNF

• https://dnf.readthedocs.io/en/latest/index.html
• http://linoxide.com/linux-how-to/dnf-commands-manage-rpm-linux/

Configuration

• http://www.2daygeek.com/install-enable-rpm-fusion-repository-on-centos-fedora-rhel/
• http://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/
• http://www.cyberciti.biz/tips/rhel5-fedora-core-add-new-yum-repository.html
• http://sharadchhetri.com/2014/02/22/yum-command-to-download-rpm-file-without-installing-in-linux-system/
• http://www.linuxtechi.com/download-rpm-using-yumdownloader-centos-7-rhel-7/
• https://blog.packagecloud.io/eng/2015/10/13/inspect-extract-contents-rpm-packages/

Rollback or Revert YUM Changes

• https://www.if-not-true-then-false.com/2010/yum-history-list-info-summary-repeat-redo-undo-new/
• http://www.itechlounge.net/2012/08/linux-how-to-rollback-yum-updates-on-rhel-centos/
• http://www.cyberciti.biz/faq/howto-yum-downgrade-packages-on-rhel-centos-fedora-scientific-linux/

Downloading packages for RPM based OS

• https://pkgs.org/

Delta-RPMs

• https://unix.stackexchange.com/questions/277900/do-i-need-to-do-something-about-delta-rpms-disabled
• https://www.certdepot.net/rhel7-get-started-delta-rpms/

26 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Troubleshooting & Log Parsing

• https://major.io/2007/05/27/rpmdb-lock-table-is-out-of-available-locker-entries/
• http://www.serveradminblog.com/2010/10/yum-problem-rpmdb-lock-table-is-out-of-available-locker-entries/
• http://www.if-not-true-then-false.com/2012/delete-remove-old-kernels-on-fedora-centos-red-hat-rhel/
• http://xmodulo.com/check-rpm-package-dependencies-fedora-centos-rhel.html
• http://xmodulo.com/how-to-fix-yum-errors-on-centos-rhel-or-fedora.html
• https://blog.packagecloud.io/eng/2015/04/20/working-with-source-rpms/

APT

• https://askubuntu.com/questions/18804/what-do-the-various-dpkg-flags-like-ii-rc-mean
• https://unix.stackexchange.com/questions/6284/check-package-version-using-apt-get-aptitude
• https://blog.packagecloud.io/eng/2015/03/30/apt-cheat-sheet/
• https://help.ubuntu.com/community/Repositories/CommandLine
• https://www.tecmint.com/useful-basic-commands-of-apt-get-and-apt-cache-for-package-management/

Working With Packages

• https://askubuntu.com/questions/44122/how-to-upgrade-a-single-package-using-apt-get
• https://askubuntu.com/questions/80655/how-can-i-check-dependency-list-for-a-deb-package
• https://askubuntu.com/questions/17823/how-to-list-all-installed-packages

Configuration

• http://www.tecmint.com/apt-advanced-package-command-examples-in-ubuntu/
• http://www.cyberciti.biz/faq/debian-ubuntu-linux-show-detailed-description-info/

1.3. Please Note: 27

DevOps-Notes Documentation, Release 2.0.2

• https://blog.packagecloud.io/eng/2015/10/13/inspect-extract-contents-debian-packages/

Troubleshooting & Log Parsing

Is apt-get autoremove safe

• https://askubuntu.com/questions/393212/is-it-safe-to-use-the-command-apt-get-autoremove-in-this-particular-scenario/
393215

Difference between apt-get update / apt-get upgrade

• https://askubuntu.com/questions/94102/what-is-the-difference-between-apt-get-update-and-upgrade

28 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

1.3.9 Hardware

Some useful links to explain the concepts of Hardware (Dell / SuperMicro)

• http://www.thegeekstuff.com/2008/11/how-to-get-hardware-information-on-linux-using-dmidecode-command/
• http://www.tecmint.com/how-to-get-hardware-information-with-dmidecode-command-on-linux/
• http://linoxide.com/linux-command/how-to-display-system-hardware-information-in-bios/
• http://www.thegeekstuff.com/2014/04/lspci-examples
• https://linuxconfig.org/getting-know-a-hardware-on-your-linux-box
• https://blog.mattbrock.co.uk/monitoring-perc-raid-controllers-and-storage-arrays-on-dell-poweredge-servers-with-debian-and-u

Dell OMSA

Concepts

• http://cavepopo.hd.free.fr/wordpress/linux/dell-server-utility-omreport/
• http://public.support.unisys.com/pcproducts/esx/docs/delldocs5.4/en/dosa/storageug/cli.html
• https://stuff.mit.edu/afs/athena/dept/cron/documentation/OldFiles/Manuals/dell-server-admin/en/Dosa/CLI/
storage.htm
• https://discuss.zendesk.com/hc/en-us/articles/201831218-Useful-omreport-commands-for-DCA-V1
• https://cs.uwaterloo.ca/~brecht/servers/docs/PowerEdge-2600/en/Dosa/CLI/cli_cc5s.htm

Commands

• We use 2 commands to monitor / change parameters in Dell servers

1 omreport - Checks the server details via specified parameters.

2

3 omconfig - Modifies the server details via specified parameters.

1.3. Please Note: 29

 DevOps-Notes Documentation, Release 2.0.2

Examples :

• Will list all possibly available system / chassis / storage domain commands

1 sudo omreport system -? | omreport chassis -? | omreport storage -?

• Retrieve general system information

1 sudo omreport system summary | less

• Display the Hardware logs

1 sudo omreport system esmlog

• Retrieve the RAID configuration

1 sudo omreport storage vdisk controller=0

• Clearing the logs

1 sudo omconfig system esmlog action=clear (Replace esmlog with alertlog or cmdlog,
˓→esmlog is the hardware log)

Configuration

• http://cavepopo.hd.free.fr/wordpress/linux/how-to-create-a-raid-array-using-omconfig-omreport-cli/
• http://cavepopo.hd.free.fr/wordpress/linux/how-to-expand-a-raid-array-using-omconfig-omreport-cli/

IPMITool

• https://discuss.zendesk.com/hc/en-us/articles/206396927-How-to-work-on-IPMI-and-IPMITOOL

MegaCLI

• https://artipc10.vub.ac.be/wordpress/2011/09/12/megacli-useful-commands/
• https://things.maths.cam.ac.uk/computing/docs/public/megacli_raid_lsi.html

1.3.10 SSH

Some useful links to explain the concepts of SSH

Concepts

• http://cavepopo.hd.free.fr/wordpress/linux/about-ssh-the-secure-shell/
• https://www.hostinger.com/tutorials/ssh-tutorial-how-does-ssh-work
• https://www.slashroot.in/secure-shell-how-does-ssh-work
• https://www.ibm.com/developerworks/aix/library/au-sshsecurity/index.html
• https://www.ibm.com/developerworks/library/l-keyc/index.html

30 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• https://www.ssh.com/ssh/port

Commands

Configuration

SSH key-based logins

• https://www.digitalocean.com/community/tutorials/ssh-essentials-working-with-ssh-servers-clients-and-keys
• https://www.digitalocean.com/community/tutorials/how-to-configure-ssh-key-based-authentication-on-a-freebsd-server
• https://www.linode.com/docs/security/use-public-key-authentication-with-ssh
• http://blog.nowherelan.com/2014/01/04/two-step-authentication-for-ssh-on-centos-6-using-google-authenticator/
• https://www.linux.com/blog/securing-ssh-two-factor-authentication-using-google-authenticator
• http://www.thegeekstuff.com/2008/11/3-steps-to-perform-ssh-login-without-password-using-ssh-keygen-ssh-copy-id/
• https://www.digitalocean.com/community/tutorials/how-to-configure-custom-connection-options-for-your-ssh-client
• https://www.ssh.com/ssh/config/
• https://stackoverflow.com/questions/10197559/ssh-configuration-override-the-default-username/10197697#
10197697

Tuning & Hardening

• https://superuser.com/questions/718346/openssh-ssh-config-host-specific-overrides-not-working
• https://www.digitalocean.com/community/tutorial_series/how-to-troubleshoot-ssh
• https://www.digitalocean.com/community/tutorials/how-to-tune-your-ssh-daemon-configuration-on-a-linux-vps
• https://stackoverflow.com/questions/10310299/proper-way-to-sudo-over-ssh
• https://security.stackexchange.com/questions/150540/is-it-completely-safe-to-publish-an-ssh-public-key
• https://hackerific.net/2017/04/23/ssh-audit—a-tool-for-checking-ssh-server-security/
• https://superuser.com/questions/868998/how-can-i-find-a-list-of-macs-ciphers-and-kexalgorithms-that-my-openssh-client

1.3. Please Note: 31

DevOps-Notes Documentation, Release 2.0.2

Correct permissions of .ssh directory

• https://superuser.com/questions/215504/permissions-on-private-key-in-ssh-folder

32 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Run same commands via SSH on multiple servers

• https://www.linux.com/news/parallel-ssh-execution-and-single-shell-control-them-all
• http://serverfault.com/questions/321167/add-correct-host-key-in-known-hosts-multiple-ssh-host-keys-per-hostname
• http://unix.stackexchange.com/questions/19008/automatically-run-commands-over-ssh-on-many-servers
• http://www.cyberciti.biz/tips/execute-commands-on-multiple-linux-or-unix-servers.html
• http://unix.stackexchange.com/questions/107800/using-while-loop-to-ssh-to-multiple-servers
• http://stackoverflow.com/questions/20254906/bash-script-to-ssh-multiple-servers-in-a-loop-and-issue-commands
• http://unix.stackexchange.com/questions/182101/script-to-ssh-to-multiple-linux-server-and-execute-a-find-command
• http://cavepopo.hd.free.fr/wordpress/linux/ssh-the-secure-shell-practical-examples/
• http://thornelabs.net/2013/08/21/simple-ways-to-send-multiple-line-commands-over-ssh.html
• http://nerderati.com/2011/03/17/simplify-your-life-with-an-ssh-config-file/
• http://aarvik.dk/ssh-fundamentals-cssh-and-fabric/
• http://aarvik.dk/how-to-create-socks-proxy-through-ssh/

TMux

Some useful links to cover the working of TMux.

1.3. Please Note: 33

DevOps-Notes Documentation, Release 2.0.2

• https://danielmiessler.com/study/tmux/
• https://tmuxcheatsheet.com/
• http://lukaszwrobel.pl/blog/tmux-tutorial-split-terminal-windows-easily
• https://robots.thoughtbot.com/a-tmux-crash-course
• http://blog.hawkhost.com/2010/06/28/tmux-the-terminal-multiplexer/
• http://www.sitepoint.com/tmux-a-simple-start/
• http://fideloper.com/mac-vim-tmux
• http://blog.victorquinn.com/tmux-tutorial
• https://www.codementor.io/tmux/tutorial/beginners-guide-to-tmux-navigating-and-configuring-your-tmux
• https://marc.cortinasval.cat/blog/2013/11/29/tmux-the-best-choice/
• http://www.hamvocke.com/blog/a-guide-to-customizing-your-tmux-conf/
• https://github.com/rothgar/awesome-tmux

Troubleshooting & Log Parsing

1.3.11 Editors

Some useful links to explain the concepts of Editors

VIM Basics

• http://vim.rtorr.com/
• https://danielmiessler.com/study/vim/
• https://vim.swaroopch.com/

Customizing VIM

VIM Color Schemes

• https://vimcolors.com
• https://vimawesome.com/
• https://github.com/rafi/awesome-vim-colorschemes
• https://github.com/flazz/vim-colorschemes

Making your own .vimrc

• https://dougblack.io/words/a-good-vimrc.html

34 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

1.3. Please Note: 35

DevOps-Notes Documentation, Release 2.0.2

VI Editor Cheat Sheet

36 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

To delete all lines in vim

SED && AWK

• http://www.yourownlinux.com/2015/04/sed-command-in-linux-delete-lines-from-file.html

Converting New-Lines to Spaces

• http://unix.stackexchange.com/questions/26788/using-sed-to-convert-newlines-into-spaces
• https://stackoverflow.com/questions/13610639/tr-command-how-to-replace-the-string-n-with-an-actual-newline-n
• https://stackoverflow.com/questions/1251999/how-can-i-replace-a-newline-n-using-sed
• https://stackoverflow.com/questions/19151954/how-to-use-variables-in-a-command-in-sed

AWK Notes

• http://www.theunixschool.com/2012/06/awk-10-examples-to-group-data-in-csv-or.html
• https://stackoverflow.com/questions/15758814/turning-multiple-lines-into-one-line-with-comma-separated-perl-sed-awk
• https://stackoverflow.com/questions/8714355/bash-turning-multi-line-string-into-single-comma-separated
• http://linoxide.com/linux-command/awk-linux-famous-oneliners/
• http://www.linuxnix.com/category/programming/awk/

Markdown

Syntax

• https://www.markdownguide.org/basic-syntax

1.3. Please Note: 37

DevOps-Notes Documentation, Release 2.0.2

• https://paperhive.org/help/markdown

Online Editors

• https://stackedit.io/
• https://dillinger.io/
• https://typora.io
• https://markable.in
• https://hackmd.io

Atom

Configuration

• https://stackoverflow.com/questions/30006827/how-to-save-atom-editor-config-and-list-of-packages-installed
• https://discuss.atom.io/t/how-to-backup-all-your-settings/15674

1.3.12 Utilities

Environment Variables {ENV}

Some useful links to explain the concepts of Environment Variables

• https://www.guru99.com/linux-environment-variables.html
• https://www.digitalocean.com/community/tutorials/how-to-read-and-set-environmental-and-shell-variables-on-a-linux-vps
• https://www.slashroot.in/difference-between-bashrc-and-bashprofile
• https://www.cyberciti.biz/faq/linux-list-all-environment-variables-env-command/

Crontab

Some useful links to cover the working of Crontab

• http://www.thegeekstuff.com/2009/06/15-practical-crontab-examples/
• http://www.cyberciti.biz/faq/linux-show-what-cron-jobs-are-setup/
• https://www.pantz.org/software/cron/croninfo.html

38 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• http://corntab.com/crontab_syntax

1.3. Please Note: 39

DevOps-Notes Documentation, Release 2.0.2

40 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Online Utilities to generate crons

• http://cronchecker.net/
• http://crontab-generator.org/
• http://www.cronmaker.com/
• http://htmlminifiers.com/cron-maker.php
• http://blog.endpoint.com/2008/12/best-practices-for-cron.html

GREP

Some useful links to cover the working of Grep

• https://www.linode.com/docs/tools-reference/search-and-filter-text-with-grep
• https://danielmiessler.com/study/grep/
• http://www.cyberciti.biz/faq/searching-multiple-words-string-using-grep/
• http://www.thegeekstuff.com/2011/10/grep-or-and-not-operators/
• http://xmodulo.com/how-to-grep-multiple-terms-or-strings.html
• https://www.digitalocean.com/community/tutorials/using-grep-regular-expressions-to-search-for-text-patterns-in-linux
• https://alvinalexander.com/unix/edu/examples/grep.shtml
• https://stackoverflow.com/questions/2427913/how-can-i-grep-for-a-string-that-begins-with-a-dash-hyphen
• https://stackoverflow.com/questions/221921/use-grep-exclude-include-syntax-to-not-grep-through-certain-files
• https://stackoverflow.com/questions/25853722/how-to-suppress-binary-file-matching-results-in-grep

Grepping ps output without showing the grep process

• https://www.safaribooksonline.com/library/view/bash-cookbook/0596526784/ch17s18.html
• https://unix.stackexchange.com/questions/74185/how-can-i-prevent-grep-from-showing-up-in-ps-results

1.3. Please Note: 41

DevOps-Notes Documentation, Release 2.0.2

FIND

Some useful links to cover the working of Find

• https://danielmiessler.com/study/find/
• https://www.digitalocean.com/community/tutorials/how-to-use-find-and-locate-to-search-for-files-on-a-linux-vps
• http://tecadmin.net/delete-files-older-x-days/
• http://www.tecmint.com/find-and-sort-files-modification-date-and-time-in-linux/
• https://www.linode.com/docs/tools-reference/tools/find-files-in-linux-using-the-command-line
• https://alvinalexander.com/blog/post/linux-unix/find-how-multiple-search-patterns-filename-command
• https://stackoverflow.com/questions/5475905/linux-delete-file-with-size-0/18052644#18052644

User Management

• https://www.linode.com/docs/tools-reference/linux-users-and-groups
• https://linux-audit.com/unused-linux-users-delete-or-keep/
• https://askubuntu.com/questions/515103/how-can-i-display-all-users-and-groups-with-a-command

42 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• https://askubuntu.com/questions/34074/how-do-i-change-my-username

1.3. Please Note: 43

DevOps-Notes Documentation, Release 2.0.2

44 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

File Permissions

Some useful links to cover the working of File Permissions

• https://www.linux.com/learn/getting-know-linux-file-permissions
• http://www.penguintutor.com/linux/file-permissions-reference
• https://www.linux.com/learn/understanding-linux-file-permissions
• http://www.linuxnix.com/chmod-command-explained-linuxunix/
• https://unix.stackexchange.com/questions/21251/execute-vs-read-bit-how-do-directory-permissions-in-linux-work
• http://www.grymoire.com/Unix/Permissions.html
• http://www.thegeekstuff.com/2010/06/chmod-command-examples/
• https://danielmiessler.com/study/unixlinux_permissions/
• https://www.linux.com/learn/how-easily-back-and-restore-linux-file-permissions
• https://unix.stackexchange.com/questions/102286/ls-gives-no-such-file-or-directory-message

Why sudo cd <path-to-dir> doesn’t work

• https://superuser.com/questions/241129/why-wont-sudo-cd-work

1.3. Please Note: 45

DevOps-Notes Documentation, Release 2.0.2

• https://askubuntu.com/questions/291666/why-doesnt-sudo-cd-var-named-work

46 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

1.3. Please Note: 47

DevOps-Notes Documentation, Release 2.0.2

• https://askubuntu.com/questions/468901/how-to-show-only-hidden-files-in-terminal/468902#468902

Sudoers

• http://cavepopo.hd.free.fr/wordpress/linux/sudo-command-sudoers-file-concepts-and-practical-examples/
• http://serverfault.com/questions/364334/test-whether-a-user-has-sudo-privileges-without-requiring-user-input
• https://www.digitalocean.com/community/tutorials/how-to-add-delete-and-grant-sudo-privileges-to-users-on-a-debian-vps
• http://www.thegeekstuff.com/2010/09/sudo-command-examples/
• https://www.garron.me/en/linux/visudo-command-sudoers-file-sudo-default-editor.html
• https://www.digitalocean.com/community/tutorials/how-to-edit-the-sudoers-file-on-ubuntu-and-centos
• https://askubuntu.com/questions/73864/how-to-modify-an-invalid-etc-sudoers-file

48 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• https://askubuntu.com/questions/799669/etc-sudoers-file-corrupted-and-i-cant-run-pkexec-visudo-over-ssh

1.3. Please Note: 49

DevOps-Notes Documentation, Release 2.0.2

UserMod Command

• http://www.tecmint.com/usermod-command-examples/
• http://linoxide.com/linux-command/linux-usermod-command-to-modify-user-details/
• http://crybit.com/15-switches-of-usermod-command-with-example-unixlinux/
• https://muffinresearch.co.uk/linux-changing-uids-and-gids-for-user/
• http://www.htpcbeginner.com/safely-change-primary-group-group-in-linux/
• http://www.cyberciti.biz/faq/howto-linux-add-user-to-group/

SetUID and SetGID

• http://www.tutonics.com/2012/12/linux-file-permissions-chmod-umask.html
• https://www.slashroot.in/suid-and-sgid-linux-explained-examples
• https://www.thegeekstuff.com/2013/02/sticky-bit/
• http://www.linuxnix.com/suid-set-suid-linuxunix/
• https://unix.stackexchange.com/questions/28363/whats-the-difference-between-s-and-s-in-ls-la

UMask Values

• → UMask contains default permissions for newly created files / directories based on base permissions for those

50 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• → Base permissions for files is 666 (read-write permissions for owner-group-other : execute permissions are
excluded by for files as security measure)
• → Similarly, the base permissions for directories are 777 (read-write-execute permissions for owner-group-
other)
• → The UMask values are inverse to regular file permissions → i.e: chmod
• → UMask values = Base permissions - required permissions All UMasks
• https://www.computerhope.com/unix/uumask.htm
• http://www.cyberciti.biz/tips/understanding-linux-unix-umask-value-usage.html
• https://www.digitalocean.com/community/tutorials/linux-permissions-basics-and-how-to-use-umask-on-a-vps

SymLinks

Some useful links to cover the working of SymLinks

• http://www.cyberciti.biz/tips/understanding-unixlinux-symbolic-soft-and-hard-links.html
• http://www.thegeekstuff.com/2010/10/linux-ln-command-examples/
• http://bencane.com/2013/10/10/symlinks-vs-hardlinks-and-how-to-create-them/
• http://geek-university.com/linux/symbolic-links/
• http://geek-university.com/linux/hard-links/

Others

XArgs vs Exec

• http://unix.worldiswelcome.com/what-is-the-difference-between-exec-and-xargs
• https://danielmiessler.com/blog/linux-xargs-vs-exec/
• http://www.differencebetween.co.in/technology/difference-between-xargs-and-exec/
• https://stackoverflow.com/questions/16758525/make-xargs-handle-filenames-that-contain-spaces

1.3. Please Note: 51

DevOps-Notes Documentation, Release 2.0.2

• https://askubuntu.com/questions/666001/piping-find-name-to-xargs-results-in-filenames-with-spaces-not-being-passed-to/
666004

52 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

NTP

• http://support.ntp.org/bin/view/Support/TroubleshootingNTP
• https://support.rackspace.com/how-to/using-ntp-to-sync-time/
• https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.cmds4/ntpdate.htm

Renaming / Deleting multiple files

• http://www.cyberciti.biz/tips/renaming-multiple-files-at-a-shell-prompt.html
• http://tips.webdesign10.com/how-to-bulk-rename-files-in-linux-in-the-terminal
• https://www.linux.com/blog/linux-shell-tip-remove-files-names-contains-spaces-and-special-characters-such
• http://www.cyberciti.biz/faq/linux-bash-delete-all-files-in-directory-except-few/
• https://www.tecmint.com/delete-all-files-in-directory-except-one-few-file-extensions/

1.3. Please Note: 53

DevOps-Notes Documentation, Release 2.0.2

• https://askubuntu.com/questions/470134/how-to-find-the-creation-time-of-a-file
• https://www.if-not-true-then-false.com/2011/linux-display-show-file-contents-tabs-line-breaks-non-printing-characters/

Rename multiple files with different extensions

• https://superuser.com/questions/865826/linux-mv-command-for-moving-multiple-files-with-different-extensions

Command Prompt Tweaks

• http://computers.tutsplus.com/tutorials/speed-up-your-terminal-workflow-with-command-aliases-and-
profile–mac-30515
• http://blog.taylormcgann.com/2012/06/13/customize-your-shell-command-prompt/

Random Tools

• http://aarvik.dk/blacklist-check-unix-linux-utility/
• http://www.tecmint.com/progress-monitor-check-progress-of-linux-commands/
• http://www.tecmint.com/screen-command-examples-to-manage-linux-terminals/
• http://www.thegeekstuff.com/2009/04/chage-linux-password-expiration-and-aging/
• https://danielmiessler.com/study/tar/
• https://danielmiessler.com/blog/collection-of-less-commonly-used-unix-commands/

54 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• http://www.commandlinefu.com/commands/browse
• https://distrochooser.de/en
• https://www.cyberciti.biz/open-source/command-line-hacks/compgen-linux-command/
• https://www.dynacont.net/documentation/linux/Useful_SystemD_commands/

SysCTL

Some useful links to cover the working of SysCTL

• https://rtcamp.com/tutorials/linux/sysctl-conf
• https://www.cyberciti.biz/faq/linux-kernel-etcsysctl-conf-security-hardening
• https://www.slashroot.in/linux-network-tcp-performance-tuning-sysctl
• https://www.linux.com/news/kernel-tuning-sysctl

LSOF

• https://danielmiessler.com/study/lsof/
• http://www.catonmat.net/blog/unix-utilities-lsof/
• http://www.thegeekstuff.com/2012/08/lsof-command-examples/
• https://www.ibm.com/developerworks/aix/library/au-lsof.html
• https://unix.stackexchange.com/questions/253321/how-to-display-size-human-readable-in-lsof-grep-var

DIFF

• https://www.lifewire.com/compare-two-text-files-linux-3861434

Comparing difference between files / directories on 2 servers

• http://xmodulo.com/how-to-diff-remote-files-over-ssh.html
• http://zuhaiblog.com/2011/02/14/using-diff-to-compare-folders-over-ssh-on-two-different-servers/

1.3.13 Fundamentals

Some useful links to explain the concepts of Web-Servers

Concepts

• http://computer.howstuffworks.com/web-server2.htm
• https://serversforhackers.com/hosting-web-applications
• https://danielmiessler.com/study/web-performance/
• https://launchschool.com/books/http/read/introduction
• https://github.com/alex/what-happens-when

1.3. Please Note: 55

DevOps-Notes Documentation, Release 2.0.2

• https://howhttps.works/
• https://dev.to/pratikaambani/explain-http-verbs-like-im-five-b10
• http://stackoverflow.com/questions/224664/difference-between-proxy-server-and-reverse-proxy-server
• https://dev.to/swyx/every-web-performance-test-tool-naj

Load Balancing

• https://blog.envoyproxy.io/introduction-to-modern-network-load-balancing-and-proxying-a57f6ff80236
• https://avinetworks.com/what-is-load-balancing
• https://levelup.gitconnected.com/l4-vs-l7-load-balancing-d2012e271f56

HTTP request lifecycle overview

• https://dev.to/dangolant/things-i-brushed-up-on-this-week-the-http-request-lifecycle-

Difference between http 443 and https 80

• http://serverfault.com/questions/705197/http-over-port-443-vs-https-over-port-80

56 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

1.3. Please Note: 57

DevOps-Notes Documentation, Release 2.0.2

HTTP status codes

• http://www.restapitutorial.com/httpstatuscodes.html
• https://httpstatuses.com/
• https://stackoverflow.com/questions/50143518/401-unauthorized-vs-403-forbidden-which-is-the-right-status-code-for-when-the
• https://stackoverflow.com/questions/3297048/403-forbidden-vs-401-unauthorized-http-responses
• https://geekflare.com/http-status-code-infographics/

58 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

1.3. Please Note: 59

DevOps-Notes Documentation, Release 2.0.2

Check website-availability issues - Tools

• https://www.site24x7.com/tools.html
• https://geopeeker.com/
• https://httpstatus.io/

cURL

• http://www.thegeekstuff.com/2012/04/curl-examples/
• http://www.slashroot.in/curl-command-tutorial-linux-example-usage
• https://curl.haxx.se/docs/httpscripting.html
• https://ec.haxx.se/curl.html
• http://www.codediesel.com/tools/6-essential-curl-commands/
• https://robots.thoughtbot.com/back-to-basics-http-requests
• https://blog.josephscott.org/2011/10/14/timing-details-with-curl/
• https://stackoverflow.com/questions/46362284/run-multiple-curl-commands-in-parallel
• http://http-prompt.com/

wGET

• http://www.thegeekstuff.com/2009/09/the-ultimate-wget-download-guide-with-15-awesome-examples/
• http://www.tecmint.com/10-wget-command-examples-in-linux/
• http://www.labnol.org/software/wget-command-examples/28750/
• http://www.linuxtechi.com/wget-command-practical-examples/
• https://alvinalexander.com/linux-unix/how-to-make-offline-mirror-copy-website-with-wget

1.3.14 Apache

Some useful links to explain the concepts of Apache

Concepts

• http://code.tutsplus.com/tutorials/an-introduction-to-apache–net-25786
• http://fideloper.com/quick-caching-explanation
• https://anturis.com/blog/get-insight-into-your-website-performance-with-key-apache-statistics/
• http://middlewaretechnologies.blogspot.in/2013/04/have-you-ever-tried-installing-apache.html

60 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Information about MPM (Multi Processing Modules)

• https://prakash-khadka.com.np/apache-mpm/
• http://articles.slicehost.com/2010/12/3/configuring-the-apache-mpm-on-gentoo
• https://serverfault.com/questions/383526/how-do-i-select-which-apache-mpm-to-use

1.3. Please Note: 61

DevOps-Notes Documentation, Release 2.0.2

62 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Configuration

• http://fideloper.com/ubuntu-12-04-lamp-server-setup
• https://www.linode.com/docs/websites/apache/apache-web-server-on-centos-6
• https://www.linode.com/docs/websites/lamp/lamp-on-centos-6/
• http://www.erikwebb.net/blog/compile-and-install-apache-24-red-hat-enterprise-linux-rhel-6-or-centos-6/
• https://www.digitalocean.com/community/tutorials/how-to-install-linux-apache-mysql-php-lamp-stack-on-centos-6
• http://shapeshed.com/domain_forwarding_in_apache/
• https://www.linode.com/docs/websites/apache-tips-and-tricks/redirect-urls-with-the-apache-web-server
• http://www.tecmint.com/creating-your-own-webserver-and-hosting-a-website-from-your-linux-box/
• https://serversforhackers.com/video/php-fpm-configuration-the-listen-directive
• https://coderwall.com/p/hmsr5a/have-php-fpm-listen-on-unix-socket

Virtual Hosting

• http://www.thegeekstuff.com/2011/07/apache-virtual-host/
• https://support.rackspace.com/how-to/how-to-serve-multiple-domains-using-virtual-hosts/
• https://mobile.serverwatch.com/tutorials/article.php/1127571/Apache-Guide-Setting-Up-Virtual-Hosts.htm
• https://serversforhackers.com/configuring-apache-virtual-hosts
• http://www.thegeekstuff.com/2011/07/apache-virtual-host/
• https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-virtual-hosts-on-centos-6
• https://www.digitalocean.com/community/tutorials/how-to-create-a-ssl-certificate-on-apache-for-centos-6
• http://aarvik.dk/good-apache-virtualhost/

HTaccess checks and guides

• http://www.htaccesscheck.com/
• http://htaccess.madewithlove.be/
• http://www.askapache.com/htaccess/htaccess.html
• http://socreativedigital.com/7-htaccess-file-examples-that-work-for-seo-2013-05-04

Tuning & Hardening

Apache Server Status Page

• http://www.tecmint.com/monitor-apache-web-server-load-and-page-statistics/
• http://www.coscale.com/blog/apache-troubleshooting-and-monitoring
• https://anturis.com/blog/get-insight-into-your-website-performance-with-key-apache-statistics/
• https://blog.serverdensity.com/monitor-apache/

1.3. Please Note: 63

DevOps-Notes Documentation, Release 2.0.2

Other Optimizations

• http://www.tecmint.com/apache-performance-tuning/
• http://www.ubuntufree.com/how-to-optimize-apache/
• https://www.maketecheasier.com/series/apache-server-guide/
• http://linuxbsdos.com/2015/02/17/how-to-reduce-php-fpm-php5-fpm-ram-usage-by-about-50/
• http://blog.chrismeller.com/configuring-and-optimizing-php-fpm-and-nginx-on-ubuntu-or-debian
• https://serversforhackers.com/video/php-fpm-process-management
• http://linuxconfig.org/apache-web-server-ssl-authentication
• https://geekflare.com/apache-web-server-hardening-security/
• https://geekflare.com/category/web-infrastructure/apache/
• http://www.acunetix.com/blog/articles/10-tips-secure-apache-installation/
• http://www.tecmint.com/apache-security-tips/
• http://www.thegeekstuff.com/2011/03/apache-hardening/
• https://haydenjames.io/strip-apache-improve-performance-memory-efficiency/
• http://www.serverlab.ca/tutorials/linux/web-servers-linux/configuring-selinux-policies-for-apache-web-servers/
• http://geekflare.com/apache-web-server-hardening-security/

Troubleshooting & Log Parsing

Find Original IPs of sites using CDNs like Cloudflare

• https://censys.io/

Levels of traffic with Apache access log

• http://www.inmotionhosting.com/support/website/server-usage/view-level-of-traffic-with-apache-access-log
• http://www.inmotionhosting.com/support/website/website-troubleshooting/determine-cause-of-server-usage-spike
• http://www.cyberciti.biz/faq/apache-logs/
• http://www.serverwatch.com/tutorials/article.php/1127521/Apache-Guide-Logging-with-ApacheUnderstanding-Your-accesslog.
htm
• http://list.xmodulo.com/web-server-benchmarking-tools-linux.html
• http://stackoverflow.com/questions/9234699/understanding-apache-access-log
• http://logz.io/blog/apache-log-analyzer/
• http://www.the-art-of-web.com/system/logs/

64 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Commands

• Find hits by IP to server from access log in ascending order

1 sudo tail -n 10000 <path-to-log-file> | awk '{print $2}' | sort | uniq -c | sort -n
2 sudo grep 'text' <path-to-access-log> | cut -d' ' -f1 | sort | uniq -c | sort -r

• Finding connections to all server IPs source/destination & sorting in ascending order

1 sudo netstat -antulp | awk '{print $4}' | cut -d":" -f1 | sort | uniq -c | sort -n
2 sudo netstat -antulp | awk '{print $5}' | cut -d":" -f1 | sort | uniq -c | sort -n

1.3.15 Nginx

Some useful links to explain the concepts of NGinx

Concepts

• https://www.digitalocean.com/community/tutorials/understanding-nginx-server-and-location-block-selection-algorithms
• https://www.nginx.com/resources/wiki/start/
• https://www.digitalocean.com/community/tutorials/understanding-the-nginx-configuration-file-structure-and-configuration-conte
• https://www.linode.com/docs/websites/nginx/how-to-configure-nginx
• https://www.digitalocean.com/community/tutorials/understanding-nginx-http-proxying-load-balancing-buffering-and-caching
• https://www.atulhost.com/nginx

Configuration

• http://fideloper.com/ubuntu-12-04-lemp-nginx-setup
• https://blog.serverdensity.com/monitor-nginx/
• http://devdocs.io/nginx/
• http://nginx.org/en/docs/
• https://rtcamp.com/tutorials/nginx/
• http://articles.slicehost.com/nginx
• https://rtcamp.com/wordpress-nginx/tutorials/
• https://geekflare.com/category/web-infrastructure/nginx/
• https://geekflare.com/nginx-webserver-security-hardening-guide/
• https://www.digitalocean.com/community/tutorials/how-to-create-temporary-and-permanent-redirects-with-apache-and-nginx
• https://www.nginx.com/blog/creating-nginx-rewrite-rules/
• https://www.godaddy.com/garage/tech/config/how-to-install-and-configure-nginx-on-fedora/

1.3. Please Note: 65

DevOps-Notes Documentation, Release 2.0.2

Nginx with HTTPS using SSL via LetsEncrypt

• https://shawnliu.me/post/using-lets-encrypt-for-nginx-on-centos-7/
• https://www.e2enetworks.com/help/knowledge-base/free-ssl-for-ngnix-on-centos-by-lets-encrypt/
• https://www.imagescape.com/blog/2017/11/27/free-and-auto-renewing-ssl-certificates-letsencrypt-quick-setup-2017-edition/

Visual Editor for NGinx Config

• https://nginxconfig.io

Tuning & Hardening

• http://linuxbsdos.com/2015/02/17/how-to-reduce-php-fpm-php5-fpm-ram-usage-by-about-50/
• https://www.nginx.com/blog/10-tips-for-10x-application-performance/
• http://www.tokiwinter.com/building-a-highly-available-load-balancer-with-nginx-and-keepalived-on-centos/
• http://www.cyberciti.biz/faq/rhel-centos-fedora-keepalived-lvs-cluster-configuration/
• https://lincolnloop.com/blog/rate-limiting-nginx/
• https://www.nginx.com/blog/benefits-of-microcaching-nginx/
• https://anturis.com/blog/nginx-vs-apache/
• http://www.thegeekstuff.com/2013/11/nginx-vs-apache/
• https://www.fissionblue.com/blog/mask-domain-entire-site-using-nginx-ssl/
• https://www.digitalocean.com/community/tutorials/apache-vs-nginx-practical-considerations
• https://www.digitalocean.com/community/tutorials/how-to-optimize-nginx-configuration

Troubleshooting & Log Parsing

• http://logz.io/blog/nginx-log-analysis/
• http://logz.io/blog/nginx-access-log-monitoring-dashboard/

1.3.16 HAProxy

Some useful links to explain the concepts of HAProxy

Concepts

Official Documentation

• https://cbonte.github.io/haproxy-dconv/
• https://linuxacademy.com/howtoguides/posts/show/topic/12012-introduction-to-haproxy
• https://www.digitalocean.com/community/tutorials/an-introduction-to-haproxy-and-load-balancing-concepts

66 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Configuration

• https://serversforhackers.com/using-ssl-certificates-with-haproxy
• https://cbonte.github.io/haproxy-dconv/configuration-1.5.html
• https://coelhorjc.wordpress.com/2015/03/26/how-to-load-balance-an-http-server-using-with-haproxy-or-pound/
• http://blog.haproxy.com/haproxy/haproxy-and-ssl/
• https://www.digitalocean.com/community/tutorials/how-to-implement-ssl-termination-with-haproxy-on-ubuntu-14-04
• http://ubtutorials.com/tutorial/436/how-implement-ssl-termination-haproxy-ubuntu-1404
• https://www.howtoforge.com/high-availability-load-balancer-haproxy-heartbeat-debian-etch
• http://support.severalnines.com/entries/23612682-Install-HAProxy-and-Keepalived-Virtual-IP-
• http://behindtheracks.com/2014/04/redundant-load-balancers-haproxy-and-keepalived/
• https://serversforhackers.com/load-balancing-with-haproxy
• http://www.linuxnix.com/heartbeat-clustering/
• https://www.digitalocean.com/community/tutorials/how-to-create-a-high-availability-setup-with-heartbeat-and-floating-ips-on-u
• https://www.digitalocean.com/community/tutorials/how-to-create-a-high-availability-setup-with-corosync-pacemaker-and-floati
• https://marc.cortinasval.cat/blog/2013/12/04/a-cheap-web-balancer-nginxhaproxypacemaker/
• http://support.severalnines.com/entries/23612682-Install-HAProxy-and-Keepalived-Virtual-IP-
• https://www.howtoforge.com/setting-up-a-high-availability-load-balancer-with-haproxy-keepalived-on-debian-lenny
• https://www.freecodecamp.org/news/how-we-fine-tuned-haproxy-to-achieve-2-000-000-concurrent-ssl-connections-d017e61a4d
• https://delta.blue/blog/haproxy-timeouts

Tuning & Hardening

SNI on HAProxy

• http://stuff-things.net/2016/11/30/haproxy-sni/
• http://www.networkinghowtos.com/howto/reload-haproxy-config-with-minimal-downtime/
• http://blog.haproxy.com/2012/04/13/enhanced-ssl-load-balancing-with-server-name-indication-sni-tls-extension/
• https://medium.com/@siddharth.d/securing-haproxy-and-nginx-via-http-headers-54020d460283

Troubleshooting & Log Parsing

• https://www.linux.com/blog/how-analyze-haproxy-logs
• http://logz.io/blog/monitor-haproxy-elk-stack/
• https://github.com/gforcada/haproxy_log_analysis
• https://serversforhackers.com/so-you-got-yourself-a-loadbalancer
• http://www.networkinghowtos.com/howto/viewing-haproxy-statistics/
• https://www.datadoghq.com/blog/monitoring-haproxy-performance-metrics/
• https://www.datadoghq.com/blog/how-to-collect-haproxy-metrics/

1.3. Please Note: 67

DevOps-Notes Documentation, Release 2.0.2

1.3.17 SSL

Some useful links to explain the concepts of SSL

Concepts

• https://tls.ulfheim.net
• https://www.digitalocean.com/community/tutorials/a-comparison-of-let-s-encrypt-commercial-and-private-certificate-authorities
• https://scotthelme.co.uk/https-cheat-sheet/
• https://www.digicert.com/ssl.htm

68 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• http://grahamc.com/blog/openssl-madness-how-to-create-keys-certificate-signing-requests-authorities-and-pem-files/
• https://www.digicert.com/csr-creation.htm
• http://swift.siphos.be/aglara/certificates.html
• https://www.openssl.org/docs/manmaster/apps/verify.html
• http://www.openssl.org/docs/manmaster/apps/x509.html
• http://www.slashroot.in/understanding-working-secure-socket-layerssl
• http://www.slashroot.in/understanding-ssl-handshake-protocol

1.3. Please Note: 69

DevOps-Notes Documentation, Release 2.0.2

FAQs on SSL

• https://timnash.co.uk/guessing-ssl-questions/
• http://www.martfox.com/customer/knowledgebase/140/Why-a-SSL-Requires-Dedicated-IP.html
• https://dzone.com/articles/introduction-to-ssl-for-managers
• https://www.nutsandboltsmedia.com/does-your-website-really-need-ssl/
• https://www.slashroot.in/how-does-ssltls-chain-certificates-and-its-validation-work
• https://blog-cloudflare-com.cdn.ampproject.org/c/s/blog.cloudflare.com/rfc-8446-aka-tls-1-3/amp/
• https://serverfault.com/questions/9708/what-is-a-pem-file-and-how-does-it-differ-from-other-openssl-generated-key-file
• https://www.troyhunt.com/life-is-about-to-get-harder-for-websites-without-https/
• https://www.troyhunt.com/on-the-perceived-value-ev-certs-cas-phishing-lets-encrypt/
• https://www.troyhunt.com/extended-validation-certificates-are-dead/

Server Name Indication

• https://devcentral.f5.com/articles/ssl-profiles-part-7-server-name-indication
• http://wiki.apache.org/httpd/NameBasedSSLVHostsWithSNI
• https://www.digicert.com/ssl-support/apache-multiple-ssl-certificates-using-sni.htm

Configuration

Basics of OpenSSL Commands for CSR, Keys & Certs

• https://www.digitalocean.com/community/tutorials/openssl-essentials-working-with-ssl-certificates-private-keys-and-csrs

Wildcard SSL on sub-domain

• http://stackoverflow.com/questions/2115611/wildcard-ssl-on-sub-subdomain
• http://serverfault.com/questions/566426/does-each-subdomain-need-its-own-ssl-certificate
• http://serverfault.com/questions/104160/wildcard-ssl-certificate-for-second-level-subdomain

Switching from HTTP to HTTPs

• https://www.smashingmagazine.com/2017/06/guide-switching-http-https/
• http://searchengineland.com/http-https-seos-guide-securing-website-246940
• https://yoast.com/moving-your-website-to-https-ssl-tips-tricks/

Creating SAN SSL certificate

• https://geekflare.com/san-ssl-certificate/

70 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Tuning & Hardening

• http://heartbleed.com/
• http://www.troyhunt.com/2014/04/everything-you-need-to-know-about.html
• https://www.yahoo.com/tech/heres-what-you-need-to-know-about-the-heartbleed-bug-82120054478.html
• http://thehackernews.com/2014/04/heartbleed-bug-explained-10-most.html
• http://kb.odin.com/en/118918
• https://security.stackexchange.com/questions/8210/what-vulnerabilities-could-be-caused-by-a-wildcard-ssl-cert
• http://www.jamescoyle.net/how-to/1073-bash-script-to-create-an-ssl-certificate-key-and-request-csr
• https://rtcamp.com/wordpress-nginx/tutorials/ssl/multidomain-ssl-subject-alternative-names/

Hardening Your Web Server’s SSL Ciphers

• https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
• https://cipherli.st/
• https://mozilla.github.io/server-side-tls/ssl-config-generator/

Troubleshooting & Log Parsing

• https://www.sslshopper.com/ssl-certificate-tools.html
• https://cheapsslsecurity.com/ssltools/
• http://geekflare.com/ssl-test-certificate/
• https://serversforhackers.com/self-signed-ssl-certificates

Commands

• https://www.sslshopper.com/article-most-common-openssl-commands.html
• http://www.shellhacks.com/en/HowTo-Check-SSL-Certificate-Expiration-Date-from-the-Linux-Shell
• https://cryptoreport.websecurity.symantec.com/checker/
• https://www.digicert.com/help/

Free SSL Certificates : LetsEncrypt

• https://www.digitalocean.com/community/tutorials/an-introduction-to-let-s-encrypt
• https://geekflare.com/free-ssl-tls-certificate/
• https://serversforhackers.com/video/letsencrypt-for-free-easy-ssl-certificates
• https://letsencrypt.org/
• https://digitz.org/blog/lets-encrypt-ssl-centos-7-setup/
• https://certbot.eff.org/lets-encrypt/centosrhel7-nginx.html

1.3. Please Note: 71

DevOps-Notes Documentation, Release 2.0.2

1.3.18 Monitoring

Some useful links to explain the concepts of Monitoring

72 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Concepts

1.3. Please Note: 73

DevOps-Notes Documentation, Release 2.0.2

• https://www.datadoghq.com/blog/monitoring-101-alerting/
• https://blog.serverdensity.com/what-ive-learnt-from-using-ansible-exclusively-for-2-years/
• https://blog.serverdensity.com/80-linux-monitoring-tools-know/
• https://serversforhackers.com/monitoring-processes-with-supervisord
• http://supervisord.org/

Nagios

• http://nagios.sourceforge.net/docs/nagioscore/4/en/toc.html
• http://users.telenet.be/mydotcom/howto/nagios/index.html
• http://www.linuxquestions.org/questions/linux-newbie-8/nagios-internal-server-error-903798/

What Nagios does

• Monitoring of network services (SMTP, POP3, HTTP, NNTP, ICMP, SNMP, FTP, SSH)
• Monitoring of host resources (processor load, disk usage, system logs) on a majority of network operating
systems, including Microsoft Windows with the NSClient++ plugin or Check MK.
• Monitoring via remotely run scripts via Nagios Remote Plugin Executor or through SSH or SSL encrypted
tunnels.
• Contact notifications when service or host problems occur & get resolved (via e-mail, pager, SMS, or any user-
defined method through plugin system)
• The ability to define event handlers to be run during service or host events for proactive problem resolution.
Automatic log file rotation. Support for implementing redundant monitoring hosts
• An optional web-interface for viewing current network status, notifications, problem history, log files, etc. Data
storage via text files rather than database

Configuration

Host / Service Monitoring

• http://www.opensourceforu.com/2011/07/nagios-setup-guide/
• http://www.productionmonkeys.net/guides/nagios/nagios-server-installation/advanced-configuration/
service-and-servicegroups

Deploying SSH Checks For Nagios

• http://tecadmin.net/monitor-remote-linux-system-nagios-via-ssh/
• http://www.techrepublic.com/blog/linux-and-open-source/remotely-monitor-servers-with-the-nagios-check-by-ssh-plugin/
• http://www.opensourceforu.com/2011/07/nagios-setup-guide/
• http://tuxradar.com/content/nagios-made-easy
• https://geekpeek.net/nagios-plugin-bash/

74 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Check HTTP

• http://linux.101hacks.com/unix/check-http/

1.3.19 Graphing

Some useful links to explain the concepts of Graphing

Concepts

Cacti
• http://www.cacti.net/index.php
Metrics
• http://blog.4aiur.net/2012/01/installing-and-configuring-graphite-on-centos/
• http://kaivanov.blogspot.in/2014/07/metrics-visualisation-and-collection.html

Ganglia

• http://www.slashroot.in/introduction-ganglia-monitoring-and-graphing-tool
• https://www.digitalocean.com/community/tutorials/introduction-to-ganglia-on-ubuntu-14-04
• https://sachinsharm.wordpress.com/2013/08/17/setup-and-configure-ganglia-3-6-on-centosrhel-6-3/

What Ganglia does

• Graph different properties of a server such as CPU,memory,load,etc

• Compare the graphing trend of those properties with previous trend & identify which node or host is causing
the issue easily from the trend.
• Make custom metrics for graphing for different process.
• Machines from different data centers which are part of one single cluster must be represented in that single
cluster in a single interface.

Important points

• Node : SINGLE machine sending data to Ganglia monitoring daemon. (All individual servers are nodes, can
or can’t be part of a cluster)
• Cluster : All nodes that are used for any particular purpose is a CLUSTER.
• Grid : Collection of clusters is a GRID.

1.3. Please Note: 75

DevOps-Notes Documentation, Release 2.0.2

Parts of Ganglia Monitoring Tool

• 1. Gmond :
– Ganglia Monitoring daemon (Service that needs to be installed on each & every node that needs to be
monitored)
– Sends data via XML over TCP & main configuration file : /etc/gmond.conf
• 2. Gmetad :
– Collects data from Gmond daemons & stores in RRD (Round robin database)
– Main configuration file is /etc/gmetad.conf & should be installed on one node of each cluster
• 3. RRD tool :
– Used by Ganglia to store data for visualization (graphing) & store data of particular time intervals & then
graphs the same.
• 4. PHP Front-End :
– A web interface on the master node that displays graphs and metrics from data in the RRD tool.

Configuration

• http://linuxdrops.com/install-ganglia-monitoring-system-on-centos-rhel/
• https://sachinsharm.wordpress.com/2013/08/17/setup-and-configure-ganglia-3-6-on-centosrhel-6-3/
• https://sachinsharm.wordpress.com/2013/08/19/setup-and-configure-ganglia-python-modules-on-centosrhel-6-3/
• http://a4amittripathi.blogspot.in/2014/01/how-to-configure-and-install-ganglia-in.html
• http://www.ibm.com/developerworks/library/l-ganglia-nagios-1/

Troubleshooting & Log Parsing

• https://ahmadchaudary.wordpress.com/tag/ganglia-monitoring/
• http://rowsandcolumns.blogspot.in/2010/07/compiling-ganglia-errors-and-problems.html

1.3.20 Logging

Some useful links to explain the concepts of Logging

Concepts

• https://syslog-ng.org/
• https://www.loggly.com/ultimate-guide/linux-logging-basics/
• https://www.loggly.com/ultimate-guide/access-and-error-logs/

76 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Linux Logs

• http://www.thegeekstuff.com/2011/08/linux-var-log-files/
• https://www.loggly.com/ultimate-guide/managing-linux-logs/
• https://blog.logentries.com/2012/05/logging-on-linux-part-1-ive-got-a-bunch-of-linux-boxes-what-should-i-be-looking-out-for-
• https://www.eurovps.com/blog/important-linux-log-files-you-must-be-monitoring

Configuration

Log-rotate

• https://serversforhackers.com/managing-logs-with-logrotate
• http://articles.slicehost.com/2010/6/30/understanding-logrotate-on-ubuntu-part-1
• http://articles.slicehost.com/2010/6/30/understanding-logrotate-on-centos-part-1
• http://www.rackspace.com/knowledge_center/article/understanding-logrotate-utility
• http://www.softpanorama.org/Commercial_linuxes/RHEL/rhel_log_rotation.shtml
• https://www.digitalocean.com/community/tutorials/how-to-manage-log-files-with-logrotate-on-ubuntu-12-10
• http://linuxers.org/howto/howto-use-logrotate-manage-log-files
• http://linuxconfig.org/setting-up-logrotate-on-redhat-linux
• http://www.thegeekstuff.com/2010/07/logrotate-examples/
• http://www.techrepublic.com/article/manage-linux-log-files-with-logrotate/
• https://www.linode.com/docs/uptime/logs/use-logrotate-to-manage-log-files

Troubleshooting & Log Parsing

• https://www.loggly.com/ultimate-guide/troubleshooting-with-linux-logs/
• https://www.loggly.com/ultimate-guide/centralizing-windows-logs/
• https://blog.logentries.com/2016/09/avoid-these-things-when-logging-from-your-application/

ELK Stack

Some useful links to explain the concepts of the ELK stack - ElasticSearch, Logstash and Kibana
• http://logz.io/learn/complete-guide-elk-stack/
• http://logz.io/blog/10-elasticsearch-concepts/

ElasticSearch

• http://logz.io/blog/elasticsearch-tutorial/
• http://vichargrave.com/ossec-log-management-with-elasticsearch
• http://aarvik.dk/a-bit-on-elasticsearch-logstash-kibana-the-elk-stack/

1.3. Please Note: 77

DevOps-Notes Documentation, Release 2.0.2

Logstash

• http://logz.io/blog/logstash-tutorial/
• https://www.digitalocean.com/community/tutorials/how-to-use-logstash-and-kibana-to-centralize-and-visualize-logs-on-ubuntu-
• http://logz.io/blog/fluentd-logstash/
• http://www.slashroot.in/logstash-tutorial-linux-central-logging-server

Kibana

• http://logz.io/blog/kibana-tutorial/
• http://logz.io/blog/kibana-visualizations/
• https://www.digitalocean.com/community/tutorials/how-to-use-kibana-dashboards-and-visualizations
• https://marc.cortinasval.cat/blog/2014/03/05/kibana-help-us-to-analyze-cdn-logs/

1.3.21 Networking

Some useful links to explain the concepts of Networking Protocols

Concepts

OSI Model Mnemonic : Please Do Not Teach Students Pointless Acronyms

• https://www.cybrary.it/0p3n/osi-model-7-layers-basic-understanding/

78 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• https://www.thomas-krenn.com/en/wiki/Two_Default_Gateways_on_One_System
• http://cavepopo.hd.free.fr/wordpress/linux/networking-on-linux-practical-examples/
• http://www.linuxhomenetworking.com/
• https://askleo.com/why_cant_i_connect_with_a_169254xx_ip_address/

Finding optimum MTU (Maximum Transmission Unit) size

• http://homenetworkadmin.com/what-is-mtu-size/
• https://www.tp-link.com/us/FAQ-190.html
• https://www.linksys.com/us/support-article?articleNum=134914

1.3. Please Note: 79

DevOps-Notes Documentation, Release 2.0.2

Category Cables

• https://www.networkcablingdirectory.com/articles/structured-network-cabling-id_1151.htm
• http://35.154.102.183/ethernet-cable-cat5-vs-cat6-vs-cat6a-vs-cat7-vs-cat8/amp/
• https://planetechusa.com/blog/ethernet-different-ethernet-categories-cat3-vs-cat5e-vs-cat6-vs-cat6a-vs-cat7-vs-cat8/

MAC Addresses : Explained

Private / Public / LocalHost IPs:

• http://www.tech-faq.com/127-0-0-1.html
• https://www.howtogeek.com/225487/what-is-the-difference-between-127.0.0.1-and-0.0.0.0/
• http://theydiffer.com/difference-between-public-and-private-ip-address/
• https://www.iplocation.net/public-vs-private-ip-address

80 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• http://serverfault.com/questions/49765/how-does-ipv4-subnetting-work/49836
• https://danielmiessler.com/study/subnetting/
• http://computer.howstuffworks.com/internet/basics/question549.htm
• http://unixadminschool.com/blog/2014/01/ipv6-the-solution-for-ipv4-exhaustion/

TCP

States of a TCP Connection

• https://community.apigee.com/articles/7970/tcp-states-explained.html
• https://blog.confirm.ch/tcp-connection-states/
• https://www.frozentux.net/iptables-tutorial/chunkyhtml/x1425.html
• http://www.masterraghu.com/subjects/np/introduction/unix_network_programming_v1.3/ch02lev1sec6.html
• https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.halu101/constatus.htm
• https://danielmiessler.com/study/tcpflags/
• https://vincent.bernat.ch/en/blog/2014-tcp-time-wait-state-linux

LIST of TCP / UDP Ports

• https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers

UDP

ICMP

• https://danielmiessler.com/study/icmp/

1.3. Please Note: 81

DevOps-Notes Documentation, Release 2.0.2

• https://protechgurus.com/ip-routing-process-step-step-explanation/
• https://www.techrepublic.com/article/ip-routing-in-40-short-steps/

VLAN

Simple Overview

• https://www.lifewire.com/virtual-local-area-network-817357

Conceptual Overview

• http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst4500/12-2/25ew/configuration/guide/conf/vlans.
html

Difference between access & trunk ports

• https://bit.ly/2rXuh6M

82 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Commands

• http://www.alexonlinux.com/useful-linux-networking-commands
• https://www.lifewire.com/find-a-mac-address-using-an-ip-address-818132

1.3. Please Note: 83

DevOps-Notes Documentation, Release 2.0.2

• http://www.thegeekstuff.com/2012/04/ip-routing-intro/
• http://www.thegeekstuff.com/2012/04/route-examples
• http://computernetworkingnotes.com/linux/rhce6-study-guide/

PING

• https://www.computerhope.com/unix/uping.htm
• http://www.slashroot.in/what-ping-sweep-and-how-do-ping-sweep

IFCONFIG

• http://www.tecmint.com/ifconfig-command-examples/
• https://www.linux.com/learn/replacing-ifconfig-ip
• http://cavepopo.hd.free.fr/wordpress/linux/bash-command-tips-ethtool/
• http://www.cyberciti.biz/faq/check-network-connection-linux/

TRACEROUTE

• http://www.slashroot.in/how-does-traceroute-work-and-examples-using-traceroute-command
• http://www.mavetju.org/networking/basicnetworktroubleshooting.php
• https://www.maxcdn.com/one/tutorial/how-to-read-a-traceroute/
• https://archive.nanog.org/meetings/nanog47/presentations/Sunday/RAS_Traceroute_N47_Sun.pdf

TCPDUMP

• https://danielmiessler.com/study/tcpdump/
• http://www.slashroot.in/packet-capturing-tcpdump-command-linux
• http://www.tecmint.com/12-tcpdump-commands-a-network-sniffer-tool/
• http://www.rationallyparanoid.com/articles/tcpdump.html

NETSTAT

• https://www.linux.com/learn/intro-to-linux/2017/7/introduction-ss-command
• http://www.slashroot.in/netstat-command-examples-and-its-usage
• http://www.binarytides.com/linux-ss-command/
• https://pcarleton.com/2018/05/31/netstat-vs-ss/

84 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

IFTOP

• http://www.slashroot.in/linux-iptraf-and-iftop-monitor-and-analyse-network-traffic-and-bandwidth
• http://www.thegeekstuff.com/2008/12/iftop-guide-display-network-interface-bandwidth-usage-on-linux/
• http://linoxide.com/monitoring-2/iftop-network-traffic/

NMap

• https://danielmiessler.com/study/nmap/
• http://www.plugged.in/network/nmap-scanning-methods.html
• http://www.plugged.in/network/nmap-target-specification-in-detail.html
• http://www.cyberciti.biz/networking/nmap-command-examples-tutorials/
Nmap Command - Free IPs in a particular subnet

1 for i in `sudo nmap -sP <subnet/CIDR> | grep -i 'Nmap scan report for' | awk '{print
˓→$5}'`;do ping -c 1 $i;done | grep from

ARP

• http://www.thegeekstuff.com/2012/01/arp-cache-poisoning/
• http://www.cyberciti.biz/faq/linux-duplicate-address-detection-with-arping/
• https://www.linux.com/news/ping-icmp-vs-arp
• http://xmodulo.com/how-to-add-or-remove-static-arp-entry-on-linux.html
• https://www.tummy.com/articles/networking-basics-how-arp-works/
• http://www.cyberciti.biz/faq/linux-duplicate-address-detection-with-arping/

NETCAT

• http://aarvik.dk/netcat-basics-and-a-few-smart-examples/

LLDP

• http://www.marcoach.nl/lldp-network-discovery/

Configuration

Basic NIC configuration & Bonding

• http://xmodulo.com/how-to-find-ethernet-network-interface-card-information-in-linux.html
• http://www.tecmint.com/configure-network-interface-in-rhel-centos-7-0/
• https://linuxconfig.org/how-to-add-new-static-route-on-rhel7-linux

1.3. Please Note: 85

DevOps-Notes Documentation, Release 2.0.2

• http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-interface.html
• http://login2linux.blogspot.in/2015/01/how-to-remove-nic-bonding-in-linux.html
• http://www.configserverfirewall.com/ubuntu-linux/ubuntu-set-static-ip-address/
• https://www.swiftstack.com/docs/install/configure_networking.html
• http://www.itzgeek.com/how-tos/mini-howtos/change-default-network-name-ens33-to-old-eth0-on-ubuntu-16-04.
html

Virtual IPs

• http://www.cyberciti.biz/faq/linux-creating-or-adding-new-network-alias-to-a-network-card-nic/
• https://linuxconfig.org/configuring-virtual-network-interfaces-in-linux
• http://www.cyberciti.biz/faq/linux-command-to-remove-virtual-interfaces-or-network-aliases/
• http://www.jamescoyle.net/how-to/307-create-a-virtual-ip-address-in-linux
• http://www.tecmint.com/create-multiple-ip-addresses-to-one-single-network-interface/

Static Routing

• http://www.thegeekstuff.com/2012/04/route-examples/
• http://bencane.com/2013/05/12/adding-and-troubleshooting-static-routes-on-red-hat-based-linux-distributions/
• http://www.cyberciti.biz/faq/linux-route-add/
• http://www.thetechnicalstuff.com/how-to-add-and-delete-route-in-linux/

DHCP

• http://tecadmin.net/configuring-dhcp-server-on-centos-redhat/
• http://linuxconfig.org/what-is-dhcp-and-how-to-configure-dhcp-server-in-linux
• http://www.hecticgeek.com/2012/01/command-line-ip-subnet-calculator-ubuntu-linux/
• https://www.geeksforgeeks.org/how-dhcp-server-dynamically-assigns-ip-address-to-a-host/

VPN

• https://zero.pritunl.com
• https://github.com/trailofbits/algo

Tuning, Troubleshooting & Log Parsing

Advanced Configuration

• http://www.slashroot.in/linux-kernel-rpfilter-settings-reverse-path-filtering
• http://www.slashroot.in/vlan-configuration-linux-nic-interface

86 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• http://www.cyberciti.biz/faq/linux-unix-tcp-port-forwarding/
• http://www.thewindowsclub.com/how-to-fix-network-internet-connection-problems-in-windows

Digging deeper into the Linux Networking stack

• https://blog.packagecloud.io/eng/2017/02/06/monitoring-tuning-linux-networking-stack-sending-data/
• https://blog.packagecloud.io/eng/2016/10/11/monitoring-tuning-linux-networking-stack-receiving-data-illustrated/
• https://blog.packagecloud.io/eng/2016/06/22/monitoring-tuning-linux-networking-stack-receiving-data/

Disable Network Manager in Linux

• http://xmodulo.com/disable-network-manager-linux.html

Disable Network Manager from over-writing resolv.conf in Linux

• https://www.reddit.com/r/linux4noobs/comments/3keuhd/how_to_stop_networkmanager_from_overriding_
my/

1.3.22 Security

Some useful links to explain the concepts of Security in Linux OS

Concepts

• https://www.thefanclub.co.za/how-to/how-secure-ubuntu-1604-lts-server-part-1-basics
• https://github.com/imthenachoman/How-To-Secure-A-Linux-Server
• https://dev.to/danlebrero/kerberos-explained-in-pictures

Rootkits

• https://www.digitalocean.com/community/tutorials/how-to-use-rkhunter-to-guard-against-rootkits-on-an-ubuntu-vps
• http://www.woktron.com/secure/knowledgebase/79/Installation-Rootkit-Hunter-rkhunter-on-CentOS-5-and-6.
html
• http://www.tecmint.com/install-rootkit-hunter-scan-for-rootkits-backdoors-in-linux/

Fail2Ban

• https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-centos-6
• https://www.digitalocean.com/community/tutorials/how-fail2ban-works-to-protect-services-on-a-linux-server
• http://www.tecmint.com/install-fail2ban-on-rhel-centos-fedora/
• http://www.the-art-of-web.com/system/fail2ban-log/
• http://www.dghost.com/techno/internet/the-power-of-fail2ban

1.3. Please Note: 87

DevOps-Notes Documentation, Release 2.0.2

• http://www.fail2ban.org/wiki/index.php/MANUAL_0_8
• http://aarvik.dk/prevent-intrusion-with-fail2ban/

TCP Wrappers

• https://jamalahmed.wordpress.com/2010/03/19/using-etchosts-allow-and-etchosts-deny-to-secure-unix/
• http://static.closedsrc.org/articles/dn-articles/hosts_allow.html
• http://www.tecmint.com/secure-linux-tcp-wrappers-hosts-allow-deny-restrict-access/
• http://www.aboutlinux.info/2005/10/using-tcp-wrappers-to-secure-linux.html
• http://www.cyberciti.biz/faq/tcp-wrappers-hosts-allow-deny-tutorial/
• http://bastille-linux.sourceforge.net/
• https://grsecurity.net/index.php

SELinux

Concepts of SELinux

• https://www.digitalocean.com/community/tutorial_series/an-introduction-to-selinux-on-centos-7
• http://www.linuxpathfinder.com/how-to-disable-selinux-temporarily-or-permanently
• http://www.revsys.com/writings/quicktips/turn-off-selinux.html

DDoS

• http://www.digitalattackmap.com/understanding-ddos/
• https://blog.cloudflare.com/the-ddos-that-almost-broke-the-internet/
• https://www.incapsula.com/ddos/ddos-attacks/

Configuration

• http://bencane.com/2013/01/14/mitigating-dos-attacks-with-a-null-or-blackhole-route-on-linux/
• http://www.mkyong.com/linux/how-to-block-attackers-ip-with-null-route-command/
• http://www.opensourceforu.com/2011/04/securing-apache-part-8-dos-ddos-attacks/
• https://developers.google.com/speed/public-dns/docs/security#introduction-dns-security-threats-and-mitigations
• http://www.tldp.org/HOWTO/Firewall-Piercing/index.html
• http://www.ubuntufree.com/how-to-stop-a-ddos-attack-on-ubuntu/
• http://ubtutorials.com/tutorial/1139/how-stop-small-ddos-attacks-ubuntu

1.3.23 IPTables

Some useful links to explain the concepts of IPTables

88 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Concepts

• https://www.digitalocean.com/community/tutorials/what-is-a-firewall-and-how-does-it-work
• https://www.digitalocean.com/community/tutorials/how-the-iptables-firewall-works
• https://www.digitalocean.com/community/tutorials/how-to-choose-an-effective-firewall-policy-to-secure-your-servers
• http://www.thegeekstuff.com/2011/01/iptables-fundamentals/
• https://wiki.centos.org/HowTos/Network/IPTables

Configuration

• http://fideloper.com/iptables-tutorial
• http://www.cyberciti.biz/tips/linux-iptables-examples.html
• http://www.thegeekstuff.com/2011/06/iptables-rules-examples
• https://linuxconfig.org/collection-of-basic-linux-firewall-iptables-rules
• http://blog.ls20.com/securing-your-server-using-ipset-and-dynamic-blocklists/
• http://www.cyberciti.biz/faq/how-do-i-block-an-ip-on-my-linux-server/
• http://www.tecmint.com/basic-guide-on-iptables-linux-firewall-tips-commands/
• http://www.cyberciti.biz/faq/rhel-fedorta-linux-iptables-firewall-configuration-tutorial/
• http://www.howtogeek.com/177621/the-beginners-guide-to-iptables-the-linux-firewall/
• https://www.digitalocean.com/community/tutorials/how-to-list-and-delete-iptables-firewall-rules
• https://www.digitalocean.com/community/tutorials/how-to-setup-a-basic-ip-tables-configuration-on-centos-6
• https://www.digitalocean.com/community/tutorials/7-security-measures-to-protect-your-servers
• https://www.frozentux.net/iptables-tutorial/chunkyhtml/x1309.html
• https://www.digitalocean.com/community/tutorials/iptables-essentials-common-firewall-rules-and-commands
• https://jadendreamer.wordpress.com/2013/04/18/linux-tutorial-blocking-ip-addresses-from-your-server/
• http://www.linuxjournal.com/content/advanced-firewall-configurations-ipset
• https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-using-firewalld-on-centos-7
• http://blog.nintechnet.com/how-to-block-w00tw00t-at-isc-sans-dfind-and-other-web-vulnerability-scanners/

Tuning and Hardening

• https://www.digitalocean.com/community/tutorials/how-to-test-your-firewall-configuration-with-nmap-and-tcpdump

IP Conntrack

• http://www.cyberciti.biz/faq/ip_conntrack-table-ful-dropping-packet-error/
• http://www.pc-freak.net/blog/resolving-nf_conntrack-table-full-dropping-packet-flood-message-in-dmesg-linux-kernel-log/
• http://blackbird.si/ip_conntrack-table-full-dropping-packet-conclusions-about-connection-tracking/
• https://www.scalescale.com/tips/nginx/ip_conntrack-table-full-dropping-packet/

1.3. Please Note: 89

DevOps-Notes Documentation, Release 2.0.2

• http://www.rigacci.org/wiki/lib/exe/fetch.php/doc/appunti/linux/sa/iptables/conntrack.html
• https://bobcares.com/blog/2011/04/21/ip-connection-tracking-ip_conntrack/
• https://timanovsky.wordpress.com/2009/04/10/tuning-linux-firewall-connection-tracker-ip_conntrack/

1.3.24 Generic

Some useful links to explain the concepts of Programming

Concepts

• http://regex.learncodethehardway.org/book/
• https://techdifferences.com/differenece-between-for-and-while-loop.html#KeyDifferences
• https://dev.to/sandordargo/8-books-every-junior-developer-should-read–4p5h
• https://dev.to/perigk/fantastic-youtube-channels-with-top-quality-learning-for-software-engineers–263i
• https://dev.to/tapudp/github-repos-for-learners-634
• https://dev.to/codehakase/how-i-learned-go-programming
• https://dev.to/brpaz/my-linux-development-environment-of-2018-ch7
• https://dev.to/rpalo/powershell-tutorial-especially-for-people-who-hate-powershell-2g25
• https://devdocs.io
• https://books.goalkicker.com/
• https://robertheaton.com/2020/04/06/systems-design-for-advanced-beginners/

Configuration

Online Interpreter for Multiple Languages

• https://tio.run/

90 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

DevOps

1.3. Please Note: 91

DevOps-Notes Documentation, Release 2.0.2

• https://devops.zeef.com/christian.peper
• https://dev-ops-tools.zeef.com/richard.kraayenhagen
• https://dev.to/
• https://xebialabs.com/glossary/
• https://xebialabs.com/the-ultimate-devops-tool-chest/
• https://xebialabs.com/periodic-table-of-devops-tools/
• https://zachholman.com/posts/deploying-software
• https://www.ibm.com/developerworks/learn/devops/
• https://learn-anything.xyz/programming/software-development/devops
• http://itrevolution.com/devops-blog/
• https://continuousdelivery.com
• http://dev2ops.org
• https://codeascraft.com
• http://www.devopsbookmarks.com
• https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/0988262592
• https://increment.com/on-call/
• https://medium.com/cleartax-engineering/cleartaxs-requirements-from-a-deployment-pipeline-1b77a55ba08c
• http://www.humanops.com/

1.3.25 Python

Some useful links to explain the concepts of Python

Concepts

• http://www.codeconquest.com/blog/the-50-best-websites-to-learn-python/
• http://simeonfranklin.com/python-fundamentals/
• https://python.zeef.com/yakup.keskindag
• https://www.tutorialspoint.com/python/index.htm

Official Python Documentation

• https://docs.python.org/2/index.html
• https://docs.python.org/

Python Books

A Byte of Python - https://python.swaroopch.com/

Learn Python the Hard Way - https://learnpythonthehardway.org/book/

92 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Online Courses

• https://www.codecademy.com/learn/python
• https://www.udacity.com/course/programming-foundations-with-python–ud036
• http://campus.codeschool.com/courses/try-python/level/1/section/1/video/1
• https://www.coursera.org/learn/python
• https://dev.to/jessicagarson/resources-for-learning-python-hd6
• https://www.youtube.com/watch?v=_uQrJ0TkZlc

Intermediate Python

• http://book.pythontips.com/en/latest/
• https://automatetheboringstuff.com/
• https://gist.github.com/diyan/f3c24653e63c24c99137

Solving Python Problems to Learn Concepts

• https://www.hackerrank.com/domains/python/py-introduction

Configuration

• http://aarvik.dk/python-for-system-administration/
• https://github.com/h2oai/h2o-2/wiki/installing-python-2.7-on-centos-6.3.-follow-this-sequence-exactly-for-centos-machine-only
• https://superuser.com/questions/143119/how-do-i-add-python-to-the-windows-path

1.3. Please Note: 93

DevOps-Notes Documentation, Release 2.0.2

• https://stackoverflow.com/questions/6318156/adding-python-path-on-windows-7?

94 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• https://techglimpse.com/error-executing-python3-5-command-solution/
• https://stackoverflow.com/questions/24205624/how-do-i-resolve-not-fully-installed-package-python3-setuptools
• https://stackoverflow.com/questions/38132755/importerror-no-module-named-encodings

Visual IDE for Python

• http://pythontutor.com/
• http://www.skulpt.org/

1.3.26 Bash

Some useful links to explain the concepts of Bash Scripting

Concepts

• http://linuxcommand.org
• http://tldp.org/LDP/Bash-Beginners-Guide/html/index.html
• http://www.linuxnix.com/category/programming/bash/
• https://dev.to/thiht/shell-scripts-matter
• http://www.thegeekstuff.com/2010/08/bash-shell-builtin-commands/
• http://bash.cyberciti.biz
• https://github.com/dylanaraps/pure-bash-bible
• https://dev.to/awwsmm/101-bash-commands-and-tips-for-beginners-to-experts-30je

1.3. Please Note: 95

DevOps-Notes Documentation, Release 2.0.2

Conditional Statements, Loops

• http://bencane.com/2014/01/27/8-examples-of-bash-if-statements-to-get-you-started/
• https://superuser.com/questions/615043/while-loop-in-bash-until-file-size-does-not-change
• https://unix.stackexchange.com/questions/247187/bash-if-not-multiple-conditions-without-subshell
• https://www.lifewire.com/write-bash-while-loops-2200576
• http://www.compciv.org/topics/bash/loops/
• https://dev.to/rpalo/bash-brackets-quick-reference-4eh6
• https://www.cyberciti.biz/faq/howto-check-if-a-directory-exists-in-a-bash-shellscript/
• https://unix.stackexchange.com/questions/52800/how-to-do-an-if-statement-from-the-result-of-an-executed-command

Editing & Formatting Text

• http://misc.flogisoft.com/bash/tip_colors_and_formatting
• https://google.github.io/styleguide/shell.xml
• https://stackoverflow.com/questions/11102360/read-lines-starting-from-a-line-number-in-a-bash-script
• https://www.cyberciti.biz/faq/unix-howto-read-line-by-line-from-file/

Configuration

Online IDE for Bash Scripts

• https://www.shellcheck.net/
• https://github.com/koalaman/shellcheck

Operators in Bash

• https://unix.stackexchange.com/questions/159513/what-are-the-shells-control-and-redirection-operators
• http://www.tecmint.com/chaining-operators-in-linux-with-practical-examples/
• http://bencane.com/2014/09/02/understanding-exit-codes-and-how-to-use-them-in-bash-scripts/
• https://dev.to/oyagci/shell-redirections-explained-2h9

Avoid Temp files in Shell scripts

• https://dev.to/philgibbs/avoiding-temporary-files-in-shell-scripts

Variables, Functions in Bash

• http://www.compciv.org/topics/bash/variables-and-substitution/
• https://stackoverflow.com/questions/1809899/how-can-i-assign-the-output-of-a-function-to-a-variable-using-bash
• https://ryanstutorials.net/bash-scripting-tutorial/bash-functions.php

96 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

• https://www.digitalocean.com/community/tutorials/an-introduction-to-useful-bash-aliases-and-functions
• https://www.shellscript.sh/functions.html

Configure Bash History Retention

• https://stackoverflow.com/questions/9457233/unlimited-bash-history/12234989#12234989

• https://unix.stackexchange.com/questions/1288/preserve-bash-history-in-multiple-terminal-windows/48113#
48113

1.3. Please Note: 97

DevOps-Notes Documentation, Release 2.0.2

• https://askubuntu.com/questions/15926/how-to-avoid-duplicate-entries-in-bash-history/15929#15929

98 Chapter 1. Overview
 DevOps-Notes Documentation, Release 2.0.2

Troubleshooting & Log Parsing

• https://www.tutorialkart.com/bash-shell-scripting/write-output-of-bash-command-to-log-file-example
• http://www.shellhacks.com/en/7-Tips-Tuning-Command-Line-History-in-Bash
• https://github.com/kward/shunit2
• https://github.com/bmizerany/roundup
• https://github.com/sstephenson/bats

1.3.27 Ruby

Some useful links to explain the concepts of Ruby

Concepts

Official Ruby Documentation

• https://www.ruby-lang.org/en/documentation/
• https://ruby-doc.org/

Configuration

• https://stackoverflow.com/questions/15708916/use-rvmrc-or-ruby-version-file-to-set-a-project-gemset-with-rvm
• https://stackoverflow.com/questions/4667150/what-is-the-best-way-to-uninstall-gems-from-a-rails3-project
• https://dev.to/molly_struve/how-to-setup-a-readonly-rails-console-1j1a
• http://syndicode.co/2018/01/29/books-and-tutorials-for-learning-and-mastering-ruby-language

Troubleshooting & Log Parsing

• https://stackoverflow.com/questions/14289217/im-getting-rvm-is-not-a-function-error-on-mac-os-x-and-no-posted-solutions-w
• https://www.exceptionalcreatures.com/bestiary/Net/OpenTimeout.html
• https://dev.to/exampro/testunit-writing-test-code-in-ruby-part-1-of-3-44m2
• http://engineering.pivotal.io/post/debugging-ruby-memory-issues-cloud-foundry-cloud-controller/

1.3.28 GIT

Some useful links to explain the concepts of GIT

Concepts

Official Git Documentation

• https://git-scm.com/book/en/v2/
• https://danielmiessler.com/study/git/

1.3. Please Note: 99

DevOps-Notes Documentation, Release 2.0.2

• https://about.gitlab.com/2015/05/18/simple-words-for-a-gitlab-newbie/
• http://slidetocode.com/2013/08/25/how-git-works/
• http://juristr.com/blog/2013/04/git-explained/
• https://betterexplained.com/articles/aha-moments-when-learning-git/
• http://readwrite.com/2013/09/30/understanding-github-a-journey-for-beginners-part-1/
• http://2ndscale.com/rtomayko/2008/the-thing-about-git
• http://eagain.net/articles/git-for-computer-scientists/
• https://www.sbf5.com/~cduan/technical/git/git-1.shtml
• https://launchschool.com/books/git/read/introduction
• http://githowto.com/
• http://gitimmersion.com/lab_01.html
• http://swift.siphos.be/aglara/centralcmdb.html
• http://www.gitguys.com/topics/
• https://robots.thoughtbot.com/tags/git
• https://www.linux.com/learn/finding-everything-git
• http://slidetocode.com/2013/08/25/how-git-works/
• http://juristr.com/blog/2013/04/git-explained/
• http://pcottle.github.io/learnGitBranching/
• http://rogerdudler.github.io/git-guide/
• https://dev.to/raha198/how-to-master-the-art-of-git
• http://www.sitepoint.com/git-for-beginners/
• https://dev.to/maxwell_dev/the-git-rebase-introduction-i-wish-id-had

Configuration

• https://www.atlassian.com/git/tutorials/
• http://eev.ee/blog/2015/04/24/just-enough-git-to-be-less-dangerous/
• http://kushagragour.in/blog/2014/01/build-git-learn-git/
• http://wildlyinaccurate.com/a-hackers-guide-to-git/
• https://www.digitalocean.com/community/tutorials/how-to-use-git-effectively
• https://www.digitalocean.com/community/tutorials/how-to-use-git-branches
• http://redmine.jamoma.org/projects/1/wiki/Working_with_GIT_branches
• https://try.github.io/levels/1/challenges/1
• https://help.github.com/articles/which-remote-url-should-i-use/
• https://help.github.com/articles/using-ssh-over-the-https-port/
• https://medium.com/@haydar_ai/learning-how-to-git-using-ssh-instead-of-https-91f09cff72de
• https://rtcamp.com/tutorials/git/

100 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

• https://git-scm.com/book/en/v2/Git-Basics-Viewing-the-Commit-History
• https://coderwall.com/p/fasnya/add-git-branch-name-to-bash-prompt
• Update multiple git repos : https://github.com/earwig/git-repo-updater

Self-hosted GitHub Clones

• https://gogs.io/
• https://gitea.io/en-US/
• https://gitlab.com/

Reviews || Configuration of the above

• https://blog.ifloop.org/2017/10/12/gitlab-vs-gogs-vs-gitea/
• https://www.reddit.com/r/git/comments/6y68vr/gitlab_vs_bitbucket_server_vs_gitea_vs_gogs/
• https://github.com/MartinThoma/MartinThoma.github.io/blob/pelican/content/
2017-09-06-git-repository-managers.md
• https://gitbucket.github.io/gitbucket-news/gitbucket/2017/03/29/benchmark-of-gitbucket.html
• https://blog.hypriot.com/post/run-your-own-github-like-service-with-docker/

Troubleshooting

• http://www.gitguys.com/topics/merging-with-a-conflict-conflicts-and-resolutions/
• https://help.github.com/articles/error-permission-denied-publickey/
• http://www.ikriv.com/blog/?p=1905

Remove local commits NOT pushed to remote

• http://stackoverflow.com/questions/927358/how-to-undo-last-commits-in-git
• http://stackoverflow.com/questions/1611215/remove-a-git-commit-which-has-not-pushed

Understanding .gitignore file

• https://www.git-tower.com/learn/git/faq/ignore-tracked-files-in-git
• https://github.com/github/gitignore
• https://www.gitignore.io/
• https://stackoverflow.com/questions/1274057/how-to-make-git-forget-about-a-file-that-was-tracked-but-is-now-in-gitignore

1.3. Please Note: 101

DevOps-Notes Documentation, Release 2.0.2

Git Operations (Stash, Merge, Diff, Rebase)

• https://www.atlassian.com/git/tutorials/saving-changes/git-stash
• https://www.git-tower.com/learn/git/faq/save-changes-with-git-stash
• https://stackoverflow.com/questions/19003009/how-to-recover-stashed-uncommitted-changes
• https://www.atlassian.com/git/tutorials/using-branches/git-merge
• https://stackoverflow.com/questions/20106712/what-are-the-differences-between-git-remote-prune-git-prune-git-fetch-prune
• https://blog.algolia.com/master-git-rebase/
• https://stackoverflow.com/questions/4099742/how-to-compare-files-from-two-different-branches

102 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

Git on Windows

• https://gist.github.com/adamjohnson/5682757
• https://stackoverflow.com/questions/11000869/command-line-git-on-windows
• http://guides.beanstalkapp.com/version-control/git-on-windows.html

Better GIT Workflow

• https://sandofsky.com/blog/git-workflow.html
• https://pixelbrackets.github.io/git_cheat_sheet/
• https://github.com/git-tips/tips

1.3. Please Note: 103

DevOps-Notes Documentation, Release 2.0.2

• https://github.com/so-fancy/diff-so-fancy
• https://dev.to/shreyasminocha/how-i-do-my-git-commits-34d?
• https://gist.github.com/PurpleBooth/109311bb0361f32d87a2
• http://michaelwales.com/articles/make-gitconfig-work-for-you/
• https://blog.scottnonnenberg.com/better-git-configuration/
• https://howtogit.net/
• https://githooks.com/
• https://nvie.com/posts/a-successful-git-branching-model/
• https://dev.to/_darrenburns/8-productivity-tips-for-github-44kn
• https://scotch.io/tutorials/using-git-hooks-in-your-development-workflow
• https://stackoverflow.com/questions/3651860/which-characters-are-illegal-within-a-branch-name/3651867#
3651867
• https://stackoverflow.com/questions/3580013/should-i-use-past-or-present-tense-in-git-commit-messages/
3580764#3580764

Random Notes

• https://svnvsgit.com

1.3.29 Ansible

Some useful links to explain the concepts of Ansible

Concepts

• https://blog.josephkahn.io/articles/ansible/
• http://engineering.waveapps.io/post/80595462671/an-ansible-primer
• http://tjheeta.github.io/2015/04/15/ansible-vs-chef/
• https://andidog.de/blog/2017-04-24-ansible-best-practices

Official Documentation

• https://docs.ansible.com/ansible/2.4/index.html

Configuration

Ansible.cfg

• https://raw.githubusercontent.com/ansible/ansible/devel/examples/ansible.cfg
• http://techadminblog.com/get-execution-time-of-each-task-ansible/
• https://blog.ssdnodes.com/blog/step-by-step-ansible-guide/
• https://stackoverflow.com/questions/27733511/how-to-set-linux-environment-variables-with-ansible

104 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

Run one Task - Ansible Playbook

• https://stackoverflow.com/questions/23945201/how-to-run-only-one-task-in-ansible-playbook

1.3. Please Note: 105

DevOps-Notes Documentation, Release 2.0.2

106 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

Basic Ansible Tutorials

• https://serversforhackers.com/c/an-ansible2-tutorial
• https://leucos.github.io/ansible-files-layout
• https://www.linode.com/docs/applications/configuration-management/automatically-configure-servers-with-ansible-and-playboo
• https://www.digitalocean.com/community/tutorials/how-to-manage-multistage-environments-with-ansible
• http://codeheaven.io/15-things-you-should-know-about-ansible/
• https://symfonycasts.com/screencast/ansible/handlers

Ansible Roles

• https://www.digitalocean.com/community/tutorials/how-to-use-ansible-roles-to-abstract-your-infrastructure-environment

Managing Users with Ansible

• https://www.alibabacloud.com/blog/managing-system-users-using-ansible_593861
• https://serverfault.com/questions/830302/ensure-only-specific-list-of-users-exist-with-ansible
• http://minimum-viable-automation.com/ansible/managing-users-accounts-ansible/

NGinx Setup via Ansible

• https://symfonycasts.com/screencast/ansible/nginx-conf-template
• https://blog.serverdensity.com/deploying-nginx-with-ansible/
• https://meritocracy.is/blog/2017/07/24/manage-nginx-configurations-ansible/
• https://liquidat.wordpress.com/2016/01/26/howto-introduction-to-ansible-variables/
• https://mydbops.wordpress.com/2019/04/17/jinja2-for-better-ansible/

Troubleshooting

• https://stackoverflow.com/questions/20563639/ansible-playbook-shell-output
• https://docs.ansible.com/ansible/latest/user_guide/playbooks_delegation.html
• https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html

1.3.30 Puppet

Some useful links to explain the concepts of Puppet

Concepts

• http://logz.io/blog/chef-vs-puppet/
• https://www.linkedin.com/learning/mastering-puppet-for-large-infrastructures/the-course-overview

1.3. Please Note: 107

DevOps-Notes Documentation, Release 2.0.2

Official Documentation

• https://docs.puppet.com/puppet/
• https://docs.puppet.com/guides/style_guide.html

Others

• http://www.harker.com/puppet/BayLISA100715.html
• http://www.slashroot.in/puppet-tutorial-introduction-puppet-configuration-management-tool
• http://www.slashroot.in/puppet-tutorial-how-does-puppet-work
• http://www.slashroot.in/puppet-hiera-tutorial-example-configuration
• http://www.example42.com/tutorials/PuppetTutorial/
• http://www.pindi.us/blog/getting-started-puppet
• https://www.youtube.com/watch?v=73xeSTz8AeA&list=PLtNErhYMkHnEu1_ZHSJt1xF–zU35dRgZ
• https://www.digitalocean.com/community/tutorials/how-to-install-puppet-to-manage-your-server-infrastructure
• https://www.digitalocean.com/community/tutorials/getting-started-with-puppet-code-manifests-and-modules
• http://codingbee.net/tutorials/puppet/puppet-what-is-puppet/
• http://www.puppetcookbook.com/
• http://opensourceforu.efytimes.com/2009/06/puppet-show-automating-unix-administration/
• http://www.linuxuser.co.uk/tutorials/puppet-server-management
• https://www.linux.com/learn/tutorials/325201-introduction-to-puppet-streamlined-system-configuration
• http://www.linuxjournal.com/magazine/automate-system-administration-tasks-puppet
• http://www.infoworld.com/article/2614204/data-center/puppet-or-chef–the-configuration-management-
dilemma.html
• http://puppetlunch.com/contents/

1.3.31 Chef

Some useful links to explain the concepts of Chef

Concepts

• https://www.hpe.com/us/en/insights/articles/what-is-chef-a-primer-for-devops-newbies-1704.html
• https://learn.chef.io

Configuration

Troubleshooting

1.3.32 Docker

Some useful links to explain the concepts of Docker

108 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

Official Docker Documentation

• https://docs.docker.com/

Concepts

• https://docker.zeef.com/lim
• https://medium.freecodecamp.org/a-beginner-friendly-introduction-to-containers-vms-and-docker-79a9e3e119b
• https://dev.to/softchris/5-part-docker-series-beginner-to-master-3m1b?
• https://hackernoon.com/beginners-guide-to-microservices-explaining-it-to-a-5-year-old-4481f5aba466
• https://jvns.ca/blog/2019/11/18/how-containers-work–overlayfs

Good Explanation of Namespaces : needed for Docker

• https://www.toptal.com/linux/separation-anxiety-isolating-your-system-with-linux-namespaces

Difference between containers and VMs

• https://stackoverflow.com/questions/16047306/how-is-docker-different-from-a-normal-virtual-machine

1.3. Please Note: 109

DevOps-Notes Documentation, Release 2.0.2

110 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

Docker Image Layers : Easy Explanation

• http://tuhrig.de/layering-of-docker-images/

Docker Swarm Working

• https://docs.docker.com/engine/swarm/how-swarm-mode-works/services/

Configuration

• http://bencane.com/2015/12/01/getting-started-with-docker-by-dockerizing-this-blog/
• https://docs.docker.com/config/containers/resource_constraints/
• https://stackoverflow.com/questions/29480099/docker-compose-vs-dockerfile-which-is-better

1.3. Please Note: 111

DevOps-Notes Documentation, Release 2.0.2

112 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

• https://www.digitalocean.com/community/tutorials/how-to-use-traefik-as-a-reverse-proxy-for-docker-containers-on-ubuntu-18-0
• https://blog.codeship.com/orchestrate-containers-for-development-with-docker-compose/

Troubleshooting & Log Parsing

• https://stackoverflow.com/questions/28320134/how-to-list-all-tags-for-a-docker-image-on-a-remote-registry
• https://stackoverflow.com/questions/24481564/how-can-i-find-a-docker-image-with-a-specific-tag-in-docker-registry-on-the-doc
32622147
• https://codefresh.io/docker-tutorial/not-ignore-dockerignore/

Tuning & Hardening

• https://cloudvedas.com/how-to-cap-or-limit-memory-usage-of-a-docker-container/
• https://56k.cloud/blog/put-the-brakes-on-docker-containers/

Training & Certification

Docker Training

• http://training.play-with-docker.com/ops-landing/
• http://training.play-with-docker.com/alacart/
• https://katacoda.com/login
• http://container.training/
• https://github.com/veggiemonk/awesome-docker
• https://docker-curriculum.com/

Docker Certified Associate

• https://www.bretfisher.com/docker-certified-associate/
• https://www.linode.com/docs/applications/containers/when-and-why-to-use-docker/
• https://www.reddit.com/r/docker/comments/3yo0e8/is_there_any_advantage_to_using_docker_for/

1.3.33 Kubernetes

Some useful links to explain the concepts of Kubenetes

Concepts

• https://dev.to/techworld_with_nana/kubernetes-simply-explained-for-beginners-33em
• https://dev.to/aurelievache/kubernetes-sketchnotes-pods-4ib0
• https://speakerdeck.com/thockin/the-ins-and-outs-of-networking-in-google-container-engine

1.3. Please Note: 113

DevOps-Notes Documentation, Release 2.0.2

Official Documentation

• https://kubernetes.io/docs/home
• https://kubectl.docs.kubernetes.io

Configuration

• https://medium.com/@jmarhee/managing-kubernetes-config-contexts-fbe710fa756a
• https://nikgrozev.com/2019/10/03/switch-between-multiple-kubernetes-clusters-with-ease/
• https://sysdig.com/blog/kubernetes-limits-requests/
• https://speakerdeck.com/thockin/bringing-traffic-into-your-kubernetes-cluster

Tuning & Hardening

• https://github.com/derailed/popeye

Troubleshooting

• https://stackoverflow.com/questions/57161481/kubectl-how-to-display-pod-logs-without-specyfing-the-pod-name-explicitly/
57161894#57161894
• https://stackoverflow.com/questions/54821044/how-to-stop-pause-a-pod-in-kubernetes/54822866#54822866
• https://stackoverflow.com/questions/35453792/pods-stuck-in-terminating-status/38178833#38178833

• https://learnk8s.io/troubleshooting-deployments

114 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

1.3. Please Note: 115

DevOps-Notes Documentation, Release 2.0.2

1.3.34 AWS

Some useful links to explain the concepts of AWS

Official AWS Documentation

• https://docs.aws.amazon.com/

Concepts

• https://dev.to/itscoderslife/aws-in-5-minutes-lif
• https://dev.to/helenanders26/aws-series-aws-from-a-to-z-45b1
• https://start.jcolemorrison.com/aws-iam-policies-in-a-nutshell/

Configuration

• https://serverfault.com/questions/778426/aws-ec2-describe-instances-filtering-by-multiple-ec2-tags
• https://serverfault.com/questions/560337/search-ec2-instance-by-its-name-from-aws-command-line-tool
• https://www.onica.com/blog/using-aws-cli-to-find-untagged-instances/
• https://stackoverflow.com/questions/24938971/list-public-ip-addresses-of-ec2-instances

s3cmd

• http://tecadmin.net/install-s3cmd-manage-amazon-s3-buckets/
• http://tecadmin.net/s3cmd-file-sync-with-s3bucket/
• http://jayendrapatil.com/tag/s3/

AWS Instances

• https://ec2instances.info/

Troubleshooting & Log Parsing

• http://www.fizerkhan.com/blog/posts/Resize-EBS-volume-without-rebooting-in-AWS.html
• https://linuxroutes.com/how-to-resolve-instance-does-not-have-a-volume-attached-at-root-dev-sda1-error/

Tuning & Hardening

• https://www.sohamkamani.com/blog/2016/12/21/web-security-cors/
• https://www.altcademy.com/questions/amazon-aws-s3-error-no-access-control-allow-origin-header-is-present-on-the-requested-
• http://virtuallyhyper.com/2013/08/cors-with-cloudfront-s3-and-multiple-domains/

116 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

Training & Certification

• https://www.quora.com/What-is-the-AWS-Certified-Cloud-Practitioner-exam
• https://www.quora.com/I-am-a-fresher-Which-AWS-certification-is-better-for-me-AWS-solution-architect-or-AWS-developer
• http://blog.rowanudell.com/what-aws-certification-should-you-get/
• https://acloud.guru
• https://www.kylegalbraith.com/learn-aws/
• https://cantrill.io/2018/07/05/Passing-the-aws-certified-sysops-administrator-associate-exam.html

1.3.35 MySQL

Some useful links to explain the concepts of MySQl

Concepts

• http://etutorials.org/SQL/MySQL/
• https://serversforhackers.com/series/mysql
• http://www.rathishkumar.in/2016/04/understanding-mysql-architecture.html?

Difference between MyISAM and InnoDB Storage engines

• http://blog.danyll.com/myisam-vs-innodb/
• http://www.rapidprogramming.com/questions-answers/differences-between-innodb-and-myisam-in-mysql-innodb-vs-myisam-15

Commands

• http://www.mysqltutorial.org/

Configuration

• https://www.percona.com/blog/2014/11/12/log-rotate-and-the-deleted-mysql-log-file-mystery/
• http://blog.oneiroi.co.uk/mysql/mysql-slow-query-log-rotation/
• http://mysql.az/2015/05/12/mysql-logrotate-script/
• https://www.question-defense.com/2009/12/20/configure-logrotate-to-rotate-and-flush-mysql-logs-without-a-password
• http://etutorials.org/SQL/MySQL/Part+III+MySQL+Administration/Chapter+11.+General+MySQL+
Administration/Maintaining+Log+Files/
• https://opensourcedbms.com/dbms/how-to-upgrade-mysql-5-5-to-mysql-5-6-on-centos-6-3-red-hat-fedora/

1.3. Please Note: 117

DevOps-Notes Documentation, Release 2.0.2

Backing Up | Restore Databases via Command Line

• https://in.godaddy.com/help/backup-mysql-databases-on-your-server-linux-17547
• http://www.mysqltutorial.org/how-to-backup-database-using-mysqldump.aspx
• https://www.liquidweb.com/kb/how-to-back-up-mysql-databases-from-the-command-line/
• https://www.tecmint.com/mysql-backup-and-restore-commands-for-database-administration/

Configuring MariaDB for remote access

• https://mariadb.com/kb/en/library/configuring-mariadb-for-remote-client-access/

Removing a MySQL user with his privileges

• https://nsaunders.wordpress.com/2007/04/30/removing-a-mysql-user/
• https://www.a2hosting.in/kb/developer-corner/mysql/managing-mysql-databases-and-users-from-the-command-line

Information on MySQL Bin logs

• http://www.cyberciti.biz/faq/what-is-mysql-binary-log/

Replication (Master-Master)

• https://www.digitalocean.com/community/tutorials/how-to-set-up-mysql-master-master-replication

Replication (Master-Slave)

• http://www.tecmint.com/how-to-setup-mysql-master-slave-replication-in-rhel-centos-fedora/
• https://www.rackspace.com/knowledge_center/article/mysql-master-slave-replication
• https://www.digitalocean.com/community/tutorials/how-to-set-up-master-slave-replication-in-mysql
• http://sharadchhetri.com/2013/11/21/setup-mysql-master-slave-replication-in-centos-6/
• http://aarvik.dk/how-to-set-up-master-slave-replication-in-mysql/
• http://plusbryan.com/mysql-replication-without-downtime
• https://blog.marceloaltmann.com/en-how-does-mysql-replication-works-pt-como-funciona-a-replicacao-no-mysq/

Reset forgotten MySQL password

• https://www.digitalocean.com/community/tutorials/how-to-reset-your-mysql-or-mariadb-root-password
• http://www.cyberciti.biz/faq/mysql-change-root-password/
• https://www.liquidweb.com/kb/change-a-password-for-mysql-on-linux-via-command-line/
• https://www.codeenigma.com/community/blog/restoring-mysql-root-user

118 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

Tuning & Hardening

• http://blog.webyog.com/2012/11/20/how-to-monitor-mysql-replication/
• https://www.digitalocean.com/community/tutorials/how-to-secure-mysql-and-mariadb-databases-in-a-linux-vps
• https://www.percona.com/blog/2013/04/18/rotating-mysql-slow-logs-safely/
• http://www.pontikis.net/blog/how-and-when-to-enable-mysql-logs
• https://serversforhackers.com/c/mysql-network-security
• http://www.proxysql.com
• https://severalnines.com/resources/database-management-tutorials/mysql-load-balancing-haproxy-tutorial

To change the value of expire_log_days without MySQL restart

• https://www.sebastien-han.fr/blog/2013/02/15/purge-mysql-binary-logs/

To enable various logs via my.cnf or on the fly without restart

• http://www.pontikis.net/blog/how-and-when-to-enable-mysql-logs

Perl script to analyses system MySQL variables & optimize accordingly

• https://major.io/mysqltuner/

Tool to convert xls files into SQL

• https://sqlizer.io/

Troubleshooting

• https://alvinalexander.com/blog/post/mysql/how-show-open-database-connections-mysql
• https://www.tecmint.com/mysqladmin-commands-for-database-administration-in-linux/
• http://www.plugged.in/databases/mysql-server-wont-start-pid-file-errors.html
• https://major.io/2008/06/24/mysql-error-1040-too-many-connections/
• https://www.digitalocean.com/community/tutorials/how-to-use-mytop-to-monitor-mysql-performance

Checking for replication Lags

• https://www.percona.com/blog/2007/10/12/managing-slave-lag-with-mysql-replication/
• https://www.percona.com/blog/2014/05/02/how-to-identify-and-cure-mysql-replication-slave-lag/

1.3. Please Note: 119

DevOps-Notes Documentation, Release 2.0.2

Various MySQL error codes

• http://www.fromdual.com/mysql-error-codes-and-messages
• https://major.io/2007/08/09/mysql-error-codes/

Fix for the ERROR 1396

• https://stackoverflow.com/questions/5555328/error-1396-hy000-operation-create-user-failed-for-jacklocalhost

1.3.36 PostgreSQL

Some useful links to explain the concepts of PostgreSQl

Concepts

• http://swift.siphos.be/aglara/databaseserver.html
• https://postgresql.zeef.com/dennis.brouwer

Commands

• http://www.postgresqltutorial.com/

Configuration

• http://www.postgresql.org/docs/9.3/static/high-availability.html
• http://www.uptimemadeeasy.com/databases/centos-install-postgres-9-3/
• http://tecadmin.net/install-postgresql-on-centos-rhel-and-fedora/
• http://www.davidghedini.com/pg/entry/install_postgresql_9_on_centos

120 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

• https://chartio.com/resources/tutorials/how-to-list-databases-and-tables-in-postgresql-using-psql
• http://www.thegeekstuff.com/2009/04/15-practical-postgresql-database-adminstration-commands/
• https://launchschool.com/blog/how-to-install-postgres-for-linux
• https://www.digitalocean.com/community/tutorials/how-to-use-roles-and-manage-grant-permissions-in-
postgresql-on-a-vps–2

Replication using Slony

• http://www.hack.net.br/2015/02/24/postgresql-replication-with-slony/
• https://www.howtoforge.com/configuring-slony-i-cascading-replication-on-postgresql-8.3
• https://opensourcedbms.com/dbms/setup-replication-with-postgres-9-2-on-centos-6redhat-el6fedora/
• http://www.linuxjournal.com/article/7834?page=0,0
• http://raghavt.blogspot.in/2012/07/simple-slony-i-replication-setup.html
• https://miceliux.com/blog/2013/02/09/postgresql-replication-with-slony/

Tuning & Hardening

• https://www.digitalocean.com/community/tutorials/how-to-secure-postgresql-against-automated-attacks
• https://spapas.github.io/2016/11/02/postgresql-backup/

Troubleshooting

• https://stackoverflow.com/questions/17443379/psql-fatal-peer-authentication-failed-for-user-dev/21889759

FAQ - When pg-cancel-backend / pg-terminate-backend doesn’t work

• https://serverfault.com/questions/35332/what-do-i-do-when-pg-cancel-backend-doesnt-work

Removing PostgreSQL queries on TCP Level - when pg-cancel-backend / pg-terminate-backend fails

• https://tech.zalando.com/blog/hack-to-terminate-tcp-conn-postgres/?gh_src=4n3gxh1

Official Links

• http://slony.info/documentation/index.html
• http://slony.info/documentation/tutorial.html

1.3.37 NoSQL

Some useful links to explain the concepts of NoSQL

1.3. Please Note: 121

DevOps-Notes Documentation, Release 2.0.2

Concepts

• https://nosql.zeef.com/stefan.edlich
• https://dev.to/mushketyk/should-you-use-dynamodb-5m5

Configuration

Troubleshooting

1.3.38 Redis

Some useful links to cover the concepts of Redis

Concepts

Official Redis Documentation

• https://redis.io/documentation

Configuration

Tuning & Hardening

• http://ozzimpact.github.io/development/redis-setup-and-high-availability

Troubleshooting & Log Parsing

• https://redis.io/commands

1.3.39 Kafka

Some useful links to cover the concepts of Kafka

Concepts

• https://www.cloudkarafka.com/blog/2016-11-30-part1-kafka-for-beginners-what-is-apache-kafka.html

Official Kafka Documentation

• https://kafka.apache.org/documentation

122 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

Configuration

Tuning & Hardening

Troubleshooting & Log Parsing

1.3.40 Email

Some useful links to explain the concepts of Email

Concepts

E-Mail status Codes

• https://tools.ietf.org/html/rfc1893

Article on E-Mail Ports

• http://blog.mailgun.com/25-465-587-what-port-should-i-use/

1.3.41 Postfix

Some useful links to explain the concepts of Postfix

Concepts

Configuration

• http://blog.schaal-24.de/mail/postscreen-im-kampf-gegen-spam/?lang=en
• https://www.howtoforge.com/hardening-postfix-for-ispconfig-3
• https://workaround.org/ispmail/squeeze/postfix-domain-types
• https://workaround.org/article/postfix-database-configuration
• https://www.howtoforge.com/virtual_postfix_antispam
• https://rtcamp.com/tutorials/mail/
• https://www.linode.com/docs/email
• https://www.linode.com/docs/email/running-a-mail-server

SSL/TLS on Postfix

• https://blog.kruyt.org/postfix-and-tls-encryption/

1.3. Please Note: 123

 DevOps-Notes Documentation, Release 2.0.2

Troubleshooting & Log Parsing

Queue Management

• http://www.postfix.org/QSHAPE_README.html
• https://easyengine.io/tutorials/mail/postfix-queue/
• http://wiki.zimbra.com/wiki/Managing_The_Postfix_Queues
• http://www.tullyrankin.com/managing-the-postfix-queue
• https://www.wirehive.net/blog/2014/11/07/5-top-tips-for-reviewing-your-postfix-mail-queue

Commands

• Sorting queued mails by From address

1 sudo mailq | awk '/^[0-9,A-F]/ {print $7}' | sort | uniq -c | sort -n

• Holding queued mails by From address

1 sudo mailq| grep '^[A-Z0-9]'| grep <sender-ID>| cut -f1 -d' ' | tr -d \*|sudo
˓→postsuper -h -

• Holding queued mails by To address

1 sudo mailq | tail -n +2 | grep -v '^ *(' | awk 'BEGIN { RS = "" } { if ($8 == "
˓→<recipient>") print $1 } ' | tr -d '*!' | sudo postsuper -h -

• Holding queued mails by Domain

1 sudo mailq| grep '^[A-Z0-9]'| grep @<domain>| cut -f1 -d' ' | tr -d \*|sudo postsuper
˓→-h -

• Holding emails from the [active|deferred] queue based on subject

1 sudo find /var/spool/postfix/[active|deferred]/ -type f -exec grep -il '<subject>' '

˓→{}' \; | xargs -n1 basename | sudo postsuper -h -

• Removing Mails based on sender Address

1 sudo mailq| grep '^[A-Z0-9]'| grep <sender-ID>| cut -f1 -d' ' | tr -d \*|sudo
˓→postsuper -d -

• Removing Mails based on Domain

1 sudo mailq| grep '^[A-Z0-9]'| grep @<domain>| cut -f1 -d' ' | tr -d \*|sudo postsuper
˓→-d -

• Delete mails to a specific mail address

1 sudo mailq | tail -n +2 | grep -v '^ *(' | awk 'BEGIN { RS = "" } { if ($8 == "
˓→<recipient-ID>") print $1 } ' | tr -d '*!' | sudo postsuper -h -

124 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

1.3.42 Exim

Some useful links to explain the concepts of Exim

Concepts

Configuration

Queue Management

• http://www.cyberciti.biz/faq/exim-remove-all-messages-from-the-mail-queue/
• http://www.electrictoolbox.com/show-exim-mail-queue/
• https://www.ndchost.com/wiki/mail/exim-management

1.3.43 Dovecot

Some useful links to explain the concepts of Dovecot

Concepts

• http://www.stefan-seelmann.de/wiki/mailserver-postfix-dovecot
• http://wiki2.dovecot.org/FrontPage

Configuration

• http://bobcares.com/blog/dovecot-sendmail-perfect-mail-server/
• http://www.stefan-seelmann.de/wiki/mailserver-postfix-dovecot
• https://www.digitalocean.com/community/tutorials/how-to-set-up-a-postfix-email-server-with-dovecot-dynamic-maildirs-and-lm
• https://workaround.org/ispmail/squeeze/setting-up-dovecot

Troubleshooting & Log Parsing

1.3.44 Milters

This section covers the basics of how Milters works

Concepts

Email RBLs (real-time block-hole lists) Primer

• https://en.wikipedia.org/wiki/DNSBL
• https://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists
• https://www.spamhaus.org/whitepapers/dnsbl_function/
• https://kb.mailchimp.com/delivery/spam-filters/how-blacklists-work

1.3. Please Note: 125

DevOps-Notes Documentation, Release 2.0.2

• https://www.pinpointe.com/blog/how-do-i-know-if-im-on-a-spam-blacklist
• https://rbltracker.com/blog/2015/07/what-are-rbls-and-how-do-they-work-part-1/
• https://blog.returnpath.com/blacklist-basics-the-top-email-blacklists-you-need-to-know-v2/
• https://techtalk.gfi.com/dns-blacklists-work/
• https://sendgrid.com/blog/email-blacklist/

Configuration

Check multiple global RBLs online

• http://multirbl.valli.org/lookup/
• http://mxtoolbox.com/
• http://www.dnsbl.info/

126 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

Troubleshooting & Log Parsing

1.3.45 SPF & DKIM

Some useful links to explain the concepts of SPF & DKIM

Concepts

SPF Records

• http://blog.postmarkapp.com/post/86506131503/explaining-spf
• http://blog.hyperfive.com/2011/05/spf-records-explained.html
• http://www.openspf.org/SPF_Record_Syntax

Configuration

• https://www.digitalocean.com/community/tutorials/how-to-use-an-spf-record-to-prevent-spoofing-improve-e-mail-reliability
• https://cheriches.com/linux/mail-server-series/2014-09-07-dedicated-mail-server-hosting-multiple-domains-step-5-antispam.
html
• https://kura.io/2015/01/03/debian-wheezy-tls-mailserver-with-mysql-clamav-domainkeys-dkim-spf-solr-imap-search/
#ssl
• http://www.stevejenkins.com/blog/2010/09/how-to-get-dkim-domainkeys-identified-mail-working-on-centos-5-5-and-postfix-us
• https://easyengine.io/tutorials/mail/setup-opendkim/

Troubleshooting and Log Parsing

1.3.46 ClamAV

Some useful links to explain the concepts of ClamAV

Concepts

• https://forums.freebsd.org/threads/can-anyone-explain-clamav-to-me.29798/
• https://cheriches.com/linux/mail-server-series/2014-09-16-clamav-milter.html

Configuration

• https://z0z0.me/2014/10/26/install-postfix-dovecot-auth-tls-mysql-postfixadmin-spamassassin-and-clamav-on-centos7/
• http://swimminginthought.com/postfix-and-clamav-milter-gotchas-how-to-fix-them-and-keep-your-sanity/
• http://linux.die.net/man/8/clamd
• http://www.devguerrilla.com/notes/2014/09/linux-speeding-up-clamav-with-clamd-on-rhel/
• http://pantestmb.blogspot.in/2013/10/fedora-19-systemctl-sendmail-clamav.html

1.3. Please Note: 127

 DevOps-Notes Documentation, Release 2.0.2

Troubleshooting & Log Parsing

1.3.47 DNS

Some useful links to explain the concepts of DNS

Concepts

• Authoritative NS
– When a DNS query is made to a server which has the domain’s data, it is an authoritative NS, other-
wise it will point to other NS or serve cached copies of other NS
• Zone file
– simple text file containing the mapping between domain names and IP addresses, e.g :
www.google.com
• Root Servers
– 13 servers - a to h, routed to the nearest mirror of the server
• TLD servers :
– .com [others are : .org, .net, .edu etc]
• Domain Level NS
– the server containing the actual records of the requested domain (ns1.google.com, ns2.google.com
etc)
• TTL - Time to live
– A timer. Caching name servers can use this until the TTL runs out
• Records

1 domain.com. IN SOA ns1.domain.com. admin.domain.com. (

2 12083 ; serial number - incremented on zone file change, slave NS checks if master
˓→NS serial > cached serial & if yes, slave NS requests for updated zone else serves

˓→same zone file.

3 3h; refresh interval - Slave NS waits this period to poll the master NS for changes
4 30m; retry interval - Slave NS will retry querying master NS every this period for
˓→zone transfer updates

5 3w; expiry period - if slave NS can not contact master for this time, it will no
˓→longer return authoritative response for the queried zone

6 1h ; negative TTL - a NS will cache errors for this period

7 )

Domain Transfer (AXFR)

• The original DNS specifications RFC-1034 & RFC-1035 envisaged that slave (or secondary) DNS servers would
poll the master.
• The time between such ‘polling’ is determined by the refresh value on the domain’s SOA Resource Record
• The polling process is accomplished by the ‘slave’ sending a query to the master and requesting its current SOA
record.

128 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

• If serial number of this record is higher than the current one maintained by the slave a zone transfer (AXFR) is
requested & done on TCP Port 53.

DNS uses UDP for DNS queries over port 53

• DNS uses UDP for to replying to client DNS queries such as client asking DNS server for a Name to IP or IP to
NAME resolution.
• The reason is that UDP is not connection oriented, so its light-weight & fast, resulting in faster data transmission
of results to client compared to TCP.
• At the same time, if needed then DNS can also work over TCP to serve the DNS queries, but UDP is always
preferred because of greater speed.

Why DNS uses TCP for Zone files transfer over port 53

• DNS uses a master & slave architecture, in which one main authoritative Name server having all the entries &
others are replicated (zone files transferred) from master & also serve DNS queries.
• As there can’t be any inconsistency in Zone files, so to transfer these Zone files DNS uses TCP as the commu-
nication protocol, which makes sure that the zone files are transferred reliably.

Resource Records

• A record
– map a host to an IP address
host IN A IPv4_address host IN AAAA IPv6_address
• MX Record
– map a mail exchange used for the domain
IN MX 10 mail.domain.com. (where 10 is record priority. Priority is given to
MX with lower values at DNS lookup)
• PTR
– maps an IP address to a reverse name

How do resolvers work

• What happens when you set resolvers in PC (Windows) And / Or Router

– A browser 1st checks its internal cache of recent queries which it checks initially otherwise it asks the
system resolver for DNS queries (/etc/hosts) else it forwards requests to another resolver.

1.3. Please Note: 129

DevOps-Notes Documentation, Release 2.0.2

Types of DNS Servers

• Recursive:
– A DNS server which queries other servers until it finds answer to the queried domain. They maintain
a cache which is initially checked before sending the app’s query to another NS.
• Iterative:
– To be explained
• Authoritative-Only :
– Only answers those queries for which it stores the zones. Does not respond to recursive queries &
cache query results.
• Caching :

130 Chapter 1. Overview

 DevOps-Notes Documentation, Release 2.0.2

– It handles recursive queries from clients which handles queries received from the OS stub resolver
(/etc/hosts).
• https://gitlearning.wordpress.com/2015/06/23/dns-server/
• https://danielmiessler.com/study/dns
• https://support.google.com/a/answer/48090?hl=en
• http://www.slashroot.in/what-dns-zone-file-complete-tutorial-zone-file-and-its-contents
• https://ns1.com/blog/glue-records-and-dedicated-dns
• http://www.slashroot.in/mx-record-dns-explained-example-configurations
• http://www.slashroot.in/dns-root-servers-most-critical-infrastructure-internet
• http://www.slashroot.in/difference-between-iterative-and-recursive-dns-query
• http://www.slashroot.in/what-is-dns-cname-record
• https://www.digitalocean.com/community/tutorial_series/an-introduction-to-managing-dns
• https://www.digitalocean.com/community/tutorials/an-introduction-to-dns-terminology-components-and-concepts
• http://technify.me/systems/dns-explained-so-you-can-understand/
• https://luxsci.com/blog/understanding-domain-name-service-dns.html
• http://www.menandmice.com/support-training/support-center/knowledgehub/dns-glossary/
• http://computer.howstuffworks.com/dns.htm
• https://en.wikipedia.org/wiki/Wildcard_DNS_record
• DNS Explained via YouTube

Why are there are only 13-root DNS servers

• https://www.netnod.se/i-root/what-are-root-name-servers
• https://techiemaster.wordpress.com/2016/06/09/why-only-13-root-dns/amp/
• https://www.lifewire.com/dns-root-name-servers-3971336

Configuration

Bind Configuration / Tweaks

• https://www.digitalocean.com/community/tutorials/how-to-configure-bind-as-a-private-network-dns-server-on-ubuntu-14-04
• https://www.digitalocean.com/community/tutorials/how-to-configure-bind-as-a-private-network-dns-server-on-centos-7

Managing DNS Better

• https://blog.powerdns.com/2015/03/11/introducing-dnsdist-dns-abuse-and-dos-aware-query-distribution-for-optimal-performanc
• https://www.digitalocean.com/community/tutorials/how-to-deploy-and-manage-your-dns-using-dnscontrol-on-ubuntu-18-04

1.3. Please Note: 131

DevOps-Notes Documentation, Release 2.0.2

Troubleshooting & Log Parsing

• http://www.tecmint.com/10-linux-dig-domain-information-groper-commands-to-query-dns/
• http://www.cyberciti.biz/faq/linux-unix-dig-command-examples-usage-syntax/
• http://www.thegeekstuff.com/2012/02/dig-command-examples/
• https://mediatemple.net/community/products/dv/204644130/understanding-the-dig-command
• http://anouar.adlani.com/2011/12/useful-dig-command-to-troubleshot-your-domains.html
• http://www.cyberciti.biz/faq/dnstop-monitor-bind-dns-server-dns-network-traffic-from-a-shell-prompt/

Check DNS Propagation Issues

• https://intodns.com/
• https://www.nslookup.io/
• https://www.site24x7.com/dns-lookup.html
• http://viewdns.info/

132 Chapter 1. Overview