Shell scripting for system administrators: advanced techniques
Swayam Prakasha concludes his excellent three part series on shell scripting with some of the more advanced concepts, including two very useful commands…
In issues 91 and 92 of Linux User & Developer, we had a look at various aspects of shell scripting. We covered topics such as redirection, variables, arithmetic expansions, control constructs, various operations on strings etc.
We also had a detailed look at some of the popular shell commands such as ‘find’ and ‘grep’. Thus an in-depth understanding of shell scripting is very essential if someone wants to become an expert in system administration tasks. It is recommended to go through those two articles first via the links at the bottom of this page.
In this part, our focus will be on various advanced concepts of shell scripting and see how they are going to be beneficial for software developers and system administrators.
Two important concepts in shell scripting are ‘sed’ and ‘awk’. Let us take a detailed look at each of these.
sed – This is a non-interactive stream editor. A stream editor is used to perform various forms of text transformations on an input stream. In its basic form, it reads input files line by line and applies the operation specified via the command line. Therefore, if you want to write a program to make changes in a file, sed should be your ultimate choice. IT professionals have considered sed as the successor to the popular ‘grep’ command. Fig 1 gives more information about this command.
Please go through the complete man page to find out more about this specific command. If you want to learn more about how to use the sed command, please take a look at Fig 2.
Let’s understand this command with some examples. sed has several commands and the simplest one is the substitute command, ‘s’.
Consider a sample test file with these contents:
[sourcecode language=”bash”]This is a sample program.
abc def ghi klm abc opq abc
Now let’s use sed for a well-known in-place editing of files:
[sourcecode language=”bash”]cvs@cn-setta-2:~> sed -i ‘s/abc/ABC/’ test_file
cvs@cn-setta-2:~> cat test_file
This is a sample program.
ABC def ghi klm abc opq abc
The above example shows that the sed command has replaced the first occurrence of ‘abc’ with ‘ABC’. If we want to replace all occurrences of ‘abc’ by ‘ABC’, then the following command will do the work for us:
[sourcecode language=”bash”]cvs@cn-setta-2:~> sed -i ‘s/abc/ABC/g’ test_file
It is important to note here that ‘sed -i’ overwrites the original file with the new one. Let us look at another typical usage of sed:
[sourcecode language=”bash”]cvs@cn-setta-2:~> sed -e ‘s/abc/DEF/g’ test_file > test_file_out
cvs@cn-setta-2:~> cat test_file_out
This is a sample program.
DEF def ghi klm DEF opq DEF
It is not always needed to specify the -e option. Here, ‘s’ stands for substitute and ‘g’ stands for global. Thus, as expected, all occurrences matching the specified criteria are replaced and the output is redirected to another file.