Basic Shell and Basic Scripting

Objectives:

  • Explain the features and capabilities of bash shell scripting.
  • Know the basic syntax of scripting statements.
  • Be familiar with various methods and constructs used.
  • Test for properties and existence of files and other objects.
  • Use conditional statements, such as if-then-else blocks.
  • Perform arithmetic operations using scripting language.

The return value of a script is stored in the environment variable represented by $? A non-zero value represents failure, while a zero value represents success.

Special characters used in bash scripts:

  • # – comment
  • \ – continuation on  a new line; splitting long commands on multiple lines
  • ; – what follows is a new command to be executed
  • $ – indicate a variable
  • > – redirect output
  • >> – append output
  • < – redirect input
  • | – pipe result into the new command
  • && – abort subsequent commands when an earlier one fails

Typing help in the shell, will display the built-in commands and help.

Script parameters are depicted by the $ followed by a number.

  • $0 – script name
  • $1 – 1st argument
  • $2, $3 – 2nd, 3rd  argument
  • $* – all arguments
  • $# – number of arguments

Command substitution

  • by enclosing the inner command in $(   ) – prefered form
  • by enclosing the inner command with backtick ( ` ) – deprecate form
$ ls /lib/modules/$(uname -r)

Environment Variables

Standard environment variables:

  • PATH
  • HOME
  • HOST

Print a list of environment variables with printenv or env.

$ printenv 
$ env

Set environment variables with the set command.

Variables can be exported from a local variable to a global (environment) variable using the export keyword.

$ export VAR=value
# or
# VAR=value; export VAR

Typing just export without any arguments will show all the exported environment variables.

Functions

Declaration of a function:

function_name () {
    command ...
}

Example of a function named display.

display () {
    echo "This is a sample function"
}

If statement

if TEST-COMMANDS; then CONSEQUENT-COMMANDS; fi

or

if condition
then
       statements
else
       statements
fi

File condition expressions can be viewed with man 1 test command.

Boolean Expressions

  • && – AND
  • || – OR
  • ! – NOT

Linux File Operations

Linux File Operations

Objectives:

  • explore the file system
  • explain the file architecture
  • compare files and identify different file types
  • backup and compress data

In Linux “almost” everything is a file, or treated as such.

Linux native file systems:

  • ext3
  • ext4
  • squashfs
  • btrfs

Linux can implement other file systems:

  •  windows (ntfs, fat, vfat)
  • sgi
  • macos (hfs, hfs+)

On a hard disk, partitions can organize data. Each partition can have its own file system.

A file system can be mounted at a mount point (usually an empty directory). If the directory is not empty, the contents of that directory are covered-up by the new file system and will not be available until the file system is unmounted.

# mount a file system
$ sudo mount /dev/sda5 /home

# unmount a filesystem/partition
$ sudo unmount /home

To automatically mount a filesystem every time the system starts up edit the /etc/fstab file.

# show mounted filesystems
$ df -hT

# display read-only or writable status of mounted file system
$ mount

NSF service

# enable NFS
$ sudo systemctl enable nfs

# start NSF
$ sudo systemctl start nsf

File /etc/exports contains the directory permissions. After modifying /etc/exports, run exportfs -av to notify Linux about directories that can be remotely mounted.

Directories

/proc contains virtual files that permit viewing constantly changing kernel data.

/dev contains device nodes.

/var contains files  that are expected to change in size as the system is running.

  • /var/logs – system logs
  • /var/lib – package and database files
  • /var/spool – print queues
  • /var/tmp – temporary files

/etc directory contains system configuration files; no binary programs

/boot directory contains essential files needed to boot the system. For every alternative kernel installed on the system there are four files:

  1. vmlinuz – compressed linux kernel
  2. initramfs – initial ram file system required fr booting (aka initrd)
  3. config – configuration file
  4. system.map – kernel symbol table (used for debugging)

GRUB files are also located in the /boot folder

  • /boot/grub/grub.conf
  • /boot/grub2/grub2.conf

/lib folder contains libraries for programs located in the /bin or /sbin folders.

Comparing Files

diff [options] <fileName1> <fileName2>

diff3 – compare changes make to 3 different files.

# compare the content of 3 files
$ diff3 myFile refFile yourFile
# creating a patch file
$ diff -NUr origFile newFile > patchFile

# apply the patch file
patch -p1 > patchFile
patch originalFile patchFile
# file utility; get information about the file
$ file fileName

Backing up data

Use rsync to backup data.

$ rsync --progress -avrxH sourceDir destDir

Compressing data

  • gzip – most common compression utility
  • bzip2 – produces smaller files than gzip
  • xz – most space efficient
  • zip – similar to gzip
  • unzip – extract all files in the backup file
  • tar – group files in an archive before compressing

Prefix compression commands with time to display compression times and sizes.

$ time tar zcf include.tar.gz include

Copying disk-to-disk

# backup the Master Boot Record
$ dd if=/dev/sda of=sda.mbr bs=512 count=1

# copy one disk to another
$ dd if=/dev/sda of=/dev/sdb

 

Set Static IP Address in FreeBSD

Before you begin, find out what is the interface name using the ifconfig command.

$ ifconfig -a
em0: flags=8843 metric 0 mtu 1500 options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
ether 7a:98:a6:53:cd:7a
hwaddr 7a:98:a6:53:cd:7a
inet 192.168.0.103 netmask 0xffffff00 broadcast 192.168.0.255
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049 metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo

In this case the interface name is em0.

a. set temporary static IP (until next reboot)

# set temporary IP address to 192.168.0.103
$ ipconfig em0 inet 192.168.0.103

b. set permanent static IP address (persistent across reboots)

Static IP configuration must be included in /etc/rc.conf. Edit rc.conf using your favorite editor.

Delete or comment out the following line

#ifconfig_em0="DHCP"

Add the following lines after hostname directive:

ifconfig_em0="inet 192.168.0.103 netmask 255.255.255.0"
defaultrouter="192.168.0.1"

Restart network interface service

$ sudo service netif restart 

Note:

Be aware that this method of setting a static IP address in rc.conf will disable the DHCP server. If you have custom DNS servers defined in /etc/dhclient.conf, they will not work. In this case, the custom DNS servers must be added in the /etc/resolv.conf file.

Give a local user ‘sudo’ privileges in Debian Linux

In Debian GNU/Linux users do not have ‘sudo’ privileges by default.

In order to allow ‘sudo’ privileges for a user, there are three options:

  1. add the user to the ‘sudo’ group.
  2. create a sudousers file in the /etc/sudoers.d folder, and individually allow ‘sudo’ privileges to each user or group
  3. edit the /etc/sudoers file directly to add users or groups

Option 3 is not my favorite, as I do not like to edit the sudoers file directly, therefore we are left with the first two options:

Option 1: add the user to the ‘sudo’ group

This is the easiest method to add sudo privileges.

//login as root
$ su -

//add the user to the sudo group
# usermod -aG sudo username

Option 2: create a sudousers file in /etc/sudoers.d

//login as root
$ su -

//create a sudousers file in /etc/sudoers.d
# nano /etc/sudoers.d/sudousers

//add user and group privilege specifications
user    ALL=(ALL:ALL) ALL
%group    ALL=(ALL:ALL) ALL

//save the file

//set file mode to 0440
# chmod 0440 sudousers

//edit the /etc/sudoers file to include files from /etc/sudoers.d
//the sudoers file should always be edited as root with visudo
# visudo

//the pound sign(#) is NOT a commented line; it has to prefix includedir, otherwise a parsing error will occur
#includedir /etc/sudoers.d


//save the file

Log out from system and log back in order to activate the changes.

Test the sudo privileges for the new user(s).

Changing the DNS nameservers in FreeBSD

Adding the nameservers to the /etc/resolv.conf file will only temporarily change the DNS nameservers. After the system is restarted, FreeBSD will default to the DNS servers obtained from the DHCP server.

To permanently alter the DNS nameservers to the ones you want to use, they need to be added to the /etc/dhclient.conf file.

$ sudo nano /etc/dhclient.conf

Add the following to the dhclient.conf file

interface "em0" {
    supersede domain-name-servers 192.168.0.100, 192.168.0.101;
}

Replace the interface “em0” with your default interface name, and obviously the IP address of the DNS nameservers with your own.

For more information read the man pages for dhclient and dhclient.conf.

Note:

If you disable the DHCP server in /etc/rc.conf, for example, by setting a static IP address, the supersede directive will not work. You must then edit the resolv.conf file manually and add the nameservers.

 

Services running on Linux

service is an application that runs in the background to perform some tasks or waiting for requests from other applications to perform tasks.

Some services are required by the operating system, others are initiated by the user. OS specific services are networking, cron, dbus, ssh, while some of the user specific services are web servers such as Apache or Nginx, or database server such as mysql.

SysVinit vs Systemd

SysVinit is the old and test proven init system, while the Systemd is the newer init system used in many Linux distros.

Location of SysVinit scripts:

Debian: /etc/init.d

Red Hat: /etc/rc.d

Location of Systemd scripts:

{lib/etc}/systemd/

SysVinit uses:

service <service-name> command <options>

Systemd uses:

systemctl [options …] command [service-name …]

Examples:

List all services in /etc/init.d

$ ls -al /etc/init.d

List all services (SysVinit)

$ sudo service --status-all

To start a service (SysVinit)

$ sudo service networking start
$ sudo service cron --full-restart

To stop a service (SysVinit)

$ sudo service networking stop

To show the status of a service (SysVinit)

$ sudo service cron status

List all services, including inactive ones (systemd)

$ systemctl -a --type=service list-units

Start a service (systemd)

$ systemctl start httpd.service

Stop a service (systemd)

$ systemctl stop networking.service

To show the status of a service (systemd)

$ systemctl status cron.service

See also Systemd-vs-SysVinit-cheatsheet

 

apt-cache Debian package query utility

apt-cache description:

  1. performs a variety of operations on the APT’s package cache
  2. does not manipulate the state of the system
  3. perform read-only operations on the APT package cache
  4. extract useful information from the APT package cache
  5. the metadata is acquired and updated via the ‘apt-get update’

apt-cache commands:

  • stats displays statistics about the cache
  • dump shows a short listing of every package in the cache
  • umet displays a summary of all unmet dependencies in the package cache
  • show pkg displays the package record for the named package
  • search regex performs a full text search on all available package lists
  • depends pkg shows a listing of each dependency a package has

Relevant Files and Folders

/etc/apt/sources.list
/etc/apt/sources.list.d/
/var/lib/apt/lists/
/var/lib/apt/lists/partial

 

How to find out what version of Linux Debian are you running?

Open Terminal and run any of the following commands:

  • $ cat /etc/issue
  • $ cat /etc/debian_version
  • $ lsb_release -a
$ cat /etc/issue
Debian GBU/Linux 9 \n \l
$ cat /etc/debian_version
9.3
$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.3 (stretch)
Release: 9.3
Codename: stretch

To find out more about the lsb (linux standard base) command read the man page or issue the -h (help) option.

$ lsb_release -h
Usage: lsb_release [options]
Options:
-h, --help show this help message and exit
-v, --version show LSB modules this system supports
-i, --id show distributor ID
-d, --description show description of this distribution
-r, --release show release number of this distribution
-c, --codename show code name of this distribution
-a, --all show all of the above information
-s, --short show requested information in short format