Basic Shell and Basic Scripting

Objectives:

  • Explain the features and capabilities of bash shell scripting.
  • Know the basic syntax of scripting statements.
  • Be familiar with various methods and constructs used.
  • Test for properties and existence of files and other objects.
  • Use conditional statements, such as if-then-else blocks.
  • Perform arithmetic operations using scripting language.

The return value of a script is stored in the environment variable represented by $? A non-zero value represents failure, while a zero value represents success.

Special characters used in bash scripts:

  • # – comment
  • \ – continuation on  a new line; splitting long commands on multiple lines
  • ; – what follows is a new command to be executed
  • $ – indicate a variable
  • > – redirect output
  • >> – append output
  • < – redirect input
  • | – pipe result into the new command
  • && – abort subsequent commands when an earlier one fails

Typing help in the shell, will display the built-in commands and help.

Script parameters are depicted by the $ followed by a number.

  • $0 – script name
  • $1 – 1st argument
  • $2, $3 – 2nd, 3rd  argument
  • $* – all arguments
  • $# – number of arguments

Command substitution

  • by enclosing the inner command in $(   ) – prefered form
  • by enclosing the inner command with backtick ( ` ) – deprecate form
$ ls /lib/modules/$(uname -r)

Environment Variables

Standard environment variables:

  • PATH
  • HOME
  • HOST

Print a list of environment variables with printenv or env.

$ printenv 
$ env

Set environment variables with the set command.

Variables can be exported from a local variable to a global (environment) variable using the export keyword.

$ export VAR=value
# or
# VAR=value; export VAR

Typing just export without any arguments will show all the exported environment variables.

Functions

Declaration of a function:

function_name () {
    command ...
}

Example of a function named display.

display () {
    echo "This is a sample function"
}

If statement

if TEST-COMMANDS; then CONSEQUENT-COMMANDS; fi

or

if condition
then
       statements
else
       statements
fi

File condition expressions can be viewed with man 1 test command.

Boolean Expressions

  • && – AND
  • || – OR
  • ! – NOT

Linux File Operations

Linux File Operations

Objectives:

  • explore the file system
  • explain the file architecture
  • compare files and identify different file types
  • backup and compress data

In Linux “almost” everything is a file, or treated as such.

Linux native file systems:

  • ext3
  • ext4
  • squashfs
  • btrfs

Linux can implement other file systems:

  •  windows (ntfs, fat, vfat)
  • sgi
  • macos (hfs, hfs+)

On a hard disk, partitions can organize data. Each partition can have its own file system.

A file system can be mounted at a mount point (usually an empty directory). If the directory is not empty, the contents of that directory are covered-up by the new file system and will not be available until the file system is unmounted.

# mount a file system
$ sudo mount /dev/sda5 /home

# unmount a filesystem/partition
$ sudo unmount /home

To automatically mount a filesystem every time the system starts up edit the /etc/fstab file.

# show mounted filesystems
$ df -hT

# display read-only or writable status of mounted file system
$ mount

NSF service

# enable NFS
$ sudo systemctl enable nfs

# start NSF
$ sudo systemctl start nsf

File /etc/exports contains the directory permissions. After modifying /etc/exports, run exportfs -av to notify Linux about directories that can be remotely mounted.

Directories

/proc contains virtual files that permit viewing constantly changing kernel data.

/dev contains device nodes.

/var contains files  that are expected to change in size as the system is running.

  • /var/logs – system logs
  • /var/lib – package and database files
  • /var/spool – print queues
  • /var/tmp – temporary files

/etc directory contains system configuration files; no binary programs

/boot directory contains essential files needed to boot the system. For every alternative kernel installed on the system there are four files:

  1. vmlinuz – compressed linux kernel
  2. initramfs – initial ram file system required fr booting (aka initrd)
  3. config – configuration file
  4. system.map – kernel symbol table (used for debugging)

GRUB files are also located in the /boot folder

  • /boot/grub/grub.conf
  • /boot/grub2/grub2.conf

/lib folder contains libraries for programs located in the /bin or /sbin folders.

Comparing Files

diff [options] <fileName1> <fileName2>

diff3 – compare changes make to 3 different files.

# compare the content of 3 files
$ diff3 myFile refFile yourFile
# creating a patch file
$ diff -NUr origFile newFile > patchFile

# apply the patch file
patch -p1 > patchFile
patch originalFile patchFile
# file utility; get information about the file
$ file fileName

Backing up data

Use rsync to backup data.

$ rsync --progress -avrxH sourceDir destDir

Compressing data

  • gzip – most common compression utility
  • bzip2 – produces smaller files than gzip
  • xz – most space efficient
  • zip – similar to gzip
  • unzip – extract all files in the backup file
  • tar – group files in an archive before compressing

Prefix compression commands with time to display compression times and sizes.

$ time tar zcf include.tar.gz include

Copying disk-to-disk

# backup the Master Boot Record
$ dd if=/dev/sda of=sda.mbr bs=512 count=1

# copy one disk to another
$ dd if=/dev/sda of=/dev/sdb

 

Give a local user ‘sudo’ privileges in Debian Linux

In Debian GNU/Linux users do not have ‘sudo’ privileges by default.

In order to allow ‘sudo’ privileges for a user, there are three options:

  1. add the user to the ‘sudo’ group.
  2. create a sudousers file in the /etc/sudoers.d folder, and individually allow ‘sudo’ privileges to each user or group
  3. edit the /etc/sudoers file directly to add users or groups

Option 3 is not my favorite, as I do not like to edit the sudoers file directly, therefore we are left with the first two options:

Option 1: add the user to the ‘sudo’ group

This is the easiest method to add sudo privileges.

//login as root
$ su -

//add the user to the sudo group
# usermod -aG sudo username

Option 2: create a sudousers file in /etc/sudoers.d

//login as root
$ su -

//create a sudousers file in /etc/sudoers.d
# nano /etc/sudoers.d/sudousers

//add user and group privilege specifications
user    ALL=(ALL:ALL) ALL
%group    ALL=(ALL:ALL) ALL

//save the file

//set file mode to 0440
# chmod 0440 sudousers

//edit the /etc/sudoers file to include files from /etc/sudoers.d
//the sudoers file should always be edited as root with visudo
# visudo

//the pound sign(#) is NOT a commented line; it has to prefix includedir, otherwise a parsing error will occur
#includedir /etc/sudoers.d


//save the file

Log out from system and log back in order to activate the changes.

Test the sudo privileges for the new user(s).

Enable root login via SSH on Linux/Unix systems

Most Linux/Unix systems have root login disabled by default.

It is a huge security risk to allow root login via SSH, so if for some particular reason you need root login via SSH, enable it temporarily to perform need tasks, then disable it right away to secure your system.

FreeNAS:

  • go to Services -> SSH config
  • enable Login as Root with password
  • OK to save

Linux/FreeBSD:

  • edit the /etc/ssh/sshd_config file
    • $ sudo nano /etc/ssh/sshd_config
  • set PermitRootLogin yes
  • restart sshd
    • $ systemctl restart sshd

For more information on sshd_config settings read the man page.

How to find out what version of Linux Debian are you running?

Open Terminal and run any of the following commands:

  • $ cat /etc/issue
  • $ cat /etc/debian_version
  • $ lsb_release -a
$ cat /etc/issue
Debian GBU/Linux 9 \n \l
$ cat /etc/debian_version
9.3
$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.3 (stretch)
Release: 9.3
Codename: stretch

To find out more about the lsb (linux standard base) command read the man page or issue the -h (help) option.

$ lsb_release -h
Usage: lsb_release [options]
Options:
-h, --help show this help message and exit
-v, --version show LSB modules this system supports
-i, --id show distributor ID
-d, --description show description of this distribution
-r, --release show release number of this distribution
-c, --codename show code name of this distribution
-a, --all show all of the above information
-s, --short show requested information in short format

 

Install VirtualBox Guest Additions on Debian Linux to View Guest OS in Full Screen

When you first install Debian in VirtualBox you can’t adjust the size of the guest OS. In order to be able to adjust the screen size of the guest OS you must install VirtualBox Guest Additions on the guest OS. This will allow you to use the guest OS just like you were using a normal OS.

Here is how to do this:

  • log in as root or with an account that has root privileges
  • update & upgrade the system
apt-get update && apt-get upgrade
  • install required packages for building kernel modules
apt-get install dkms build-essential module-assistant linux-headers-$(uname -r)
  • prepare your system for building kernel modules
m-a prepare
  • download the same version of VirtualBox Guest Additions that matches your VirtualBox version.
  • in the virtual machine window go to Devices -> Insert Guest Additions CD image …
  • mount the CD
mount /media/cdrom
  • run the VboxLinuxAdditions.run script
sh /media/cdrom/VboxLinuxAdditions.run
  • restart Debian

Important Note:

  • Do not Enable 3D Acceleration in Settings > Display. After installing the VirtualBox Linux Additions, the dispaly manager will crush and you won’t be able to see anything on the screen.

Configure Static DNS with DHCP on Debian

Purpose : configure your system to use your choice of DNS servers.

Why update DNS servers addresses?

  • you wish to use your own private DNS server
  • access your network devices by using a FQDN (fully qualified domain name) instead of an IP address
  • your ISP’s DNS servers are slow, creating a sluggish browsing experience

There are many free/public DNS servers. Here is a small list:

  1. Freenom: 80.80.80.80, 80.80.81.81
  2. Google: 8.8.8.8, 8.8.4.4
  3. OpenDNS: 208.67.222.222, 208.67.220.220
  4. FreeDNS: 37.235.1.174, 37.235.1.177
  5. Level3: 209.244.0.3, 209.244.0.4
  6. Cloudflare: 1.1.1.1, 1.0.0.1
  7. Quad9: 9.9.9.9, 149.112.112.112

When updating your DNS servers, always check with the provider to make sure they do not log any private information (e.g. requested IP addresses, ISP, or your geographic location)

To update the DNS servers on Debian you need to:

  1. edit the /etc/dhcp/dhclient.conf file
  2. restart the networking service

Simply adding the DNS servers to the /etc/resolv.conf file is not going to make the changes permanent. They will only last until your restart your dhcp client or reboot your system. This method is useful if you need to temporarily use or test new DNS servers.

Procedure:

1. Edit the /etc/dhcp/dhclient.conf file

To add static DNS servers that are used in addition to the ones provided by DHCP, use prepend

prepend domain-name-servers aaa.aaa.aaa.aaa, bbb.bbb.bbb.bbb

To completely override the DNS servers provided by DHCP, and entirely force the system to use only the ones you provide, use supersede

supersede domain-name-servers aaa.aaa.aaa.aaa, bbb.bbb.bbb.bbb

2. Restart the networking service

sudo /etc/init.d/networking restart

3. Check the /etc/resolv.conf file to make sure the new DNS servers where added.