Learning Linux: The Essentials Notes

Creating a new user account

The user name should be "your full last name followed by the dash and then followed by the first two letters of the first name". Set the password to be: netlab123

sudo useradd -m -G sudo pollard-be
sudo passwd pollard-be
su - pollard-be

Working with files

Viewing

pwd - prints working directory

ls - list files in the directory

-l - long view, shows metadata

-a - show all files, even hidden files

-h - output human-readable file sizes

tree - show the file tree of a directory

less - provides advanced paging capability but is not included with all Linux distributions

more - has fewer features than less, but is available in Linux distributions

head - show lines from the start of a file

-n 10 - example of showing a specified number of lines

-n -10 - exclude that many lines from the end and only give you the first

tail - show lines from the end of a file

-n 10 - example of showing a specified number of lines

-n +10 - exclude that many lines from the start and only give you the first

wc - provides the number of lines, words and bytes

-l - just the numebr of lines

-w - show the number of words

-c - number of bytes

Sorting

By default, the ls command sorts files alphabetically by file name. Add these options to change that:

-S - sort by file size, large-to-small

-t - sort by time stamp, newest-to-oldest

--full-time - print full time

-r - reverse the defined sort

Copying

cp - copy files

cp [source(s)] [destination]

-i - will make the action interactive

-v - will make the action output verbose result information

-r - will allow copying directories (and copying files inside)

-p - will preserve file attributes, example date and permission modes

Moving

mv - move and/or rename file

mv [source(s)] [destination]

rename = mv helloworld.txt hello.txt

-i - will make the action interactive

-n - will not overwrite files in destination

-v - will make the action output verbose result information

Removing

Danger zone. Always use -i to see what you're doing.

rm - remove files or empty directories

rmdir - remove directory

Creating

touch - create an empty file

cat - displays file contents, create with syntax cat > file.txt and enter file contents

mkdir - make directory

Input/Output Control

Pipes

The pipe | character can be used to send the output of one command to another. The result of the command will be passed to the second command this way. The last command will show the output. Pipe always connects two commands, you can pipe as many as you need through.

cat /etc/psswd | head -n 5 | tail -n 2 | sort - The output will be the sort command

Redirection

Deals with streams of data, how is the data being sent into the computer system.

Three streams of data:

STDIN - Standard input, or STDIN, is information entered normally by the user via the keyboard

STDOUT - Standard output, or STDOUT, is the normal output of commands. > >>

STDOUT - Standard error, or STDERR, are error messages generated by commands 2> 2>>

STDOUT & STDERR - &> &>>

Input Redirection

This is the default flow. The following is the same:

cat < example.txt = cat example.txt

Output Redirection

By default output is shown in the terminal. Send the output to a file instead. This is useful for huge outputs.

Separating output and error is useful. Yuu can use 2> and 2>> to separate the errors.

> - if the file does not exist, the file will be made, if it does exist it will be overwritten.

>> - if output file does not exist, the file will be made, if does exist the new content will be appended.

date > d.txt - will redirect the output to a d.txt file, will contain what the date command would normally output

ls / blah 2> ls.err.txt - will create an error file with all the errors of the command

ls / blah &> ls.all.txt - output both errors and output to a file

Null device

find / -name linux 2> /dev/null - it works like a black hole, it doesn't go to the hard drive. If you will never review the errors why bother saving them? Discard them, banish them to the void.

Glob characters

Glob characters are often referred to as wild cards. These are symbol characters that have special meaning to the shell.

* - represent zero or more of any character in a filename

? - represents any single character, each question mark character matches exactly one character

[ ] - used to match a single character by representing a range of characters that are possible match characters

example: /etc/*[0-9]* attern displays any file that contains at least one number

! - used in conjunction with the square brackets to negate a range

Paths

In Linux, everything is stored in files. Files are used to store data such as text, graphics, and programs. Directories are a type of file used to store other files. Directories are used to provide a hierarchical organization structure.

A relative path is a path that specifies the location of a file or directory relative to the current working directory

An absolute path always begins from the absolute start of your hard drive and describes every step you must take through the filesystem to end up at the target location.

./ means this directory, always represents the current directory.

../ means up to parent directory, represents one directory higher relative to the current directory.

/ means the root directory, creating an absolute path.

~ means relative to home, path starts from home (not root) directory, creating an absolute path.

Regular Expressions

Resources:

learnbyexample: GNU BRE/ERE cheatsheet

BRE

. - match any single character

[ ] - match any of these characters once

^ - restricts the match to the start of the string

$ - restricts the match to the end of the string

* - repetition char, match 0 or more times (similar to {0,})

{x,y} - repetition char, from x to y many times

\{x,y\} - curly braces need to be escaped in BRE

{x,} - repetition char, from x many times

{,y} - repetition char, up to y many times

{x} - repetition char, exactly m many times

^$ is an empty line

ERE

? - repetition char, repeat 0 or 1 time (similar to {0,1})

+ - repetition char, repeat at least 1 time (similar to {1,})

(...) - grouping operator, apply action to multiple characters

| - or operator

ste(v|ph)en

Filtering & grep

Filter

cut - extract columns of text from a file or standard input

grep / egrep

This command is used to filter lines in a file or the output of another command that matches a specified pattern using regex.

grep [options] pattern [files]

grep -i '^[0-9].*[ts]' /etc/hosts

-c - prints only a count of the lines that match a pattern

-n - show line numbers in the output

-w - exact value search (word search), result will not pick up words containing search string

-i - switch to case insensitive mode

-e - EGREP mode, works with BRE (grep, basic regex) and ERE (extended regex)

-f - FGREP mode (fast grep, disables regex)

Archiving and compressing

Compression

Compression makes the files smaller by removing redundant information. The most common compression tool for Linux is gzip.

gzip - compress and decompress a file

gzip example.txt

-l - provides info about the compression action

-d - decompresses .gz files

gunzip - decompress a file

bzip - compress and decompress a file

Bash scripting

VI editor

vi mynewfile - create a new file "mynewfile" and edits it in vi.

i - insert mode

ESC - exit mode

j - move cursor down (arrow ↓)

k - move cursor up (arrow ↑)

l - move cursor right (arrow →)

h - move cursor left (arrow ←)

w - move cursor beginning of the next word

e - move cursor to the end of the word

b - move cursor to the beginning of previous word

$ - move cursor to the end of line

0 - move cursor to the beginning of line

SHIFT+G - move cursor to the beginning of line

dw - delete word 2dw - delete two word

dd - delete line

yk - copy/yank current word

P - paste last deleted/yanked

x - delete character

u - undo

o - add blank line

:%s/text //g - search for and delete the word "text"

:e! - discard changes and reload

:wq - write to disc and exit vi (save and quit)

:q! - quit without saving

To make it clear that this is a BASH shell script, you need to include a special line at the top of the file called a "shbang" (or "shebang"). This line starts with #! and then contains the path to the BASH shell executable #!/bin/bash.

bash sample.sh - to run a script.

You can avoid having to type bash in front of the filename by making the file "executable" for all users like this:

chmod a+x sample.sh
./sample.sh

Variables

It is important that there are no spaces between the name of the variable, the equals sign, and the item to be assigned to the variable.

To assign to a variable, just use the name of the variable. To access the contents of the variable, prefix it with a dollar sign.

#!/bin/bash

ANIMAL="penguin"
echo "My favorite animal is a $ANIMAL"

Another way to assign to a variable is to use the output of another command as the contents of the variable by enclosing the command in back ticks:

#!/bin/bash

CURRENT_DIRECTORY=`pwd`
echo "You are in $CURRENT_DIRECTORY"

The read command can accept a string right from the keyboard or as part of command redirection

#!/bin/bash

echo -n "What is your name? "
read NAME
echo "Hello $NAME!"

A dollar $ sign followed by a number N corresponds to the Nth argument passed to the script. If you call the following example with ./test.sh World the output will be Hello World. The $0 variable contains the name of the script itself.

#!/bin/bash

echo "Hello $1"

#!/bin/bash

if [ $# -ne 2 ]
  then
    echo "You need to give TWO args"
    exit 1
fi

let sum=$1+$1
echo "The sum of $1 and $2 is $sum"

Conditionals

A basic if statement looks like this:

#!/bin/bash

if grep -q root /etc/passwd; then
echo root is in the password file
else
echo root is missing from the password file
fi

The if statement has a final form that lets you do multiple comparisons at one time using elif (short for else if).

#!/bin/bash

if [ "$1" = "hello" ]; then
  echo "hello yourself"
elif [ "$1" = "goodbye" ]; then
  echo "nice to have met you"
  echo "I hope to see you again"
else
  echo "I didn't understand that"
fi

The case statement provides a different way of making multiple tests easier:

#!/bin/bash

case "$1" in
hello|hi)
  echo "hello yourself"
  ;;
goodbye)
  echo "nice to have met you"
  echo "I hope to see you again"
  ;;
*)
  echo "I didn't understand that"
esac

The test command gives you easy access to comparison and file test operators.

test –f /dev/ttyS0 gives a 0 if the file exists

test ! –f /dev/ttyS0 gives a 0 if the file does not exists

test –d /tmp gives a 0 if the directory exists

test 1 –eq 1 0 if numeric comparison succeeds

test 1 –ne 1 test for numeric inequality

test “a” = “a” string comparison, 0 if succeed

test 1 –eq 1 –o 2 –eq 2 test OR

test 1 –eq 1 –a 2 –eq 2 test AND

Loops

for loops are used when you have a finite collection over which you want to iterate.

#!/bin/bash

SERVERS="servera serverb serverc"
for S in $SERVERS; do
  echo "Doing something to $S"
done

for NAME in Sean Jon Isaac David; do
  echo "Hello $NAME"
done

for S in *; do
  echo "Doing something to $S"
done

A while loop, operates on a list of unknown size. Can think of it as “while some condition is true, do stuff.”

#!/bin/bash

i=0
while [ $i -lt 10 ]; do
echo $i
i=$(( $i + 1))
done
echo “Done counting”