linux.txt

Linux Commands:
Operating System:
    -- OS works with CPU
    
CLient OS vs Server OS:
    -- client, an OS made for individual, private user
        -- EG: Win 11, Mac OS

    -- Server OS works for mutliple users
        -- EG: RHEL, Win Server 2022 

Linux OS:
    -- windows is GUI based
    -- Mac is made upon UNIX Enterprise (shell based)
    -- Linus while working on UNIX created Linux
    -- aws due to its scalable nature, names elastic

Linux Working:
    -- linux core is its kernel, through shell
    -- kernel is written in C, that talks to hardware

    Shell:
        -- command is written on shell, its tranlation is forwarded to kernel, and kernel talks to hardware
        -- shell command, list is shell scripting,  
        -- sh standard, bash borne again (p), csh corn , zsh, fish are all types
        -- / is known as root directory

    Sudo:
        -- Its a group that has super user permission

    systemctl:
        -- tool that controls installed services

Pacakge Manager:

    apt:
        -- application package manager that ubuntu uses
        -- RHEL uses yum, dnf

    apt update:
        -- fetches packages

    apt upgrade:
        -- downloads packages

VIM:

    vim file_name:
        -- open / create a file

    i:
        insert mode

    esc:
        -- escape to come out of a mode
        
    :wq: 
        -- w -write, q -quit

    :qa!:
        -- if a file is readme only the use sudo with vim - sudo vim index.html
        -- quit without saving


Directory Structure:

    var:
        -- variable information like cache, logs

    tmp:
        -- temporary folder, any user can access
        -- its green bcz any one can acesss

    mnt:
        -- mount storage and devices

    home:
        -- where users are placed

    sbin:
        -- system level binary files are stored here
        -- machine convert commadn to binary 0,1
        -- blue for system level

    root:
        -- root is super user

    media:
        -- images and others

    etc:
        -- all configs stay here
        
    boot:
        -- system run program in ram, to start os

    bin:
        -- binaries


Basic Commands:
    clear:
        -- clear the screen

    echo "asd":
        -- prints asd on screen

    echo "text" > new_file.txt:
        -- put the text and push the output in the new file
        -- removes older output
        
    echo "text" >> new_file.txt:
        -- it appends into file with existing data

    pwd:
        -- present working directory

    mkdir dev:
        -- make directory named dev
        -- in linux is either a file or directory

    touch list.txt:
        -- make txt file, .sh for shell scripts
    
    ls:
        -- list files and folders
        -- folders are blue

    cd dev:
        -- change directory
        -- cd .. - move one step back
        -- cd or cd ~ will bring to current user

    man ls:
        -- manual about ls

    cp src dest:
        -- copy files and directory
        
    mv src dest:
        -- move files 

    rm file:
        -- remove file
        -- -r, recursively delete a dir
        -- -f, fore delete

    rmdir:
        -- remove directory

    cat:
        -- display file contents
        -- /etc/passwd - shows all users

SSH: 
    $SHELL: 
        -- the current shell type

    SSH: [Secure Shell]
        -- ssh -i "Web-Server-One-Key.pem" ubuntu@ec2-54-227-2-14.compute-1.amazonaws.com
            -- -i is for key location, ubuntu is username
            -- DNS is ec2-54-227-2-14.compute-1.amazonaws.com

    Bashton Host | Jump server:
        -- server that allows connection to other servers

    ssh-keygen:
        -- create a secret and public key
        -- private key must be on server A
        -- public keys are in .ssh/authorized_keys
        -- then we will add the publi ckey from server a to server b authorized_keys using vim

    scp -i prv_key src dest:
        -- secure copy files to another server

  
Users and groups:

    whoami:
        -- currently logged in user

    sudo useradd -m user_name:
        -- add user to system
        -- -m , make a home directory folder

    sudo useradd -m user_name -s /bin/bash:
        -- this opens in bash 
        -- when we add a user, group is also greated

    sudo passwd user_name:
        -- set password on a user

    sudo userdel user_name:
        -- delete a user

    sudo su:
        -- switch to root user

    su - user_name:
        -- switch / substitute user
        -- we enter into the user 
        -- but shell is changed from bash to sh

    cat /etc/passwd: 
        -- list all users

    cat /etc/group:
        -- list all group
    
    sudo groupadd devops:
        -- create a group

    sudo gpasswd:
    sudo gpasswd -M john,jack devops:
        -- add multiple users to a single group

    sudo usermod -aG devops john:
        -- -a append to a group, -G is for group


File Permissions:

    ls -l:
        -- list files verbose
        -- -r--------  1 safi safi  387 Feb 18 21:20 Web-Server-One-Key.pem
        -- first safi is user, second safi is group
        -- 387 is size        

    drwxr-xr-x:
        -- d = directory, r = read, w = write, x = execute, permissions
        -- file is read or wrtie, shell script is executable
        -- first three (rwx) are for user, second  three (r-x) is group, other users have (r-x)

    -rw-r--r--:
        -- - is for file
        -- first three (rw-) are for user, second  three (r--) is group, other users have (r--)
        -- 644 is the number

    Table:
        -- r    w   x   |   
           0    0   0   |   0
           0    0   1   |   1
           0    1   0   |   2
           0    1   1   |   3
           1    0   0   |   4
           1    0   1   |   5
           1    1   0   |   6
           1    1   1   |   7
        
    chomd 400 first.txt:
        -- we make the file readonly of user

    chomd 477 first.txt:
        -- making a file executable gives it green color

    sudo chown first.txt user:
        -- change ownershop of text file to user

    sudo chgrp first.txt user_group:
        -- change ownershop of group to user_group


Daily (awk, grep, sed, find)

    grep: [Global Regular Expression Print]

        grep -i "search_this" first.txt:
            -- -i is for case unsensitive

        grep -i "search_this" first.txt | head -n 10:
            -- we pass the output to head, to show 10 rows

        grep -i "search_this" first.txt | head -n 10:
            -- search last 10

    awk: 
        -- programmatic behaviour, we can use if, ==, $ 
    
        awk '{print}' first.txt
            -- print the file

        awk '/nginx/ {print $1,$2,$3}' first.txt
            -- $ represent the words seperated with space
            -- this is to filter and get other data

        awk '/nginx/ {$1="sudo"; print $1, $2, $3}' ./documentation/linux.txt
            -- Pattern Matching → /nginx/ ensures only lines containing "nginx" are processed.
            -- Field Replacement → $1="sudo" correctly replaces the first column.
            -- Correct Print Statement → print $1, $2, $3 prints the modified first three fields.

        awk '/nginx/ {$1="sudo"; print $1, $2, $3}' ./documentation/linux.txt > nginx_cmd.txt:
            -- we can forward output in a file
        
    sed: [Stream Editor]
        
        sed "s/user/username":
            -- /s is for string
            -- convert user to username in every stream

        awk '/nginx/ {$1="nginx"; print $1, $2, $3}' ./documentation/linux.txt | sed 's/\bnginx\b/http/'
            -- \b in sed → Ensures "nginx" is replaced as a whole word, avoiding partial replacements.
            -- Keeps awk logic intact → $1="sudo" modifies the first field and prints selected fields.
            -- Ensures correct piping → sed processes only the printed fields, not the full original line.

        sed 's/\bnginx\b/http/g' file.txt:
            -- /g changes file globaly

        sed 's/\bnginx\b/http/g' file.txt | uniq:
            -- uniq gives non-repeated data

    find:

        find . -name file.txt:
            -- ding the file in current path (.)
    
            
Networking:
    Domain: 
        -- used to reach an IP address

    nslookup:
        sudo apt update && sudo apt install bind9-dnsutils -y
            -- nslookup is a package of bind9-utils

    nslookup safi-siddiqui.com:
        -- gives name and address of domain

    ping 3.33.251.168:
        -- ping for trainwuthshubham.com
        -- ping uses ICMP protocol and it doesnot have ports
        -- we have to allow inbound in our ec2

    traceroute:
        sudo apt install traceroute

        debug:
            -- in case i used wrong ip
            -- i need to know how i went to this ip
            -- this shows how request goes from our ip to requested ip

    dig url|ip:
        -- gives detials and history

    wget url_to_file:
        -- download a file from server

    curl:
        Verbs:
            GET, POST, PUT, DELETE

        curl -X GET "https://jsonplaceholder.typicode.com/todos/1"
            -- perform a get request

        curl -X GET "https://jsonplaceholder.typicode.com/todos/1 | jq"
            -- pretty json

Volumes:
    -- for storage, blocks are required, like discs, in order to store data
    -- AWS EC2 internal volumes, AWS ECB [Eastic Block Store] store external volumes

    lsblk:
        -- show attached volumes

    df -h:
        -- shows mounted volumes
        -- also show disk space

    mount:
        -- bind volume to location, in order make it usable

    attach:
        -- atach volume to instance

    snapshot:
        -- backup volume

    attaching:
        -- in order to attach, give the name /dev/sdf, as abcde are already in use for root volume
        -- /dev/sdf will be changed to /dev/xvdf in instance
        -- when we attach adn us lsblk its shown 
        -- but there mount point is empty

    physical volume:
        -- in order to use volume, we must convert it into physical volume

    volume group:
        -- if we have attached two or more volumes, then we can combine them in a group 
        
    Logical Volumes:
        -- we can take small slices from groups

    LVM: [Logical Volume Manager]
        -- it manages root volume, physical and logical
        -- physical make group, group make logical

        Enter LVM:
            pvcreate /dev/xvdf /dev/xvdg /dev/xvdh:
                -- create physical volumes of 3 attached blocks

            pvs:
                -- list all physical volumes

            pvdisplay:
                -- list detail about physical volume

            vgcreate safi_vg_one /dev/xvdf /dev/xvdg:
                -- create volume group with two pvs

            vgs:
                -- list volume group

            vgdisplay:
                -- list detail about volume groups

            lgcreate -L 10G -n safi_lv_one safi_vg_one:
                -- create a logical volume from volume group

            lvs:
                -- list logical volumes

            lvdisplay:
                -- deatil list

    Mount Logical Volume:
        
        mkdir /mnt/safi_lv_one_mount:
            -- in order to monut lv we need to make a directory
        
        mkfs.ext4 /dev/safi_vg_one/safi_lv_one:
            -- then i need to format my lv inside the group

        mount /dev/safi_vg_one/safi_lv_one /mnt/safi_lv_one_mount/:
            -- then we mount the lv source to dest

        unount /mnt/safi_lv_one_mount:
            -- unmount makes it non-usable
            -- but data is persisted

    AWS EBS on EC2:
        -- if we need to directly mount an EBS volume without making a group, or ligical volume

        mkdir /mnt/safi_disk_one_mount:
            -- make dir for disk to mount

        mkfs -t ext4 /dev/xvdh:
            -- make file system of ext4 in the disc storage

        mount /dev/xvdh /mnt/safi_disk_one_mount/:
            -- mount disk to dir
        

    Dynamic Storage:
        -- in order to make our logical volume bigger

        lvextend -L +5G /dev/safi_vg_one/safi_lv_one:
            -- extend more space into  lv 
            -- this happens when our lv is connected to a group
            -- and we have enough space on our group
            -- this way lv extends from group from both pvs


Shell Scripting:
    -- shell commands in a sequence

    mkdir -p app-four:
        -- if not present then make dir

    .sh:
        -- extension for shell scripts

    echo $SHELL:
        -- current shell

    ls -l:
        -- view if file has execute permissions

    chmod 764 first.sh:
        -- to give execute permission to file, and file becomes green

    ./first.sh:
        -- run the file using path

    printenv:
        -- print all env variables

    ./first.sh some_line:
        -- both of these are arguments
        -- used using $0, $1
        -- $@ prints all arguments excluding $0
        -- $# counts all arguments excluding $0
        -- $? tells teh resut of exit, 0 on false 1 on true

    if [ -f data.txt ]:
        -- checks if a file exists

    ${#VAR}:
        -- gives count of variables

    scripts:
    
        #!:
            -- # is for comment, !is for denoting sh type
            -- #! is called as shebang

        title="title":
        echo $title:
            -- set and print variable

        $USER:
            -- predefined env variables

        read -p "Your msg" my_msg:
            -- read takes input from using, by prompting the string and saving it in my_msg variable

        <<help   <<any_word
        help      any_word  :
            -- multiline comments, we can use any word but must end with same name

        sudo apt install nginx -y > /dev/null:
            -- this doesnot dispkay the install proccess

        function show_name {
            echo "My name is safi"
        }
        show_name:
            -- make a function and call it

        for((num=1; num <=10; num++))
        do 
            show_name
        done

Backup:
    -- when we terminate an instance, all data is removed
    -- zip to compress and then move to backups dir

    zip -r shell_scripts.zip ./scripts:
        -- zip with all firs and files in scripts dir

    zip -r /home/safi/backups/backup1.zip /home/safi/app-four/scripts
    zip -r /home/safi/documentation/shell_scripts.zip /home/safi/app-four/scripts

    date '+%Y-%m-%d':
        -- date shows full date
        -- format it using 'expression'


CRON:
    -- schedule a task 

    crontab -e:
        -- enter into cron app

    * * * * * bash /home/safi/scripts/four.sh
        -- this creates file every minute

    watch ls /home/safi/app-scripts:
        -- we watch the directory getting populated 
    
    
Packages:

    Nginx:
        a reverse-proxy web server to hsot web apps

        sudo apt install nginx:
            -- install nginx package

        systemctl status nginx:
            -- status about nginx

        sudo systemctl start nginx
            -- start nginx service

        sudo systemctl stop nginx
            -- stop nginx service

        sudo systemctl enable nginx:
            -- when we stop our system, the services stop
            -- enable allows to run by restart
        
Wifi:
    sudo lshw -C network:
        -- check your wifis

    sudo ip link show:
        -- verify your wifi

    sudo find /sys/ -name "wlp1s0":
    sudo find /sys/ -name "wlp2s0":
        -- whatever your wifi name is

    sudo ls /sys/class/net/wlp1s0/:
        -- list all fiels in folder and find phy80211

    sudo ls /sys/class/net/wlp1s0/phy80211:
        -- list under this

    sudo cat /sys/class/rfkill/rfkill3/soft:
        -- if its "1", then all networks are blocked

    sudo su - root:
    sudo echo 0 > /sys/class/rfkill/rfkill3/soft:
        -- switch to root user and perfomr this
    
    sudo su - dev:
    sudo ip link set wlp2s0 up:
        -- switch back to user role and enable wifi

    sudo vim /etc/netplan/50-cloud-inti.yaml:
        -- this maybe defailt file, but the name can change

            network:
              version: 2
              renderer: networkd
              wifis:
                wlp2s0:
                dhcp4: true
                access-points:
                      "Your_SSID":
                      password: "Your_Password"

    sudo netplan apply:
        -- apply yaml configs