Common and useful commands

The best way to learn commands is to use them. Staring at lists of commands and trying to memorize them is both boring and ineffective. Commands are usually memorized through repetition of use. For rarely used commands, a browser's bookmark feature is a great solution, although it should be noted that obscure blogs can go away, get rewritten, links changed, and the same can even happen for large forums, such as happened with ubuntuforums.org, rendering bookmarks to greatly useful posts worthless. Another option is to save the webpages to a directory, keep notes somewhere convenient, or contribute to this wiki!

Note that nearly everything in Linux is case-sensitive, and most commands are entered as lower-case, including all of the commands below.

Common commands

cd

Used to change directory of the command prompt and is discussed in the Intro to command line article.

chown

The chown command is used to change the owner or group a file or directory belongs to. For example, chown username directoryorfilename will assign username as the owner of the directory or file specified. Using chown username:groupname directoryorfilename will assign username as the owner and groupname as the group of the directory or file specified. Using the recursive option by entering chown -R when changing user or group ownership on a directory will apply the changes to all files and subdirectories in the targeted directory.

Note that, as with all Linux commands, using chown will depend on file and directory permissions, and typically will require root privileges to execute.

chmod

The chmod command is used to change the permission set of a file or directory. The permissions can be entered as symbolic notation or octal notation, but octal notation is typically preferred for speed of entry. A change is made by entering chmod {permission annotation} filenameordirectoryname, example chmod 750 /home/username/test sets the permissions on the test file to drwxr-x---. The recursive option can used by entering -R after chmod (before permission annotation).

Note that chmod often requires root privileges to execute.

setgid

It is possible to force all new entries created in a directory to always belong to the same group the parent directory belongs to. This is enabled by entering g+s instead of the permission annotation, for example, chmod g+s /targetdirectory/. When a directory has an s in the group write permissions, this means that setgid has been enabled. (. is always the current directory and .. is the parent directory) Example:

username@servername:~$ mkdir testdirectory
username@servername:~$ sudo chown username:adm testdirectory/
username@servername:~$ chmod g+s testdirectory/
username@servername:~$ touch testdirectory/test
username@servername:~$ ll testdirectory/
drwxrwsr-x 7 username adm      4096 May 12 0915 .
drwxr-x--- 3 username username 4096 May 10 1548 ..
-rw-rw-r-- 1 username adm         0 May 12 0916 test
username@servername:~$ rm -rf testdirectory/

setuid

Note that despite what some sites on the Internet may state, although setuid can be flagged in Linux, the OS will ignore the flag and new files will always be owned by their creators.

cp

The cp command is used to copy files or directories. When using the command, the new file or directory can also be given a new name. For example, cp /path/to/fileordirectory /new/path/newfileordirectoryname. The new file or directory can be in the same directory as the one being copied. This is useful for backing up configuration files before editing or copying configuration files that are being applied to new items, such as virtual servers in nginx.

exit

The exit command is used to exit the current shell session. For example, if a root session was entered, exit would return to the session to user session.

ll

The ll command is an alias of ls -alF and is used to list the contents of a directory. While there are other commands, such as ls, to accomplish the same thing, ll is typically more useful as it will list ownership and permissions for each file and directory as well as listing hidden files and directories. If the directory being queried is particularly large, using the |less option after the directory location will allow for viewing the contents of the directory (navigated with page up/page down) more easily and can be exited by pressing q.

logout

The logout command will log the current user out of the session.

mkdir

The mkdir command is used to create a new directory. Example: mkdir /path/to/directory/newdirectoryname or mkdir newdirectoryname to make a new sub-directory of the current directory.

mv

The mv command is used to move files or directories. For example, mv /path/to/fileordirectory /new/path/neworsamefileordirectoryname. As with the cp command, the moved file or directory may be in the same directory. This is usually the easiest way to rename a directory or file.

nano

The nano text editor is discussed in the Intro to command line article.

poweroff

This command will shutdown Ubuntu and turn the computer's power off. It is only used on physical machines, never virtual machines such as VPS and cloud servers. Powering off a VPS or cloud server will only shut down Ubuntu while the hosting company will continue to charge as if the server were running.

shutdown

The shutdown command is not usually used as it only shuts down Ubuntu without powering off the enclosure, but is mentioned here as many new administrators mistakenly use it.

reboot

The reboot command reboots the server and requires root privileges.

rm

The rm command removes the targted file.

rm -rf

Running the rm command with options -rf can be used to delete a directory and its contents. Caution! Linux has few built-in safety nets and will execute this command as permissions allow, so using root privilege to delete a directory containing vital system files will destroy the system! Additionally, there is no 'trash', so all deletions are permanent. On top of that, the user will not be asked to verify whether the target should be deleted, so pressing enter is all that's required to permanently delete the target!

rmdir

The rmdir command removes the specified empty directory.

service

It is often necessary to restart services after making changes to configuration files. This can be accomplished with service servicename restart, and nearly always requires root privileges.

su

The su command is used to change user sessions. A common usage is to change to a root session using sudo su -, but it can also be to change to a different user, for example, sudo su www-data.

sudo

The sudo command is discussed in the Intro to command line article.

Useful commands

The commands below are used somewhat infrequently, but are convenient when working towards certain goals.

apg

The Automated Password Generator is very convenient for generating random password strings. Although in the Ubuntu Main repositories, apg is not installed by default.

user@servername:~$ sudo aptitude install apg

Generate Wi-Fi password

The below sample command demonstrates how to create a random Wi-Fi password.

user@servername:~$ apg -s -a 1 -m 63 -n 4

Please enter some random data (only first 16 are significant)
(eg. your old password):>
U4/aw_L5.Oi"Upi[8N55=>FxQ;vB6N\{&V8ZS?rxDnhpfYSl]^ssX(aqm;c1*M:
gA"3"veeZkak43dc;F*"|8;zu^&Xy4m/a,S,WNxb}oJ,]AQoc;"%jKvLghC9\{4
)!qACtXenpiil!%Pm6soDw%QD8E],aX,j^/OTJXJXJ-(mj`%ClhmI*$<u='Ko1H
M!}DR1cC+Q!G-K>3}rB9T}jV9SM%0=FAX_'Q.NXl*M?LgNp*XC!x<J>c";Rp|-2

It is also possible to create a password that only includes alphanumeric characters:

user@servername:~$ apg -s -a 1 -M NCL -m 63 -n 4

Please enter some random data (only first 16 are significant)
(eg. your old password):>
koBKIwEbHeAt5wy8mJ3W99tFFI9P5O3tdssqibISGytplGOEBHeNhQxdqQ5yKzL
bisXjXY1hOYW6kiMWlgHGVl73U07woKhtXc9WN2rYvThd3MOehYfe2Njkts6f8w
JKCr4onlxIUPsspv0nVjdGe7Hv9JpMmNilPrNTCEOp7Bn7sXqJ0JZtKVnOWrHi3
8zVd5hmq7YGTTbSR49dg5QNhaeC0H2tly6fAbRJo0ihn19ywyFOYjgdnmrSMuQr

The random data can be mashed in from the keyboard or copied and pasted from secure sites such as GRC | Ultra High Security Password Generator.

dd

The dd command is a very useful and powerful tool that can perform a range of functions.

Write image to disk

This command will write the image file to a drive. This is very convenient for writing install images to flash media, eliminating the need to burn optical disks:

root@servername:~# dd if=filename.iso of=/dev/sdX

Erase boot sectors of a drive

This command will erase the boot sectors of a drive, which is a method for sanitizing new storage media that may come with undesirable software pre-installed to the drive:

root@servername:~# dd if=/dev/zero of=/dev/sdX bs=512 count=1000

Write zeros to a drive

This command will write zeros to a drive:

root@servername:~# dd if=/dev/zero of=/dev/sdX

dig

As its man page description states:

dig (domain information groper) is a flexible tool for interrogating DNS name servers. It performs DNS lookups and displays the answers that are returned from the name server(s) that were queried. Most DNS administrators use dig to troubleshoot DNS problems because of its flexibility, ease of use and clarity of output.

It can be particularly useful for seeing what other admins have done when setting up various different DNS records. The +short modifier can be used to provide a terse output instead of the default verbose output. Below are some examples of the many options for viewing specific records (note that a subdomain should be used only when appropriate):

A records

username@servername:~$ dig +short example.com

AAAA records

username@servername:~$ dig +short aaaa example.com

CNAME records

username@servername:~$ dig +short cname subdomain.example.com

DKIM records

username@servername:~$ dig +short -t txt selector._domainkey.example.com

DMARC records

username@servername:~$ dig +short txt _dmarc.example.com

DMARC RUF/RUA reporting address different from organization domain

username@servername:~$ dig +short txt example.com._report._dmarc.example.org

MX records

username@servername:~$ dig  +short mx example.com

PTR records

username@servername:~$ dig +short -x <IPv6 or IPv4 address>

SPF records

username@servername:~$ dig  +short txt example.com

hdparm

The hdparm tool can be used to get or set various parameters from SATA and IDE devices. To view various informations about a device, the following command may be used, substituting sda as desired:

username@servername:~$ sudo hdparm -I /dev/sda

Using hdparm to erase drives

Probably the most commonly used function of hdparm is to perform a secure erase of a drive. This is especially useful when it is desired to erase a drive, particularly the sectors of a drive not normally available to an OS, including damaged sectors. Note that some researchers have been finding that not all manufacturers are properly implementing the ATA standard for this command, so when using the command for the purposes of securely erasing data, it may be a good idea to first wipe the drive using Darik's Boot and Nuke (DBAN), as current research indicates this will satisfactorily erase SSDs even when factoring in wear-leveling.

Since it is assumed that this is for purposes other than server administration, such as wiping a drive before resale or installation in another device, it is likely easiest to run this from an OS with a GUI, and a very simple solution is to run Ubuntu 14.04 from a live DVD. It may also be a good idea to disconnect all drives not being wiped, which is standard before running DBAN.

The security parameters of a drive are an option in the ATA specification. It is possible that a drive's manufacturer has decided not to include these features on a drive. When the above identification info command is run, and the Security section is blank or missing, that means that the drive does not include this option.

To issue a secure erase, ensure the drive is not frozen. A drive is frozen when the word not is not in front of frozen in the Security section of the results from the previously issued command:

Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
                frozen
        not     expired: security count
        not     supported: enhanced erase
        6min for SECURITY ERASE UNIT.

This is the normal condition of the drive. For most systems, the easiest way to make the drive not frozen is to simply put the system in standby, then restore from standby. A procedure that takes less than 15 seconds in a live CD Ubuntu 14.04 session, and can actually be done in the menu screen that appears on initial boot that asks whether the user wants to install Ubuntu or try Ubuntu. Some systems do not properly support recovering to Ubuntu from standby, and using this option nearly always avoids this issue.

Verify the disk is not frozen:

Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
        not     frozen
        not     expired: security count
        not     supported: enhanced erase
        6min for SECURITY ERASE UNIT.

If going to standby is not an option, the drive status can be changed by unplugging the drive's SATA cable or power cable, then plugging it back in. This procedure will probably not work with IDE drives, which would need to be installed in a USB drive enclosure or a computer that supports going to and returning from standby.

Issue secure erase command

To issue the secure erase command, create a user password, then use the user password to issue the command:

username@servername:~$ sudo hdparm --user-master u --security-set-pass password /dev/sda
username@servername:~$ sudo time hdparm --user-master u --security-erase password /dev/sda

The password used does not matter because that information will be wiped during the operation.

The time command is used as a sort of verification that the task completed, as hdparm otherwise provides no confirmation, other than releasing the command prompt.

Issue enhanced secure erase command

Some drives support enhanced secure erase, which restores the drive to the condition the manufacturer intended it to leave the factory with. A drive will report supporting this in the Security section:

Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
        not     frozen
        not     expired: security count
                supported: enhanced erase
        54min for SECURITY ERASE UNIT. 54min for ENHANCED SECURITY ERASE UNIT.

To issue the enhanced secure erase command, create a user password, then use the user password to issue the command:

username@servername:~$ sudo hdparm --user-master u --security-set-pass password /dev/sda
username@servername:~$ sudo time hdparm --user-master u --security-erase-enhanced password /dev/sda

With both the secure erase and enhanced secure erase, do not issue hdparm or smartctl commands, even in other terminals, to the system until after the erasure is completed and the command prompt released.

rename

Some devices create files with upper case letters in the file extension which may cause problems when linking to those files from other applications. When there are high quantities of the files, such as file1.JPG, file2.JPG, etc., changing each one manually can be cumbersome. The rename command is useful for such cases.

Using the following command will convert all .JPG file extensions to .jpg for the current directory:

username@servername:~$ rename 's/\.JPG$/.jpg/' *.JPG

rsync

The rsync command has a number of uses, and a somewhat less common but useful option set allows for monitoring the status of copying a large from one location to another. Running the following command will perform the copy/sync action along with stating updated data amount transferred, percent data transferred, transfer rate, and estimated time to completion.

username@servername:~$ rsync -ah --progress source destination
destination
      47.74G  20%   72.48MB/s    0:42:22

Smartmontools

The Smartmontools packages contain two utilities to control and monitor storage systems using the Self-Monitoring, Analysis and Reporting Technology System (S.M.A.R.T.). One of the utilities, smartctl, can be used to execute S.M.A.R.T. tests and extract S.M.A.R.T. data from a drive. When the only purpose for installing the package is one of these two purposes, it is better to install only the smartmontools package and not its recommended packages, which include postfix.

username@servername:~$ sudo aptitude install smartmontools --no-install-recommends

View S.M.A.R.T. data for a drive

To view all S.M.A.R.T. data on a drive, including the logs, the following command may be run:

username@servername:~$ sudo smartctl -a /dev/sdX

Many drives will have a considerable amount of information printed from that command, so it may be more useful to read the output in a file:

username@servername:~$ sudo smartctl -a /dev/sdX > sdX.smart.data.txt

Run extended self-test

Note that the drive will perform the test in the background, and its results will be published to the drive's log after the time stated in the printed output, which varies by drive:

username@servername:~$ sudo smartctl -t long /dev/sdX
smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.13.0-39-generic] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Warning: device does not support Self-Test functions.

Sending command: "Execute SMART Extended self-test routine immediately in off-line mode".
Drive command "Execute SMART Extended self-test routine immediately in off-line mode" successful.
Testing has begun.
Please wait 1 minutes for test to complete.
Test will complete after Sun May 22 14:30:19 2016

Use smartctl -X to abort test.

tar

The tar package is installed by default in Ubuntu. It is both a file format and a program. The program can be used to create and extract compressed tar files, which may be further compressed with the gzip (archivename.tar.gz) or bzip2 (archivename.tar.bz2) programs, which are also installed by default in Ubuntu. Files created with tar are commonly referred to as tarballs.

The biggest advantage tar has over zip is that the archive can be created to retain information such as file ownership and permissions. The biggest disadvantage is that tar files are generally only usable by *nix operating systems, so users of Windows will have to download some additional utility, such as 7-Zip, to extract the files.

Using tar

xkcd comic demonstrating common difficulty with tar options.

As popular as tar is, remembering the myriad options can be difficult, and it pretty much can't be used without options. Below is an option set that will allow compression and extraction to retain file ownership and permissions. Note that the options and their usage may not function as stated below in non-Ubuntu Linux distributions and other *nix operating systems.

Options for compressing with tar and gzip

This command will create archivename.tar.gz from the files or directories specified, compress them using the gzip program and will preserve ownership and permission information.

username@servername:~$ tar -zcvf archivename.tar.gz fileordirectoryname1 fileordirectoryname2

Options for compressing with tar and bzip

To compress using bzip, simply use the above commands, but add the j option:

username@servername:~$ tar -jcvf archivename.tar.bz2 fileordirectoryname1 fileordirectoryname2

Extracting tarballs

This command will extract tarballs to the current directory and retain ownership and file permissions. Executing without sudo will perform the command, but will assign the current user as owner while retaining file permissions. Note that it may be somewhat inconvenient to use sudo unless it is known that the archive was created with correct ownership and permissions.

username@servername:~$ sudo tar -xvf archivename.tar.gz

Note that this option set will extract both gzip and bzip archives.

Adding the -C option will extract the archive to /directoryname/.

username@servername:~$ sudo tar -xvf archivename.tar.gz -C /directoryname/

List the files in a tarball

To view the contents of a tarball without decompressing it, use the following commands:

List the contents of a tar file:

username@servername:~$ tar -tvf filename.tar

List the contents of a tar.gz file:

username@servername:~$ tar -ztvf filename.tar.gz

List the contents of a tar.bz file:

username@servername:~$ tar -jtvf filename.tar.bz

For archives with very long file lists, it may be easier to print them to a text file using >:

username@servername:~$ tar -ztvf filename.tar.gz > filename.txt

tmux

The tmux command can be used to run tasks after an SSH session has been closed. When invoked, tmux will open a new terminal which is running from the same location and with the same user as invoked the session. Any command run from the new tmux session will continue running even after returning to the original session by detaching from tmux by pressing Ctrl+b then d. Note that the tmux session will exist until it is opened again, even if all jobs in the session are completed, so it is good practice to return to the session and exit.

whois

Installation and usage of whois is covered in the Using Ubuntu Server 16.04 LTS article.

External links

DiG HOWTO

How to keep processes running after ending ssh session? - Ask Ubuntu

A tmux Crash Course