Common and useful commands
The best way to learn commands is to use them. Staring at lists of commands and trying to memorize them is both boring and ineffective. Commands are usually memorized through repetition of use. For rarely used commands, a browser's bookmark feature is a great solution, although it should be noted that obscure blogs can go away, get rewritten, links changed, and the same can even happen for large forums, such as happened with ubuntuforums.org, rendering bookmarks to greatly useful posts worthless. Another option is to save the webpages to a directory, keep notes somewhere convenient, or contribute to this wiki!
Note that nearly everything in Linux is case-sensitive, and most commands are entered as lower-case, including all of the commands below.
Common commands
cd
Used to change directory of the command prompt and is discussed in the Intro to command line article.
chown
The chown command is used to change the owner or group a file or directory belongs to. For example, chown username directoryorfilename will assign username as the owner of the directory or file specified. Using chown username:groupname directoryorfilename will assign username as the owner and groupname as the group of the directory or file specified. Using the recursive option by entering chown -R when changing user or group ownership on a directory will apply the changes to all files and subdirectories in the targeted directory.
Note that, as with all Linux commands, using chown will depend on file and directory permissions, and typically will require root privileges to execute.
chmod
The chmod command is used to change the permission set of a file or directory. The permissions can be entered as symbolic notation or octal notation, but octal notation is typically preferred for speed of entry. A change is made by entering chmod {permission annotation} filenameordirectoryname, example chmod 750 /home/username/test sets the permissions on the test file to drwxr-x---. The recursive option can used by entering -R after chmod (before permission annotation).
Note that chmod often requires root privileges to execute.
setgid
It is possible to force all new entries created in a directory to always belong to the same group the parent directory belongs to. This is enabled by entering g+s instead of the permission annotation, for example, chmod g+s /targetdirectory/. When a directory has an s in the group write permissions, this means that setgid has been enabled. (. is always the current directory and .. is the parent directory) Example:
username@servername:~$ mkdir testdirectory username@servername:~$ sudo chown username:adm testdirectory/ username@servername:~$ chmod g+s testdirectory/ username@servername:~$ touch testdirectory/test username@servername:~$ ll testdirectory/ drwxrwsr-x 7 username adm 4096 May 12 0915 . drwxr-x--- 3 username username 4096 May 10 1548 .. -rw-rw-r-- 1 username adm 0 May 12 0916 test username@servername:~$ rm -rf testdirectory/
setuid
Note that despite what some sites on the Internet may state, although setuid can be flagged in Linux, the OS will ignore the flag and new files will always be owned by their creators.
cp
The cp command is used to copy files or directories. When using the command, the new file or directory can also be given a new name. For example, cp /path/to/fileordirectory /new/path/newfileordirectoryname. The new file or directory can be in the same directory as the one being copied. This is useful for backing up configuration files before editing or copying configuration files that are being applied to new items, such as virtual servers in nginx.
exit
The exit command is used to exit the current shell session. For example, if a root session was entered, exit would return to the session to user session.
ll
The ll command is an alias of ls -alF and is used to list the contents of a directory. While there are other commands, such as ls, to accomplish the same thing, ll is typically more useful as it will list ownership and permissions for each file and directory as well as listing hidden files and directories. If the directory being queried is particularly large, using the |less option after the directory location will allow for viewing the contents of the directory (navigated with page up/page down) more easily and can be exited by pressing q.
logout
The logout command will log the current user out of the session.
mkdir
The mkdir command is used to create a new directory. Example: mkdir /path/to/directory/newdirectoryname or mkdir newdirectoryname to make a new sub-directory of the current directory.
mv
The mv command is used to move files or directories. For example, mv /path/to/fileordirectory /new/path/neworsamefileordirectoryname. As with the cp command, the moved file or directory may be in the same directory. This is usually the easiest way to rename a directory or file.
nano
The nano text editor is discussed in the Intro to command line article.
poweroff
This command will shutdown Ubuntu and turn the computer's power off. It is only used on physical machines, never virtual machines such as VPS and cloud servers. Powering off a VPS or cloud server will only shut down Ubuntu while the hosting company will continue to charge as if the server were running.
shutdown
The shutdown command is not usually used as it only shuts down Ubuntu without powering off the enclosure, but is mentioned here as many new administrators mistakenly use it.
reboot
The reboot command reboots the server and requires root privileges.
rm
The rm command removes the targted file.
rm -rf
Running the rm command with options -rf can be used to delete a directory and its contents. Caution! Linux has few built-in safety nets and will execute this command as permissions allow, so using root privilege to delete a directory containing vital system files will destroy the system!
Additionally, there is no 'trash', so all deletions are permanent. On top of that, the user will not be asked to verify whether the target should be deleted, so pressing enter is all that's required to permanently delete the target!
rmdir
The rmdir command removes the specified empty directory.
service
It is often necessary to restart services after making changes to configuration files. This can be accomplished with service servicename restart, and nearly always requires root privileges.
su
The su command is used to change user sessions. A common usage is to change to a root session using sudo su -, but it can also be to change to a different user, for example, sudo su www-data.
sudo
The sudo command is discussed in the Intro to command line article.
Useful commands
The commands below are used somewhat infrequently, but are convenient when working towards certain goals.
apg
The Automated Password Generator is very convenient for generating random password strings. Although in the Ubuntu Main repositories, apg is not installed by default.
user@servername:~$ sudo aptitude install apg
Generate Wi-Fi password
The below sample command demonstrates how to create a random Wi-Fi password.
user@servername:~$ apg -s -a 1 -m 63 -n 4
Please enter some random data (only first 16 are significant)
(eg. your old password):>
U4/aw_L5.Oi"Upi[8N55=>FxQ;vB6N\{&V8ZS?rxDnhpfYSl]^ssX(aqm;c1*M:
gA"3"veeZkak43dc;F*"|8;zu^&Xy4m/a,S,WNxb}oJ,]AQoc;"%jKvLghC9\{4
)!qACtXenpiil!%Pm6soDw%QD8E],aX,j^/OTJXJXJ-(mj`%ClhmI*$<u='Ko1H
M!}DR1cC+Q!G-K>3}rB9T}jV9SM%0=FAX_'Q.NXl*M?LgNp*XC!x<J>c";Rp|-2
It is also possible to create a password that only includes alphanumeric characters:
user@servername:~$ apg -s -a 1 -M NCL -m 63 -n 4 Please enter some random data (only first 16 are significant) (eg. your old password):> koBKIwEbHeAt5wy8mJ3W99tFFI9P5O3tdssqibISGytplGOEBHeNhQxdqQ5yKzL bisXjXY1hOYW6kiMWlgHGVl73U07woKhtXc9WN2rYvThd3MOehYfe2Njkts6f8w JKCr4onlxIUPsspv0nVjdGe7Hv9JpMmNilPrNTCEOp7Bn7sXqJ0JZtKVnOWrHi3 8zVd5hmq7YGTTbSR49dg5QNhaeC0H2tly6fAbRJo0ihn19ywyFOYjgdnmrSMuQr
The random data can be mashed in from the keyboard or copied and pasted from secure sites such as GRC | Ultra High Security Password Generator.
curl
The curl command can be used for a broad range of handy functions. One particular function can be to determine what web server software a site is served from:
username@servername:~$ curl -I www.unpm.org HTTP/1.1 200 OK Server: nginx Date: Sat, 28 Jan 2017 14:27:10 GMT Content-Type: text/html Content-Length: 13938 Last-Modified: Mon, 16 Jan 2017 02:39:49 GMT Connection: keep-alive Keep-Alive: timeout=10 Vary: Accept-Encoding ETag: "587c3275-3672" Accept-Ranges: bytes
dd
The dd command is a very useful and powerful tool that can perform a range of functions.
Write image to disk
This command will write the image file to a drive. This is very convenient for writing install images to flash media, eliminating the need to burn optical disks:
root@servername:~# dd if=filename.iso of=/dev/sdX
Write disk image to file
This command will write a drive device image to file.
root@servername:~# dd if=/dev/sdX bs=1M | pv | of=filname.img
The pv command is used to show the file transfer rate during the write. Note that writing mounted images, especially disks with operating systems, can cause problems with the image file.
This command will compress the image file while writing:
root@servername:~# dd if=/dev/sdX bs=1M | pv | pigz -c > filename.img.gz
The pigz command is used for multi-thread processing of compression. For single thread, gzip will work just as well.
The block size (bs) of 1M or larger (10M) can greatly speed up operations of large file sizes.
Erase boot sectors of a drive
This command will erase the boot sectors of a drive, which is a method for sanitizing new storage media that may come with undesirable software pre-installed to the drive:
root@servername:~# dd if=/dev/zero of=/dev/sdX bs=512 count=1000
Write zeros to a drive
This command will write zeros to a drive:
root@servername:~# dd if=/dev/zero of=/dev/sdX
dig
As its man page description states:
dig(domain information groper) is a flexible tool for interrogating DNS name servers. It performs DNS lookups and displays the answers that are returned from the name server(s) that were queried. Most DNS administrators use dig to troubleshoot DNS problems because of its flexibility, ease of use and clarity of output.
It can be particularly useful for seeing what other admins have done when setting up various different DNS records. The +short modifier can be used to provide a terse output instead of the default verbose output. Below are some examples of the many options for viewing specific records (note that a subdomain should be used only when appropriate):
A records
username@servername:~$ dig +short example.com
AAAA records
username@servername:~$ dig +short aaaa example.com
CNAME records
username@servername:~$ dig +short cname subdomain.example.com
DKIM records
username@servername:~$ dig +short txt selector._domainkey.example.com
DMARC records
username@servername:~$ dig +short txt _dmarc.example.com
DMARC RUF/RUA reporting address different from organization domain
username@servername:~$ dig +short txt example.com._report._dmarc.example.org
MX records
username@servername:~$ dig +short mx example.com
PTR records
username@servername:~$ dig +short -x <IPv6 or IPv4 address>
SPF records
username@servername:~$ dig +short txt example.com
growisofs / wodim
In some cases, it may be prudent to use an optical drive to install an operating system. One of the easiest and fastest ways to burn an optical drive is to use the wodim tool for CDs and the growisofs tool for DVDs.
To burn an ISO to a CD:
username@servername:~$ wodim -eject -tao dev=/dev/srX -v -data filename.iso
To burn an ISO to a DVD:
username@servername:~$ growisofs -dvd-compat -Z /dev/dvd=filename.iso
hdparm
The hdparm tool can be used to get or set various parameters from SATA and IDE devices. To view various informations about a device, the following command may be used, substituting sda as desired:
username@servername:~$ sudo hdparm -I /dev/sda
Using hdparm to erase drives
Probably the most commonly used function of hdparm is to perform a secure erase of a drive. This is especially useful when it is desired to erase a drive, particularly the sectors of a drive not normally available to an OS, including damaged sectors. Note that some researchers have been finding that not all manufacturers are properly implementing the ATA standard for this command, so when using the command for the purposes of securely erasing data, it may be a good idea to first wipe the drive using Darik's Boot and Nuke (DBAN), as current research indicates this will satisfactorily erase SSDs even when factoring in wear-leveling.
Since it is assumed that this is for purposes other than server administration, such as wiping a drive before resale or installation in another device, it is likely easiest to run this from an OS with a GUI, and a very simple solution is to run Ubuntu 14.04 from a live DVD. It may also be a good idea to disconnect all drives not being wiped, which is standard before running DBAN.
The security parameters of a drive are an option in the ATA specification. It is possible that a drive's manufacturer has decided not to include these features on a drive. When the above identification info command is run, and the Security section is blank or missing, that means that the drive does not include this option.
To issue a secure erase, ensure the drive is not frozen. A drive is frozen when the word not is not in front of frozen in the Security section of the results from the previously issued command:
Security:
Master password revision code = 65534
supported
not enabled
not locked
frozen
not expired: security count
not supported: enhanced erase
6min for SECURITY ERASE UNIT.
This is the normal condition of the drive. For most systems, the easiest way to make the drive not frozen is to simply put the system in standby, then restore from standby. A procedure that takes less than 15 seconds in a live CD Ubuntu 14.04 session, and can actually be done in the menu screen that appears on initial boot that asks whether the user wants to install Ubuntu or try Ubuntu. Some systems do not properly support recovering to Ubuntu from standby, and using this option nearly always avoids this issue.
Verify the disk is not frozen:
Security:
Master password revision code = 65534
supported
not enabled
not locked
not frozen
not expired: security count
not supported: enhanced erase
6min for SECURITY ERASE UNIT.
If going to standby is not an option, the drive status can be changed by unplugging the drive's SATA cable or power cable, then plugging it back in. This procedure will probably not work with IDE drives, which would need to be installed in a USB drive enclosure or a computer that supports going to and returning from standby.
Issue secure erase command
To issue the secure erase command, create a user password, then use the user password to issue the command:
username@servername:~$ sudo hdparm --user-master u --security-set-pass password /dev/sda username@servername:~$ time sudo hdparm --user-master u --security-erase password /dev/sda
The password used does not matter because that information will be wiped during the operation.
The time command is used as a sort of verification that the task completed, as hdparm otherwise provides no confirmation, other than releasing the command prompt.
Issue enhanced secure erase command
Some drives support enhanced secure erase, which restores the drive to the condition the manufacturer intended it to leave the factory with. A drive will report supporting this in the Security section:
Security:
Master password revision code = 65534
supported
not enabled
not locked
not frozen
not expired: security count
supported: enhanced erase
54min for SECURITY ERASE UNIT. 54min for ENHANCED SECURITY ERASE UNIT.
To issue the enhanced secure erase command, create a user password, then use the user password to issue the command:
username@servername:~$ sudo hdparm --user-master u --security-set-pass password /dev/sda username@servername:~$ time sudo hdparm --user-master u --security-erase-enhanced password /dev/sda
With both the secure erase and enhanced secure erase, do not issue hdparm or smartctl commands, even in other terminals, to the system until after the erasure is completed and the command prompt released.
ImageMagick
The imagemagick package contains very powerful image management tools and is used by a wide variety projects, including MediaWiki.
Convert
The convert command can be used to perform basic image editing functions from the command line. Note that the functions can be performed on nearly all image file format types and extensions. Note that to overwrite the existing file with the altered file, simply make the destination file and source file the same.
Resize image
Use the -resize option to resize an image based on a simple percent change:
username@servername:~$ convert -resize 50% sourcefile.jpg destinationfile.jpg
Use the mogrify command to resize a director of imgages:
username@servername:~$ mogrify -resize 25% *.jpg
Rotate image
To rotate an image, use the -rotate option and enter the degrees with positive being clockwise and negative being counterclockwise:
username@servername:~$ convert -rotate "-90" sourcefile.jpg destinationfile.jpg
rename
Some devices create files with upper case letters in the file extension which may cause problems when linking to those files from other applications. When there are high quantities of the files, such as file1.JPG, file2.JPG, etc., changing each one manually can be cumbersome. The rename command is useful for such cases.
Using the following command will convert all .JPG file extensions to .jpg for the current directory:
username@servername:~$ rename 's/\.JPG$/.jpg/' *.JPG
rsync
The rsync command has a number of uses.
Monitor copy progress
Somewhat less common but useful option set allows for monitoring the status of copying a large from one location to another. Running the following command will perform the copy/sync action along with stating updated data amount transferred, percent data transferred, transfer rate, and estimated time to completion.
username@servername:~$ rsync -ah --progress source destination
destination
47.74G 20% 72.48MB/s 0:42:22
Local backup of remote directory
Among the more popular uses is to create and manage a local backup for a remote directory through ssh.
username@servername:~$ rsync -avh --progress --exclude '/optional/path/to/directory/not/archived/' remoteserver:/path/to/source/directory/ /path/to/target/directory/ --delete
The --exclude option is used for directories not desired to have copied over. The --delete option will delete local files in the target directory that are not found on the source directory.
It is common to use the -z option which compresses the file at the remote server before transfer and decompresses at the local server before saving. However, for servers on the local network, this typically costs more time unless moving exceptionally large files.
Smartmontools
The Smartmontools packages contain two utilities to control and monitor storage systems using the Self-Monitoring, Analysis and Reporting Technology System (S.M.A.R.T.). One of the utilities, smartctl, can be used to execute S.M.A.R.T. tests and extract S.M.A.R.T. data from a drive. When the only purpose for installing the package is one of these two purposes, it is better to install only the smartmontools package and not its recommended packages, which include postfix.
username@servername:~$ sudo aptitude install smartmontools --no-install-recommends
View S.M.A.R.T. data for a drive
To view all S.M.A.R.T. data on a drive, including the logs, the following command may be run:
username@servername:~$ sudo smartctl -a /dev/sdX
Many drives will have a considerable amount of information printed from that command, so it may be more useful to read the output in a file:
username@servername:~$ sudo smartctl -a /dev/sdX > sdX.smart.data.txt
Calculate TBW
To calculate TBW of a drive, use the following equation:
Total_LBAs_Written * Sector Size / 1024^4 = TBW
Example (where S.M.A.R.T. attribute 241 = 993101570):
993101570 * 512 / 1099511627776 = 0.46245
Run extended self-test
Note that the drive will perform the test in the background, and its results will be published to the drive's log after the time stated in the printed output, which varies by drive:
username@servername:~$ sudo smartctl -t long /dev/sdX smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.13.0-39-generic] (local build) Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net === START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION === Warning: device does not support Self-Test functions. Sending command: "Execute SMART Extended self-test routine immediately in off-line mode". Drive command "Execute SMART Extended self-test routine immediately in off-line mode" successful. Testing has begun. Please wait 1 minutes for test to complete. Test will complete after Sun May 22 14:30:19 2016 Use smartctl -X to abort test.
tar
The tar package is installed by default in Ubuntu. It is both a file format and a program. The program can be used to create and extract compressed tar files, which may be further compressed with the gzip (archivename.tar.gz) or bzip2 (archivename.tar.bz2) programs, which are also installed by default in Ubuntu. Files created with tar are commonly referred to as tarballs.
The biggest advantage tar has over zip is that the archive can be created to retain information such as file ownership and permissions. The biggest disadvantage is that tar files are generally only usable by *nix operating systems, so users of Windows will have to download some additional utility, such as 7-Zip, to extract the files.
Using tar
As popular as tar is, remembering the myriad options can be difficult, and it pretty much can't be used without options. Below is an option set that will allow compression and extraction to retain file ownership and permissions. Note that the options and their usage may not function as stated below in non-Ubuntu Linux distributions and other *nix operating systems.
Options for compressing with tar and gzip
This command will create archivename.tar.gz from the files or directories specified, compress them using the gzip program and will preserve ownership and permission information.
username@servername:~$ tar -zcvf archivename.tar.gz fileordirectoryname1 fileordirectoryname2
Options for compressing with tar and bzip
To compress using bzip, simply use the above commands, but add the j option:
username@servername:~$ tar -jcvf archivename.tar.bz2 fileordirectoryname1 fileordirectoryname2
Extracting tarballs
This command will extract tarballs to the current directory and retain ownership and file permissions. Executing without sudo will perform the command, but will assign the current user as owner while retaining file permissions. Note that it may be somewhat inconvenient to use sudo unless it is known that the archive was created with correct ownership and permissions.
username@servername:~$ sudo tar -xvf archivename.tar.gz
Note that this option set will extract both gzip and bzip archives.
Adding the -C option will extract the archive to /directoryname/.
username@servername:~$ sudo tar -xvf archivename.tar.gz -C /directoryname/
List the files in a tarball
To view the contents of a tarball without decompressing it, use the following commands:
List the contents of a tar file:
username@servername:~$ tar -tvf filename.tar
List the contents of a tar.gz file:
username@servername:~$ tar -ztvf filename.tar.gz
List the contents of a tar.bz file:
username@servername:~$ tar -jtvf filename.tar.bz
For archives with very long file lists, it may be easier to print them to a text file using >:
username@servername:~$ tar -ztvf filename.tar.gz > filename.txt
tmux
The tmux command can be used to run tasks after an SSH session has been closed. When invoked, tmux will open a new terminal which is running from the same location and with the same user as invoked the session. Any command run from the new tmux session will continue running even after returning to the original SSH session.
Detach from current tmux session
Press Ctrl+b then d.
Note that the tmux session will exist until it is opened again, even if all jobs in the session are completed, so it is good practice to return to the session and exit.
See current tmux sessions
$ tmux info
Attach to tmux session
$ tmux attach -t <session number>
End tmux session from within session
$ exit
whois
Installation and usage of whois is covered in the Using Ubuntu Server 16.04 LTS article.
External links
How to keep processes running after ending ssh session? - Ask Ubuntu