More actions
Introduction
Save large files like code, binary files, movies, etc. The storage media are things like HDDs, SSDs, magnetic tape.
Storage is not memory, where memory caches memory for quick access, things like any type of RAM: SRAM, SDRAM, VRAM, etc.
Swap storage is fake memory that is actually storage, but pretends to be memory. You need a partition for this, and so you run
root #typemkswap /dev/sda2to make the partition a swap file, then run
root #swapon /dev/sda2to enable swap. Run top or htop to see your swap and memory usage.
Looking at things
- lsblk - "list block devices", shows partitions
lsblk -fgives same info asblkid
- blkid - "block device ids", shows storage UUIDs
- fdisk - list all your storage media using
fdisk - df -h - shows you the free space on each drive.
When trying to save space, use
du -sh ./* to look at folder sizes and work your way down to delete the things that aren't needed.Mounting systems
findmnt
Finds all the mounts in your system.
Anything in /sys or /proc are kernel stuff. The kernel opens up files for you to look at. I don't know anything else about it! Look it up yourself, cuz I don't know what it means. It's not important to working with storage systems.
/dev is your stuff... mostly. Some system stuff that you don't use: /dev/pts, /dev/shm, /dev/mqueue.
Mounting commands
You're working with storage media, so every single command will always be run as root.
/dev/sdX are your drives! /dev/sda is your first, /dev/sdb is your second, etc.
Mount a drive to look at:
root #mount /dev/sdb1 /mntUnmount it when you're done:
root #umount /mntIf you plug in a USB and you want to access it, use lsblk -f, then mount /dev/sdXN /mnt or wherever you want the USB files. Then, umount /mnt to safely remove.
If you're testing this on VirtualBox, your virtual drives have no format, so just run
root #mkfs.ext4 /dev/sdXBare-metal drives
We use fdisk.
root #fdisk /dev/sda| Command | What it does |
|---|---|
| g | sets disk as GUID Partition Table (GPT). This supports UEFI and up to 8 ZB (it goes GB, TB, PB, EB, ZB.. so a lot). |
| o | Sets disk as a Master Boot Record MBR. This supports legacy boot, NO UEFI, and up to 2 TB size. Easier to set up than GPT. |
| n | New partition. |
| p | Primary partition. |
| t | Sets the file type. Key ones are: (MBR) 82=Linux swap, 83=Linux, fd=RAID, 8e=LVM. (GPT) 19=Linux swap, 20=Linux, 29=Linux RAID, 30=Linux LVM, 1=EFI. The default on fdisk is usually Linux. |
| w | Write your changes. |
Demo setup
Follow this setup to test.
Device Start End Sectors Size Type
/dev/sda1 2048 526335 524288 256M EFI System
/dev/sda2 526336 2623487 2097152 1G Linux swap
/dev/sda3 2623488 19400703 16777216 8G Linux filesystem
I'd recommend playing with this in VirtualBox because it is very difficult to change these on your filesystem once you have data on it. In fact, you destroy all data when running these commands, so you'd just move everything off the drives before doing this. It's a big pain.
Logical volume management (LVM)
LVMs abstract away drives quite a bit.
/---------------\ /-----\ /--------\
Logical volumes | movies | code | home |
\---------------/ \-----/ \--------/
/----------------------------------\
Volume groups | lvm |
\----------------------------------/
/----------\ /---------\ /----------\
Physical volumes | /dev/sda1 | /dev/sdb1 | /dev/sdc1 |
\----------/ \---------/ \----------/
The main commands are:
| Command | What it does |
|---|---|
| pvcreate | Add a drive to the physical volumes. |
| pvdisplay | View physical volumes in the LVM. |
| vgcreate | Create a logical group with specified physical volumes. |
| vgextend | Add physical volumes to the volume group. |
| vgdisplay | View logical groups in the LVM. |
| lvcreate | Create a logical volume within the volume group. |
| lvextend | Expand a logical volume within the volume group. |
| lvreduce | Decrease a logical volume within the volume group. |
| lvdisplay | View all logical volumes. |
I'd highly, highly recommend following along with this tutorial within an LVM. You will be very comfortable with all of these commands after this.
Demo setup
Let us assume we have /dev/sda3, /dev/sdb1/, /dev/sdc1, 3 drives ready to be added to an LVM group. We will name our volume group lvm because why not? This will create a folder, /dev/lvm that contains our logical volumes.
First, add your drives to the physical volume list:
root #pvcreate /dev/sda3 /dev/sdb1 /dev/sdc1Then create a volume group with them all:
root #vgcreate lvm /dev/sda3 /dev/sdb1 /dev/sdc1Say you want your home directory in a different volume:
root #lvcreate -L 30G lvm -n lv_homeAnd then use the rest of the logical volume:
root #lvcreate -l 100%FREE lvm -n lv_rootYou got a new hard drive? Cool. It's /dev/sdd1. Add it to the volume group:
root #vgextend lvm /dev/sdd1You ran out of storage in a volume group? Resize it:
root #lvextend -L 35G /dev/lvm/lv_homeRAID
RAID is for data protection. This is not a backup. It is just for protection. Instead of one drive dying and you lose all your data, you can use a RAID system so you have protection against it dying.
Instead of a table of the different RAID systems, here's a list of them. I'm going to assume we use RAID 1 or RAID 10.
RAID is done all under mdadm. It's straightforward comapred to LVM.
root #mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1root #mdadm --create /dev/md0 --level=10 --raid-devices=2 /dev/sdb1 /dev/sdc1ZFS
You can use LVM and RAID by manually making sure you don't go over 50% or whatever RAID you want requires, then using mdadm on your logical volumes. Or just use ZFS to create a pool. Even less work.
root #zpool create pool /dev/sdb /dev/sdcRAID 1:
root #zpool create pool mirror /dev/sdb /dev/sdcRAID 10:
root #zpool create pool mirror /dev/sdb /dev/sdc mirror /dev/sdd /dev/sdeCheck out what you did:
root #zpool statusHere is our Linux mirrors configuration:
pool: lug
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
lug ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
label/lug-HGST-4TB-01 ONLINE 0 0 0
label/lug-HGST-4TB-02 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
label/lug-HGST-4TB-03 ONLINE 0 0 0
label/lug-HGST-4TB-04 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
label/lug-HGST-4TB-05 ONLINE 0 0 0
label/lug-HGST-4TB-06 ONLINE 0 0 0
mirror-3 ONLINE 0 0 0
label/lug-HGST-4TB-07 ONLINE 0 0 0
label/lug-HGST-4TB-08 ONLINE 0 0 0
mirror-4 ONLINE 0 0 0
label/lug-HGST-4TB-09 ONLINE 0 0 0
label/lug-HGST-4TB-10 ONLINE 0 0 0
errors: No known data errors
pool: zroot
state: ONLINE
scan: scrub repaired 0B in 00:01:47 with 0 errors on Sun Sep 26 16:42:05 2021
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
da10p3 ONLINE 0 0 0
da11p3 ONLINE 0 0 0
errors: No known data errors
fstab
Automatically mount your file systems at start up and tells you which file system corresponds to which drive, etc.
<fs> <mountpoint> <type> <opts> <dump/pass>.
It is recommended you use UUIDs for non-LVM non-ZFS. It is recommended you use the device mapper symlink for LVM or ZFS. I am not smart enough to know the benefits for either.
Options are seen in "Filesystem independent mount options" and "Filesystem dependent mount options" in the mount(8) man page. I'd just use defaults. Use _netdev for a network drive (cough David cough).
The numbers at the end are not too necessary, but read the man page on the 5th and 6th fields if you really want to know. Gentoo recommends having / at 0 1 and everything else as 0 2.
Here's my computer's setup:
/dev/lvm/lv_root / ext4 defaults 0 1
/dev/lvm/lv_home /home ext4 defaults 0 2
UUID=46ef4b37-b2bc-4a3a-b821-604c9fb64787 /boot ext4 defaults,noatime 0 2
UUID=61518d0a-1a81-4936-85e0-6bb7e1d69155 none swap sw 0 0