Intermittent Technology

HOME IT, NETWORKING, STORAGE

HOW TO INSTALL PROXMOX AND SETUP A

ZFS POOL
AUGUST 30, 2019 | [email protected] | 8 COMMENTS

How to install Proxmox and setup a ZFS Pool

This article is to accompany my video about setting up Proxmox, creating a ZFS Pool and
then installing a small VM on it. It’s a tutorial from start to finish!

Part of multiple articles

The article index for this 100TB 10Gbit server series can be found by clicking here.

Video

How to install Proxmox and setup a ZFS Pool + VM

 Home Networking: 100TB 10Gbit Server - Installing …

During this video we’re installing proxmox, creating a ZFS pool and installing a VM, here
are the parts in order with the video. It should contain everything you need to accomplish
the same as in the video!

Again, main explanations will be in the video, below text is to go along with it!

Download links
Proxmox ISO file – https://www.proxmox.com/en/downloads
BalenaEtcher – https://www.balena.io/etcher/

BIOS Settings
Some BIOS settings need to be changed for Proxmox do be able to do it’s virtualization,
mainly those are:

SVM Mode – Enabled (Enables CPU Virtualization)

Storage Boot Option Control – Disabled (Disables loading PCIe add-in card ROMs)
Other PCIe Device ROM Priority – Disabled

Fixing Update Repository

By Default Proxmox is configured for the Enterprise repository, without a license you can’t
use those so to reconfigure it to the community repository, do the following.

cd /etc/apt/sources.list.d/
ls

rm pve-enterprise.list
 cd ..

After this, we are going to edit the “sources.list” file and replace it’s content. You can
delete lines in nano by using “ctrl+k”.

nano sources.list

deb http://ftp.debian.org/debian buster main contrib

deb http://ftp.debian.org/debian buster-updates main contrib

# PVE pve-no-subscription repository provided by proxmox.com,

# NOT recommended for production use
deb http://download.proxmox.com/debian/pve buster pve-no-
subscription

# security updates
deb http://security.debian.org buster/updates main contrib

Save the file by hitting “ctrl+x” and choosing yes

Erasing partition table on disks

First we need to delete the partitions on the disks, we do this using a tool called “gdisk”
which is installed by default

First look up the disks listed in /dev/disk/by-id:

cd /dev/disk/by-id
ls

Then, select the disks you want to clear and run the following command as you see in the
video:

gdisk /dev/disk/by-id/xxxxxxxxxxx

There you can clear the disk by listing the partitions with “p” and then deleting them with
“d”. Once you are done deleting the partitions, exit with “w”.

Creaintg a simple RAIDz pool

To setup a RAIDz2 pool use the following command (the amount of disks can vary to your
needs)

zpool create NAME -o ashift=12 raidz2 /dev/disk/by-id/DISK1

/dev/disk/by-id/DISK2 /dev/disk/by-id/DISK3 /dev/disk/by-id/DISK4
/dev/disk/by-id/DISK5
To look at the pool status run:

zpool status

To turn compression on for the pool run:

zfs set compression=lz4 POOLNAME

Creating ISO storage

Here we create a dataset using the command-line using:

zfs create POOL/ISO

Video transcript
For people who don’t enjoy videos and would rather just read, here is the script I used
while preparing for the video. It’s raw and not edited to be in an easy to read post form! If
you find this useful, please let me know and I’ll try to post it more often!

Script
Hey guys, in this video we’re going to take a look at installing proxmox. This is going to be a
bit more basic level tutorial video but I have some more in depth stuff planned for upcom-
ing videos!

In this video we’ll run through downloading the ISO and creating a USB stick. Then we are
going to look at some BIOS settings and after that run through the installation process
together.

Once that’s done, we are also going to run through creating a simple RAIDz pool from
5x2TB disks. In a future video we are going to dive a bit more in depth into some more ad-
vanced ZFS Mirror pools with SSD caching.

In the last part of the video we do some basic setup stuff for a new proxmox server and to
end we’re going to install a small Linux VM together!

I’ll make sure to include time jump links again, Ok, let’s go!

—— Intro

Right, so to get started we need to download the newest version of proxmox. Links to ev-
erything and command-line commands will be put in a blog post you’ll find in the descrip-
tion, please check that out if you want to perform any of these steps yourself! On the
download page download the ISO file, we are going to use the 6.0 version.

While that’s downloading we also need a tool called “balenaetcher”.

Once both are downloaded and etcher is installed, insert your USB stick (Anything 1GB or
more should be fine) and start etcher. Point it to the downloaded ISO file and then to the
USB stick, hit write and wait.

You might get some partition popups, just click cancel on those.

Ok, stick done! Let’s go to the server and check some BIOS settings to make sure we avoid
some little problems.

— start server and hit del like a moron

In the BIOS it’s probably best to load optimized defaults and then make the following
changes:

First we want to enable “SVM Mode”, in my BIOS that’s located under “Advanced
Frequency Settings” and then “Advanced CPU Core Settings”. There you just set “SVM
Mode” to enabled. This enables certain virtualization features inside of your CPU which
Proxmox wants to use.

Then in the BIOS tab you want to set “Storage Boot Option Control” to Disabled and also
set “Other PCI Device ROM Priority” to disabled.

This disables the loading of option ROMs during boot. Since we won’t be booting from the
network or any storage connected to the LSI controller, we don’t need to load their
BIOSses while booting. If you do want to boot from the LSI controller you need to set the
first option to “Legacy Only”. Setting it to UEFI can work but that means you need to do
certain Proxmox stuff differently and I don’t recommend that right now!

Ok, hit F10 to save all changes and hit F12 to get into the boot override menu. There you
should see your USB stick listed, now it depends on your system if you need to select the
UEFI version of your USB Stick or not. In my case I need to, so I select “UEFI USB Disk 2.0
PMAP”.

A thing of note, a mouse could actually be easy to use during the graphical installer part, if
you have one, make sure it’s connected!

Once booted you should be greeted with the following screen. There hit the first option to
Install Proxmox.
Ofcourse we agree with the license agreement

Then we get to the storage configuration for where you want to install Proxmox. Here we
are going to make a few changes!

Hit Options and change EXT4 to ZFS (Raid 1). As you can see all the disks Proxmox detects
are now shown and we want to select the SSDs of which we want to create a mirror and in-
stall Proxmox onto. Set all the others to “– do not use –“. Then we want to do a little tweak-
ing in the advanced options.

Only change compression from on to LZ4. This shouldn’t be necessary but I like to be sure.
Then we also want to change the hdsize value. These are 256GB SSDs but I never use the
full size if I can avoid it. So in this case I’m going to enter a value of 200GB.

Hit ok and Next!

Fill in your Country details and once everything is correct hit Next again.

Set a password and enter an e-mail address. This address is important because the server
will send you messages when something is wrong!

Select the correct network card you connected and make sure to fill in a correct FQDN
hostname like you see here. Then also fill in all the IP details, it’s best to always use a static
IP for your virtual machine host machine.

Once you hit Next you’ll see a quick summary screen and once you hit Install the installa-
tion starts!

Ok, all done, let’s reboot!

Once the installation is done, Promxox should automatically boot. If it doesn’t, check your
BIOS that you have the Linux BootLoader or SSDs used selected as first boot devices.
Since we are running a mirror, make sure to select both so that if one fails it will still boot
from the other one!

OK, that’s done. Great! Almost done with all the basics! Let’s configure a few things more,
but using the web interface and remote sessions!

First we are going to make sure you can update Proxmox to the latest version. To do that
we need to enable to community repository.

To do this we log into the web interface, the address and port number for this are listed on
the screen. The first time you will get a security warning, you can ignore this safely. After
we proceed you’ll see the proxmox login prompt, there we log in with “root” and the pass-
word used during the installer.

Every time you log into this interface you will get a pop-up that you don’t have an official li-
cense. There are ways to remove this pop-up but I don’t really condone those methods.
You can use the software without paying for it and they only give you a single pop-up when
you log in. If you hate the pop-up that much, buy a license.

Ok, once in the interface, the first thing we want to do is get the latest updates. But this
doesn’t work out of the box, you need to reconfigure proxmox to use the community
repository.

To do this, open a shell and choose xterm. This didn’t work while recording so I opened a
putty session instead, basically the same thing. Just connect to the IP of the proxmox host
and login with root and your password again.

There we are going to change the repositories available. All these steps will also be type
out in a blog article so you can repeat them more easily.

Basically, we are removing the pve-enterprise repository file and then editing the default
one to include the community version.

After that we can run a “apt update” to get all the latest information and then install it us-
ing “apt dist-upgrade”.

Once that’s done, give the machine a reboot and you’re done, updating will now also work
from the GUI. To make sure the GUI reflects all changes, give it a refresh once the server is
back up.

Ok, next let’s configure a simple RAIDz pool. I’ll be doing videos on more complex ZFS
technical details and how to create more advanced pools but for now this is fine to start
out with.

My disks have already been used so although they show up under “disks” I can’t configure a
pool on them, let’s fix that. Also, although configuring a pool from the GUI now is available
I’d rather still do it from the command-line. That way I have a bit more control over it and I
also like to use /dev/disk/by-id links instead of /dev/sda sdb etc.. Using the /dev/disk/by-id
links it doesn’t matter how you reshuffle your disks, even between controllers or systems,
they will always work!

Ok, let’s first clear those disks. To do that we go back to command-line and use a tool
called “gdisk” to basically delete all partitions from the disks.
These are my Samsung disks I want to use. Let’s erase them. To erase you look at which
partitions exist using “p” and then d both those partitions and exist using w. And then we
repeat that for all the disks!

Now if we check in the GUI we can see that all disks are available, but as said, I’d rather use
the command-line anyway.

Again, commands will be listed in the description if you want to be able to read them more
without the video.

I’m not going to explain the zpool create command all too much. Just know you should ba-
sically always use the -o ashift=12. The “rz2TB” is the name I’m giving the pool and I’m
choosing a raidZ2 which uses 2 parity disks, so basically equivalent to RAID6 protection
wise. I’m using old 2TB Samsung disks which have been in service for over 8 years now, but
safe then sorry.

Ok, zpool created and available! Let’s make sure compression is turned on for the whole
pool. LZ2 compression is basically free on modern processors so it’s good to always enable
it!

Ok, the GUI is seeing our new pool and everything looks correct! But it’s not available to
use yet. You still need to add it as “storage” within Proxmox. You do that by adding it in the
storage menu. I enable thin provision because that works best if you want to host VM’s on
it.

Ok, all done and online! Next, lets create a place where we can put ISO files and make
backups to install a Ubuntu VM.

To do so we create whats called a ZFS dataset on the command-line. A dataset is a sort of

mix between a partition and a directory. If you are going to create directories with a spe-
cific purpose and want to maybe share them later on or set a quota on it, it’s good to make
it a dataset.

Then we are going to add that dataset as a directory to Proxmox so we can upload our
Linux ISO install file to it!

In storage we now add a directory, call it ISO and point it toward the dataset on the pool.
Here we say we want it to house everything except disk images and containers and we’ll
allow up to 10 backups.

Ok, go to the ISO storage and upload your favorite Linux ISO file, in my case a netinstall of
Ubuntu 18.04LTS. I won’t go through all the settings right now, but we’re using the ISO im-
age just uploaded and putting it on the boot SSDs for speed.
 Ok, once created, click on the VM, 100 and start it. Then you can click the console button
to view it. Ok, I’ll run through a very quick install of Ubuntu!

And well, that’s it! You’ve now setup a Proxmox machine, made it so we can run updates,
created some ZFS storage and even installed your first virtual machine!

Hopefully this tutorial was helpful to you. I’m quickly going to continue using my server to
migrate data from the old server to the new one! This was a bit of a “basic” video to get
started, more advanced and in depth videos coming up soon! A like and subscribe is always
appreciated, a share even more and I hope to see you back next time!

Concluding remarks
Again, this post if part of a larger series, click here to go to the index page.

BUILD COMPUTER GUIDE IRONWOLF L2ARC PROXMOX RYZEN SEAGATE SERVER

SSD TUTORIAL ZFS ZIL

8 THOUGHTS ON “HOW TO INSTALL PROXMOX AND SETUP A ZFS POOL”

Pingback: Building a 100TB 10Gbit Home Server - Intermittent Technology

iVerlaek
SEPTEMBER 21, 2019 AT 19:07

Great info, I am following it, for my own system.

f.y.i.
1.
please note but there are a few UPPER/lowercase errors in the commands to create the RAID.
2.
I also had to gdisk, then use the command “o” and the command “w” to completely empty the
disks and write the info to the disk.
3.
As I have a 24 disk system using /dev/sda to /dev/sdn (in my case) was easier for this gdisk com-
mand. And indeed I used the by-disk also, which is what you should do when you have hot swap-
pable disks.

 [email protected]
SEPTEMBER 30, 2019 AT 20:56
 Thnx for the comment, good points

Pingback: Proxmox – ZFS-Pool erstellen – nerd.WIKI

Mancuso
JANUARY 31, 2021 AT 17:14

Excellent guide! I sincerely appreciate the time and effort you’ve put into this (and your other
projects). You’ve made the command line fun again.

easyronny
JULY 7, 2021 AT 17:40

I used this manual to install Proxmox 7 only for the update repo there is a change in file
/etc/apt/sources.list

vi sources.list
(press A to Add / change)

deb http://ftp.debian.org/debian bullseye main contrib

deb http://ftp.debian.org/debian bullseye-updates main contrib

# PVE pve-no-subscription repository provided by proxmox.com,

# NOT recommended for production use
deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription

# security updates
deb http://security.debian.org/debian-security bullseye-security main contrib

When this is added press ESC and then Shift + Z and release Shift press enter

Julio Mendez
SEPTEMBER 24, 2021 AT 06:48

Can you make a youtube video on how to replace a rpool hard drive?

Pingback: Set up a zfs Pool on Proxmox - Nancy is a Nerd