Tutorial: How to configure squid to use a separate hard drive as its cache

on: December 24, 2009, 08:27:47 pm

I've spent literally weeks trying to get my pfsense/squid setup finished... and its finally nearly
complete! I tell you what, if someone was paying me only $7.00/hour to get this thing set up,
they would be spending hundreds.
Before I get into anything, I'll just say that I'm trying to make this as noob proof as possible.
That said, I hope this will help those who are new to FreeBSD/pfsense.
The hardest part for me was that I was having a very hard time configuring squid to use a
different hard drive. As some of you know, using a separate hard drive is faster... and you
usually get more space that way too. The problem is, however, that caching on a separate hard
drive is not officially supported by pfsense. Therefore, you might run into problems and you
might have to spend hours reading the freeBSD handbook and searching through threads,
maybe even posting threads... sometimes getting "RTF's." Nevertheless, I think the process is
quite straightforward and its definitely worth a try!
Here it is:
If you've not done so already, connect your hard drive and configure jumper settings (master,
slave) if needed.
You will need to delete all data from the drive. Then you have to create a slice, or partition. You
can do this manually in the console but I've personally ran into many problems doing it that way.
A much better way, I've found is to use a pfsense liveCD.
Insert the CD and reboot the computer. When you get to the part where it asks if you want to
enter Recovery Mode or Installer Mode, hit "I" to enter the installer. When you get to the man
install screen and it asks you if you want to do a quick install or a custom install, hit 'custom
install.' We will not be installing pfsense now, but just partitioning the hard drive.
Next, it will ask you what hard drive you want to install it on. Be sure to choose the right one. If
your drive is a IDE like mine and its a slave, then it will be ad1. Just keep continuing through
the setups... it will go to the disk geometry screen, and you hit "accept this geometry." And then
it will want you to create a slice (I think thats what its called), so you'll create one. Then it will
want you to install bootblocks. You should just skip this step. Then it will want you to partition it.
When you get to this screen, you need to delete the default swap partition and change the
mount point to something like /squid/cache or /cache/squid. Or, it might just work with /cache.
Then you will hit continue...

At this point, it will want to format the partition and begin the installation of pfsense. I've found
that for some reason, pfsense does not format the partition correctly and it comes back with an
error. This might be because it is confused because there is no swap partition, I don't know. But,
this problem is easily remedied by manually formatting the hard drive.
So, at this point you need to keep on hitting the "return to..." buttons until you are back to the
main install page. There you will hit 'reboot' and once its rebooted, remove the CD. Then, it
should load back into your pfsense HD install.
From there, you'll need to hit 8 to enter the shell where you can type in commands. Here, you
need to type

Code: [Select]
newfs /dev/ad1s1a Replace the '/dev/ad1s1a' with the proper name for your hard drive and the
slice on your hard drive.
After that is finished, we need to edit /etc/fstab. Type "ee /etc/fstab" (without quotes) and add
the following line:

Code: [Select]
/dev/ad1s1a

/cache/squid/

ufs

rw

1 Again, change the

"/cache/squid" to match the mountpoint that you want to set for your squid cache.
Create the /cache directory (for pfsense to mount your hard drive on). Do that with

Code: [Select]
mkdir /cache/squid again, chaning the "/cache/squid" so that it matches your setup.

Mount the hard drive by typing

Code: [Select]
mount -a . Then type in df -h to make sure its mounted. it should look something like this:
Code: [Select]
# df -h
Filesystem

Size

/dev/ad0s1a
devfs

devfs

1.9G

1.0K

/dev/md0

Used Avail Capacity Mounted on

1.0K

3.6M
1.0K

/dev/ad1s1a

177M

28K

1.0K

54G

22M

1.6G

10%

0B 100%
3.3M

50G

/dev

1%

0B 100%

/var/run
/var/dhcpd/dev

0%

/cache/squid

Now all thats left to do is to enter your pfsense GUI as you normally would, and go to Services
--> Squid --> Cache Management. Where it asks you where the cache is to be stored, enter
"cache/squid"
Now, if all went well, you should be done! Check and make sure that there are no errors in your
System Log... Satus --> System Logs. It may be necessary for you to manually create the swap
directories. In which case, go back to the command prompt and type in

Code: [Select]
squid -z .

Its also a good idea to check and make sure that it create your swap directories. Of course, if
squid-z comes back with an error, then it obviously did not. But, if it did then typing in ls
/cache/squid will show you something like this:

Code: [Select]
# ls /cache/squid
.snap

05

0B

11

17

1D

00

06

0C

12

18

1E

01

07

0D

13

19

1F

02

08

0E

14

1A

swap.state

03

09

0F

15

1B

04

0A

10

16

1C

If it did not work, it will only show .snap and nothing else. So, if ls /cache/squid shows the above
then congratulations! Now the only thing left for you to do is to delete your other squid cache by
typing

Code: [Select]
rm -R /var/squid/cache/*
Some Troubleshooting:
If you were unable to create swap directories (as I was), it might say something like this:

Code: [Select]
# squid -z
2009/12/24 20:56:44| parseConfigFile: squid.conf:62 unrecognized: 'delay_pools'
2009/12/24 20:56:44| parseConfigFile: squid.conf:63 unrecognized: 'delay_class'
2009/12/24 20:56:44| parseConfigFile: squid.conf:64 unrecognized: 'delay_parameters'
2009/12/24 20:56:44| parseConfigFile: squid.conf:65 unrecognized:
'delay_initial_bucket_level'

2009/12/24 20:56:44| parseConfigFile: squid.conf:66 unrecognized: 'delay_access'

Code: [Select]
Dec 24 20:53:07
Dec 24 20:53:08

squid[43341]: Squid Parent: child process 43504 started

cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid
for the first time.
Dec 24 20:53:08

kernel: pid 43504 (squid), uid 62: exited on signal 6

Dec 24 20:53:08

squid[43341]: Squid Parent: child process 43504 exited due to signal

6
Dec 24 20:53:08

squid[43341]: Exiting due to repeated, frequent failures

Dec 24 20:53:55

Squid_Alarm[43599]: Squid has exited. Reconfiguring filter.

Dec 24 20:53:55

Squid_Alarm[43601]: Attempting restart...

Dec 24 20:53:55

squid[43608]: Squid Parent: child process 43611 started

Dec 24 20:53:55

squid[43611]: Failed to verify one of the swap directories, Check

cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid
for the first time.
Dec 24 20:53:55

kernel: pid 43611 (squid), uid 62: exited on signal 6

Dec 24 20:53:55

squid[43608]: Squid Parent: child process 43611 exited due to signal

6
Dec 24 20:53:58

Squid_Alarm[43615]: Reconfiguring filter...

Dec 24 20:53:58

squid[43608]: Squid Parent: child process 43618 started

Dec 24 20:53:58

squid[43618]: Failed to verify one of the swap directories, Check

cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid
for the first time.
Dec 24 20:53:58

kernel: pid 43618 (squid), uid 62: exited on signal 6

Dec 24 20:53:58

squid[43608]: Squid Parent: child process 43618 exited due to signal

Dec 24 20:53:59

Squid_Alarm[43693]: Squid has resumed. Reconfiguring filter.

Dec 24 20:54:01

squid[43608]: Squid Parent: child process 43763 started

Dec 24 20:54:02

squid[43763]: Failed to verify one of the swap directories, Check

cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid
for the first time.
Dec 24 20:54:02

kernel: pid 43763 (squid), uid 62: exited on signal 6

Dec 24 20:54:02

squid[43608]: Squid Parent: child process 43763 exited due to signal

6
Dec 24 20:54:05

squid[43608]: Squid Parent: child process 43768 started

Dec 24 20:54:05

squid[43768]: Failed to verify one of the swap directories, Check

cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid
for the first time.
Dec 24 20:54:05

kernel: pid 43768 (squid), uid 62: exited on signal 6

Dec 24 20:54:05

squid[43608]: Squid Parent: child process 43768 exited due to signal

6
Dec 24 20:54:08

squid[43608]: Squid Parent: child process 43772 started

Dec 24 20:54:08

squid[43772]: Failed to verify one of the swap directories, Check

cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid
for the first time.
Dec 24 20:54:08

kernel: pid 43772 (squid), uid 62: exited on signal 6

Dec 24 20:54:08

squid[43608]: Squid Parent: child process 43772 exited due to signal

6
Dec 24 20:54:08

squid[43608]: Exiting due to repeated, frequent failures

Dec 24 20:54:55

Squid_Alarm[43880]: Squid has exited. Reconfiguring filter.

Dec 24 20:54:55

Squid_Alarm[43882]: Attempting restart...

Dec 24 20:54:56

squid[43889]: Squid Parent: child process 43892 started

Dec 24 20:54:56

squid[43892]: Failed to verify one of the swap directories, Check

cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid
for the first time.
Dec 24 20:54:56

kernel: pid 43892 (squid), uid 62: exited on signal 6

Dec 24 20:54:56

squid[43889]: Squid Parent: child process 43892 exited due to signal

6
Dec 24 20:54:59

Squid_Alarm[43896]: Reconfiguring filter...

Dec 24 20:54:59

squid[43889]: Squid Parent: child process 43899 started

Dec 24 20:54:59

squid[43899]: Failed to verify one of the swap directories, Check

cache.log for details. Run 'squid -z' to create swap directories if needed, or if running Squid
for the first time.
Dec 24 20:54:59

kernel: pid 43899 (squid), uid 62: exited on signal 6

For this, it might be worthwhile to read the squid faq page where it talks about
this: http://wiki.squid-cache.org/SquidFaq/TroubleShooting.
For me, however, that did not solve my problem. I simply had to change privileges of the file by
typing in

Code: [Select]
chown proxy:proxy /cache/squid . After that, I was able to do squid -z properly and everything
went smoothly!
So, I hope this will help other people. I'm sorry, I'm not very good at making how-tos but
hopefully this will work. If someone wants to rewrite it and post it in with the pfsense tutorials
and howtos then be my guest!