(Nearly) All You Need to Know About Proxy Autoconfiguration, WPAD and PAC files

The principle behind the automatic configuration of proxy settings is obvious To enable users to automatically obtain their proxy settings, without the requirement of having the manually configure their browser (or internet application) settings. There are other benefits, such as the ability to quickly update proxy information, as well as the ability to specify fail-over proxies, in the event that a primary proxy is not available, and Ill go through those as well, but first PAC stands for Proxy Automic Configuration, and PAC files are the files that WPAD uses to pull down the proxy information. PAC files can be published via the WPAD protocol, or alternatively they can be manually configured in the browser by providing a path or URL to their location. WPAD stands for Web Proxy Automatic Discovery, and is a method published by either DHCP, DNS or both in order to enable browsers to automatically detect the proxy settings required for the network that they are on.

PAC Files
Ill begin with talking about PAC files, as that is where the proxy information is actually stored, and everything else is just the distribution method of your PAC files. PAC files are written in the Javascript language. They primarily contain the following information

The proxy server(s) to use The port of the proxy server(s) A list of sites or hosts that the proxy bypasses (the requests go directly out to the internet, without bypassing the proxy)

An example of a pretty basic PAC file, is this (dont worry, Ill break it down further) function FindProxyForURL(url, host) { if (isPlainHostName(host)) { return DIRECT; } if (isInNet(host, 192.168.0.0, 255.255.255.0)) { return DIRECT; } if (shExpMatch(host, www.host.com)) { return DIRECT; } if (shExpMatch(url, http://www.url.com*)) { return DIRECT; }

return PROXY proxy1:8080; PROXY proxy2:8080; DIRECT; } This is a standard PAC file, that retrieves the URL or host that the users browser has accessed, and compares it to a list of exceptions to determine whether it will allow direct access to the internet, or whether it will pass the connection through a proxy server. If the URL or host doesnt match anything in the exclusion list, it will then pass the connection through the first proxy. If the first proxy (proxy1) doesnt respond on port 8080, it will try to pass the connection through the second proxy (proxy2) on port 8080, and if that also fails, it will then allow the connection to pass out through the internet directly, as its least preferred option. You do not need to allow the connection to pass through to the internet directly if the proxy servers dont respond In fact, if you do not specifically include it as a fallback option, the connection will instead time out and the user will not be able to establish their connection until a proxy server responds. In case youre having difficulty identifying what does what in my PAC file example, Ill break it down a bit further function FindProxyForURL(url, host) This line obtains the URL and host information from the browser, so that it has information to compare against its exception list For example, if the user was trying to access Google, then http://www.google.com/ would be the URL and www.google.com would be the host. if (isPlainHostName(host)) { return DIRECT; } This essentially checks if this is a single label host, which means there are no full stops (periods). If a single label host can be resolved, its pretty much going to be internal (such as http://intranet for the URL or just intranet for the host. If the host fits these conditions, then the connection is not passed through the proxy This is what the return DIRECT part of the line means. if (isInNet(host, 192.168.0.0, 255.255.255.0)) { return DIRECT; } This checks if the IP address of the host is on the internal network (assuming 192.168.0.0 192.168.0.255 is your internal network) and therefore also bypasses the proxy for this connection. if shExpMatch(host, www.host.com)) { return DIRECT; } This is a direct string comparison, and grabs the host variable specified earlier, and compares it against www.host.com If it matches, then the connection bypasses the proxy.

if shExpMatch(url, http://www.url.com*)) [ return DIRECT; } This is another direct string comparison, but ends in a wildcard (*) which means that if the URL begins with http://www.url.com then it bypasses the proxy. return PROXY proxy1:8080; PROXY proxy2:8080; DIRECT; This is where your preferred proxy servers are listed. The connection will try proxy1 on port 8080 first, proxy2 on port 8080 second, and then go out directly to the internet if the first two proxies gave no response. PAC files can be called anything you want to call them, except if you are using the DNS WPAD implementation, in which case it MUST be called wpad.dat in lower case.

Publishing Your PAC File

There are essentially two ways to publish your PAC file On a file system (network share, or local machine) or via HTTP (on a web server). The file system method is pretty simple. Just copy the PAC file to the desired location. This could be the users local machine, or a file server. If you are using the users local machine, then I would assume this would either be a temporary measure, or that you are deploying PAC files to each of the machines for a specific requirement. The more common file system method, would be to copy the PAC file up to a file server, and reference it via a network share. In either of the file system based scenarios, you need to reference the PAC file by appending file://// to the beginning of the location, so a file stored in the users C: would become file:////C:/proxy.pac Notice the use of forward slashes, rather than backslashes in the path. This isnt specifically required, but makes sense. The same is true for the PAC file hosted on a file server, in that the reference would be file:////SERVER/SHARE/proxy.pac I have seen some cases where an extra fifth slash is required after file: particular in Firefox. Youll need to find the right balance here, depending on what your application prefers. If youre using the HTTP method, you need a MIME type of application/x-ns-proxy-autoconfig for the file extension, which should either be .dat or .pac and then reference the file using the full URL, for example http://server/proxy.pac

Manually Configuring PAC Files In Your Browser

This method assumes a few things about your environment

Your only have a few machines to manage, and manually configuring each one is no big deal; or You have a method of managing the browser configuration, such as Group Policy AND your are confident there are no internet capable application outside of your management (or they dont matter) AND your environment does not cater for external users roaming on your network (or its no big deal to configure these machines as they come on to your environment, and to de-configure them before they leave)

If you arent happy with these conditions, then the WPAD method (in the Deploying Automatic Configuration Using WPAD section) might be the way to go for you. For the purpose of this article, Im only going to go through Internet Explorer 7 (which is the same for Internet Explorer 6) and Firefox. Any other compatible applications will have a very similar configuration method. Lets start with Internet Explorer. Fire up IE, click on Tools -> Internet Options, go to the Connections tab and click on the LAN Settings button. You should clear any previous proxy configuration from here, and then tick Use automatic configuration script and provide the path or URL to your PAC file in here (see the section above Publishing Your PAC File if you havent already). You should also ensure no other tick boxes are checked, such as the Automatically detect settings box, as this will slow things down (it will be looking for the WPAD implementation, and when it cant find it, it will load your proxy script There is a noticeable delay). This is similar in Firefox. Click on Tools -> Options, click on the Advanced icon up the top, click the Network tab and then click the Settings button. Clear any previous proxy information here, select the Automatic proxy configuration URL radio button and provide the path or URL to your PAC file in here (see the section above Publishing Your PAC File if you havent already). As I mentioned before, you can manage these settings using Group Policy. For Internet Explorer, there are built-in configuration options in Group Policy, but in Firefox you need to rely on the use of third party tools, or alternatively an in-house developed option. There is a fantastic Firefox setting management add-on for Group Policy called FirefoxADM which can be downloaded from SourceForge at http://sourceforge.net/project/showfiles.php?group_id=129699

Deploying Automatic Configuration Using WPAD

As mentioned at the beginning of this article, you can publish your wpad.dat file using DNS and/or DHCP, although DHCP is probably the more preferred method because it is more flexible and is more easily distributed to your client machines than DNS is.

The DNS method requires the HTTP distribution of the wpad.dat file, and also requires that a CNAME alias record called wpad is created in the root domain in DNS and points to the web server that hosts your wpad.dat file. Ill go in to specifics shortly. The DHCP method is much more flexible, as it supports both the file system and HTTP based methods of wpad.dat distribution, and requires that you add an extra scope option to your DHCP server. DNS You need to have uploaded your wpad.dat (remember, lower casing for compatibility reasons) to an HTTP server and added the MIME type of application/x-ns-proxy-autoconfig for .dat file extensions. Also, its important that the file can be downloaded via the IP address, rather than the hostname (which means you CANT use host headers) because some applications actually resolve the host themselves, the then use the IP address to obtain the wpad.dat file from the server. Basically, if you cant download http://<ipaddress>/wpad.dat, then youre probably going to run in to issues. If you can get to your wpad.dat this way, then youre nearly there The second requirement is that you need to manage your own DNS services internally, and you need to add a CNAME alias record called wpad which points to the hostname of your HTTP server where your wpad.dat file is stored. This CNAME record needs to exist in the domain that you have recorded in your clients DNS suffix configuration on their NIC settings. If this doesnt exist, you need to populate that information on the NIC settings to avoid problems. If you are on a Windows domain, this should already be configured. From the clients, ensure you can ping browse to http://wpad/wpad.dat and download the file. If you can, then skip over the next DHCP section down to the part about browser configuration for WPAD. DHCP This is my preferred method, because

It supports both file system and HTTP based hosting of the PAC file; It supports custom ports; It doesnt require internally managed DNS; It doesnt require NIC settings modification to allow remote or misconfigured machines to resolve the WPAD DNS entry

To deploy your PAC file via DHCP, you need to add an extra scope option 252 to your DHCP scope. If you are using Windows 2003 DHCP, then you can following this article http://www.microsoft.com/technet/isa/2004/help/SRSP1_H_Create252.mspx If youre running a different DHCP server, you need to ensure that it supports the addition of custom scope options. If it does, create the 252 option, and then add it to your scope populating

the information with the location to your PAC file, but its important to add a trailing space to the location of your file, as there are some cases where the last character is truncated and therefore the PAC file is not loaded correctly. Youll need to renew the DHCP lease on the clients in order for them to obtain this information. Unfortunately, the only way to verify that your clients are receiving this information, is to use network capturing software, such as Microsofts NetMon, to monitor the DHCP lease negotiation. Configuring the browser The last step with the WPAD implementation, is just to ensure that the Automatically detect settings box is checked in the browser (called Auto-detect proxy settings for this network in Firefox). You can do this by Group Policy, if thats an option, or make the change for/advise your users. Tags: auto-config, auto-configuration, auto-detect, auto-discovery, autoconfig, autoconfiguration, autodetect, autodiscovery, pac, pac file, proxy, wpad
This entry was posted on Saturday, May 17th, 2008 at 2:55 PM and is filed under Network, Windows. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

40 Responses to (Nearly) All You Need to Know About Proxy Autoconfiguration, WPAD and PAC files
1. c0vert says: November 7, 2008 at 6:48 AM great article! this helped alot, thanks! 2. Mat Mirabito says: November 7, 2008 at 7:27 AM Thanks! Glad you found it useful. 3. Configurazione automatica proxy | Problemi e Soluzioni di un sistemista informatico says: December 5, 2008 at 3:03 AM [...] http://techblog.mirabito.net.au/?p=21 [...]

4.

Josh says: December 10, 2008 at 7:13 AM Did not know if you were still monitoring this or not, but in case you are here is my question: We are using the proxy pac file stored on local machines outside of our network as a whitelist, anything that does not match one of the listed sites gets redirected back to 127.0.0.1. The problem I am having is when I update the proxy file for a new web address, the update does not take affect, or it is not recognized by IE. Is there something I should be doing to re-register it locally?

5.

Mat Mirabito says: December 10, 2008 at 7:30 AM Hi Josh, By default, Internet Explorer will use automatic proxy caching. Have a read of this Microsoft KB article for more information on how this works, and how to modify the behaviour http://support.microsoft.com/kb/271361 Let me know if you need further info!

6.

Jim Argo says: January 22, 2009 at 8:30 AM Hi I am using Firefox v3 along with a wpad.dat on a web/IIS server. I have the DNS records setup to point to my IIS server. From the XP client I get prompted to download the wpad.dat file when I type http:///wpad.dat in the address bar, so I think everything is configured properly. My question is why doesnt the Firefox auto-detect work but Internet Explorer works fine? htttp://wpad/wpad.dat also prompts as a file download in Firefox but when auto-detect is selected and I restart FF it cant get anywhere, any suggestions? Thanks.

7.

Mat Mirabito says: January 22, 2009 at 8:37 AM Hi Jim,

Try manually setting Firefox to use http://wpad/wpad.dat and see if this works. If it doesnt work using this method, then the problem lies with the wpad.dat file itself rather than the delivery method. 8. SuD says: January 23, 2009 at 9:24 PM How can I know if me or a customer has been hijacked? I mean, apart from configuration of WPAC in IE where can I see current status of WPAC in IE?

9.

Mat Mirabito says: January 23, 2009 at 9:53 PM Hi SuD, As far as I know, theres no feature in any browser that will show you that option easily. What you can do though, is run netstat -b from the command line of the machine, and check the connections that your browser is making. There are a few GUI tools you can use that might make things a bit easier I recommend TCPView

10.

Michael M. says: January 28, 2009 at 12:05 AM Excellent stuff this.. One question though. I am considdering changing the wpad.dat file our ISA server issue, however this seem to be somewhat complex in comparison to your example above. My main goal would be to implement failover via inserting a second proxy in the proxy array (we have 3 isa servers worldwide). I dont know if you are familiar with the ISA servers wpad.dat (its a coupple of hundred lines long), but in case you are is there any catches to exchanging it with a shorter and simpler one (like a modified version of yours).

11.

Mat Mirabito says: January 28, 2009 at 7:57 AM Hi Michael, I am somewhat familiar with the wpad.dat file in ISA, although I do exactly what you are suggesting, in creating my own WPAD file instead. There arent any fundamental issues with this. Youll obviously no longer make changes via the ISA GUI for things that it will normally put in the wpad.dat file itself, but I prefer to have direct control over it myself anyway; it means its easier to troubleshoot!

12. Setting up proxy autodetect Michael Ellerbeck says: March 17, 2009 at 3:58 AM [...] page explains this well http://techblog.mirabito.net.au/?p=21 actually its a great post about WPAD and PAC [...] 13. Guruprasad says: March 20, 2009 at 12:57 AM We are using wpad.dat via DHCP (scope id 252) and found when a client is static IP configured, IE automatically detect settings doesnt work.

14.

Mat Mirabito says: March 20, 2009 at 8:32 PM Hi Guruprasad, This is, of course, the expected behaviour in that if you are not using DHCP to assign client information, they cannot obtain the WPAD options defined in your 252 scope option either. The recommendation is to either use DHCP reservations in place of static IP address, or to use DNS instead of DHCP to publish your proxy settings to your clients (or even a combination of both).

15.

Hamid says:

December 23, 2009 at 4:12 AM This is a great post! I hope it is still being monitored as I have a question. If I wanted a specific network to go through a different proxy, how could I add this to my existing wpad.dat? Thanks, Hamid

16.

Mat Mirabito says: December 23, 2009 at 9:21 AM Hi Hamid, You can use this method, which will resolve the local IP address of your WPAD client, and then return a proxy based on the subnet that the client is in: clientip=dnsResolve(host) if (isInNet(clientip, 10.0.0.0, 255.255.255.0)) { return PROXY proxy1:8080 }

17.

Danny Maguire says: January 8, 2010 at 2:57 AM Hi, and happy NY, We use a HTTP URL based proxy.pac in IE6 browsers. My question, if you are at all able to answer, is how often or how does the browser query that particular network file. Is it cached on teh machine locally when the browser is opened or is it queried across the LAN / WAN for every url request? many thanks in advance. Danny

18.

Mat Mirabito says: January 8, 2010 at 9:08 AM Hi Danny,

The proxy.pac file is not queried per URL request; it is queried based on your browser session. In Internet Explorer, the PAC is cached for a period of time after the browser loads the .pac file. There are registry settings to modify the cache period for IE. 19. Kristof says: January 29, 2010 at 6:46 PM Very userfull post, Still one question. When de do the setup using DHCP and the 252 option, this means that every PC has to be verified if the option auto-detect proxu-settings is checked or not. Im looking for a system where I can push our proxy-settings to every PC that enters our network and receive a DHCP address. Thanks in advance ! 20. Mat Mirabito says: January 30, 2010 at 1:23 PM Hi Kristof, You have at least two options in this scenario: 1) If all of the machines under your management are on an Active Directory domain, you can use group policy to enforce the Internet Explorer auto-detect checkbox 2) You can investigate the use of a transparent proxy, which effectively intercepts all outbound internet traffic on port 80 and redirects it automatically through your proxy server This requires no client-side configuration, but there are other disadvantages that you should research You can also use a combination of the two, or develop your own solution to the problem using scripting or simply advising external users that might not be in your Active Directory domain to tick the auto-detect box manually. I hope this helps.

21.

Chad Deeds says: April 27, 2010 at 7:36 AM To go along with your reply to Guruprasad, is it possible to run a combination of the DHCP option 252 configuration as well as DNS (CNAME record for wpad)? If so, are there any advantages or disadvantages to this? Weve been using DNS to resolve wpad for clients, but Im looking to change this over to DHCP because it seems like it would be a more reliable option. Any recommendations or tips would be greatly appreciated! Great article BTW!

22.

Mat Mirabito says: April 27, 2010 at 8:33 AM Hi Chad, You can use a combination of DHCP and DNS in your environment if you like, however be aware that different browser and browser versions will prefer one option over the other. Using DHCP you can assign a different WPAD file to clients in different DHCP scopes to redirect them to different proxy servers. With DNS, to accomplish this same result, youd need to build the client location logic in to the WPAD file itself (see my response to Hamid earlier) as you can only provide one WPAD file per domain.

23.

Chad Deeds says: April 28, 2010 at 4:31 AM Thanks for the reply! Would you mind elaborating more on the different browsers and browser versions preference or link to further reading on this? Ive searched online but havent found any details as of yet.

24.

Mat Mirabito says: April 28, 2010 at 11:21 AM If you havent already, check out the Wikipedia article for WPAD, as its quite good http://en.wikipedia.org/wiki/Web_Proxy_Autodiscovery_Protocol

Regarding the browser implementations of WPAD, my understanding is that the WPAD draft states that the browser should first try DHCP before trying DNS. I cant remember if I read somewhere, or discovered through personal use, that a version of Firefox I was running preferred DNS to DHCP (i.e. would check DNS before checking DHCP). Id suggest checking the release notes and technical documentation for each browser you intend to deploy to verify how it handles WPAD requests. 25. Chad Deeds says: April 29, 2010 at 2:06 AM Very good info! Thanks again! 26. Bruce Larson says: May 25, 2010 at 3:37 AM Can the wpad file be used to se exceptions? That is, can you set a list of URLs that should never go through the proxy server? Thanks great post. 27. Stefan says: June 1, 2010 at 12:42 AM Hi Mat Very useful article, thanks! Question: How do you deal with laptops in the internet, not finding the proxy.pac file on the internal webserver becouse there are not allowed to? There are extremly long timeouts depending on the webbrowser. I considered providing an nearly empty pac file (only return DIRECT) on an external webserver which is resolved with anonther ip address on the external DNS Servers then on the internal dns servers. Regards Stefan 28. Mat Mirabito says:

June 1, 2010 at 6:23 AM Hi Bruce, You can certainly configure exceptions in the WPAD file. In fact, the article above even has some examples. 29. Mat Mirabito says: June 1, 2010 at 6:25 AM Hi Stefan, The timeout shouldnt be too noticeable, as it only occurs when the browser is first opened. Once the WPAD file cannot be located, the rest of the browsing experience isnt affected by the WPAD file not being accessible. You can certainly point the browser to a blank WPAD file using split DNS if you wish, however typically this isnt required. Generally, the convenience of not having to manually configure the proxies in a large, distributed and mobile organisation outweighs the initial delay while the browser tries to locate the WPAD file. 30. Bongz says: August 17, 2011 at 5:59 PM Hi Mat I have got my web server configured with https: can I still use this option or do I need to configure an http webserver ? 31. Chuttu Shah says: August 23, 2011 at 3:24 AM Is there a way for me to configure the PAC file and/or configure the browser to failover to the next proxy server should the first proxy server does not respond in a certain time (say 15 secs). Thank you for your help with this.

32.

Prabu says: December 5, 2011 at 11:01 PM I have a issues on my ISA can you please help me.

33.

Prabu says: December 5, 2011 at 11:04 PM My client is not pointing the exact ISA firewall which i mentioned in the DHCP server, instead pointing to another firewall. Can you please help me how to solve this issue. ([email protected]).

34.

Brian says: February 11, 2012 at 3:02 AM dont no if this is still being checked or not, but is there anywhere I can check on the local PC to see what version of the wpad file it has? For example if I make a change to wpad, i would like to somehow check something on the PC to see if it was the new wpad or see if it has a cached version of the old one?

35.

Mat Mirabito says: February 11, 2012 at 2:38 PM Hi Brian, You can clear the cached WPAD configuration by deleting the DefaultConnectionSettings and SavedLegacySettings registry keys from HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections This is a pretty good article on Microsoft TechNet for further reading http://technet.microsoft.com/en-us/library/cc302643.aspx

36.

Mat Mirabito says: February 11, 2012 at 2:39 PM Prabu, the above post to Brian may also help you out with your query.

37.

Mat Mirabito says: February 11, 2012 at 2:40 PM Chuttu Shah, I am not sure if the timeout values are configurable. My understanding is that the browser makes this decision.

38.

Mat Mirabito says: February 11, 2012 at 2:40 PM Bongz, I believe this would depend on the browser.

39.

Zulu says: February 17, 2012 at 9:11 PM Hi Mat, wonder if you could help me please? I have a Win2008 server running DHCP with WPAD (252) enabled on the scope pointing to our proxy server. All PC clients are picking up the proxy settting automatically and can browse onto the web no problem without manual intervention. An additional requirement surfaced asking to perform the same functionality, but for Ipads and Iphones. At present (using the same DHCP scopes) the Ipads and Iphones will not pick up the proxy address automatically (manually works fine) What do I need to do to get Ipads/Iphones proxy setting automatically, but without disrupting the PC side of things? Many Thanks

40.

Mat Mirabito says: February 17, 2012 at 9:35 PM Hi Zulu, Maybe something in this thread will help you (its for Mac OS X, but I reckon the protocol is still being handled the same way): http://forums.macrumors.com/showthread.php?t=991781

The options provided were to manually append a null character (%00) to either the wpad.dat file, or as a redirect in IIS, or as the second option was to use the DNS alias WPAD