Configuration: Windows native DialUP VPN client & Netscreen

Posted by Ahsan Tasneem | 12:30 AM | , , , , , | 0 comments »

As the subject of this topic states I want to create a dialup VPN to a Netscreen Firewall. To make it more complex I will choose the onboard Windows VPN client. At the moment only a few documentations exist and most of them contain only pieces of the whole setup process. Different threads in discussion groups out there state that the whole XP-VPN/Netscreen thing will not be possible but they lack the proof that this cannot be done.

STEP 1: Checking prerequisitesAt the moment there are only two software requirements that have to be met. You will need a Netscreen with ScreenOS 5.1 and higher. This is the first firmware that will support a VPN client behind a router that has no official internet IP address (NAT-T). On the client side you will need Windows XP with at least SP2. It contains the so called L2TP/IPSec-NAT-T-Update (MS KB818043) that will enable the client to handle this situation correctly.
From the design aspect we will have to use a certificate based IPsec authentication. Preshared keys will not work for Windows XP roadwarriors as the builtin client only sends its IP or its FQDN as Phase 1 ids. This does not matter as certificates should always be the choice to go.

STEP 2: Creating a root certificate.
When working with certificates we will have to obtain a root certificate (or certificate authority = CA) that can be used to check the trustworthiness of any user or firewall certificate that we need in the later setup process. As we want to take the cheap way, we will generate the root certificate ourselves with openssl on a linux machine. Nothing easier than that. First we will create a private key for the root certificate.

linux > openssl genrsa -des3 -out ca.key 4096
Generating RSA private key, 4096 bit long modulus
e is 65537 (0x10001)
Enter pass phrase for ca.key:
Verifying - Enter pass phrase for ca.key:
linux > ls -l
total 4
-rw-r--r-- 1 root root 3311 Jan 13 19:40 ca.key 

And voila ca.crt is our root certificate. This could be a good basis for printing money if we convince someone to sign his certificates by us. But the only thing we want to do at the moment is to load this certificate into the Netscreen. Simply logon to your firewall an go to Objects > Certificates. Search for your ca.crt File and click on load. When you change the view to Show CA afterwards you should see your just uploaded certificate.

STEP 3: Certificate Revocation List
When you give away certificates that where signed with a root certificate it may be required to mark the certificate invalid although it's desired lifetime is not over yet. This ist done by a CRL (certificate revocation list). A CRL of a root certificate is nothing more than all the signed certificates that should no longer be trusted. A Netscreen will handle CRLs but do we need them? In my simple setup this will never be the case for the following reason. In IKE phase 1 a client will send its certificate that the netscreen has to match in its user database. This is done by comparing the contents of the certificate (Email, CN, Username, ...) with the information that is stored in each dialup user. The handshake will fail if no corresponding user data is found. In the reversal conclusion we can say a certificate will be invalid as soon as the designated user is deleted from the Netscreen.
To disable the CRL for a root certificate we have to logon to the device by telnet or ssh and have to determine the internal certificate id. It si only because I did not find a feature in the web interface to accomplish the task.

netscreen-> get pki x509 list ca-cert

Getting CA CERT ...
IDX ID num X509 Certificate Subject Distinguish Name
0003 89129012 CA CERT friendly name <52>
CN=ACME Inc.,O=ACME Inc.,L=Tampa,ST=Florida,C=US,
Expire on 01- 8-2028 18:45, Issued By:
CN=ACME Inc.,O=ACME Inc.,L=Tampa,ST=Florida,C=US,

Our root certificate is stored with the id 89129012. With this we can disable the revocation check in the following way.

netscreen-> set pki authority 89129012 cert-status revocation-check none

To save the changes we just have to logoff from the device

netscreen-> exit
Configuration modified, save? [y]/n y
Now our root certificate is in place.

STEP 4: Firewall certificate
Creating an L2TP over IPsec Dialup VPN with certificates means that both sides (client and firewall) need their own certificate. Luckily the netscreen supports the generation of a private key and a signing request. So we will jump into the web interface to objects > certificates > new and create a new one. Here you normally can enter any text you like. Nevertheless I want to implement the FQDN feature that signal15 described in his thread (MAC OS VPN client & Netscreen). What is the idea behind that? Let's assume our firewalls outside interface can be reached via DNS name If the client tries to do the IPSec handshake it will receive the certificate of the firewall. Anywhere in this certificate the DNS name should occur. Only if this condition is met the client can trust its partner because the certificate was created exclusively for this domain. So we will insert the following data into the certificate subject information:

Name : ACME Firewall
Organization : ACME Inc.

Don't forget to set the FQDN to what the client will point to later. Set the key to RSA 2048 bits and let the netscreen generate a private key and a certificate signing request. We will copy the request out of the following screen and store it side by side to our root certificate as file fw.csr. Tries over tries led me to the conclusion that an offical signing request will place the FQDN automatically into the additional certificate field called alternative name (subjectAltName). Unfortunately this does not apply if one is using self signed certificates. So we will have to add the field manually. For this reason we have to create the file subj.txt with the following content.


With this file we finally can sign our firewall certificate
linux > openssl x509 -req -days 7300 -in fw.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out fw.crt -extfile subj.txt
Signature ok
subject=/O=ACME Inc./CN=0096052005000522/CN=rsa-key/ Firewall
Getting CA Private Key
Enter pass phrase for ca.key:


The above step creates the new certificate file fw.crt with a lifetime of 20 years that is "officially" signed by our own root certificate. Pay attention that the serial you provide in the command line is not used by another certificate that is already loaded into your netscreen. Before transferring the file to the firewall we can take the last chance to check if the subjectAltName has found its way into the certificate.

linux > openssl x509 -in fw.crt -text -noout
X509v3 extensions:
X509v3 Subject Alternative Name:


This worked well and we can log on to the netscreen web interface again and go to objects > certificates. Provide the path to the file fw.crt and upload it to the firewall. I think this was not to difficult and saved quite a lot of money. Btw. if anyone experiences problems when generating and signing certificates with 64bit Linux machines should try the same on a 32bit device.

STEP 5: Creating a user group

Before we can setup the IKE firewall configuration we need a user group. For this we simply jump to objects > users > local groups in the web interface of the netscreen. There we create a new group ACME Teleworkers. Remote users will be assigned to the group later.

STEP 6: IKE gateway

Now it is time for the configuration of IKE phase 1. The webinterface provides the required dialog at location VPNs > Autokey Advanced > Gateway. As one would expect we create a new entry and call it GW-ACME-Teleworkers. The dialup users will connect to our firewall from changing IP addresses and so the only solution for setting a remote gateway type will be Dialup User Group. Choose the new item ACME Teleworkers from the group drop down list. All other fields on this page are not from interest. Preshared keys will not be used and our local id is stored in the firewall certificate. 
From here we directly go to the advanced configuration page. For simplicity I set the security level to custom and provide only rsa-g2-3des-sha as phase 1 proposal. The tree other proposal fields can be set to none. The Windows VPN client only supports main mode negotation (a handshake with more phases) so we have to enable that radio button. Activate the Enable NAT-Traversal checkbox and set keepalive frequencies or heartbeat settings to whatever fits your needs best. Our szenario does not need any XAuth settings so we can set this parameter to none.
Finally we must choose the preferred certificates for this gateway. The local certificate will be the firewall certificate (CN=ACME Firewall) and the peer CA must be the corresponding root certificate (CN=ACME Inc). Don't forget to set the certificate type to X509-SIG. Selecting the Return button we come back to the basic settings where we choose Ok to save the new gateway.

STEP 7: IKE phase 2
Now it is time to define the phase 2 parameters for the new gateway (phase 1). This can be done in VPNs > AutoKey IKE. For readability we call this VPN IKE-ACME and assign the predefined gateway GW-ACME-Teleworkers to it. In the advanced options there is again some work to do. We set the security level to custom with the proposals nopfs-esp-3des-sha, nopfs-esp-3des-md5, nopfs-esp-des-sha and nopfs-esp-des-md5. I think we won't need them all, but this time a had not leisure to test it out. The flag Transport Mode has to be checked because the encryption of packet headers in tunnel mode would break the communication in a NAT environment. 
Binding has to be set to none and not only proxy ids but also the local and remote ips have to be left blank. The reason should be clear. We are building a policy based routing and the client will connect with a random IP as proxy id. Leave the other fields as they are and click on Return. Back in the normal Autkey IKE data we choose Save to make the changes permanent.

STEP 8: Defining an IP pool

When providing an dialup network the outside teleworker will receive an IP address on its virtual adapter VPN that is different from the inside network. This is due to the fact that otherwise an inside machine would not answer incoming TCP/IP packets by sending them back to the gateway (the Netscreen) that will forward them via the VPN tunnel. Let's assume ACME Inc. has an inside network with IP adresses in the range The teleworkers should be assigned the address range from In Objects > IP Pools one can define the required address intervall. We choose New and name the IP pool IPPool-Teleworkers. Insert as the desired interval and save the settings.
Make sure that there are no overlapping networks when creating an IP pool. If you have multiple gateways also remember that the default gateway in your inside topology for the VPN network is pointing to the internal Netscreen interface.

STEP 8: L2TP Tunnel

After defining the IPsec parameters for our connection it is time to manage the L2TP tunnel parameters. Enter VPNs > L2TP > Tunnel and create a new one called Tunnel-ACME-Teleworkers. We don't want to bother with L2TP default settings of the Netscreen so we choose custom settings this time. The users will be maintained in the firewall so the authentication server must be set to local and we will allow access for the dialup user group ACME Teleworkers. Everything that deals with interface, peer IP, host and secret is not in our focus. Just leave the fields as they are. For the IP pool name we take the newly created IPPool-Teleworkers. The ACME company maintains a DNS/WINS server which can be reached by IP and that is exactly what we will provide in the corresponding fields of the web dialogue. Choose save and again a step is finished.

STEP 9: Policy

To allow traffic from the VPN notebooks to our internal network we still need a policy. This will be the simplest part of the whole configuration. Open the Policies page, choose the direction "from untrust to trust" and create the object Policy-ACME-Teleworkers. The source address is our address book entry Dial-Up VPN. It is an predefined entry in your Netscreen and it will fit to any address that matches not otherwise. Your local LAN should be the destination address and the allowed services can be set to ANY - supposed you want to give full access to the dialup clients. Because we have not defined a tunnel interface yet we will choose the action Tunnel. For this tunnel the VPN has to be set to IKE-ACME and L2TP needs be Tunnel-ACME-Teleworkers. Activate logging if you like and choose Ok to save your changes.
With this the firewall internal ruleset is built and we will care for the client configuration in the next posts

STEP 10: Creating dialup users

Now that everything is in place on the firewall we can create the users that will be allowed to connect to the ACME company network. For each user we go to Objects > Users > Local and hit the New button. The first person we want to grant access is our employee of the month John Doe. So we name the user John.Doe and enable him. This will be the user id the client has to provide in the username field of the dialup dialogue. As this entry should be taken into account for IKE handshakes we check the flag IKE User. Mark the Use Distinguished Name for ID radio button and enter John.Doe.01 for the CN and as the email. These texts will have to match the contents of the user certificate we will hand out to the client. You may ask why I chose to enter the number 01 into these text fields but the reason behind that is simple. If John Does laptop will be stolen one day and someone else has access to his certificate I simply change the 01 to a 02 and create a new user certificate. This is nothing more than the above mentioned "manual management" of certificate revocation lists. If the firewall is hit during IKE phase 1 with the leaked certificate entries CN=John.Doe.01 & it will not find a matching user id and aborts the IKE phase. Check L2TP User so that the account will be allowed to open an L2TP tunnel after its IPsec negotiation is over. Finally we provide the user password that has to be entered on the client afterwards and save the the account. Jump to Objects > Users > Local Groups and add John Doe to the group ACME Teleworkers.

STEP 11: Client certificate

A user who wants to identify himself to the firewall needs a certificate that matches an entry in the firewall. For this the firewall will compare any subject field of the certificate (CN,email,OU,...) with the user data in its database and will grant access if it finds an entry that has the same field contents. So we have to create a certificate that fits to our new user John Doe. Again I use openssl to generate a private key file john.doe.key for the user.

linux > openssl genrsa -des3 -out john.doe.key 4096
Generating RSA private key, 4096 bit long modulus
e is 65537 (0x10001)
Enter pass phrase for john.doe.key:
Verifying - Enter pass phrase for john.doe.key:


Insert any password you like. It is only needed during the creation process of the certificate and needs not to be hand out to the user. The next step is clear. To obtain a certificate from a private key we need a signing request john.doe.csr. Nothing easier than this.

linux > openssl req -new -key john.doe.key -out john.doe.csr -subj "/CN=John.Doe.01/"
Enter pass phrase for john.doe.key:


Hopefully you did not throw away your root certificate. We need it at this point to sign the user certificate. In contrast to the root certificate I only create a user certitificate with a validity of one year. This will help to keep the maintenance process easy if users leave ACME Inc. and the firewall administrator is not informed of this.

linux > openssl x509 -req -days 365 -in john.doe.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out john.doe.crt
Signature ok
Getting CA Private Key
Enter pass phrase for ca.key:


Now we have a client certificate john.doe.crt but sadly the Windows client cannot handle this type of file correctly. What we need is a PKCS12 certificate file. This file is a password encrypted bundle of an unencrypted private key and the matching certificate. Or to say it the other way round we take the john.doe.key file and merge it with the john.doe.crt file into a new john.doe.p12 file that is secured by a new password.

linux > openssl pkcs12 -inkey john.doe.key -in john.doe.crt -export -out john.doe.p12
Enter pass phrase for john.doe.key:
Enter Export Password:
Verifying - Enter Export Password:


I guess you can see clearly that we have to hand out the PKCS12 password to the user afterwards and not the key password. This is because the key will be decrypted before storing it in the PKCS12 file.

STEP 12: Import certificates to user laptop

A user that wants to logon to the ACME intranet needs two certificates. His user certificate (stored in john.doe.p12) and the root certificate root.crt. It is not possible to hand out the user certificate alone because we need a so called valid certificate chain. This means a program that wants to check the validity of a certificate will go back the signing chain until it finds a trusted root certificate. In our case we have only one dependency and therefore we need two certificates. Start the Microsoft management console (mmc.exe) on your windows client and open the dialogue Add/Remove Snap-in. Choose Add and select Certificates in the following window to insert the certificate snap-in to the management console.
You now have the choice which certificates you want to manage. Here you must select the computer account. If your company has high security standards this should concern you. The Windows VPN client will always search for the certificates in the user independend computer account. This means that anyone who has access to the machine can at least finish the IKE handshake (phase 1 and 2) if he knows the IP or DNS name of the Netscreen outside interface. To build a connection via a L2TP tunnel to your network he needs a username and a password of course. This should always be remembered when implementing a VPN with the standard Windows client. If you want to proceed click on Next. Mark the radiobutton This Snap-in manages the local computer and return to the previous dialogue by selecting Finish. You don't need any other snap-ins so click on Close and shut the last open window with Ok. 

You should now see the certificate tree in the console. Pay attention that the root node in the left pane reads Certificates (Local Computer). Everything else will be the wrong place for your certificates. To import the root certifcate root.crt go down the tree Trusted Root Certification Authorities > Certificates. There you will see a lot of root certificates your machine already trusts. Right click on the leaf Certificates in the left pane and select All Tasks > Import ... A wizard will open and asks for the location of the certificate you want to import. Provide the full path to the file root.crt. Click on Next and select the radio button Import certificates into the following store. Don't change anything else. Just select Next and afterwards Finish. The root certificate of ACME Inc. is now in place and should show up in the right pane.
We have to repeat the same steps for the user certificate. Therefore right click on the leaf Personal > Certificates and select All tasks > Import ... again. Do everything what you made before for the root certificate but this time select the file john.doe.p12. As the PKCS12 certificate is password protected the wizard will ask for that. Simply enter the password you chose during the creation process. After finishing the wizard your user certificate will be on the laptop too.
This guide is slowly coming to its end and I must admit that I'm still working hard on the NAT-T miracle. I don't like the idea to be defeated by this lousy oakley.dll doing its FQDN handshake all the time.

STEP 13: Create the VPN connection

Now that the laptop is prepared we can setup the new VPN connection. Open the network configuration wizard and choose Connect to the network at my workplace as the network type. On the next page we will select VPN connection (what else) and continue to name the connection ACME VPN (or whatever you like). As we will manage the standard internet connection (dialup or via router) ourselves we do not want to dial an initial connection automatically. In the next input field we will enter the address of the Netscreen outside interface. In this case it is On the last page we say that we don't need a smardcard and finish the wizard. As the connection is configured we have to adapt the VPN type. Right click on ACME VPN, choose Properties and switch to the Network tab. It is strange that the default Automatic VPN type cannot build a L2TP over IPsec connection. Maybe Microsoft knows... So we select L2TP over IPsec manually and save the changes.

STEP 14: Connect ot the VPN

The time has come to build the initial connection to the ACME VPN network. If you are directly connected to the internet and the laptop has an official IP address this will be possible by now. Otherwise you have to wait for my upcoming posts that will cover two unsolved issues. Open the newly created connection and provide the username and the password. Both of them must match the user settings in the Netscreen. Hit Connect and the connection should be established.

ISSUE 1: Split Tunneling

Already mentioned above all client traffic will be routed by default into the VPN connection. This is a security aspect that one cannot neglect. When you are connected to the network of ACME Inc. the infrastructure of the company should provide you all the neccesary services you need for working. So if you want to surf the internet when being connected by VPN the requests should go the way laptop > VPN > company proxy > internet. Additionaly one can argue that a normal PC in the ACME building underlies the same restrictions. It is not allowed to participate in the intranet on the one hand and have an second network card to be connected to the an insecure network of the neighbour company. So by default the configuration will be absolutely correct for the person with security in mind. 
Let's assume you want to allow the roadwarriors to acces your intranet and use their current network connections at the same time. This could be helpful if the companies internet connection has only a slow uplink and a lot of teleworkers do not only work on the inside resources but also surf the internet through the VPN. In this case the uplink bandwidth represents a bottleneck that can cause severe slowdown.
The magic word here is split tunneling. It describes the ability of the client to route all traffic to the company network via VPN and all other traffic to the normal network connection. Basically it is nothing more than enhancing the default routing table with a new entry that routes all traffic to the ACME intranet ( through the VPN. This can be done manually by the user every time he connects - a solution no one really wants - or automatically. Sadly the automatism is not implemented in the windows client by default. Therefore we need CMAK.
CMAK or known as Connection Management Administration Kit is a sofware component on the Windows 2003 Server CD. It provides a wizard to create a self installing binary that an user can execute without any further input on his laptop. It will create the VPN connection and all its settings so that the user may directly launch the VPN afterwards. No other efforts required. A detailed description of this tool will go far beyond the scope of this article. Therefore I just reference to the best documentation I found throughout the net in my opinion. If you are interested take some time and read more at

The essential point of the configuration process is the step Routing Table Update. There you can provide a file with a new routing table. Everytime a client will connect to the CMAK created VPN connection it will automatically update its routing table corresponding to the contents of this file. The exact syntax of this file is more or less an abbreviation of the windows route command. In our case the file wll contain only one line:

ADD MASK default METRIC default IF default

It tells the client that only the traffic to the ACME intranet will go through the VPN. I guess you cannot make it easier than this. Nevertheless it took me some time to find this secret solution any normal VPN client is capable of by default.


Starting this topic I had never thought that it would be so challenging. Now I know why it has been covered so rarely in the past. The less work you want to have on the client side, the harder you have to fight on all the other fronts. But why should I use a memory wasting VPN client bloated with features I do not need. Of course all programs have their existence authorization but the longer I work in the IT the more I'm attracted to clean and easy tools on my workstation. Different strokes for different folks.
After all I managed to fix the problem that kept me busy the last days. While writing these lines I feel a great satisfaction. Man has won over machine. But what it is all about you may ask. So we will go into detail for the last time.
In phase 2 of the IKE handshake the client and the server of a VPN connection have to decide how they will identify themselfes. Possible values are email address, IP address or FQDN, ... The Netscreen accepts an IP address and this is no problem if you use the Netscreen client. The Windows native client provides an IP address as long as it has an official IP address and NAT-T is not required. If the client works behind a router this behaviour changes dramatically. All of a sudden it will simply send an FQDN and the Netscreen will block any further packet. I asked why so often during my analysis but it led me to no conclusion. All the normal tricks faded away as soon as I tried them out. If you look at the interesting part in the IKE debug trace it reads like this:

12:31:22 : IKE Recv*: [HASH] [SA] [NONCE] [ID] [ID] [NAT_OA]
12:31:22 : IKE extract payload (352):
12:31:22 : IKE QM in state OAK_QM_SA_ACCEPT.
12:31:22 : IKE ERROR: Cannot handle this id type, 2!


So what to do if everything fails? We have to fix the client! Call it what you want for me it is nothing more than a program enhancement to fit my needs. To show you what I mean I will give you an extract of the oakley.dll:

push eax
mov [ebp+var_10C], 100h
call _GetComputerFQDN
add esp, 0Ch
cmp eax, edi
jz short loc_756CD5F7
push eax
push offset aGetcomputerfqd ; "GetComputerFQDN failed %d"
push edi
push edi
push edi
push edi
push 80000000h
call _IKEDbgMsg
add esp, 1Ch
jmp loc_756CD55B

If you are a little bit into programming you can see what happens here. The library tries to determine the FQDN of the client. If this succeeds (return code in register EAX=0) it will continue in the default way otherwise it will write a message to the log. The normal behaviour will be that the GetComputerFQDN finishes without errors and the client can use the FQDN in IKE phase 2. And exactly at this point we will place a small modification (nothing more than a single bit). We will convert the conditional JZ jump into a JNZ. This means the client thinks it cannot determine the FQDN. Luckily the routine will not abort but will send its IP address to the firewall instead. I cried out loud when I saw the first NAT-T connect to my rebellious Netscreen. To make the solution easy for the non assembler oriented reader I will include the neccessary conversion you have to apply to the oakley.dll. But remember: YOU WILL DO THIS ON YOUR OWN RISK:

system32\oakley.dll (Windows XP 32 Bit): replace 3BC7741C5068 by 3BC7751C5068
system32\oakley.dll (Windows XP 64 Bit): replace 85C0743389 by 85C0753389

Pay attention that the byte sequences only occur once in the oakley.dll. If you want to patch the file you will have to take care of the safety copies Windows stores in system32\dllcache and/or ServicePackFiles. Just search all occurrences of the library and fix them in the right order. Reboot your machine and check that the changes in system32\oakley.dll have been made permanent. Finally you have a fully working VPN client for your Netscreen.

Hopefully you enjoyed reading this article as much as it was fun for me writing it.


Related Posts Plugin for WordPress, Blogger...