VNC over STUNNEL with a Linux server and Windows 2000 client HOWTO

19 February 2001

ver 1.0

by Craig Furter and Arno van der Walt

We assume that you have already downloaded VNCServer and VNCViewer.

First of all there is a step by step HOWTO and then we'll look at the theory behind all this.

Download and install OpenSSL, SSLeay, and Stunnel on the Linux/Unix box. Download the modules.

a) [root@anthrax$]gunzip openssl-x.xx.tar.gz (repeat for all 3 the modules)

b) [root@anthrax$]tar – xvf openssl-x.xx.tar (repeat for all 3 the modules)

Copy the following to Notepad and save the file as VNCRegEdit.REG on the Windows 2000 box

--cut here and copy to VNCRegEdit.REG then double click the file to import--
REGEDIT4

[HKEY_LOCAL_MACHINE\Software\ORL\WinVNC3]
AllowLoopback=dword:00000001

[HKEY_LOCAL_MACHINE\Software\ORL\WinVNC3\Default]
AllowLoopback=dword:00000001
--stop here--

Install Stunnel on the Windows 2000 machine by copying the following files to your \WINNT\SYSTEM32\ directory

a)libeay32.dll

b)libssl.dll

c)stunnel.pem

On the Linux box execute the following command as root and let it run in its own terminal.

./stunnel -d 5900 -r 5901

Execute vncserver (it should run as display:1 when you execute the ps aux |grep vnc command)

Now on the Windows 2000 machine execute the following command and let it run in its own terminal.

stunnel -d 5900 -r unix.ip.address:5900 -c

And on the Windows 2000 machine open VNCviewer and connect to localhost specifying no display

ie. 10.10.1.53 in the window

For each additional display repeat steps 4 – 6 and increment the specified ports with 2 ie. The Linux command will look as follows:

./stunnel -d 5902 -r 5903

and the Windows 2000 command as follows:

stunnel -d 5902 -r unix.ip.address:5902

and remember to start another vncserver on the Linux box for each VNC display

The display number on the vncviewer must also be incremented with two ie:

10.10.1.53:2 etc.

The THEORY

Tunneling:

What this means is that software (daemon) runs on the client and server machine. In this case, the Windows 2000 machine is the client and the server is the *NIX machine. Stunnel will then run as client on Windows 2000 and server mode on the UNIX box.

eg:
Windows:
stunnel -d 5900 -r unix.ip.address:5900 -c

UNIX
stunnel -d 5900 -r 5901

This means that connecting to VNC display 0 in the localhost will transfer all the calls to the *NIX machine on display 1. So the VNC server on the *NIX machine must be running on display 1. Not display 0. If you run stunnel before VNC, VNC will automatically move to display 1 noticing that port 5900 ("display" 0) is already in use).

What happens now is that when you connect to port 5900 on the Windows machine via an "unsecured" connection, a secure "tunnel" is opened from Windows 2000 to the *NIX machine on port 5900. The *NIX machine then opens a "unsecured" connection to itself on port 5901. We now have a secure tunnel available.

A bit about VNC and displays

The -d is the listening IPaddress:port and the -r is the remote IPaddress:port. VNC uses port 5900 for display 0. That means that display 1 will be 5901. If you want VNC server to listen for a connection on port 80 then the display number will be 80 - 5900 = -5820. If you want VNC server to
listen on port 14000 then the display number is 14000 - 5900 = 8100.

So all you have to do is run stunnel on the UNIX machine and VNC on the desired "display" number.

VNC on the Windows 2000 machine

To connect from the client machine you need to enter the client machine's IP address and the "display" (from the port conversion). But VNC will think that you are trying to connect to the local machine and does not allow this. To override this add the following to your registry.

--cut here and copy to anything.reg. then double click the file to import--
REGEDIT4

[HKEY_LOCAL_MACHINE\Software\ORL\WinVNC3]
AllowLoopback=dword:00000001

[HKEY_LOCAL_MACHINE\Software\ORL\WinVNC3\Default]
AllowLoopback=dword:00000001
--stop here--

Now VNC will not complain. So you need to always run stunnel in client mode on the Windows machine and then connect with VNCViewer to the localhost on the correct "display". By the way, *NIX doesn't complain about this. There is no setting needed if *NIX to *NIX.

VNC's Java client

Unfortunately this will not work well with the built-in web version. If you did not known about it, try http'ing into a machine running VNC server on it, to port 58XX (where XX is the display number), and the Java client will be loaded.