What is a VPN?

I've had two interesting VPN-related experiences in the past couple months.

The first was at work — we needed to connect network printers to the private network of an external system we use. This resulted in a moderately difficult networking project, involving connecting an AWS Transit Gateway to the private network, and then connecting VPNs to it one by one.

The second was in my personal life. I'm a huge baseball fan, and as baseball season rolled around, it became clear the MLB and surrounding companies (looking at you Bally Sports) don't care at all about creating new baseball fans, and they wanted to make it really hard to watch baseball. There is no real streaming service that you can watch it on, except AT&T TV, which is very expensive and involves long contracts. It's basically CoIP, cable over IP.

There's also MLB.tv, but the kicker here is that they block regional games. So for myself, living in Atlanta and being a Braves fan, cannot watch the Braves on MLB.tv. Pretty much the opposite user experience from what you'd want.

So how does a VPN come in? Well, I want to watch Braves games in Atlanta. I want to hide my IP address.

And a VPN does that, I can buy MLB TV, use a VPN to change my location to somewhere that's not Atlanta, and watch Braves games. The issue is that we want to watch the games on our Roku TVs, and Roku does not have a VPN client app.

So I had to buy a router running dd-wrt, setup a second WiFi network in my home, connect the router to a VPN provider, and connect all the TV's to that network.

That worked, and I drank beer and laughed at my minor victory over the knuckleheads at Bally Sports.

So, what is a VPN?

VPN's can provide access to private networks, like in my work example. They can hide your location, like in my baseball use case. It's not a private network, but it simulates one with encryption. A key aspect is that VPN traffic flows over a public network, usually the internet.

The tunnel is built by encrypting packets or frames and then encapsulating them in regular IP traffic between the 2 hosts or networks.


Encapsulation is a fundamental idea in networking. In fact, the network stack itself uses encapsulation, even without considering a more complicated encapsulation use case like a VPN.

Here is a typical TCP/IP scenario.

Packets grow in size as we go down the stack. The closer you are to physical things like electrical signals on a copper wire or light in a fiber optic cable, the bigger the unit of data actually is, because it's getting encapsulated at each layer.

VPN's use encapsulation in a way that messes with the clean layering we see above. We take an IP packet, encrypt it, and send it over UDP/IP—with a new IP header. Something like this.

(A, B, and C are used for encryption and authentication. Sparing you some details here. For UDP / ESP relationship see here.)

Now let's talk about how a VPN does this, with IPSec.

What is IPSec?

IPSec is a set of protocols that allow us to create VPNs. It's not the only way to create a VPN, but it is the most common. It's an example of a tunneling protocol, which involves repackaging data — where the cleanliness of only a lower level packet encapsulating a higher-level packet is shattered.

IPSec itself involves a bunch of protocols

  • Authentication Headers (AH)
  • Encapsulating Security Payloads (ESP)
  • Internet Security Association and Key Management Protocol (ISAKMP), of which the most common in Internet Key Exchange (IKE)


Internet Key Exchange, the heart of IPSec, is a protocol defining what happens when we initiate a VPN connection. This is what actually sets up the "secure tunnel".

They negotiate a number of parameters (called Security Associations), things such as

  • How should we authenticate each other? (pre-shared key, RSA, etc.)
  • How should we encrypt data?
  • Should we be in Tunnel mode or Transport mode? (more on this in a bit).

For example, we could have a VPN that uses a pre-shared key for authentication, ESP for encapsulation, and tunnel mode (among many other less important agreed upon security associations). This would be a fairly common setup.

The result of the 2 phases of IKE is a shared session secret, which both sides of the VPN can use to symmetrically encrypt data. Not unlike TLS.

Tunnel Mode vs. Transport Mode

One interesting thing to note is that VPNs can operate in one of two modes — tunnel or transport. In short, tunnel mode encrypts the original IP header (which contains the source IP address!), and transport mode does not.

Tunnel mode is definitely the most common, and is what we mean when we say "site-to-site VPN" or "encrypted tunnel". In my baseball example, having an encrypted IP header was a critical requirement, because I don't want the MLB to know I'm in Atlanta.

It might be fine to use transport mode for a remote access VPN, like when a remote employee wants to access a company's private network.

Downsides of IPSec

IPSec definitely has some problems.

  1. It has to run in the kernel! It breaks the OS Ring Architecture by operating in ring 0. This makes it quite the security risk.

    One video I watched claimed that if there's malware on a device on an IPSec home network that it could infect devices on the corporate network the home network is connected to via IPSec. I would like to look more into this, but that seems pretty bad if that's truly the case.
  2. There's a lot of configuration.


Interestingly, there is another way to configure a VPN — over SSL/TLS. This can have significant advantages, including

  • Operates in user-space, far more secure
  • Built on top of OpenSSL, which is probably the most battle-tested security library in existence
  • It's more portable to different OS's, due to the above reasons

OpenVPN is the most prevalent example of an SSL VPN. It really is quite similar to IPSec — we use Public Key Cryptography to establish a symmetric key that we can then encrypt data with.

It's also what I used in my baseball example. 😈

Note: You Don't Want To Do TCP over TCP

VPNs normally run over UDP or TCP — UDP (or TCP) can encapsulate the ESP packets, and then IP encapsulates the UDP (or TCP) packets. However, if you're system uses TCP, and you have that connected to a VPN, you want to make sure the VPN doesn't use TCP.

The retransmission algorithm for TCP is not made to occur at two separate layers of a network stack — they end up conflicting and causing general congestion known as TCP meltdown. See here for more details.

Let me know if I got anything wrong or if you have any comments!