Carlos' Corner » networking

Simulating a more interesting environment with Vyatta and VMware ESXi – pt 2

cars — Thu, 18 Feb 2010 22:37:37 +0000

In an earlier post I went through the process of coming up with a solution to be able to test an Exchange 2003 to Exchange 2010 migration using VMs. In order to simulate a multi-site AD environment I wanted to use Vyatta based routers to create my network infrastructure. In this post I’ll actually walk through the process of setting up the ESX and theinternal router. In the next post(s) I’ll go into configuring the DMZ and internet routers and firewalls. As a reminder the environment I want to set up will look like this…

In order to actually be able to implement this I first had to go configure the appropriate networking configuration on each of the ESXi hosts.

First I needed to create a virtual switch utilizing the NIC attached to the crossover cable. This is done by going to the “Configuration” tab for the ESX host within the vSphere Client.

Clicking the “Add Networking” option will walk one through the wizard to configure the new switch. I started by choosing “Virtual Machine” on the Connection Type Screen.

On the next page I choose to create a new virtual switch and pick the appropriate physical NIC that will be used to communicate with the other host. (If I had the capacity to put all the VMs one one host I could create the vSwitch without having to specify a network adapter)

Then I created an initial Port Group and specified a VLAN ID for it. In this case for the Remote Site (192.168.3.X/24) I’m specifying VLAN ID 23.

Once completed the new virtual switch should look similar to the one shown below.

Now that the vSwitch has been created, I can add port groups for the other networks: DMZ (192.168.254.X), Internet (10.0.0.X) and HQ(192.168.2.X). Each one of these should have a unique VLAN ID associated with it which is also used when these port groups get created on the second host.

In addition I need to add port groups for the network between the DMZ and Internet router (192.168.5.X), as well as the one between the internal router and the DMZ (192.168.4.X). Because I’ll put all three routers on the same host, these last two port groups need only exist on that one host.

When all is said and done there should be 6 port groups defined on the vSwitch on the host where I’ll put the routers and 4 port groups on the other host. Notice that the VLAN IDs for each network match up across the two switches.

I initially tried this setup using an alpha version 6 of Vyatta but the routers felt a little slow to me (though I have no empirical evidence to back that up) so I went back to using version 5 LiveCDs for the routers as discussed in the earlier entry.

Building the internal router

I started by creating a custom VM for the internal or home router with 4 network adapters configured. I used the custom configuration because this VM won’t actually have a virtual hard disk. The four network adapters will be attached to my home LAN (lostroncos_01), the HQ Site Network, the Remote Site Network and the intranet to DMZ network. (Note: This post has a more detailed description of the process I used to create Vyatta routers. )

After powering on the router and opening the VM console I like to enable the NIC on my home network and ssh so I can do the rest of the configuration via an ssh client such as Putty. Once the VM has initially been powered on, I can go back and look at the settings and determine the MAC address of the NIC attached to my home network (lostroncos_01) is 00:0c:29:76:e2:07. Upon logging into the VM console I can go into configure mode by entering the command “configure” and then get a list of the interfaces the router knows about by entering “show interfaces”.

The one I’m interested in initially configuring is eth0. I can tell because the MAC matches the one I took note of earlier. To configure it to use 192.168.1.254 as it’s address and enable ssh I can do the following:

Let’s look at these commands in a little more detail.

The set interfaces command can take a wide variety of arguments. Here I’m specifying that I want to work on an ethernet interface. Other interface types include adsl, bonding, bridge, loopback, multilink, openvpn, serial, tunnel and wirelessmodem.

Even after specifying the type as ethernet and the particular interface (eth0) I can still use one of several subcommands/options.

Because I’m using VMs and a fairly generic setup I really only need to worry about the address option. So I specify the address and netmask notation by specifying the number of bits to use for the netmask (24).

The next command “set service ssh” simply enables the SSH server.

Now I’ve configured the interface and enabled SSH, but the settings haven’t been made active. That’s what the “commit” command does.

Executing the “show interfaces” command again I can see that the IP(v4) address is set for eth0 but not any of the other interfaces. Now I can use my ssh client to connect to the VM to finish the configuration. (One can also use the VMware console to perform all of the configuration steps, I just prefer an external SSH client).

After making sure the MAC addresses and networks match up the way I think they should, I can set the address for each of the other interfaces and commit the changes. Then I can exit the configuration mode by typing “exit” and save the configuration to the virtual floppy using the “init-floppy” command. This then ensures that my configuration will survive reboots of the router.

One of the other things than can be helpful to do is to use the “description” option with set interfacse to provide a little more information when logged into the router.

After doing this for each interface and then committing the changes I can see the descriptions when doing show interfaces from either the configure mode

Or from the actual router shell

Since I happen to already have VMs on both the HQ and Remote networks I can verify basic connectivity from the router by pinging those VMs as well as my PC.

The next trick is to validate that I can ping the HQ and Remote subnets from my physical PC. Since both of those subnets are directly connected to the router I don’t need to add any static routes on the router. However I do need to add a route on my PC to reach those subnets. Otherwise it’ll try to use the default gateway out to my ISP (as shown below) which doesn’t know anything about this lab environment.

To reach the 192.168.[2,3,4] subnets I want to use the interface on rtr-home that’s attached to my home network (192.168.1.254). So using the “route” command from a command prompt on my Windows7 machine I can add the route to 192.168.4.X.

The route command in this case takes a few arguments…

#1 add tells route I’m adding a new one

#2 Next I specify the destination, and since ultimately I’m trying to get to a subnet we’ll specify 192.168.4.0.

#3 & #4 MASK says the next argument is the netmask relative to the destination

#5 is the gateway for this destination, in this case the interface on my virtual router that’s attached to the home network.

Once that’s done I can then try to ping & tracert the 192.168.4.2 interface on the router again.

Executing the “route print” command from the Command Prompt I can see the entry for the 192.168.4.X network as I would expect.

Now that this particular routing entry appears to be working properly I can re-run the route command with the -p option so that it’s persistent across reboots of my home PC.

Next I need to add routes for the 192.168.2.X and 192.168.3.X networks to the PC as well.

At this point, I can try to do a traceroute from my PC to the VM on the HQ Site network that I was able to ping earlier from the router.

Because I’m occasionally paranoid, the next step for me was to connect to the console of the VMs on the HQ (shown below) and Remote (not shown) subnets and verify that I can ping machines on my home LAN across the internal router, rtr-home.

At this point my internal Vyatta router is routing traffic between subnets it is directly connected to (Home LAN, the HQ Site and Remote Site). Once the DMZ and Internet routers are set up some changes will need to be made on the internal router to get traffic to those non-connected subnets. I’ll go through that process in a subsequent post.

-crt

Simulating a more interesting environment with Vyatta and VMware ESXi

cars — Thu, 18 Feb 2010 07:05:08 +0000

[Part 2 of this series which involves the actual configuration of the Vyatta routers is now up here -crt]

At work we’ve recently made the decision to migrate to Exchange 2010 from Exchange 2003. While we do have an environment that we can use for some testing of the migration it doesn’t mimic our production environment closely enough for me to be comfortable using it as the sole test area. Given the changes in how Exchange 2010 (E2KX) works vs 2003 I wanted to be able to simulate multiple (2) Active Directory sites (i.e. subnets), a DMZ, and the “Internet” including some really simple firewalls.

I wanted to use virtual machines to go through this exercise so that I could take snapshots and repeat the various steps and/or variations of them if necessary. In order to do this I utilized the Vyatta Community Edition based routers to help create my virtual “enterprise” environment. I’ve talked about Vyatta before in this article. In this post I’ll talk a little about the process I went through to get to my final configuration (shown below). In subsequent articles I’ll go through the actual router and VMware configuration process.

The final environment as laid out on two interconnected servers

My lab environment at home consists of two Dell PowerEdge servers (one a PE2850, the other a 2950 each with 8Gigs of RAM). Both servers are running ESXi 4.0. Since the 2850 can’t run 64 bit VMs I was going to install the Exchange 2003 servers and Windows 2003 DCs on it. Then I’d install VMs running Server 2008R2 on the 2950 with Exchange 2010. Both servers are connected to my home network and since I was going to be using both I wanted to have some way for VMs on each host to be able to communicate with others without necessarily having all the traffic come across my home network. Since both Dells have multiple NICs I connected them with a crossover cable ending up with something like this:

Physical view of the network

In initially penciling out a plan for what I wanted to do I had nine VMs scattered across four subnets.

Isolated environment with multiple gateways per subnet

Considering my limited resources and my need to keep some other unrelated VMs up and running while I’m testing, I trimmed this down to 7 by combining the Domain Controllers and Exchange 2003 servers together in the HQ and Remote subnets.

Consolidating functions to reduce # of VMs

In further looking at this from a networking perspective, I was hit with the realization the initial configuration with two routers attached to the HQ and DMZ subnets would require me to manage routing on each individual VM in each of those subnets as well as on each of the routers. As an example one can look at the Exchange 2010 server in the HQ site/subnet.

In this particular case the Exchange server would need to be able to route some traffic to the DMZ via 192.168.2.253 and other traffic to the “Remote” site via 192.168.2.254. (If the server was going to communicate directly with machines in our fake “Internet” I’d have to add yet another routing entry.) I can of course configure a default gateway when configuring the NIC, but still have to manually add a route for the other gateway. This process then has to be repeated on each machine. It then gets more complicated if I want to be able to use Terminal Services (RDP) to connect to the VMs rather than using the VM remote console because I now have to figure out how to connect the virtual routers to my home network and potentially add yet another routing entry.

I decided I’d rather have a single gateway on each subnet (so I only had to specify a default gateway on each VM) and then rely on the routers to do all the routing. I considered a couple of different ways to do this. One option was do something “meshy” where the router for each subnet was connected to common shared subnet.

This would have had the desired effect a single gateway for each subnet regardless of where traffic was going, but would have required 5 virtual routers which seemed a little excessive. Going in the other direction, another option was to have a single router connected to everything.

Single-router solution

I ended up with a solution somewhere between the “mesh” and the single mongo router. It employs three routers. I decided on this partly because I wanted to keep things relatively simple especially since I was going to be enabling the firewall functionality between the DMZ and Internet (and wanting to limit the damage I could do to myself when working late at night).

Again, when all was said and done the environment I ended up with looks like the one below. In the next couple of entries I’ll go through the actual process of building the networking side of this.

Physical/Logical view