Exemple: Link Aggregations and VLAN Configurations with Oracle VM for Sparc

31.07.2013

I already wrote an similar article to Solaris 11 and Zones (link). Today I will describe how to configure several Ldoms Guest an emphasis on network configuration (several vlan).

In this example, there are 3 Ldoms running on dedicated systems that are exposed to the external networks.

The Ldom control runs in 4 vlan (front, admin, backup, interconnect) - OS Solaris 11.1
Ldom Guest 1 runs in 4 vlan (front, admin, backup, interconnect) - OS Solaris 10u11
Ldom Guest 2 runs in 3 vlan (front, admin, backup) - OS Solaris 10u10

Vlans informations:

Vlan id 1 : address 192.168.1.0/24 - front
Vlan id 2 : address 192.168.2.0/24 - admin
Vlan id 3 : address 192.168.3.0/24 - backup
Vlan id 4 : address 192.168.4.0/24 - interconnect

Address for Ldom control

Vlan id 1 : 192.168.1.10 - defaultrouter 192.168.1.1
Vlan id 2 : 192.168.2.10
Vlan id 3 : 192.168.3.10
Vlan id 4 : 192.168.4.10

Let's go... Just wait... The network configuration of switch must be already configured (please contact network team !?)

Step 1: Create link aggregation and vlan configuration on Ldom control

My system (Sparc T4-2) includes 2 NICs (10G). There is no network configuration yet (I connect on ILOM).

# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
[...]
net8 Ethernet unknown 0 unknown ixgbe1
net9 Ethernet unknown 0 unknown ixgbe0
[...]

I create a basic link aggregation (I use LACP) with 2 NICs.

# dladm create-aggr -P L2,L3 -L active -l net8 -l net9 aggr0

I check quicly the status of the aggregation.

# dladm show-link
LINK CLASS MTU STATE OVER
[...]
net8 phys 1500 up --
net9 phys 1500 up --
[...]
aggr0 aggr 1500 up net8 net9

# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr0 -- 10000Mb full up 90:xx:xx:xx:xx:x8 --
net8 10000Mb full up 90:xx:xx:xx:xx:x8 attached
net9 10000Mb full up 90:xx:xx:xx:xx:x9 attached

Yet, I create 1 virtual card for each vlan id.

# dladm create-vlan -l aggr0 -v 1 front0
# dladm create-vlan -l aggr0 -v 2 admin0
# dladm create-vlan -l aggr0 -v 3 backup0
# dladm create-vlan -l aggr0 -v 4 interco0

# dladm show-vlan
LINK VID OVER FLAGS
front0 1 aggr0 -----
admin0 2 aggr0 -----
backup0 3 aggr0 -----
interco0 4 aggr0 -----

# ipadm create-ip front0
# ipadm create-addr -T static -a local=192.168.1.10/24 front0/v4
# ipadm create-ip admin0
# ipadm create-addr -T static -a local=192.168.2.10/24 admin0/v4
# ipadm create-ip backup0
# ipadm create-addr -T static -a local=192.168.3.10/24 backup0/v4
# ipadm create-ip interco0
# ipadm create-addr -T static -a local=192.168.4.10/24 interco0/v4

# ipadm

NAME CLASS/TYPE STATE UNDER ADDR

admin0 ip ok -- --

admin0/v4 static ok -- 192.168.2.10/24

backup0 ip ok -- --

backup0/v4 static ok -- 192.168.3.10/24

front0 ip ok -- --

front0/v4 static ok -- 192.168.1.10/24

inter0 ip ok -- --

inter0/v4 static ok -- 192.168.4.10/24

lo0 loopback ok -- --

lo0/v4 static ok -- 127.0.0.1/8

lo0/v6 static ok -- ::1/128
[...]

Don't forget, the configuration of router.

# route add -p default 192.168.1.1 -ifp front0

Step 2: Create link virtual switch and configuration vnic for each Ldoms Guest

I create one virtual switch for all vlan

# ldm add-vswitch net-dev=aggr0 vid=1,2,3 primary-vsw0 primary

For Ldom Guest 1, I create 4 vnic (see definition)

# ldm add-vnet pvid=1 id=0 vnet0 primary-vsw ldom1
# ldm add-vnet pvid=2 id=0 vnet1 primary-vsw ldom1
# ldm add-vnet pvid=3 id=0 vnet2 primary-vsw ldom1
# ldm add-vnet pvid=4 id=0 vnet3 primary-vsw ldom1

For Ldom Guest 2, I create 3 vnic (see definition)

# ldm add-vnet pvid=1 id=0 vnet0 primary-vsw ldom2
# ldm add-vnet pvid=2 id=0 vnet1 primary-vsw ldom2
# ldm add-vnet pvid=3 id=0 vnet2 primary-vsw ldom2

Conclusion: We hope this step-by-step guide will give you some ideas for future consolidation with Oracle VM for Sparc. With Oracle Solaris 11 capabilities (aka Crossbow), you can easily set up fairly complex environments (simply network configuration).