White paper summary :: Deploying 10 Gigabit Ethernet on VMware vSphere 4.0 with Cisco Nexus 1000V and VMware vNetwork Standard and Distributed Switches

http://www.vmware.com/files/pdf/techpaper/WP-VMW-Nexus1000v-vSphere4-Deployment.pdf


My notes on the white paper above: using it for review and jotting down things I find important.
Please read the white paper for concepts I may gloss over since it’s a review for myself.
The reason I am going over this is even though vsphere5.x Is out, the concepts are probably relevent.




Design guidance for implementing 10 Gigabit Ethernet networking with VMware vSphere 4.0 (including VMware ESXi 4.0 and ESX 4.0 and associated updates) in a Cisco network environment.


Design Goals
1. Availability: The design should be capable of recovery from any single points of failure in the network outside the VMware ESX or ESXi server. Traffic should continue to flow if a single access or distribution switch, cable, or network interface fails.
2. Isolation: Each traffic type should be logically isolated from every other traffic type.
3. Performance: The design should provide the capability to impose limits on some traffic types to reduce the effects on other traffic types.


VMware ESX and ESXi Network Adapter Configurations
4. the most common configurations
5. Two 10 Gigabit Ethernet interfaces (converged network adapter [CNA], network interfacecard [NIC], or LAN on motherboard [LOM]).
6. Two 10 Gigabit Ethernet interfaces (CNA or NIC) plus two Gigabit Ethernet LOM ports (used for management)
7. the most common design scenario, all traffic is converged to two 10 Gigabit Ethernet interfaces.




Traffic Types in a VMware vSphere 4.0
8. Management: very low network utilization, but it should always be available and isolated from other traffic types through a management VLAN
9. Vmotion :: separate VLAN specific to VMware Vmotion :: a single VMotion migration can use up to approximately 2.6 Gbps bandwidth, with 2 running at the same time
10. Fault-tolerant logging:  latency less than 1ms, separate vlan
11. (iSCSI) :: two iSCSI vmkernel ports canbe bonded to allow iSCSI traffic over both physical network interfaces:: typically iSCSI-specific VLAN , although targets may reside on another VLAN accessiblethrough a Layer 3 gateway.
12. NFS: typically NFS-specific VLAN , although targets may reside on another VLAN accessible through a Layer 3 gateway.
13. Virtual Machines:distributed over more than one VLAN and be subject to different policies defined in port profiles and distributed virtual port groups.


Cisco Nexus 1000V 10 Gigabit Ethernet Network Design
Network architects can use two different approaches for incorporating the Cisco Nexus 1000V into the data center network environment: virtual PortChannel (vPC) and MAC pinning
Both design approaches provide protection against single-link and physical-switch failures,
they differ in theway that the virtual and physical switches are coupled and the way that the VMware ESX or ESXi server traffic is distributed over the 10 Gigabit Ethernet links.
vPC recommended when vPC or clustered physical switches are available at thephysical access layer. MAC pinning should be chosen when these options are not available.


VPC
allows the aggregation of two or more physical server ports to connect to a pair of Cisco Nexus 5000 or 7000 switches to make the connection look like one logical upstream switch.
provides better bandwidthutilization and redundancy
10 Gigabit Ethernet uplinks from the Cisco Nexus 1000V are aggregated in asingle logical link (PortChannel) to the two adjacent physical switches.
The adjacent physical switches require vPC  theyappear as a single logical switch distributed over two physical chassis.


MAC Pinning
uplinks from the Cisco Nexus 1000V are treated as stand-alone links.
each 10 Gigabit Ethernetinterface is connected to a separate physical switch with Layer 2 continuity on all IEEE 802.1Qtrunked VLANs between the two switches.
Virtual Ethernet ports supporting virtual machines, andvmkernel ports are allocated in a round-robin fashion over the available 10 Gigabit Ethernet uplinks.
Each MAC address is pinned to one of the uplinks until a failover event occurs


Traffic Isolation and Prioritization
1000V can provide consistent traffic isolation for the various VMware traffic types using port profiles.
Port profiles map to distributed virtual port groups (Dvport)
Within port profiles, parameters can be set that apply to a specific traffic type such as management, IP storage, VMware VMotion, or virtual machine traffic.
parameters cover suchdetails as port security, VLAN, and ACLs.
Policy maps for QoS treatment can be set on a per-portprofile basis to enable policing and prioritization


Limit traffic
critical that any one type of traffic does notoverconsume the bandwidth.
limit the ingressor egress bandwidth down to the virtual Ethernet port level
can be applied as part of a port profile for a particular type of traffic (ie: Vmotion )
can also be applied on a per-virtual Ethernet interfac




PortChannel Technology
A MEC capability such as vPC, VSS, or VBS is required on the adjacent physical switches to enable the PortChannel to span both physical switches and still maintain availability
When PortChannels are spread across more than one physical switch, the switches are deemed to be clustered.
clustering is transparent to the Cisco Nexus 1000V Switch
When the upstream switches are clustered, the Cisco Nexus 1000V Series Switch should be configured to use an LACP PortChannel with the two 10 Gigabit Ethernet uplinks defined by one port profile.
Traffic is distributed over the available links (two 10 Gigabit Ethernet links in this case) according to the load-balancing algorithm configured at each end of the PortChannel.




VMware vSS and vDS Configuration
 configuration for 10 GigabitEthernet with both VMware vSS and vDS is similar.


Teaming Policy Options
14. Originating virtual port ID ::Uplinks in same Layer 2 domain on all trunked VLANs   best practice recommendation
15. IP hash :: Static IEEE 802.3ad PortChannel required on uplinks (no LACP) :: Traffic distributed according to SRC-IP or DST-IP hash ::
requires the uplinks to be aggregated into a staticPortChannel.
16. Source MAC hash :: Uplinks in same Layer 2 domain on all trunked VLANs :: should be used only if you have multiple MAC addresses assigned to a vnic and you require additional load distribution over the available uplinks.
17. Explicit failover order :: Uplinks in same Layer 2 domain on all trunked VLANs :: highest-order uplink from the list of active adapters that pass failover detection. If one link fails, the next link from the list of standby adapters is activated.




Teaming Policy for Two 10 Gigabit Ethernet Interfaces
a deterministic way of directing traffic on a per-portgroup or per-distributed-virtual-port-group basis to a particular 10 Gigabit Ethernet uplink.
Virtual switch trunking (VST) mode: Trunk the required VLANs into the VMware ESX or ESXi hosts over both 10 Gigabit Ethernet interfaces and make sure that there is Layer 2 continuity between thetwo switches on each of those VLANs.
Virtual machine port groups or distributed virtual port groups: Make these active on one vmnic and standby on the other
vmkernel port groups or distributed virtual port groups: Make these active on one vmnic and standby on the other in reverse to that for the virtual machines
With both NICs active, all virtual machine traffic will use vmnic1, and all the vmkernel ports will use vmnic0. If a switch, link, or NIC failure occurs affecting one uplink, then all traffic will converge to the remaining vmnic
Another variation spreading the virtual machine traffic over both uplinks through the originating virtual port ID policy with both 10 Gigabit Ethernet uplinks active in that port group or
distributed virtual port group


Using Traffic Shaping to Control and LimitTraffic
If you have concerns about one traffic type dominating through oversubscription
traffic shaper controls and limits traffic on a virtual port.
VMware VMotion, management traffic, and fault-tolerant logging are effectively capped,
this process really concerns only iSCSI and NFS
The traffic shaper is configured on the port group (or distributed virtual port group).
On vSS, the shaper applies only to ingress traffic
vDS supports bidirectional traffic shaping.
do not specify a value greater than 4 Gbps :: reason :: 4gb Mod on value: ie: 5gb =1Gb value