snippets101

Showing posts with label ace. Show all posts

Wednesday, August 13, 2008

Cisco ACE and Private VLANs in Switch Mode
This one comes from the esigence to create a housing zone with a shared load balancer. The idea is to keep things as simple as possible providing isolation and all advanced load balancing feature. So there will be only one vlan for one-armed servers and isolation will be provided by Private VLANs. However, being the ACE shared across the pvlans I had to add access lists to control traffic from one pvlan through another's vip. Another element of simplicity in this design is the ACE module in switch-mode, this way providing a unique default gateway for all the servers (loadbalanced and not). The network diagram :

The ACE configuration :

switch-mode rserver host SERVER-10.1 ip address 10.0.10.1 inservice rserver host SERVER-10.2 ip address 10.0.10.2 inservice serverfarm host LAB-A_20.1:80 rserver SERVER-10.1 80

inservice rserver SERVER-10.2 80 inservice serverfarm host LAB-B_21.1:80 rserver SERVER-10.3 80 inservice rserver SERVER-10.4 80 inservice

class-map match-any L4-SNAT 2 match source-address 1.0.10.0 255.255.255.0 class-map match-all L4-LAB-A_20.1:80 2 match virtual-address 10.0.20.1 tcp eq www class-map match-all L4-LAB-B_21.1:80 2 match virtual-address 10.0.21.1 tcp eq www

policy-map type loadbalance first-match L7-LAB-A_20.1:80 class class-default serverfarm LAB-A_20.1:80

policy-map type loadbalance first-match L7-LAB-B_21.1:80 class class-default serverfarm LAB-B_21.1:80

policy-map multi-match L4-POLICYMAPMULTI class L4-SNAT nat dynamic 10 vlan 100 class L4-LAB-A_20.1:80 loadbalance vip inservice loadbalance policy L7-LAB-A_20.1:80 loadbalance vip icmp-reply active class L4-LAB-B_21.1:80 loadbalance vip inservice loadbalance policy L7-LAB-B_21.1:80 loadbalance vip icmp-reply active service-policy input L4-POLICYMAPMULTI

interface vlan 100 description SERVERSIDE ip address 1.1.1.200 255.255.255.0 no normalization no icmp-guard nat-pool 10 10.0.100.1 10.0.100.15 netmask 255.255.255.224 pat no shutdown interface vlan 150

description FIREWALLSIDE ip address 1.0.0.2 255.255.255.0 no normalization no icmp-guard no shutdown ip route 0.0.0.0 0.0.0.0 1.0.0.1

The router's configuration :

svclc module 11 vlan-group 100 svclc vlan-group 100 10,20,100,150 vlan private-vlan vlan private-vlan vlan name SERVERSIDE private-vlan private-vlan vlan name P2P-C65-ACE interface ip association 10 community 20 community 100 primary 10,20 150

vrf private-vlan

forwarding mapping

Vlan100 LAB 10,20 Vlan150 LAB

interface ip vrf ip address 10.0.0.1 255.255.255.240

forwarding

As usual, on the ACE there's a nat pool to permit server to server load balancing, further information here and here. "Horizontal" connections across the VLAN's broadcast are prevented by the private vlans' mechanism. However, a server belonging to the PVLAN A can connect to a VIP belonging (logically, as VIPs on the ACE are not PVLAN aware) to PVLAN B. Being the ACE the default gateway, default behavior must be permit ip any any. Using accesslists and object-groups I created a matrix of negations between real servers on each pvlan versus all the VIPs of the other PVLANs :
object-group network GROUPA host 10.0.10.1 host 10.0.10.2 object-group network GROUPB host 10.0.10.3 host 10.0.10.4

object-group network VIPSGROUPA host 10.0.20.1 object-group network VIPSGROUPB host 10.0.20.2 access-list ACL01 line 7 extended deny ip object-group GROUPA object-group VIPSGROUPB access-list ACL01 line 11 extended deny ip object-group GROUPA object-group VIPSGROUPB access-list ACL01 line 15 extended permit ip any any access-group input ACL01

Again, many thanks to Daniele and Marco for support on the lab. Posted by Alberto at 4:44 AM No comments: Labels: ace, cisco, configuration, private vlan, pvlan, switch mode

Tuesday, August 12, 2008

Asymmetric Server Normalization on Cisco ACE
Summertime, I've some time to spend in the lab to try some new scenarios, optimization or try something read on the release notes. This scenario is mostly a proof of concept as in our DataCenter many applications use server to server load balancing and it requires a too messy configuration to make it work with ASN. From the Cisco doc : Asymmetric Server Normalization (ASN) allows the ACE to load balance an initial request from the client to a real server; however, the server directly responds to the client bypassing the ACE. This behavior allows the acceleration of server to client communications and is transparent to the client. When the ACE operates in ASN, it does not perform any network translation when receiving packets destined to the VIP address. Traffic from the client hits the VIP address and the ACE uses the address as the destination address but rewrites the destination MAC address to the address of the real server. So the match is speed vs. capabilities. Since the ACE do not control the entire flow (it sees only the packets vs. the VIP) most of the advanced load balancing features aren't usable. However, in some cases, where speed is the most important thing and the application flow is quite simple it could be a good choice. Here's my implementation :

On the ACE :
access-list ACL01 line 8 extended permit ip any any ! The probe should check for the health status of the loopback interface ! on the real server. probe icmp ICMP ip address 10.0.20.1 interval 5 rserver host SERVER-10.1 ip address 10.0.10.1 inservice rserver host SERVER-10.2 ip address 10.0.10.2 inservice ! The transparent command prevents the ACE to make a destination NAT, ! sending the packet as is to the real server. serverfarm host LAB_20.1:80 transparent probe ICMP rserver SERVER-10.1 80 inservice rserver SERVER-10.2 80 inservice class-map match-all L4-LAB_20.1:80 2 match virtual-address 10.0.20.1 tcp eq www policy-map type loadbalance first-match L7-LAB_20.1:80 class class-default serverfarm LAB_20.1:80 policy-map multi-match L4-POLICYMAPMULTI class L4-LAB_20.1:80 loadbalance vip inservice loadbalance policy L7-LAB_20.1:80 loadbalance vip icmp-reply active

service-policy input L4-POLICYMAPMULTI access-group input ACL01 !The ACE is used as a bare load balancer, diasbling normalization (to make ASN work), ! and icmp-guard. interface vlan 100 description SERVERSIDE ip address 1.1.1.199 255.255.255.0 no normalization no icmp-guard no shutdown interface vlan 150 description FIREWALLSIDE ip address 1.0.0.2 255.255.255.0 no normalization no icmp-guard no shutdown ip route 0.0.0.0 0.0.0.0 1.0.0.1

On the router :
ip route vrf LAB 10.0.20.0 255.255.255.0 10.0.0.2

This way everything works fine. Now it's time to make server to server load balancing work in this scenario. What I wanted to work is a load balanced server that calls for its VIP being load balanced on itself or on another real server in the same area. Note that this scenario is different since here real server's are one-armed. The choice to have a point-to-point vlan between the router and the ACE comes with the esigence to control traffic for some feature I needed described further. For simple ASN scenarios the ACE could easily have only one arm on the real servers' vlan (100).

Here's some observations I made : 1) Without any further configuration, server to server loadbalancing doesn't work as there's a loopback interface on the server with the VIP configured on it, you'll never get out of it. 2) You will need both source NAT and destination NAT to make it work. 3) You will have to make the ACE see all of the flow to make the NAT work. 4) You can't use ASN for server to server load balancing I will not enumerate all of the tries I made to make it work (from which come the observations), here's my solution : double NAT (source and destination) on the router (6513 in this case) so that the ACE treats the connection as a normal external client request.

Here's the router's configuration :

interface Vlan100 ip vrf forwarding LAB ip address 10.0.10.200 255.255.255.0 ip nat outside ! interface Vlan150 ip vrf forwarding LAB ip address 10.0.0.1 255.255.255.0 ip nat inside ip nat inside source static 10.0.20.1 4.4.4.1 ip nat outside source static 10.0.10.1 6.6.6.1 ip route vrf LAB 10.0.20.0 255.255.255.0 10.0.0.2 ip route vrf LAB 4.4.4.0 255.255.255.0 10.0.0.2 ip route vrf LAB 6.6.6.1 255.255.255.255 10.0.10.199

The last route is a little tricky : as returning packets come from an outside zone to an inside one, packets need to be first routed and the natted. So you'll have to put a "fake" route versus an inside endpoint to make the packet then match the nat rule. On the real servers you have a route to make all NATted request return by the ACE and not by the default gateway (router). These requests are easily identified by the source address (6.6.6.1). Note that ASN and server to server load balancing with double NAT (source and destination) can be implemented both on the same box. Thanks to Daniele and Marco for helping me in the lab. Posted by Alberto at 3:42 AM 9 comments: Labels: ace, asn, asymmetric, cisco, configuration, normalization, server

Wednesday, December 12, 2007

Source NAT on Cisco ACE

I've already described the scenario for this issue here.

My first approach on Source NAT for this scenario was a little too CSS-minded, so wasting vip addresses and not considering some new features on the ACE. For each serverfarm which had to be reached from the same subnet I created two vips, one for everybody, one for SNATted connections from the same subnet. Now I changed approach, SNATting everything coming from the INTERNAL subnet, no matter which vip is looking for. This way I have not to use different vips depending on source address.

class-map match-all L4-MAP-SNAT-INTERNAL 2 match source address 10.0.11.0 255.255.255.0 class-map match-all L4-MAP-YELLOW-SERVERFARM_20.2:80 2 match virtual-address 10.0.20.2 tcp eq www policy-map type loadbalance first-match L7-FARM_20.2:80 class class-default serverfarm YELLOW-SERVERFARM_20.2:80
policy-map multi-match L4-POLICYMAPMULTI-LOADBALANCE class L4-MAP-SNAT-INTERNAL

nat dynamic 100 vlan 101 class L4-MAP-FARM_20.1:21 loadbalance vip inservice loadbalance policy L7-FARM_20.2:80 loadbalance vip icmp-reply active service-policy input L4-POLICYMAPMULTI-LOADBALANCE interface vlan 101 description SERVERSIDE ip address 10.0.10.199 255.255.255.0

nat-pool 100 10.0.21.100 10.0.21.115 netmask 255.255.255.0 pat no shutdown interface vlan 151 description FIREWALLSIDE ip address 10.0.0.2 255.255.255.240 no shutdown In this scenario each connection coming from any server on the 10.0.11.x subnet is subnetted. Then, going down on the multimatch policy, it reaches the vip class which loadbalance on the real servers. Note that the L4-MAP-SNAT-INTERNAL must be the first class defined in the policy for have things work well.