Вы находитесь на странице: 1из 51

#CLUS

Troubleshooting ACI
Configuration Settings to Objects

Mioljub Jovanovic
Technical Leader, Customer Delivery
BRKACI-2643

#CLUS
Agenda
• Overview
• Classes in the real World
• Isn’t it easier with CLI?
• Configuring the APIC
• From APIC to the Switches
• Common Config Issues and Examples
• Summary

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Cisco Webex Teams
Questions?
Use Cisco Webex Teams to chat
with the speaker after the session

How
1 Find this session in the Cisco Live Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKACI-2643


by the speaker until June 16, 2019.

#CLUS © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Everything is an object!
apic1(config)# tenant mio10ant
apic1(config-tenant)# bridge-domain exampleBD-CLI
apic1(config-tenant-bd)# end

Regarless if you’re simply logging in from UI or performing configuration


Using CLI, we’re essentially leveraging objects.

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
APIC Management Information Model Reference

From the WebUI

direct URL
https://apic/doc/html/
#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Distributed Management Information Tree (dMIT)
• Objects are structured in a tree-based hierarchy
• Everything is an object topRoot

• Objects referred to as “managed objects” (MO)


• Every object has a parent, with exception of Root (top polUni /api/node/mo/uni.json?query-target=self

dn: uni
of tree, class: topRoot)
• Objects can be linked through relationships ctrlInst fvTenant fabricInst
dn: uni/controller dn:uni/tn-mgmt dn: uni/fabric
Ex: fvRsBD links EPG (fvAEPg) to desired BD (fvBD)
• Distributed: Across all Fabric Node devices fvAp fvBD
dn: uni/tn-mgmt/BD-

Ex: class: fabricNode


dn: uni/tn-mgmt/ap-mgmt-app inb

fvAEPg fvRsBD
dn: uni/tn-mgmt/ap-mgmt-app/epg-mgmt- dn:
epg tDn:uni/tn-mgmt/BD-inb
name: EPG1
pcTag: 16386
modTs: 2017-06-22T08:52:35.502+00:00

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Object Naming
• Objects have a Relative Name (RN) and Distinguished Name (DN)
• Similar to file system structure
• RN = name of object; unique within the context of parent object
• DN = used a globally unique ID for an object

• DN formed by appending RN to parent RN until root of tree is reached


• dn = {rn}/ {rn}/ {rn}/ {rn} … fvAp fvAEPg

polUni fvTenant vzFilter vzEntry

Example: vzBrCP vzSubj


uni/tn-tenant/ap-app1/epg-epg1
fabricPathE
topRoot fabricPathEp
pCont
fabricTopology fabricPod
fabricNode

vmmProvP vmmDomP vmmCtrlrP

* credit: Burns & Pita #CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Managed Object (MO) in ACI
• Everything in ACI is represented by a Managed Object (MO)
• Managed object is just an instance of some Class of objects
• MOs are organized in a Managed Information Tree (MIT)
• You can query or view the MIT in many different ways:
• Visore : https://apicIP/visore.html
• Browsing MIT in shell : cd /mit/… or cd /aci
• moquery : cli query utility to the DB
• REST : postman, curl GET and POST
• icurl (local REST client on apic/leaf)
• Python SDK (ACI Toolkit, Cobra etc)

Understanding APIC MIT, Managed Objects is highly


recommended to improve interactions between APIC
component and improve troubleshooting efficiency.
#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Classes in the real
world
... Does it seem too abstract and difficult

… How do we map classes/objects to the real world?


Object Class Car is representing “data model” / template of a Car
Class Car with all properties we need to create computer model of a car

{ Enlisted properties are just


selected based on our choice
property => value
and desired set of information
dn => distinguished name – exact location of we wanted to know about the
the car object in our pool of cars cars, for the purpose of this
presentation.
make => describing the car manufacturer Obviously, if we wanted to
model => specific model represent detailed object
model of real car we would
colour => car colour have added many more
properties such as tires,
coolness => Subjective grade of the actual object
engine etc.
modTs => Date … modification TimeStamp Properties in ACI Classes are
obviously predefined as part
} of ACI Policy Model.

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Example object instance of a class Car
{
dn: “bru-airport/expo-bmw-1”
make: “BMW”,
model: “550i”,
colour: “gold”,
coolness: “fancy”,
price: 50000,
modTs: “Jan/09/2016”,
imgUrl: https://...
}

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Another object instance of a Class Car
{
dn: “carHistory/yugo55-1”
make: “Yugo”,
model: “55”,
colour: “red”,
coolness: “NA”,
price: 3990,
modTs: “01/01/1985”,
imgUrl: “https://...”
}
* photo source: Alden Jewell

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
Array of objects
[
{ id: 1, make: “BMW”, model: “550i”, colour: “gold”, coolness:
“high”,price: “50000”, modTs: “01/01/2016”},
{ id: 2, make: “Yugo”, model: “55”, colour: “red”, coolness: “NA”,
price: “3990”, modTs: “01/01/1985” }

]
Single object instance is contained within curly braces: { property: value }
Array of objects is contained within square braces, delimited by comma:
[ {object 1}, {object 2}, {object 3} … ]

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
Great … but how is
this related to ACI
object model?
moquery -c fabricNode -f 'fabric.Node.id=="1"' -o json
{

Class fabricNode
"imdata": [
{
"fabricNode": {
"attributes": {
"dn": "topology/pod-1/node-1",
"uid": "0",
"fabricSt": "active",
"lastStateModTs": "2019-05-01T07:10:31.649+00:00",
"serial": "FCH2019SANCL",
"id": ”1",
"version": ”A0",
"role": ”controller",
"modTs": "2019-03-28T13:04:57.155+00:00",
"rn": "node-1",
"vendor": "Cisco Systems, Inc",
"address": "10.0.0.1",
"adSt": "on", model: APIC-SERVER-M2
"name": "bdsol-aci20-apic1",
"apicType": "apic",
"model": ”APIC-SERVER-M2"
} Class fabricNode represents some of the
}
} device properties / attributes in the real world.
], In the left pane we’re observing an object
"totalCount": "1" instance of the fabricNode class,
}
#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
moquery -c fabricNode -f 'fabric.Node.id=="101"' -o json
{

Class fabricNode
"imdata": [
{
"fabricNode": {
"attributes": {
"dn": "topology/pod-1/node-101",
"uid": "0",
"fabricSt": "active",
"lastStateModTs": "2019-05-01T07:10:31.649+00:00",
"serial": "FDO2019CLUS",
"id": "101",
"version": "",
"role": "leaf",
"modTs": "2019-05-01T07:10:38.366+00:00",
"rn": "node-101",
"vendor": "Cisco Systems, Inc",
"address": "10.0.120.69",
"adSt": "on", model: N9K-C93180YC-EX
"name": "bdsol-aci20-leaf1",
"apicType": "apic",
"model": "N9K-C93180YC-EX"
} Class fabricNode represents some of the
}
} device properties / attributes in the real world.
], In the left pane we’re observing an object
"totalCount": "1" instance of the fabricNode class,
}
#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Topology is just graphical
representation of objects

admin@apic1:~> moquery -c fabricLink



# fabric.Link
n1 : 203
s1 : 1
p1 : 1
n2 : 101

• APIC WebUI and API inspector s2


p2
:
:
1
51
• Identify which objects are used dn
lcOwn
:
:
topology/pod-1/lnkcnt-101/lnk-203-1-1-to-101-1-51
local
to plot topology linkState
modTs
:
:
ok
2015-03-13T14:26:39.526+01:00 Class fabricLink represents links between
• Re-using fabricLink objects to monPolDn : uni/fabric/monfab-default
devices. Along with fabricNode can be
rn : lnk-203-1-1-to-101-1-51
identify the links status :

• We could create our own tool wiringIssues : used to plot the topology diagram
for topology, monitoring or admin@bdsol-aci2-apic1:~> moquery -c fabricLink | egrep -e ^dn | head -5
dn : topology/pod-1/lnkcnt-1/lnk-102-1-2-to-1-2-2
troubleshooting dn
dn
: topology/pod-1/lnkcnt-2/lnk-102-1-4-to-2-2-2
: topology/pod-1/lnkcnt-3/lnk-102-1-6-to-3-2-2
dn : topology/pod-1/lnkcnt-201/lnk-102-1-49-to-201-1-34
dn : topology/pod-1/lnkcnt-202/lnk-102-1-50-to-202-1-34

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Isn’t it easier with
CLI?
• What is easier for human
Is it easier to use
TCL/Expect to tell computer
what you want or to use an
Object?
SDN vs FDN
Dilemma For computers it’s easier to use
Objects and for Humans it’s
easier CLI
Finger defined Networking 

BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Wait a minute …

Automation
CLI Parsing
if CLI is easier …

why do we care REST API REST API


about objects in
the first place?

BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Everything that is displayed is an object of some class

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
API Inspector is good to see UI requests

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
APIC
Configuration using WebUI

Creating Objects using WebUI
PC REST API APIC Server
Browser
Request Method

TCP:443

Policy manager
IFM Intra Fabric
TCP:7777
TCP:443

Messaging
WebUI Debugging:
1. API Inspector Status Code
2. Developer Tools in
the Browser Logs on APIC (tail –f, tail -1000):
1. egrep exampleBD /var/log/dme/log/access.log
2. zegrep exampleBD /var/log/dme/log/nginx.bin.log*
Network debugging: 3. zegrep exampleBD /var/log/dme/log/svc_ifc_policymgr.bin.log.*
tcpdump, sniffer etc (policymgr logs may be on different APIC in the cluster)
#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
Configuration using WebUI

Example API Inspector
Creating Objects using WebUI
PC REST API APIC
Browser
Request Method

TCP:443

Policy manager
tcpdump

IFM Intra Fabric


TCP:7777
span

Messaging
or similar
Example Chrome
Developer Tools (F12) Status Code

# egrep exampleBD /var/log/dme/log/access.log


::ffff:10.48.16.90 - - [1/Jun/2019:09:34:20 +0000]
"POST /api/node/mo/uni/tn-mio10ant/BD-exampleBD.json HTTP/1.1" 200 30 …

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Configuration using WebUI

Creating Objects using CLI
apic1(config)# tenant mio10ant
apic1(config-tenant)# bridge-domain exampleBD-CLI
apic1(config-tenant-bd)# end
REST API APIC Server
Request Method

TCP:443

Policy manager
IFM Intra Fabric
TCP:7777
TCP:443

Messaging
CLI Debugging:
1. /var/log/dme/log/decoy.log
2. /var/log/dme/log/decoy_server.log
3. /var/log/dme/log/zsh-term.log Status Code

Logs on APIC (tail –f, tail -1000):


1. egrep exampleBD-CLI /var/log/dme/log/access.log
2. zegrep exampleBD-CLI /var/log/dme/log/nginx.bin.log*
3. zegrep exampleBD-CLI /var/log/dme/log/svc_ifc_policymgr.bin.log.*
(policymgr logs may be on any APIC in the cluster)
#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
How to get object DN from GUI

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
GUI download XML do not show all you need ?
Download Xml from that page
We only see dhcp info

Nothing about cert ???

<dhcpClient childAction="" clientEvent="assigned"


decomissioned="no" dn="client-[SAL1811NN5S]" fabricId="1"
fwVer="" hwAddr="00:00:00:00:00:00" id="SAL1811NN5S«
ip="10.0.168.92/32" lcOwn="local" modTs="2015-10-
08T11:12:27.175+02:00" model="N9K-C9336PQ" name="pod2-
spine1" nodeId="201" nodeRole="spine" podId="1" spineLevel="1"
status="" supported="yes"/>

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
Example:
How does Leaf get
config from the
controller?

What to check?
Logical, Resolved and Concrete Object models
•ACI objects ACI objects used for ACI objects upon
Logical model

Resolved model

Concrete model
configured by User passing information which which Switch
in APIC using and configuration operating systems
WebUI, CLI or REST from APIC to the acts. Concrete
based API. switches. These objects also user
objects are user visible but not
visible but not configurable.
configurable.
Resolved objects
are implicitly
configured, derived
from Logical model
by APIC.

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
From Logical Objects to Hardware - Workflow
2. 3. 4. Triggers sent
1.Configured
Logical
Converted to NXOS for
Resolved
Converted
by User by APIC by APIC/PE Concrete hardware
Hardware
Model Model Model programming

Some models are directly converted from logical to concrete by agent


running on the laves & spines.

1. Logical Model: Configured by User.

2. Logical Model is converted to Resolved Model by APIC. This resolved model is then
sent to the leaves & spines.
Resolved Model exists only for sending tenant configuration such as Epg, BD, Ctx.
Other configurations like Selectors are sent as is to the fabric nodes (Represented
by red line).

3. Resolved Model is converted by Policy Element (APIC agent running on


leaves/spines) to Concrete Model.

4. Creation of Concrete Model sends notifications to the iNXOS for programming


hardware.
#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
Logical Model Configured By User
EPg Containment Hierarchy

fvTenant
tn-coke

ap-default fvAp fvBD fvCtx

epg-App fvAEPg fvAEPg fvRsCtx


Objects can also be
just definition of the
relationship between
two other objects.
fvRsBd fvRsBd
Example of
relationship which is
child of EPG and used
to point to relevant BD.

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
Logical Model Converted to Resolved Model
Endpoint Profile (EpP)
fvEpPCont epp
• Internal deployment profile for Epg
• Deployed to switch / policyelement

fv-[uni/tn-coke/ap-default/epg-App] fvEpP fvEpP

fv-[uni/tn-coke/ap-default/epg-App]/node-102 fvLocale fvLocale

fv-[uni/tn-coke/ap-default/epg-App]/node-102/stpathatt-[eth1/10] fvStPathAtt fvDyPathAtt

fv-[uni/tn-coke/ap-default/epg-App]/node-102/stpathatt-
[eth1/10]/conndef/conn-[vlan-19]-[0.0.0.0] fvIfConn fvIfConn

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Logical Model Converted to Resolved Model
Endpoint Profile (fvBDDef & fvCtxDef)

uni

fvBDDef fvCtxDef

fvBDDef • Summarized internal representation of fvBD. fvCtxDef • Summarized internal representation of fvCtx.
• Naming property is the DN of the • Naming property is the DN of the
corresponding fvBD. corresponding fvCtx. Ctx identifiers like
• BD identifiers: Segment ID, PcTag (policy Segment ID and PcTag.
control tag used for Contracts. • Summarized internal representations of other
• Summarized internal representations of other Ctx policies like End Point Retention Policy
BD policies like IGMP, DHCP, End Point etc.
Retention Policy etc.

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Resolved Model Converted to Concrete Model
sys topSystem, it’s the root of concrete model

Private network (aka VRF).


l3Ctx l3Ctx sys/ctx-[vxlan-2555904]

Bridge Domain
l2BD l2BD l2BD sys/ctx-[vxlan-2555904]/bd-[vxlan-15826914]

Encap i.e. VLAN OR VXLAN


vlanCktEp vlanCktEp
sys/ctx-[vxlan-2555904]/bd-[vxlan-15826914]/vlan-[vlan-17]

Association from Encap to Port/Port-Channel


l2RsPathDomAtt sys/ctx-[vxlan-2555904]/bd-[vxlan-15826914]/vlan-[vlan-
17]/rspathDomAtt-[sys/conng/path-[eth1/16]]

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Great, so we now
know how it’s
supposed to work …
have a great day 
Config doesn’t exist on the switches
or is not programmed as anticipated
1. Very important: Always check for faults in the system. If anything fails
deployment then faults are raised. For instance, if EPg deployment fails for
any reason then faults will be raised on the EPg.

2. Verify concrete objects.

3. Verify switch using the iBash shell commands.


(more details in other sessions)

Previously explained ideal


scenario lacked some
conventional realism … 

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
How to check for configuration related faults

Select
desired
fault for
more
details

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
Example Fault F0467 details
Cause: configuration-failed

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Physical Interface Configuration Workflow
Global
Policy Interface
VMM (AAEP) Policies
Domain vSwitch Policies
Policies (settings)

Pools Interface
VLAN / Policies
VXLAN / Policy
Multicast Group Interface
Global
Policies
Physical Policy
Profiles
and (AAEP)
Port
External Phys, L2,
Blocks
Domains L3
(physical
ports)

If you miss some steps when


preparing interfaces to be assigned Profiles
to EPG … Switch Switch
Selectors Policies
(physical Profiles
Config fault such as F0467 will give switches)
you a hint!

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Using CLI / moquery to check/sort active faults
(class faultInst) quickly sorts all active faults

admin@apic1:~> moquery -c faultInst | egrep -e "^descr" | sort | uniq –c | sort -n

1 descr : Power supply shutdown. (serial number DCB1936Y3V7)


2 descr : Address configuration failure
2 descr : Configuration is invalid due to VlanInstP … Allocation mode should be dynamic.
2 descr : Configuration is invalid due to internal error occured …
2 descr : Failed to form relation to MO uni/phys-TO_N3K of class physDomP
2 descr : Service graph for tenant FG-Test could not be instantiated. …
4 descr : Deployment of EPG failed on Controller: …
4 descr : power supply missing

Now we could query all faults details by criteria – such as fault description
fault.Inst.descr
moquery -c faultInst -f 'fault.Inst.descr=="Address configuration failure"'

show faults ? show commands also available as more user friendly than moquery

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Using CLI / moquery to check/sort active faults
(class faultInfo) quickly sorts all active faults
moquery -c faultInfo -f 'fault.Info.type=="config"' | egrep "^cause" | sort | uniq -c
485 cause : configuration-failed
4 cause : fsm-failed
1 cause : product-not-registered
142 cause : resolution-failed
2 cause : wiring-check-failed
moquery -c faultInfo -f 'fault.Info.type=="config"' | egrep "^code" | sort | uniq -c
8 code : F0053
2 code : F0454
6 code : F0467
129 code : F0956
1 code : F0977
467 code : F1298
2 code
Now we could query: all
F1299
faults details by criteria – such as fault description
4 code : F609528

fault.Inst.descr
Class faultInfo includes both classes faultInst and faultDelegate

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Class Property Property
Name name value

dn
cause
what changed

description
direct URL
https://apic1.cisco.com/visore.html
rule which triggered the fault

type
#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
Significant improvements in Visore UI in 4.1 and newer releases

Class Name Direct link to Class documentation

direct URL
https://apic1.cisco.com/visore.html

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Summary
Summary
• When troubleshooting configuration issues always check for config
faults first
• Familiarity with ACI Classes greatly helps in troubleshooting
• Objects and Classes are facilitating automation and scaling
• Understanding of the configuration steps and dependencies to
accomplish desired outcome

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Complete your
online session • Please complete your session survey
evaluation after each session. Your feedback
is very important.
• Complete a minimum of 4 session
surveys and the Overall Conference
survey (starting on Thursday) to
receive your Cisco Live water bottle.
• All surveys can be taken in the Cisco Live
Mobile App or by logging in to the Session
Catalog on ciscolive.cisco.com/us.
Cisco Live sessions will be available for viewing
on demand after the event at ciscolive.cisco.com.

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
Continue your education

Demos in the
Walk-in labs
Cisco campus

Meet the engineer


Related sessions
1:1 meetings

#CLUS BRKACI-2643 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
Thank you

#CLUS
#CLUS

Вам также может понравиться