LCSinActionWorkshop-Operate5 CrashDumpAnalysis V2

Microsoft Dynamics AX
Lifecycle Services: Operate Phase: Crash Dump Analysis

Last Updated: August 2015
AX 2012 R3 / Version 2.1.0
Yuthana Arngmaneekul
Introduction ...............................................................................................................................................................................................3
Demo of Crash Dump Analysis.............................................................................................................................................................3
Possibilities to create Memory Dumps from crashing processes ................................................................................... 15
What is a crash and why do I need to collect a Memory Dump? ....................................................................................... 15
Did the AOS really crash? ............................................................................................................................................................... 16
Unexpected termination ............................................................................................................................................................. 16
Forced termination....................................................................................................................................................................... 16
Controlled shutdown ................................................................................................................................................................... 17
What do I need to create a Crash Dump? ................................................................................................................................. 18
Debug Diagnostic Tool v1.2 (DebugDiag)............................................................................................................................. 18
Windows Error Reporting (WER) .............................................................................................................................................. 18
ADPlus.............................................................................................................................................................................................. 19
Crash or High Memory Usage?..................................................................................................................................................... 19
So your AOS crashed, is hanging, or you just want to see what it's doing ................................................................. 20
Disclaimer ................................................................................................................................................................................................ 21
Microsoft Dynamics AX 2012 R3 | Lifecycle Services: Operate: Crash Dump Analysis | Last Updated: August 2015
Version
2.0
2.1
Description
Initial version (Yuthana Arngmaneekul)
Additional information and updated document with August update (Yuthana)
Date
July 2015
August 2015
[https://technet.microsoft.com/en-us/library/dn268616.aspx]
Introduction
Demo of Crash Dump Analysis
Operate Phase Crash Dump Analysis

Starting with the March release we've introduced a new feature in LCS for Crash and Hang analysis. Currently
the feature is in the Beta phase, so it will need to be activated from the "Preview feature management" menu "Public preview features" - "Crash and Hang Analysis".
After you activate the feature, it will appear as a tab in the System diagnostic tool.
To use this tool, collect a process dump file (.DMP), ZIP it and upload it onto the server using the steps below.
It will take between 30 minutes and two hours for the file to be processed on our servers. The result is an
HTML report that contains details about the crash such as crash signature, AX kernel information, list of
known issues, list of active sessions, X++ and native call stacks for all threads.
The search for known issues is done based on the names of the functions from the faulting call stack. For
instance, if the call stack of the crashing thread contains the method 'Ax32Serv!SqlConnectBase::removeStmt',
the report will look for hot-fixes that patch this method, such as KB 3017292, and provide a link to the Issue
Search tool on LCS.
Create Crash rule for all instances of AX32.exe
Create job, and run

static void Kaboom(Args _args)
{
DLL kDll = new DLL("kernel32");
int hSrc, hDest;
int memSrc, memDest;
DLLFunction CopyMemory = new DLLFunction(kDll,
"RtlMoveMemory");
//Alloc 12345 bytes for Source and Destination
hSrc = WinAPI::globalAlloc(0, 12345);
hDest = WinAPI::globalAlloc(0, 12345);
memSrc = WinAPI::globalLock(hSrc);
memDest = WinAPI::globalLock(hDest);
CopyMemory.arg(ExtTypes::DWord, ExtTypes::DWord,
ExtTypes::DWord);
//Copy 23456 from Source to Destination
CopyMemory.call(memDest, memSrc, 23456);
}
From the tool's main page, press Add to create a new

job
Provide a name and an optional description for the
10
Press Add files
analysis job
Crash dump analysis can analyse both ax32 and ax32serv
Select .ZIP file containing one or more .DMP files and

press Upload
Press Analyze dumps
11
Confirm start of analysis
Wait until processing completes. This can take

between half and hour and two hours depending on
the complexity of the dump files
12
After the job is marked as Completed, an HTML

report is available. If you can't see it, refresh the web
page
13
In this report you can see information about:
AX kernel version
Relevant hot-fixes
Faulting call stack
X++ and native call stacks for all threads
14
Possibilities to create Memory Dumps from crashing processes
EMEADAXSupport
Sometimes it happens that during regular operation the Dynamics AX Application Object Server (AOS) is
suffering from an abnormal program termination, which is better know as a crash. Users connected with the
Dynamics AX Client to the affected AOS Instance are noticing such an issue very soon as they get
disconnected from the AOS.
In such a situation maybe trying out a more recent Dynamics AX Kernel Hotfix may help. However if
you decide to open a Support Request to get help identifying what concretely causes the AOS to crash, you
will get asked to create a Memory Dump of the crash, and if you have a Memory Dump ready by that this will
speed up the process a bit.
What is a crash and why do I need to collect a Memory Dump?
At a very top level a process is crashing when it is performing an illegal operation that is not captured /
handled by the process itself. This unhandled exception forces the Operating System to terminated the process.
Examples for an illegal operation are for example a division by zero or accessing an invalid memory location.
When you are creating a Memory Dump from a crashing process (Crash Dump) at a high level the dump is a
snapshot from the process memory and register context (together with some more information) at the time of
the crash. So the dump shows for example the very last operation being executed that caused the process to
crash.
15
By doing a post mortem analysis of the memory dump it is (usually) possible to identify what the AOS was
doing (e. g. was X++ code executed) when the crash happened, so the memory dump is the key to solve a
crash situation!
Did the AOS really crash?
Before I go into detail about how to create a Memory Dump I would like to show you however some
possibilities to confirm you are definitely running into a crash situation and here the Windows Event Log of
the AOS is the key.
Unexpected termination
The most common event you see in a crash situation is the Error event 1000 from Application Error in the
Application Event Log. This is definitely a confirmation that an AOS Instance was terminating unexpectedly.
Event Type: Error
Event Source: Application Error
Event ID: 1000
Description:
Faulting application Ax32serv.exe, version 4.0.2163.0, time stamp 0x48b574e5, faulting module Ax32serv.exe,
version 4.0.2163.0, time stamp 0x48b574e5, exception code 0xc0000005, fault offset 0x001630bf, process id
0x1a90, application start time 0x01c944075355b99b.
Sometimes there is no entry in the Application Event Log however only in the System Event Log. The Error
event 7034 from Service Control Manager is another confirmation that an AOS Instance was terminating
unexpectedly.
Event Type: Error
Event Source: Service Control Manager
Event ID: 7034
Description:
The Dynamics Server$01-AOS1 service terminated unexpectedly. It has done this 2 time(s).
Forced termination
In contrast to an unexpected termination there is also a forced termination whose effects are however similar
not to say the same. The difference to an unexpected termination is that a forced termination is caused by the
program logic of the Dynamics AX AOS Kernel itself because some error checking code determined a
situation the AOS cannot resolve; however continuation in such a situation would most likely cause some more
16
serious effects up to data corruption, so it is better to terminate at this stage and log this fact to the Application
Event Log.
In such a case an Error event 110 from Dynamics Server is logged having the wording Unexpected situation
in the Description.
Event Type: Error
Event Source: Dynamics Server 01
Event ID: 110
Description:
Object Server 01: Unexpected situation
More Information: Session Allocation Failed: Session is already allocated.
Controlled shutdown
We have seen so far confirmations for an unexpected termination and a forced termination, however there exist
also proves for a controlled shutdown where the AOS in fact did not crash.
The Information event 108 in the Application Event Log is always logged when the AOS has been requested to
stop in a controlled way. If you see this event someone or something has requested the AOS to stop by
intention.
Event Type: Information
Event Source: Dynamics Server 01
Event ID: 108
Description:
Object Server 01: Dynamics Server has been stopped
The Information events 7035 and 7036 from Service Control Manager in the System Event Log are another
prove of a controlled shutdown of the AOS service. Again someone or something requested to AOS to stop by
intention.
Event ID: 7035
Description:
The Dynamics Server$01-AOS1 service was successfully sent a stop control.
Event ID: 7036
Description:
The Dynamics Server$01-AOS1 service entered the stopped state.
17
What do I need to create a Crash Dump?
A Crash Dump is a Memory Dump that is automatically created when a process is executing an operation
(CPU instruction) that causes the process to crash.
Lets assume you have confirmed your AOS is indeed crashing and you need to create a Memory Dump (Crash
Dump).
There exist several tools you can use to create a Memory Dump from a crashing process, and even the
operating system is providing some built in functionality.
Debug Diagnostic Tool v1.2 (DebugDiag)
Highlight: Once installed the tool runs as a windows service.

To create a dump with the Debug Diagnostic Tool v1.2 (DebugDiag) you need to download and install it first.
Download appropriate 32bit or 64bit version depending on Operating System in use.
When you launch the tool you are presented with a Graphical User Interface (GUI) and a Wizard the helps you
to create a new Crash Rule.
The advantage of using DebugDiag is that it has a GUI and with the Crash Rule created not only will it attach
to the currently running AOS Instances but also AOS Instances started after the creation of the Crash Rule.
A more detailed instruction about the Debug Diagnostic Tool (written for v1.1 but applies to v1.2 as well) can
be found in the following article How to Use the Debug Diagnostic Tool v1.1 (DebugDiag) to Debug User
Mode Processes.
Windows Error Reporting (WER)
Highlight: No software is required to be downloaded.

The Windows Error Reporting (WER) that is built into Windows 2008/2008 R2 can be configured to
automatically create and store memory dumps from an AOS crash (This cannot be used on Windows Server
2003).
The configuration happens by making some Registry modifications, more concrete creating the following
Registry Key:
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows Error Reporting\LocalDumps\Ax32Serv.exe
18
In this new Registry Key several Registry Values have to be added as well (DumpFolder, DumpCount,
DumpType, CustomDumpFlags).
Please note:
Do NOT set the DumpType to 1 or 2, but set the DumpType to 0 and set CustomDumpFlags to 7015
decimal (0x1B67 hexadecimal). Invalid settings will generate dumps without the required information.
A detailed instruction about the needed Registry Values is described in the following article Collecting UserMode Dumps.
ADPlus
To create a dump with ADPlus you first need to download and install the Debugging Tools for Windows and
launch adplus.vbs from the command line with the -crash and -pn ax32serv parameters.
adplus.exe crash pn ax32serv
After executing the script you will see new command windows showing up one command window per
running AOS Instance. Actually this are no command windows but this is the command line debugger CDB
that is attached to the AOS Instance.
It is very important that you dont close those CDB instances nor log off from the machine because a closing
of the CDB instances will also terminate the attached AOS Instance!
Please note:
ADPlus will only attach to the currently running AOS Instances. If you start another AOS Instance afterwards
or restart a running AOS Instance you need to execute ADPlus again!
A more detailed instruction about ADPlus can be found in the KB article How to use ADPlus to troubleshoot
"hangs" and "crashes".
PLEASE NOTE: This tool requires an active logged on user session the whole time. You can lock your
user session. However as soon as the user logs off it will terminate the tool and with that will terminate
any attached processes.
Crash or High Memory Usage?
As indicated before a (Full) Memory Dump is containing the process memory and register context, so usually
the file size of the dump file is about the size of the currently allocated virtual memory.
19
If the file size of the memory dump is about or larger than 1.6 GB and your are not running a 64 bit version of
Dynamics AX the chances are very high the crash was caused by a High Memory Usage situation or because
of a Memory Leak.
In this case please also have a look at my earlier post Why does the server need to free resources and
terminates client sessions?
So your AOS crashed, is hanging, or you just want to see what it's doing
EMEADAXSupport
Ever been in a situation where your AOS crashed? And all you had to show for it was a lousy event log
message 1000 which didn't tell you anything except that it crashed?
Or perhaps you've just wanted to see what your AOS is actually running at a given moment in time? It could
be very useful to have the ability to take a snapshot and see what's going on inside the AOS process.
Well now this is possible. We have the first of a series of posts out now which describe the steps you need to
get up and running with generating memory dumps from your AOS and analysing them using WinDbg. The
posts are linked below, with a quick description to show the steps required:
First you need to generate a memory dump to analyse, the post below explains the different options you
have for that:
http://blogs.msdn.com/b/emeadaxsupport/archive/2010/05/12/possibilities-to-create-memory-dumps-fromcrashing-processes.aspx
Next you need to install the tool "WinDbg" on your machine which will be used to analyse the dumps:
http://blogs.msdn.com/b/emeadaxsupport/archive/2011/04/10/setting-up-windbg-and-using-symbols.aspx
After that you can start pulling information from the dumps, these two posts explain how to find the
X++ call stack and the AX user:
http://blogs.msdn.com/b/emeadaxsupport/archive/2011/04/10/finding-the-x-call-stack-that-caused-a-crash.aspx
http://blogs.msdn.com/b/emeadaxsupport/archive/2011/04/10/finding-the-ax-user-that-caused-a-crash.aspx
Finally this post shows how to run scripts which we've created for you to enable you to pull information
very fast from dump files, and to pull information even when no symbols are available:
http://blogs.msdn.com/b/emeadaxsupport/archive/2011/04/10/finding-the-ax-user-and-the-x-call-stack-from-amemory-dump-the-easy-way.aspx
20
Disclaimer
2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista, Microsoft Dynamics
and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other
countries. Other names and brands may be claimed as the property of others.
The information herein is for informational purposes only and represents the current view of Microsoft
Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it
should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the
accuracy of any information provided after the date of this presentation.
Sample code included in this presentation is made available AS IS. THE ENTIRE RISK OF THE USE OR THE
RESULTS FROM THE USE OF THIS CODE REMAINS WITH THE USER.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE
INFORMATION IN THIS PRESENTATION. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR
OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS PRESENTATION.
Microsoft products are not intended for use in medical, life-saving, or life-sustaining applications.
Microsoft Dynamics is a line of integrated, adaptable business management solutions that enables you and your
people to make business decisions with greater confidence. Microsoft Dynamics works like and with familiar
Microsoft software, automating and streamlining financial, customer relationship and supply chain processes in a
way that helps you drive business success.
U.S. and Canada Toll Free 1-888-477-7989
Worldwide +1-701-281-6500 www.microsoft.com/dynamics
21

LCSinActionWorkshop-Operate5 CrashDumpAnalysis V2

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

LCSinActionWorkshop-Operate5 CrashDumpAnalysis V2

Загружено:

Авторское право:

Доступные форматы

Microsoft Dynamics AX

Lifecycle Services: Operate Phase: Crash Dump Analysis

Operate Phase Crash Dump Analysis

Create Crash rule for all instances of AX32.exe

Create job, and run

From the tool's main page, press Add to create a new

Provide a name and an optional description for the

Press Add files

Crash dump analysis can analyse both ax32 and ax32serv

Select .ZIP file containing one or more .DMP files and

Press Analyze dumps

Confirm start of analysis

Wait until processing completes. This can take

After the job is marked as Completed, an HTML

In this report you can see information about:

X++ and native call stacks for all threads

Possibilities to create Memory Dumps from crashing processes

What do I need to create a Crash Dump?

Highlight: Once installed the tool runs as a windows service.

Highlight: No software is required to be downloaded.

Вам также может понравиться