Ab Initio - Parameter Kung Fu

Remediator (IT Consultant) posted 6/18/2008 | Comments (0)

Some of you periodically shoot a question about parameters, or how to handle them and make the best of them. I've got another post here on some parameter issues, but here's a simple set of rules that can get you moving and keep you moving 1 - Never, ever, never use start or end scripts, just like you don't play basketball on the freeway - see the blog entry with "Pandora" in the title. the Edit/Parameters area is the place for custom and graph-level parameters, with environmental configuration always derived from the sandbox and project level parameters 2 - Command-line parameters should be keyword-based, never positional, and always provide a default - It provides an automatic help prompt for an operator, but more important keeps an operator from having to do too much, or type too much to get things moving 3 - Restart - when the graph launches, form a self-contained script that holds all the necessary restart/recovery information. If restart is necessary, the operator runs the the script, which automatically forms the correct command line no guesswork 4 - Don't use redundant values on the command line, such as known extensions like .xfr, .dml and .dat - if you know what these are in the graph, apply them in the graph 5 - Rather than pass 'hard' values on the command line, pass a pattern. For example, if we have an inbound file named myfile.dat, that is likewise described by the myfile.dml, and is transformed by myfile.xfr, we only need to pass the value "myfile" and snap it together with the appropriate extensions 6 - Use of parameters in the component level should arrive fully-resolved. Get it squared away in the Edit/Parameters zone. in other words, the most you should have in a DML def should be something like ${AI_DML}/${filename}.dml. Keep in mind that if these definitions exist in the start script, you have to run the graph to consume them, your vain attempts to perform a view/data in an input file, will only frustrate your troubleshooting 7 - The values of AI_DML etc that you have carefully shaped in the sandbox, do not exist on the command line. They only exist once finally resolved inside 8 - Anything that you need to promote to the next region (Dev->Test) should be part of the sandbox directories. This allows your regular check-in process to pull them in and promote them automatically 9 - When migrating your graphs from dev to test and beyond, migrate the graph's MP file but don't land them into the local directories - keep it in the EME. 10 - The entire environment is Korn shell, so Korn shell rules apply, and we can use creative Korn shell in the Edit/Parameters area, for its intended purpose - configuration control. Don't use Edit/paramters for data processing

11 - Because it is Korn shell, it is derived and resolved before the graph commences, and cannot be changed afterward. In other words, we cannot perform an in-graph operation to change a parameter's value. Data discovered in the flow won't help us here 12 - The most effective way to deal with "changing values" like this, is to leverage a parameter that will be used as a key to a lookup file, then in the flow, just use the value the key maps to 13 - Sometimes people want to preserve the enviroment for the next graph execution, or report errors in context of the existing parameters, and will launch such a reporting graph in the end-script, carrying the existing parameter values into the next graph. First, never use end scripts, and likewise never call another program - or another graph - from a graph. Second, as mentioned prior, let the EME do this kind of thing for you. The good folks at the vendor can show you how to configure the EME to launch preliminary operations and post-operations for a graph, so you can capture everything and still not lose control 14 - Parameters are human-failure points. Reduce errors by reducing command-line keystrokes.