Htcondor Python

HTCondor Python Documentation
Release 8.5.8
Brian Bockelman
Mar 01, 2019

Contents
1 Introductory Tutorial 3
1.1 ClassAds Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 HTCondor Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Submitting and Managing Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Advanced Tutorials 13
2.1 Advanced Schedd Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Scalable Job Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Interacting with Daemons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3 Module Reference 23
3.1 htcondor – HTCondor Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 classad – ClassAd reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Python Module Index 49
i
ii
HTCondor Python Documentation, Release 8.5.8
The HTCondor python modules aim to expose a high-quality, Pythonic interface to the HTCondor client libraries.
They utilize the same C++ libraries as HTCondor itself, meaning they have nearly the same behavior as the command
line tools. As the python modules are shipped with HTCondor itself, this documentation focuses on references and
tutorials on how to use the modules, rather than an install how-to.
Note: This work should be considered experimental (its aim is to be better-formatted and more intuitive); the upstream
documentation should be considered authoritative.
Contents 1
2 Contents
CHAPTER 1
Introductory Tutorial
Here, you will learn the basics of the python bindings and how to use them. This introduction provides quick overview
of the major components. Each learning module is meant to be done in sequence. Start here if you have never used
the bindings before.
1.1 ClassAds Introduction
In this module, you will learn the basics of the ClassAd language, the policy and data exchange language that underpins
all of HTCondor. Before we start to interact with the HTCondor daemons, it is best to learn ClassADs.
As always, we start off by importing the relevant module:
import classad
Great!
The python bindings - and HTCondor itself - are strongly centered on the ClassAd language.
The ClassAd language is built around values and expressions. If you know Python, both concepts are familiar. Exam-
ples of familiar values include:
• Integers (1, 2, 3),
• Floating point numbers (3.145, -1e-6)
• Booleans (true and false).
• Strings ("Hello World!").
Examples of expressions are:
• Attribute references: foo
• Boolean expressions: a && b
• Arithmetic expressions: 123 + c
• Function calls: ifThenElse(foo == 123, 3.14, 5.2)
3
Expressions can be evaluated to values. Unlike many programming languages, expressions are lazily-evaluated: they
are kept in memory until a value is explicitly requested.
The python bindings interact with the ClassAd language through the ExprTree and ClassAd objects. Let’s first examine
simple expressions:
>>> arith_expr = classad.ExprTree("1 + 4")

>>> print "ClassAd arithemetic expression: %s (of type %s)" % (arith_expr, type(arith_
˓→expr))
ClassAd arithemetic expression: 1 + 4 (of type <class 'classad.ExprTree'>)

>>> print arith_expr.eval()
5
>>> function_expr = classad.ExprTree("ifThenElse(4 > 6, 123, 456)")
>>> print "Function expression: %s" % function_expr
Function expression: ifThenElse(4 > 6,123,456)
>>> value = function_expr.eval()
>>> print "Corresponding value: %s (of type %s)" % (value, type(value))
Corresponding value: 456 (of type <type 'long'>)
Notice that, when possible, we convert ClassAd values to Python values. Hence, the result of evaluating the expression
above is the python number 456.
There are two important values in the ClassAd language that have no direct equivalent in Python: Undefined and
Error.
Undefined occurs when a reference occurs to an attribute that is not defined; it is analogous to a NameError
exception in Python (but there is no concept of an exception in ClassAds). Error occurs primarily when an expression
combines two different types or when a function call occurs with the incorrect arguments.:
>>> print classad.ExprTree("foo").eval()

Undefined
>>> print classad.ExprTree('5 + "bar"').eval()
Error
>>> print classad.ExprTree('ifThenElse(1, 2, 3, 4, 5)').eval()
Error
So what?
Expressions, values, and various error conditions are not particularly inspiring or new.
The concept that makes the ClassAd language special is, of course, the ClassAd!
The ClassAd is analogous to a python or JSON dictionary. Unlike a dictionary, which is a set of unique key-value
pairs, the ClassAd object is a set of key-expression pairs. The expressions in the ad can contain attribute references to
other keys in the ad.
There are two common ways to represent ClassAds. The “new ClassAd” format:
[a = 1;
b = "foo";
c = b
]
And the “old ClassAd” format:
a = 1
b = "foo"
c = b
4 Chapter 1. Introductory Tutorial

Despite the “new” and “old” monikers, “new” is over a decade old and HTCondor command line tools utilize the “old”
representation; the python bindings default to “new”.
A ClassAd object may be initialized via a string using one of the above representation. As a ClassAd is so similar
to a python dictionary, they may be constructed from a dictionary.
Let’s construct some ClassAds!:
>>> ad1 = classad.ClassAd("""[

... a = 1;
... b = "foo";
... c = b;
... d = a + 4;
... ]""")
>>> print ad1
[
a = 1;
b = "foo";
c = b;
d = a + 4
]
ClassAds are quite similar to dictionaries; in python, the ClassAd object behaves similarly to a dictionary and has
similar convenience methods:
>>> print ad1["a"]

1
>>> print ad1["not_here"]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'not_here'
>>> print ad1.get("not_here", 5)
5
>>> ad1.update({"e": 8, "f": True})
>>> for key in ad1:
... print key, ad1[key]
f True
e 8
a 1
b foo
c b
d a + 4
Remember our example of an Undefined attribute above? We now can evaluate references within the ad:
>>> print ad1.eval("d")

5
Note that an expression is still not evaluated until requested, even if it is invalid::
>>> ad1["g"] = classad.ExprTree("b + 5")

>>> print ad1["g"], type(ad1["g"])
b + 5 <class 'classad.ExprTree'>
>>> print ad1.eval("g")
Error
ClassAds and expressions are core concepts in interacting with HTCondor. Internally, machines and jobs are repre-
sented as ClassAds; expressions are used to filter objects and to define policy.
1.1. ClassAds Introduction 5

There’s much more to learn in ClassAds! We maintain comprehensive module documentation for classad.
For now, you have enough background to continue to the next tutorial.
1.2 HTCondor Introduction
Let’s start interacting with the HTCondor daemons!

We’ll cover the basics of two daemons, the Collector and the Schedd:
• The Collector maintains an inventory of all the pieces of the HTCondor pool. For example, each machine
that can run jobs will advertise a ClassAd describing its resources and state. In this module, we’ll learn the
basics of querying the collector for information and displaying results.
• The Schedd maintains a queue of jobs and is responsible for managing their execution. We’ll learn the ba-
sics of querying the schedd.
There are several other daemons - particularly, the Startd and the Negotiator - the python bindings can interact with.
We’ll cover those in the advanced modules.
To start, let’s import the htcondor modules.:
>>> import htcondor

>>> import classad
1.2.1 Collector
We’ll start with the Collector, which gathers descriptions of the states of all the daemons in your HTCondor pool. The
collector provides both service discovery and monitoring for these daemons.
Let’s try to find the Schedd information for your HTCondor pool. First, we’ll create a Collector object, then use
the locate() method:
>>> coll = htcondor.Collector() # Create the object representing the collector.

>>> schedd_ad = coll.locate(htcondor.DaemonTypes.Schedd) # Locate the default schedd.
>>> print schedd_ad['MyAddress'] # Prints the location of the schedd, using HTCondor
˓→'s internal addressing scheme.
<172.17.0.2:9618?addrs=172.17.0.2-9618+[--1]-9618&noUDP&sock=9_6140_4>
The locate() method takes a type of daemon and (optionally) a name, returning a ClassAd. Here, we print out
the resulting MyAddress key.
A few minor points about the above example:
• Because we didn’t provide the collector with a constructor, we used the default collector in the host’s con-
figuration file. If we wanted to instead query a non-default collector, we could have done htcondor.
Collector("collector.example.com").
• We used the htcondor.DaemonTypes enumeration to pick the kind of daemon to return.
• If there were multiple schedds in the pool, the locate() query would have failed. In such a case, we need
to provide an explicit name to the method. E.g., coll.locate(htcondor.DaemonTypes.Schedd,
"schedd.example.com").
• The final output prints the schedd’s location. You may be surprised that this is not simply a hostname:port; to
help manage addressing in the today’s complicated Internet (full of NATs, private networks, and firewalls), a
more flexible structure was needed.

– HTCondor developers sometimes refer to this as the sinful string; here, sinful is a play on a Unix data
structure name, not a moral judgement.
The locate() method often returns only enough data to contact a remote daemon. Typically, a ClassAd records
significantly more attributes. For example, if we wanted to query for a few specific attributes, we would use the
query() method instead:
>>> coll.query(htcondor.AdTypes.Schedd, projection=["Name", "MyAddress",

˓→"DaemonCoreDutyCycle"])
[[ DaemonCoreDutyCycle = 1.439361064858868E-05; Name = "jovyan@eb4f00c8f1ca"; MyType

˓→= "Scheduler"; MyAddress = "<172.17.0.2:9618?addrs=172.17.0.2-9618+[--1]-9618&noUDP&
˓→sock=9_6140_4>" ]]
Here, query() takes an AdType (slightly more generic than the DaemonTypes, as many kinds of ads are in the
collector) and several optional arguments, then returns a list of ClassAds.
We used the projection keyword argument; this indicates what attributes you want returned. The collector may
automatically insert additional attributes (here, only MyType); if an ad is missing a requested attribute, it is simply
not set in the returned ClassAd object. If no projection is specified, then all attributes are returned.
Warning: When possible, utilize the projection to limit the data returned. Some ads may have hundreds of
attributes, making returning the entire ad an expensive operation.
The projection filters the returned keys; to filter out unwanted ads, utilize the constraint option. Let’s do the same
query again, but specify our hostname explicitly:
>>> import socket # We'll use this to automatically fill in our hostname
>>> coll.query(htcondor.AdTypes.Schedd,
... constraint='Name=?=%s' % classad.quote("jovyan@%s" % socket.getfqdn()),
... projection=["Name", "MyAddress", "DaemonCoreDutyCycle"])
[[ DaemonCoreDutyCycle = 1.439621262799839E-05; Name = "jovyan@eb4f00c8f1ca"; MyType
˓→= "Scheduler"; MyAddress = "<172.17.0.2:9618?addrs=172.17.0.2-9618+[--1]-9618&noUDP&
˓→sock=9_6140_4>" ]]
Some notes on the above:

• constraint accepts either an ExprTree or str object; the latter is automatically parsed as an expression.
• We used the classad.quote() function to properly quote the hostname string. In this example, we’re
relatively certain the hostname won’t contain quotes. However, it is good practice to use the quote() function
to avoid possible SQL-injection-type attacks.
– Consider what would happen if the host’s FQDN contained spaces and doublequotes, such as foo.
example.com" || true.
1.2.2 Schedd
Let’s try our hand at querying the schedd!

First, we’ll need a Schedd object. You may either create one out of the ad returned by locate() above or use the
default in the configuration file::
>>> schedd = htcondor.Schedd()

>>> schedd = htcondor.Schedd(schedd_ad)
>>> print schedd
<htcondor.Schedd object at 0x7f388404b940>
1.2. HTCondor Introduction 7

Unfortunately, as there are no jobs in our personal HTCondor pool, querying the schedd will be boring. Let’s submit
a few jobs (note the API used below will be covered by the next module; it’s OK if you don’t understand it now)::
>>> sub = htcondor.Submit()

>>> sub['executable'] = '/bin/sleep'
>>> sub['arguments'] = '5m'
>>> with schedd.transaction() as txn:
... sub.queue(txn, 10)
We should now have 10 jobs in queue, each of which should take 5 minutes to complete.
Let’s query for the jobs, paying attention to the jobs’ ID and status::
>>> for job in schedd.xquery(projection=['ClusterId', 'ProcId', 'JobStatus']):

... print job.__repr__()
[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 2;
˓→ProcId = 0; ClusterId = 2 ]









The JobStatus is an integer; the integers map into the following states:
• 1: Idle (I)
• 2: Running (R)
• 3: Removed (X)
• 4: Completed (C)
• 5: Held (H)
• 6: Transferring Output
• 7: Suspended
Depending on how quickly you executed the notebook, you might see all jobs idle (JobStatus = 1) or one job
running (JobStatus = 2) above. It is rare to see the other codes.
As with the Collector’s query() method, we can also filter out jobs using xquery()::
>>> for job in schedd.xquery(requirements = 'ProcId >= 5', projection=['ProcId']):

... print job.get('ProcId')
5
6
(continues on next page)

(continued from previous page)

7
8
9
Astute readers may notice that the Schedd object has both xquery() and query() methods. The difference
between the two mimics the difference between xreadlines() and readlines() call in the standard Python
library:
• query() returns a list of ClassAds, meaning all objects are held in memory at once. This utilizes more mem-
ory, but the size of the results is immediately available. It utilizes an older, heavyweight protocol to communicate
with the Schedd.
• xquery() returns an iterator that produces ClassAds. This only requires one ClassAd to be in memory at
once. It is much more lightweight, both on the client and server side.
When in doubt, utilize xquery().
Now that we have a running job, it may be useful to check the status of the machine in our HTCondor pool:
>>> print coll.query(htcondor.AdTypes.Startd, projection=['Name', 'Status', 'Activity

˓→', 'JobId', 'RemoteOwner'])[0]
[
Activity = "Busy";
Name = "eb4f00c8f1ca";
RemoteOwner = "jovyan@eb4f00c8f1ca";
MyType = "Machine";
JobId = "2.3";
TargetType = "Job"
]
The Collector and Schedd APIs are large; we maintain comprehensive module documentation in htcondor.
Congratulations - you can now perform simple queries against the collector for worker and submit hosts, as well as
simple job queries against the submit host!
1.3 Submitting and Managing Jobs
The two most common HTCondor command line tools are condor_q and condor_submit; in HTCondor Intro-
duction we learning the xquery() method that corresponds to condor_q. Here, we will learn the Python binding
equivalent of condor_submit.
As usual, we start by importing the relevant modules:
>>> import htcondor
1.3.1 Submitting Jobs
We will submit jobs utilizing the dedicated Submit object.
Note: The Submit object was introduced in 8.5.6, which might be newer than your home cluster. The original API,
using the htcondor.Schedd.submit() method, utilizes raw ClassAds and is not covered here.
1.3. Submitting and Managing Jobs 9

Submit objects consist of key-value pairs. Unlike ClassAds, the values do not have an inherent type (such as strings,
integers, or booleans); they are evaluated with macro expansion at submit time. Where reasonable, they behave like
python dictionaries:
>>> sub = htcondor.Submit({"foo": "1", "bar": "2", "baz": "$(foo)"})

>>> sub.setdefault("qux", "3")
>>> print "=== START ===\n%s\n=== END ===" % sub
=== START ===
baz = $(foo)
foo = 1
bar = 2
qux = 3
queue
=== END ===
>>> print sub.expand("baz")
1
The available attributes - and their semantics - are relatively well documented in the condor_submit online help;
we won’t repeat them here. A minimal, but realistic submit object may look like the following:
>>> sub = htcondor.Submit({"executable": "/bin/sleep", "arguments": "5m"})
To go from a submit object to job in a schedd, one must do three things:

1. Create a new transaction in the schedd using transaction().
2. Call the queue() method, passing the transaction object.
3. Commit the transaction.
Since the transaction object is a python context, (1) and (3) can be achieved using Python’s with statement:
>>> schedd = htcondor.Schedd() # Create a schedd object using default

˓→settings.
>>> with schedd.transaction() as txn: # txn will now represent the transaction.
... print sub.queue(txn) # Queues one job in the current transaction;
˓→returns job's cluster ID
If the code block inside the with statement completes successfully, the transaction is automatically committed. If an
exception is thrown (or Python abruptly exits), the transaction is aborted.
1.3.2 Managing Jobs
Once a job is in queue, the schedd will try its best to execute it to completion. There are several cases where a user
may want to interrupt the normal flow of jobs. Perhaps the results are no longer needed; perhaps the job needs to be
edited to correct a submission error. These actions fall under the purview of job management.
There are two Schedd methods dedicated to job management:
• edit(): Change an attribute for a set of jobs to a given expression. If invoked within a transaction, multiple
calls to edit() are visible atomically.
– The set of jobs to change can be given as a ClassAd expression. If no jobs match the filter, then an
exception is thrown.
• act(): Change the state of a job to a given state (remove, hold, suspend, etc).

Both methods take a job specification: either a ClassAd expression (such as Owner=?="janedoe") or a list of job
IDs (such as ["1.1", "2.2", "2.3"]). The act() method takes an argument from the JobAction enum.
Commonly-used values include:
• Hold: put a job on hold, vacating a running job if necessary. A job will stay in the hold state until explicitly
acted upon by the admin or owner.
• Release: Release a job from the hold state, returning it to Idle.
• Remove: Remove a job from the Schedd’s queue, cleaning it up first on the remote host (if running). This
requires the remote host to acknowledge it has successfully vacated the job, meaning Remove may not be
instantaneous.
• Vacate: Cause a running job to be killed on the remote resource and return to idle state. With Vacate, jobs
may be given significant time to cleanly shut down.
Here’s an example of job management in action:

... clusterId = sub.queue(txn, 5) # Queues 5 copies of this job.
... schedd.edit(["%d.0" % clusterId, "%d.1" % clusterId], "foo", '"bar"') # Sets
˓→attribute foo to the string "bar".
>>> for job in schedd.xquery(requirements="ClusterId == %d" % clusterId, projection=[

˓→"ProcId", "foo", "JobStatus"]):
... print "%d: foo=%s, job_status = %d" % (job.get("ProcId"), job.get("foo",

˓→"default_string"), job["JobStatus"])
0: foo=bar, job_status = 1
2: foo=default_string, job_status = 1
>>> schedd.act(htcondor.JobAction.Hold, 'ClusterId==%d && ProcId >= 2' % clusterId)
>>> for job in schedd.xquery(requirements="ClusterId == %d" % clusterId, projection=[
˓→"ProcId", "foo", "JobStatus"]):
... print "%d: foo=%s, job_status = %d" % (job.get("ProcId"), job.get("foo",

˓→"default_string"), job["JobStatus"])
That’s it!
You’ve made it through the very basics of the python bindings. While there are many other features the python
module has to offer, we have covered enough to replace the command line tools of condor_q, condor_submit,
condor_status, condor_rm and others.
1.3. Submitting and Managing Jobs 11


CHAPTER 2
Advanced Tutorials
The advanced tutorials are in-depth looks at specific pieces of the Python modules; each is meant to be stand-alone
and only should require knowledge from the introductory tutorial.
2.1 Advanced Schedd Interaction
The introductory tutorial only scratches the surface of what the Python bindings can do with the condor_schedd;
this module focuses on covering a wider range of functionality:
• Job and history querying.
• Advanced job submission.
• Python-based negotiation with the Schedd.
2.1.1 Job and History Querying
In HTCondor Introduction, we covered the xquery() method and its two most important keywords:
• requirements: Filters the jobs the schedd should return.
• projection: Filters the attributes returned for each job.
For those familiar with SQL queries, requirements performs the equivalent as the WHERE clause while
projection performs the equivalent of the column listing in SELECT.
There are two other keywords worth mentioning:
• limit: Limits the number of returned ads; equivalent to SQL’s LIMIT.
• opts: Additional flags to send to the schedd to alter query behavior. The only flag currently defined is
AutoCluster; this groups the returned results by the current set of “auto-cluster” attributes used by the
pool. It’s analogous to GROUP BY in SQL, except the columns used for grouping are controlled by the schedd.
To illustrate these additional keywords, let’s first submit a few jobs:
13

>>> sub = htcondor.Submit({
... "executable": "/bin/sleep",
... "arguments": "5m",
... "hold": "True",
... })
... clusterId = sub.queue(txn, 10)
Note: In this example, we used the hold submit command to indicate that the jobs should start out in the
condor_schedd in the Hold state; this is used simply to prevent the jobs from running to completion while you are
running the tutorial.
We now have 10 jobs running under clusterId; they should all be identical:
>>> print sum(1 for _ in schedd.xquery(projection=["ProcID"], requirements=

˓→"ClusterId==%d" % clusterId, limit=5))
5
>>> print list(schedd.xquery(projection=["ProcID"], requirements="ClusterId==%d" %
˓→clusterId, opts=htcondor.QueryOpts.AutoCluster))
The sum(1 for _ in ...) syntax is a simple way to count the number of items produced by an iterator without
buffering all the objects in memory.
Querying many Schedds
On larger pools, it’s common to write python scripts that interact with not one but many schedds. For example, if you
want to implement a “global query” (equivalent to condor_q -g; concatenates all jobs in all schedds), it might be
tempting to write code like this:
>>> jobs = []
>>> for schedd_ad in htcondor.Collector().locateAll(htcondor.DaemonTypes.Schedd):
... schedd = htcondor.Schedd(schedd_ad)
... jobs += schedd.xquery()
>>> print len(jobs)
This is sub-optimal for two reasons:

• xquery is not given any projection, meaning it will pull all attributes for all jobs - much more data than is
needed for simply counting jobs.
• The querying across all schedds is serialized: we may wait for painfully long on one or two “bad apples”
We can instead begin the query for all schedds simultaneously, then read the responses as they are sent back. First, we
start all the queries without reading responses:
>>> queries = []
>>> coll_query = htcondor.Collector().locate(htcondor.AdTypes.Schedd)
>>> for schedd_ad in coll_query:
... schedd_obj = htcondor.Schedd(schedd_ad)
... queries.append(schedd_obj.xquery())
The iterators will yield the matching jobs; to return the autoclusters instead of jobs, use the AutoCluster option
(schedd_obj.xquery(opts=htcondor.QueryOpts.AutoCluster)). One auto-cluster ad is returned
for each set of jobs that have identical values for all significant attributes. A sample auto-cluster looks like:
14 Chapter 2. Advanced Tutorials

[
RequestDisk = DiskUsage;
Rank = 0.0;
FileSystemDomain = "hcc-briantest7.unl.edu";
MemoryUsage = ( ( ResidentSetSize + 1023 ) / 1024 );
ImageSize = 1000;
JobUniverse = 5;
DiskUsage = 1000;
JobCount = 1;
Requirements = ( TARGET.Arch == "X86_64" ) && ( TARGET.OpSys == "LINUX" ) && (
˓→TARGET.Disk >= RequestDisk ) && ( TARGET.Memory >= RequestMemory ) && ( ( TARGET.
˓→HasFileTransfer ) || ( TARGET.FileSystemDomain == MY.FileSystemDomain ) );
RequestMemory = ifthenelse(MemoryUsage isnt undefined,MemoryUsage,( ImageSize + 1023

˓→) / 1024);
ResidentSetSize = 0;
ServerTime = 1483758177;
AutoClusterId = 2
]
We use the poll() function, which will return when a query has available results:
>>> job_counts = {}
>>> for query in htcondor.poll(queries):
... schedd_name = query.tag()
... job_counts.setdefault(schedd_name, 0)
... count = len(query.nextAdsNonBlocking())
... job_counts[schedd_name] += count
... print "Got %d results from %s." % (count, schedd_name)
>>> print job_counts
The tag() method is used to identify which query is returned; the tag defaults to the Schedd’s name but can be
manually set through the tag keyword argument to xquery().
History Queries
After a job has finished in the Schedd, it moves from the queue to the history file. The history can be queried (locally
or remotely) with the history() method:

>>> for ad in schedd.history('true', ['ProcId', 'ClusterId', 'JobStatus',
˓→'WallDuration'], 2):
... print ad
At the time of writing, unlike xquery(), history() takes positional arguments and not keyword. The first
argument a job constraint; second is the projection list; the third is the maximum number of jobs to return.
2.1.2 Advanced Job Submission
In HTCondor Introduction, we introduced the Submit object. Submit allows jobs to be created using the submit
file language. This is the well-documented, familiar means for submitting jobs via condor_submit. This is the
preferred mechansim for submitting jobs from Python.
Internally, the submit files are converted to a job ClassAd. The older submit() method allows jobs to be submitted
as ClassAds. For example:
2.1. Advanced Schedd Interaction 15

>>> import os.path

>>> job_ad = { \
... 'Cmd': '/bin/sh',
... 'JobUniverse': 5,
... 'Iwd': os.path.abspath("/tmp"),
... 'Out': 'testclaim.out',
... 'Err': 'testclaim.err',
... 'Arguments': 'sleep 5m',
... }
>>> clusterId = schedd.submit(job_ad, count=2)
This will submit two copies of the job described by job_ad into a single job cluster.
Hint: To generate an example ClassAd, take a sample submit description file and invoke:
condor_submit -dump <filename> [cmdfile]
Then, load the resulting contents of <filename> into Python.
Calling submit() standalone will automatically create and commit a transaction. Multiple jobs can be submitted
atomically and more efficiently within a transaction() context.
Each submit() invocation will create a new job cluster; all attributes will be identical except for the ProcId
attribute (process IDs are assigned in monotonically increasing order, starting at zero). If jobs in the same cluster need
to differ on additional attributes, one may use the submitMany() method:
>>> foo = {'myAttr': 'foo'}
>>> bar = {'myAttr': 'bar'}
>>> clusterId = schedd.submitMany(job_ad, [(foo, 2), (bar, 2)])
>>> print list(schedd.xquery('ClusterId==%d' % clusterId, ['ProcId', 'myAttr']))
submitMany() takes a basic job ad (sometimes referred to as the cluster ad), shared by all jobs in the cluster and a
list of process ads. The process ad list indicates the attributes that should be overridden for individual jobs, as well as
the number of such jobs that should be submitted.
Job Spooling
HTCondor file transfer will move output and input files to and from the submit host; these files will move back to the
original location on the host. In some cases, this may be problematic; you may want to submit one set of jobs to run
/home/jovyan/a.out, recompile the binary, then submit a fresh set of jobs. By using the spooling feature, the
condor_schedd will make a private copy of a.out after submit, allowing the user to make new edits.
Hint: Although here we give an example of using spool() for spooling on the local Schedd, with appropriate
authoriation the same methods can be used for submitting to remote hosts.
To spool, one must specify this at submit time and invoke the spool() method and provide an ad_results array:
>>> ads = []
>>> cluster = schedd.submit(job_ad, 1, spool=True, ad_results=ads)
>>> schedd.spool(ads)
This will copy the files into the Schedd’s spool directory. After the job completes, its output files will stay in the
spool. One needs to call retrieve() to move the outputs back to their final destination:

>>> schedd.retrieve("ClusterId == %d" % cluster)
2.1.3 Negotiation with the Schedd
The condor_negotiator daemon gathers job and machine ClassAds, tries to match machines to available jobs,
and sends these matches to the condor_schedd.
In truth, the “match” is internally a claim on the resource; the Schedd is allowed to execute one or more job on it.
The python bindings can also send claims to the Schedds. First, we must prepare the claim objects by taking the slot’s
public ClassAd and adding a ClaimId attribute:
>>> coll = htcondor.Collector()

>>> private_ads = coll.query(htcondor.AdTypes.StartdPrivate)
>>> startd_ads = coll.query(htcondor.AdTypes.Startd)
>>> claim_ads = []
>>> for ad in startd_ads:
... if "Name" not in ad: continue
... found_private = False
... for pvt_ad in private_ads:
... if pvt_ad.get('Name') == ad['Name']:
... found_private = True
... ad['ClaimId'] = pvt_ad['Capability']
... claim_ads.append(ad)
Once the claims are prepared, we can send them to the schedd. Here’s an example of sending the claim to user
jovyan@example.com, for any matching ad:
>>> with htcondor.Schedd().negotiate("bbockelm@unl.edu") as session:

>>> found_claim = False
>>> for resource_request in session:
>>> for claim_ad in claim_ads:
>>> if resource_request.symmetricMatch(claim_ad):
... print "Sending claim for", claim_ad["Name"]
... session.sendClaim(claim_ads[0])
... found_claim = True
... break
... if found_claim: break
This is far cry from what the condor_negotiator actually does (the negotiator additionally enforces fairshare,
for example).
Note: The python bindings can send claims to the schedd immediately, even without reading the resource request
from the schedd. The schedd will only utilize the claim if there’s a matching job, however.
2.2 Scalable Job Tracking
The python bindings provide two scalable mechanisms for tracking jobs:
• Poll-based tracking: The Schedd can be periodically polled through the use of xquery() to get job status
information.
2.2. Scalable Job Tracking 17

• Event-based tracking: Using the job’s user log, Python can see all job events and keep an in-memory repre-
sentation of the job status.
Both poll- and event-based tracking have their strengths and weaknesses; the intrepid user can even combine both
methodologies to have extremely reliable, low-latency job status tracking.
In this module, we outline the important design considerations behind each approach and walk through examples.
2.2.1 Poll-based Tracking
Poll-based tracking involves periodically querying the schedd(s) for jobs of interest. We have covered the technical
aspects of querying the Schedd in prior tutorials. Beside the technical means of polling, important aspects to consider
are how often the poll should be performed and how much data should be retrieved.
Note: When xquery() is used, the query will cause the schedd to fork up to SCHEDD_QUERY_WORKERS
simultaneous workers. Beyond that point, queries will be handled in a non-blocking manner inside the main
condor_schedd process. Thus, the memory used by many concurrent queries can be reduced by decreasing
SCHEDD_QUERY_WORKERS.
A job tracking system should not query the Schedd more than once a minute. Aim to minimize the data returned from
the query through the use of the projection; minimize the number of jobs returned by using a query constraint. Better
yet, use the AutoCluster flag to have xquery() return a list of job summaries instead of individual jobs.
Advantages:
• A single entity can poll all condor_schedd instances in a pool; using poll(), multiple Schedds can
be queried simultaneously.
• The tracking is resilient to bugs or crashes. All tracked state is replaced at the next polling cycle.
Disadvantages:
• The amount of work to do is a function of the number of jobs in the schedd; may scale poorly once more
than 100,000 simultaneous jobs are tracked.
• Each job state transition is not seen; only snapshots of the queue in time.
• If a job disappears from the Schedd, it may be difficult to determine why (Did it finish? Was it removed?)
• Only useful for tracking jobs at the minute-level granularity.
2.2.2 Event-based Tracking
Each job in the Schedd can specify the UserLog attribute; the Schedd will atomically append a machine-parseable
event to the specified file for every state transition the job goes through. By keeping track of the events in the logs, we
can build an in-memory representation of the job queue state.
Advantages:
• No interaction with the condor_schedd process is needed to read the event logs; the job tracking
effectively places no burden on the Schedd.
• In most cases, HTCondor writes to the log synchronously after the event occurs. Hence, the latency of
receiving an update can be sub-second.
• The job tracking scales as a function of the event rate, not the total number of jobs.
• Each job state is seen, even after the job has left the queue.

Disadvantages:
• Only the local condor_schedd can be tracked; there is no mechanism to receive the event log remotely.
• Log files must be processed from the beginning. Large files can take a large amount of CPU time to
process.
• If each job writes to a separate log file, the job tracking software may have to keep an enormous number
of open file descriptors. If each job writes to the same log file, the log file may grow to many gigabytes.
• If the job tracking software misses an event (or an unknown bug causes HTCondor to fail to write the
event), then the job tracker may believe a job incorrectly is stuck in the wrong state.
At a technical level, an event log file is represented by an instance of the JobEventLog class, which is an iterator
returning instances of the JobEvent class. The following demonstrates the usage of both:
import sys
import htcondor
from htcondor import JobEventType
jel = htcondor.JobEventLog("job.log")
for event in jel.events(stop_after=60):
if event.type is JobEventType.EXECUTE:
break
else:
print("Job did not start within sixty seconds, aborting.")
sys.exit(-1)
# This (the default) waits forever for the next event.

for event in jel.events(stop_after=None):
if event.type is JobEventType.JOB_TERMINATED:
# All events have the type, cluster, proc, and timestamp attributes.
# JOB_TERMINATED events have the ReturnValue key; other event types
# will have other keys.
print("Job {0}.{1} terminated with return value {2}".format(event.cluster,
˓→event.proc, event["ReturnValue"]))
2.3 Interacting with Daemons
In this module, we’ll look at how the HTCondor python bindings can be used to interact with running daemons.
Let’s start by importing the correct modules:
>>> import htcondor
2.3.1 Configuration
The HTCondor configuration is exposed to Python in two ways:

• The local process’s configuration is available in the module-level param object.
• A remote daemon’s configuration may be queried using a RemoteParam.
The param object emulates a Python dictionary:
2.3. Interacting with Daemons 19

>>> print htcondor.param['SCHEDD_LOG'] # Prints the schedd's current log file.

/home/jovyan/condor//log/SchedLog
>>> print htcondor.param.get('TOOL_LOG') # Print None as TOOL_LOG isn't set by
˓→default.
None
>>> print htcondor.param.setdefault('TOOL_LOG', '/tmp/log') # Sets TOOL_LOG to /tmp/
˓→log.
/tmp/log
>>> print htcondor.param['TOOL_LOG'] # Prints /tmp/log, as set above.
/tmp/log
Note that assignments to param will persist only in memory; if we use reload_config() to re-read the configu-
ration files from disk, our change to TOOL_LOG disappears:
>>> print htcondor.param.get("TOOL_LOG")

/tmp/log
>>> htcondor.reload_config()
>>> print htcondor.param.get("TOOL_LOG")
None
In HTCondor, a configuration prefix may indicate that a setting is specific to that daemon. By default, the Python bind-
ing’s prefix is TOOL. If you would like to use the configuration of a different daemon, utilize the set_subsystem()
function:
>>> print htcondor.param.setdefault("TEST_FOO", "bar") # Sets the default

˓→value of TEST_FOO to bar
bar
>>> print htcondor.param.setdefault("SCHEDD.TEST_FOO", "baz") # The schedd has a
˓→special setting for TEST_FOO
baz
>>> print htcondor.param['TEST_FOO'] # Default access; should be 'bar'
bar
>>> htcondor.set_subsystem('SCHEDD') # Changes the running process to identify as a
˓→schedd.
>>> print htcondor.param['TEST_FOO'] # Since we now identify as a schedd,

˓→should use the special setting of 'baz'
baz
Between param, reload_config(), and set_subsystem(), we can explore the configuration of the local
host.
What happens if we want to test the configuration of a remote daemon? For that, we can use the RemoteParam class.
The object is first initialized from the output of the htcondor.Collector.locate() method:
>>> master_ad = htcondor.Collector().locate(htcondor.DaemonTypes.Master)

>>> print master_ad['MyAddress']
<172.17.0.2:9618?addrs=172.17.0.2-9618+[--1]-9618&noUDP&sock=378_7bb3>
>>> master_param = htcondor.RemoteParam(master_ad)
Once we have the master_param object, we can treat it like a local dictionary to access the remote daemon’s
configuration.
Note: that the param object attempts to infer type information for configuration values from the compile-time
metadata while the RemoteParam object does not:

>>> print master_param['UPDATE_INTERVAL'].__repr__() # Returns a string

'300'
>>> print htcondor.param['UPDATE_INTERVAL'].__repr__() # Returns an integer
300
In fact, we can even set the daemon’s configuration using the RemoteParam object. . . if we have permission. By
default, this is disabled for security reasons:
>>> master_param['UPDATE_INTERVAL'] = '500'

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
RuntimeError: Failed to set remote daemon parameter.
2.3.2 Logging Subsystem
The logging subsystem is available to the python bindings; this is often useful for debugging network connection
issues between the client and server:
>>> htcondor.set_subsystem("TOOL")
>>> htcondor.param['TOOL_DEBUG'] = 'D_FULLDEBUG'
>>> htcondor.param['TOOL_LOG'] = '/tmp/log'
>>> htcondor.enable_log() # Send logs to the log file (/tmp/foo)
>>> htcondor.enable_debug() # Send logs to stderr; this is ignored by the web
˓→notebook.
>>> print open("/tmp/log").read() # Print the log's contents.

12/30/16 20:06:44 Result of reading /etc/issue: \S
12/30/16 20:06:44 Result of reading /etc/redhat-release: CentOS Linux release 7.3.

˓→1611 (Core)
12/30/16 20:06:44 Using processor count: 1 processors, 1 CPUs, 0 HTs

12/30/16 20:06:44 Reading condor configuration from '/etc/condor/condor_config'
2.3.3 Sending Daemon Commands
An administrator can send administrative commands directly to the remote daemon. This is useful if you’d like a
certain daemon restarted, drained, or reconfigured.
To send a command, use the send_command() function, provide a daemon location, and provide a specific com-
mand from the DaemonCommands enumeration. For example, we can reconfigure:
>>> print master_ad['MyAddress']

<172.17.0.2:9618?addrs=172.17.0.2-9618+[--1]-9618&noUDP&sock=378_7bb3>
>>> htcondor.send_command(master_ad, htcondor.DaemonCommands.Reconfig)
>>> import time
>>> time.sleep(1) # Just to make sure the logfile has sync'd to disk
>>> log_lines = open(htcondor.param['MASTER_LOG']).readlines()
>>> print log_lines[-4:]
['12/30/16 20:07:51 Sent SIGHUP to NEGOTIATOR (pid 384)\n', '12/30/16 20:07:51 Sent
˓→SIGHUP to SCHEDD (pid 395)\n', '12/30/16 20:07:51 Sent SIGHUP to SHARED_PORT (pid
˓→380)\n', '12/30/16 20:07:51 Sent SIGHUP to STARTD (pid 413)\n']
We can also instruct the master to shut down a specific daemon:
2.3. Interacting with Daemons 21

>>> htcondor.send_command(master_ad, htcondor.DaemonCommands.DaemonOff, "SCHEDD")

>>> time.sleep(1)
>>> print log_lines[-1]
12/30/16 20:07:52 The SCHEDD (pid 395) exited with status 0
Or even turn off the whole HTCondor instance:
>>> htcondor.send_command(master_ad, htcondor.DaemonCommands.OffFast)

>>> time.sleep(1)
>>> print log_lines[-1]
12/30/16 20:07:57 The MASTER (pid 384) exited with status 0

CHAPTER 3
Module Reference
3.1 htcondor – HTCondor Reference
This page is an exhaustive reference of the API exposed by the htcondor module. It is not meant to be a tutorial for
new users but rather a helpful guide for those who already understand the basic usage of the module.
This reference covers the following:
• Common Module-Level Functions and Objects: The more commonly-used htcondor functions.
• Schedd: Interacting with the condor_schedd.
• Collector: Interacting with the condor_collector.
• Submit: Submitting to HTCondor.
• Claim: Working with HTCondor claims.
• _Param: Working with the parameter objects.
• JobEventLog: Working with user event logs.
• JobEvent: An event in a user event log.
• Esoteric Module-Level Functions: Less-commonly used htcondor functions.
• Useful Enumerations: Useful enumerations.
3.1.1 Common Module-Level Functions and Objects
htcondor.platform()
Returns the platform of HTCondor this module is running on.
htcondor.version()
Returns the version of HTCondor this module is linked against.
htcondor.reload_config()
Reload the HTCondor configuration from disk.
23
htcondor.enable_debug()
Enable debugging output from HTCondor, where output is sent to stderr. The logging level is controlled by
the TOOL_DEBUG parameter.
htcondor.enable_log()
Enable debugging output from HTCondor, where output is sent to a file. The log level is controlled by the
parameter TOOL_DEBUG, and the file used is controlled by TOOL_LOG.
htcondor.enable_debug()
Enable debugging output from HTCondor; output is sent to stderr. The logging level is controlled by the
HTCondor configuration variable TOOL_DEBUG.
htcondor.enable_log()
Enable debugging output from HTCondor; output is sent to a file.
The log level is controlled by the HTCondor configuration variable TOOL_DEBUG, and the file used is controlled
by TOOL_LOG.
htcondor.read_events(file_obj, is_xml = True)
Read and parse an HTCondor event log file.
Parameters
• file_obj – A file object corresponding to an HTCondor event log.
• is_xml (bool) – Specifies whether the event log is XML-formatted.
Returns A Python iterator which produces objects of type ClassAd.
Return type EventIterator
htcondor.poll(active_queries)
Wait on the results of multiple query iteratories.
This function returns an iterator which yields the next ready query iterator. The returned iterator stops when all
results have been consumed for all iterators.
Parameters active_queries (list[QueryIterator]) – Query iterators as returned by
xquery().
Returns An iterator producing the ready QueryIterator.
Return type BulkQueryIterator
htcondor.param
Provides dictionary-like access the HTCondor configuration.
An instance of _Param. Upon importing the htcondor module, the HTCondor configuration files are parsed
and populate this dictionary-like object.
3.1.2 Module Classes
class htcondor.Schedd
Client object for a remote condor_schedd.
__init__(location_ad=None)
Create an instance of the Schedd class.
Parameters location_ad (ClassAd) – describes the location of the remote
condor_schedd daemon, as returned by the Collector.locate() method. If
the parameter is omitted, the local condor_schedd daemon is used.
24 Chapter 3. Module Reference

transaction(flags=0, continue_txn=False)
Start a transaction with the condor_schedd.
Starting a new transaction while one is ongoing is an error unless the continue_txn flag is set.
Parameters
• flags (TransactionFlags) – Flags controlling the behavior of the transaction, de-
faulting to 0.
• continue_txn (bool) – Set to True if you would like this transaction to extend any
pre-existing transaction; defaults to False. If this is not set, starting a transaction inside
a pre-existing transaction will cause an exception to be thrown.
Returns A transaction context manager object.
query(constraint=’true’, attr_list=[], callback=None, limit=-1, opts=QueryOpts.Default)
Query the condor_schedd daemon for jobs.
Note: This returns a list of ClassAd objects, meaning all results must be buffered in memory. This may
be memory-intensive for large responses; we strongly recommend to utilize the xquery()
Parameters
• constraint (str or ExprTree) – Query constraint; only jobs matching this constraint
will be returned; defaults to 'true'.
• attr_list (list[str]) – Attributes for the condor_schedd daemon to project
along. At least the attributes in this list will be returned. The default behavior is to return
all attributes.
• callback – A callable object; if provided, it will be invoked for each ClassAd. The
return value (if note None) will be added to the returned list instead of the ad.
• limit (int) – The maximum number of ads to return; the default (-1) is to return all
ads.
• opts (QueryOpts.) – Additional flags for the query; these may affect the behavior of
the condor_schedd.
Returns ClassAds representing the matching jobs.
Return type list[ClassAd]
xquery(requirements=’true’, projection=[], limit=-1, opts=QueryOpts.Default, name=None)

Query the condor_schedd daemon for jobs.
As opposed to query(), this returns an iterator, meaning only one ad is buffered in memory at a time.
Parameters
• requirements (str or ExprTree) – provides a constraint for filtering out jobs. It
defaults to 'true'.
• projection (list[str]) – The attributes to return; an empty list (the default) signi-
fies all attributes.
• limit (int) – A limit on the number of matches to return. The default (-1) indicates
all matching jobs should be returned.
• opts (QueryOpts) – Additional flags for the query, from QueryOpts.
3.1. htcondor – HTCondor Reference 25

• name (str) – A tag name for the returned query iterator. This string will always be
returned from the QueryIterator.tag() method of the returned iterator. The default
value is the condor_schedd’s name. This tag is useful to identify different queries
when using the poll() function.
Returns An iterator for the matching job ads
Return type QueryIterator
act(action, job_spec)
Change status of job(s) in the condor_schedd daemon. The return value is a ClassAd object describing
the number of jobs changed.
This will throw an exception if no jobs are matched by the constraint.
Parameters
• action (JobAction) – The action to perform; must be of the enum JobAction.
• job_spec (list[str] or str) – The job specification. It can either be a list of job
IDs or a string specifying a constraint. Only jobs matching this description will be acted
upon.
edit(job_spec, attr, value)
Edit one or more jobs in the queue.
This will throw an exception if no jobs are matched by the job_spec constraint.
Parameters
• job_spec (list[str] or str) – The job specification. It can either be a list of job
IDs or a string specifying a constraint. Only jobs matching this description will be acted
upon.
• attr (str) – The name of the attribute to edit.
• value (str or ExprTree) – The new value of the attribute. It should be a string, which
will be converted to a ClassAd expression, or an ExprTree object. Be mindful of quoting
issues; to set the value to the string foo, one would set the value to '"foo"'
history(requirements, projection, match=1)
Fetch history records from the condor_schedd daemon.
Parameters
• requirements – Query constraint; only jobs matching this constraint will be returned;
defaults to 'true'.
• projection (list[str]) – Attributes that are to be included for each returned job.
The empty list causes all attributes to be included.
• match (int) – An limit on the number of jobs to include; the default (-1) indicates to
return all matching jobs.
Returns All matching ads in the Schedd history, with attributes according to the projection
keyword.
Return type HistoryIterator
submit(ad, count = 1, spool = false, ad_results = None)
Submit one or more jobs to the condor_schedd daemon.
This method requires the invoker to provide a ClassAd for the new job cluster; such a ClassAd contains
attributes with different names than the commands in a submit description file. As an example, the stdout
file is referred to as output in the submit description file, but Out in the ClassAd.

Hint: To generate an example ClassAd, take a sample submit description file and invoke:
condor_submit -dump <filename> [cmdfile]
Then, load the resulting contents of <filename> into Python.
Parameters
• ad (ClassAd) – The ClassAd describing the job cluster.
• count (int) – The number of jobs to submit to the job cluster. Defaults to 1.
• spool (bool) – If True, the clinent inserts the necessary attributes into the job for it
to have the input files spooled to a remote condor_schedd daemon. This parameter
is necessary for jobs submitted to a remote condor_schedd that use HTCondor file
transfer.
• ad_results (list[ClassAd]) – If set to a list, the list object will contain the job ads
resulting from the job submission. These are needed for interacting with the job spool
after submission.
Returns The newly created cluster ID.
Return type int
submitMany(cluster_ad, proc_ads, spool = false, ad_results = None)

Submit multiple jobs to the condor_schedd daemon, possibly including several distinct processes.
Parameters
• cluster_ad (ClassAd) – The base ad for the new job cluster; this is the same format
as in the submit() method.
• proc_ads (list) – A list of 2-tuples; each tuple has the format of (proc_ad,
count). For each list entry, this will result in count jobs being submitted inheriting
from both cluster_ad and proc_ad.
• spool (bool) – If True, the clinent inserts the necessary attributes into the job for it
to have the input files spooled to a remote condor_schedd daemon. This parameter
is necessary for jobs submitted to a remote condor_schedd that use HTCondor file
transfer.
• ad_results (list[ClassAd]) – If set to a list, the list object will contain the job ads
resulting from the job submission. These are needed for interacting with the job spool
after submission.
Returns The newly created cluster ID.
Return type int
spool(ad_list)
Spools the files specified in a list of job ClassAds to the condor_schedd.
Parameters ad_list (list[ClassAds]) – A list of job descriptions; typically, this is the list
filled by the ad_results argument of the submit() method call.
Raises RuntimeError – if there are any errors.
retrieve(job_spec)
Retrieve the output sandbox from one or more jobs.

Parameters job_spec (list[ClassAd]) – An expression matching the list of job output sand-
boxes to retrieve.
refreshGSIProxy(cluster, proc, filename, lifetime)
Refresh the GSI proxy of a job; the job’s proxy will be replaced the contents of the provided filename.
Note: Depending on the lifetime of the proxy in filename, the resulting lifetime may be shorter than the
desired lifetime.
Parameters
• cluster (int) – Cluster ID of the job to alter.
• proc (int) – Process ID of the job to alter.
• lifetime (int) – Indicates the desired lifetime (in seconds) of the delegated proxy. A
value of 0 specifies to not shorten the proxy lifetime. A value of -1 specifies to use the
value of configuration variable DELEGATE_JOB_GSI_CREDENTIALS_LIFETIME.
negotiate((str)accounting_name)
Begin a negotiation cycle with the remote schedd for a given user.
Note: The returned ScheddNegotiate additionally serves as a context manager, automatically de-
stroying the negotiation session when the context is left.
Parameters accounting_name (str) – Determines which user the client will start negoti-
ating with.
Returns An iterator which yields resource request ClassAds from the condor_schedd. Each
resource request represents a set of jobs that are next in queue for the schedd for this user.
Return type ScheddNegotiate
reschedule()
Send reschedule command to the schedd.
class htcondor.Collector
Client object for a remote condor_collector. The interaction with the collector broadly has three aspects:
• Locating a daemon.
• Query the collector for one or more specific ClassAds.
• Advertise a new ad to the condor_collector.
__init__(pool = None)
Create an instance of the Collector class.
Parameters pool (str or list[str]) – A host:port pair specified for the remote
collector (or a list of pairs for HA setups). If omitted, the value of configuration parameter
COLLECTOR_HOST is used.
locate(daemon_type, name)
Query the condor_collector for a particular daemon.
Parameters
• daemon_type (DaemonTypes) – The type of daemon to locate.

• name (str) – The name of daemon to locate. If not specified, it searches for the local
daemon.
Returns a minimal ClassAd of the requested daemon, sufficient only to contact the daemon;
typically, this limits to the MyAddress attribute.
Return type ClassAd
locateAll(daemon_type)
Query the condor_collector daemon for all ClassAds of a particular type. Returns a list of matching
ClassAds.
Parameters daemon_type (DaemonTypes) – The type of daemon to locate.
Returns Matching ClassAds
query(ad_type, constraint=’true’, attrs=[], statistics=”)
Query the contents of a condor_collector daemon. Returns a list of ClassAds that match the constraint
parameter.
Parameters
• ad_type (AdTypes) – The type of ClassAd to return. If not specified, the type will be
ANY_AD.
• constraint (str or ExprTree) – A constraint for the collector query; only ads match-
ing this constraint are returned. If not specified, all matching ads of the given type are
returned.
• attrs (list[str]) – A list of attributes to use for the projection. Only these attributes,
plus a few server-managed, are returned in each ClassAd.
• statistics (list[str]) – Statistics attributes to include, if they exist for the speci-
fied daemon.
Returns A list of matching ads.
advertise(ad_list, command="UPDATE_AD_GENERIC", use_tcp=True)
Advertise a list of ClassAds into the condor_collector.
Parameters
• ad_list (list[ClassAds]) – ClassAds to advertise.
• command (str) – An advertise command for the remote condor_collector. It
defaults to UPDATE_AD_GENERIC. Other commands, such as UPDATE_STARTD_AD,
may require different authorization levels with the remote daemon.
• use_tcp (bool) – When set to true, updates are sent via TCP. Defaults to True.
class htcondor.Submit
An object representing a job submit description. This uses the same submit language as condor_submit.
The submit description contains key = value pairs and implements the python dictionary protocol, including
the get, setdefault, update, keys, items, and values methods.
__init__(input = None)
Create an instance of the Submit class.
Parameters input (dict) – Key = value pairs for initializing the submit description. If
omitted, the submit class is initially empty.

expand(attr)
Expand all macros for the given attribute.
Parameters attr (str) – The name of the relevant attribute.
Returns The value of the given attribute; all macros are expanded.
Return type str
queue((object)txn, (int)count = 1, (object)ad_results = None)
Submit the current object to a remote queue.
Parameters
• txn (Transaction) – An active transaction object (see Schedd.transaction()).
• count (int) – The number of jobs to create (defaults to 1).
• ad_results – A list to receive the ClassAd resulting from this submit. As with
Schedd.submit(), this is often used to later spool the input files.
Returns The ClusterID of the submitted job(s).
Return type int
Raises RuntimeError – if the submission fails.
class htcondor.Negotiator
This class provides a query interface to the condor_negotiator; primarily, it allows one to query and set
various parameters in the fair-share accounting.
__init__(ad = None)
Create an instance of the Negotiator class.
Parameters ad (ClassAd) – A ClassAd describing the claim and the condor_negotiator
location. If omitted, the default pool negotiator is assumed.
deleteUser(user)
Delete all records of a user from the Negotiator’s fair-share accounting.
Parameters user (str) – A fully-qualified user name, i.e., USER@DOMAIN.
getPriorities([(bool)rollup = False ])
Retrieve the pool accounting information, one per entry.Returns a list of accounting ClassAds.
Parameters rollup (bool) – Set to True if accounting information, as applied to hierarchi-
cal group quotas, should be summed for groups and subgroups.
Returns A list of accounting ads, one per entity.
getResourceUsage((str)user)
Get the resources (slots) used by a specified user.
Parameters user (str) – A fully-qualified user name, USER@DOMAIN.
Returns List of ads describing the resources (slots) in use.
resetAllUsage()
Reset all usage accounting. All known user records in the negotiator are deleted.
resetUsage(user)
Reset all usage accounting of the specified user.

Parameters user (str) – A fully-qualified user name, USER@DOMAIN.

setBeginUsage(user, value)
Manually set the time that a user begins using the pool.
Parameters
• user (str) – A fully-qualified user name, USER@DOMAIN.
• value (int) – The Unix timestamp of initial usage.
setLastUsage(user, value)
Manually set the time that a user last used the pool.
Parameters
• value (int) – The Unix timestamp of last usage.
setFactor(user, factor)
Set the priority factor of a specified user.
Parameters
• factor (float) – The priority factor to be set for the user; must be greater-than or
equal-to 1.0.
setPriority(user, prio)
Set the real priority of a specified user.
Parameters
• prio (float) – The priority to be set for the user; must be greater-than 0.0.
setUsage(user, usage)
Set the accumulated usage of a specified user.
Parameters
• usage (float) – The usage, in hours, to be set for the user.
class htcondor.Startd
__init__(ad = None)
Create an instance of the Startd class.
Parameters ad (ClassAd) – A ClassAd describing the claim and the startd location. If omit-
ted, the local startd is assumed.
cancelDrainJobs(request_id = None)
Cancel a draining request.
Parameters request_id (str) – Specifies a draining request to cancel. If not specified, all
draining requests for this startd are canceled.
class htcondor.SecMan
A class, representing the internal HTCondor security state.

If a security session becomes invalid, for example, because the remote daemon restarts, reuses the same port,
and the client continues to use the session, then all future commands will fail with strange connection errors.
This is the only mechanism to invalidate in-memory sessions.
The SecMan can also behave as a context manager; when created, the object can be used to set temporary
security configurations that only last during the lifetime of the security object.
__init__()
Create a SecMan object.
invalidateAllSessions()
Invalidate all security sessions. Any future connections to a daemon will cause a new security session to
be created.
ping(ad, command=’DC_NOP’)
Perform a test authorization against a remote daemon for a given command.
Parameters
• ad (str or ClassAd) – The ClassAd of the daemon as returned by Collector.
locate(); alternately, the sinful string can be given directly as the first parameter.
• command – The DaemonCore command to try; if not given, 'DC_NOP' will be used.
Returns An ad describing the results of the test security negotiation.
Return type ClassAd
getCommandString(commandInt)
Return the string name corresponding to a given integer command.
setConfig(key, value)
Set a temporary configuration variable; this will be kept for all security sessions in this thread for as long
as the SecMan object is alive.
Parameters
• key (str) – Configuration key to set.
• value (str) – Temporary value to set.
setGSICredential(filename)
Set the GSI credential to be used for security negotiation.
Parameters filename (str) – File name of the GSI credential.
setPoolPassword(new_pass)
Set the pool password
Parameters new_pass (str) – Updated pool password to use for new security negotiations.
setTag(tag)
Set the authentication context tag for the current thread.
All security sessions negotiated with the same tag will only be utilized when that tag is active.
For example, if thread A has a tag set to Joe and thread B has a tag set to Jane, then all security sessions
negotiated for thread A will not be used for thread B.
Parameters tag (str) – New tag to set.
class htcondor.Claim
The Claim class provides access to HTCondor’s Compute-on-Demand facilities. The class represents a claim
of a remote resource; it allows the user to manually activate a claim (start a job) or release the associated
resources.

The claim comes with a finite lifetime - the lease. The lease may be extended for as long as the remote resource
(the Startd) allows.
__init__(ad)
Create a Claim object of a given remote resource. The ad provides a description of the resource, as
returned by Collector.locate().
This only stores the remote resource’s location; it is not contacted until requestCOD() is invoked.
Parameters ad (ClassAd) – Location of the Startd to claim.
requestCOD(constraint, lease_duration)
Request a claim from the condor_startd represented by this object.
On success, the Claim object will represent a valid claim on the remote startd; other methods, such as
activate() should now function.
Parameters
• constraint (str) – ClassAd expression that pecifies which slot in the startd should be
claimed. Defaults to 'true', which will result in the first slot becoming claimed.
• lease_duration (int) – Indicates how long the claim should be valid. Defaults to
-1, which indicates to lease the resource for as long as the Startd allows.
activate(ad)
Activate a claim using a given job ad.
Parameters ad – Description of the job to launch; this uses similar, but not identical attribute
names as condor_submit. See the HTCondor manual for a description of the job lan-
guage.
release(vacate_type)
Release the remote condor_startd from this claim; shut down any running job.
Parameters vacate_type (VacateTypes) – Indicates the type of vacate to perform for the
running job.
suspend()
Temporarily suspend the remote execution of the COD application. On Unix systems, this is done using
SIGSTOP.
resume()
Resume the temporarily suspended execution. On Unix systems, this is done using SIGCONT.
renew()
Renew the lease on an existing claim. The renewal should last for the value of lease_duration
provided to __init__().
deactivate()
Deactivate a claim; shuts down the currently running job, but holds onto the claim for future activation.
delegateGSIProxy(fname)
Send an X509 proxy credential to an activated claim.
Parameters fname (str) – Filename of the X509 proxy to send to the active claim.
class htcondor.ScheddNegotiate
The ScheddNegotiate class represents an ongoing negotiation session with a schedd. It is a context man-
ager, returned by the negotiate() method.
sendClaim(claim, offer, request)
Send a claim to the schedd; if possible, the schedd will activate this and run one or more jobs.

Parameters
• claim (str) – The claim ID, typically from the Capability attribute in the corre-
sponding Startd’s private ad.
• offer (ClassAd) – A description of the resource claimed (typically, the machine’s
ClassAd).
• request (ClassAd) – The resource request this claim is responding to; if not provided
(default), the Schedd will decide which job receives this resource.
disconnect()
Disconnect from this negotiation session. This can also be achieved by exiting the context.
class htcondor._Param
A dictionary-like object for the local HTCondor configuration; the keys and values of this object are the keys
and values of the HTCondor configuration.
The get, setdefault, update, keys, items, and values methods of this class have the same semantics
as a python dictionary.
Writing to a _Param object will update the in-memory HTCondor configuration.
class htcondor.JobEventLog
An iterable object (and iterable context manager) corresponding to a specific file on disk containing a user event
log.
__init__(filename)
Create an instance of the JobEventLog class.
Parameters filename – Filename of the job event log.
events(stop_after=None)
Return an iterator (self), which yields JobEvent objects. The iterator may return any number of events,
including zero, before stopping.
Parameters stop_after – Stop waiting for new events after this many seconds. If None, never
stop waiting for new events. If 0, do not wait for new events.
close()
Closes any open underlying file. Subsequent iterations on this JobEventLog object will immediately termi-
nate (will never return another JobEvent).
class htcondor.JobEvent
An immutable dictionary-like object corresponding to a particular event in the user log. All events define the
following attributes. Other type-specific attributes are keys of the dictionary. JobEvent objects support both
in operators (if "attribute" in jobEvent and for attributeName in jobEvent) and may
be passed as arguments to len.
Note: Although the attribute type is a JobEventType type, when acting as dictionary, a JobEvent object
returns types as if it were a ClassAd, so comparisons to enumerated values must use the == operator. (No
current event type has ExprTree values.)
type
Type htcondor.JobEventType
The event type.

cluster
The cluster ID.
proc
The proc ID.
timestamp
When the event was recorded.
3.1.3 Esoteric Module-Level Functions
htcondor.send_command(ad, dc, target = None)

Send a command to an HTCondor daemon specified by a location ClassAd.
Parameters
• ad (ClassAd) – Specifies the location of the daemon (typically, found by using
Collector.locate().
• dc (DaemonCommands) – A command type
• target (str) – An additional command to send to a daemon. Some commands require
additional arguments; for example, sending DaemonOff to a condor_master requires
one to specify which subsystem to turn off.
htcondor.send_alive(ad, pid = None, timeout = -1)
Send a keep alive message to an HTCondor daemon.
This is used when the python process is run as a child daemon under the condor_master.
Parameters
• ad (ClassAd) – A ClassAd specifying the location of the daemon. This ad is typically
found by using Collector.locate().
• pid (int) – The process identifier for the keep alive. The default value of None uses the
value from os.getpid().
• timeout (int) – The number of seconds that this keep alive is valid. If a new keep alive is
not received by the condor_master in time, then the process will be terminated. The default
value is controlled by configuration variable NOT_RESPONDING_TIMEOUT.
htcondor.set_subsystem(name, daemon_type = Auto)
Set the subsystem name for the object.
The subsystem is primarily used for the parsing of the HTCondor configuration file.
Parameters
• name (str) – The subsystem name.
• daemon_type (SubsystemType) – The HTCondor daemon type. The default value of
Auto infers the type from the name parameter.
htcondor.lock(file_obj, lock_type)
Take a lock on a file object using the HTCondor locking protocol (distinct from typical POSIX locks).
Parameters
• file_obj (file) – is a file object corresponding to the file which should be locked.
• lock_type (LockType) – The kind of lock to acquire.
Returns A context manager object; the lock is released when the context manager object is exited.

Return type FileLock

htcondor.log(level, msg)
Log a message using the HTCondor logging subsystem.
Parameters
• level (LogLevel) – The Log category and formatting indicator. Multiple LogLevel
enum attributes may be OR’d together.
• msg (str) – A message to log.
3.1.4 Iterator and Helper Classes
class htcondor.HistoryIterator
An iterator class for managing results of the Schedd.history() method.
next()
Returns the next available history ad.
Return type ClassAd
Raises StopIteration – when no additional ads are available.
class htcondor.QueryIterator
An iterator class for managing results of the Schedd.query() and Schedd.xquery() methods.
next(mode=BlockingMode.Blocking)
Parameters mode (BlockingMode) – The blocking mode for this call to next(); defaults
to Blocking.
Returns the next available job ad.
Return type ClassAd
Raises StopIteration – when no additional ads are available.
nextAdsNonBlocking()
Retrieve as many ads are available to the iterator object.
If no ads are available, returns an empty list. Does not throw an exception if no ads are available or the
iterator is finished.
Returns Zero-or-more job ads.
tag()
Retrieve the tag associated with this iterator; when using the poll() method, this is useful to distinguish
multiple iterators.
Returns the query’s tag.
done()
Returns True if the iterator is finished; False otherwise.
watch()
Returns an inotify-based file descriptor; if this descriptor is given to a select() instance, select
will indicate this file descriptor is ready to read whenever there are more jobs ready on the iterator.
If inotify is not available on this platform, this will return -1.
Returns A file descriptor associated with this query.

Return type int

class htcondor.BulkQueryIterator
Returned by poll(), this iterator produces a sequence of QueryIterator objects that have ads ready to be
read in a non-blocking manner.
Once there are no additional available iterators, poll() must be called again.
next()
Returns The next available QueryIterator that can be read without blocking.
Return type QueryIterator
Raises StopIteration – if no more iterators are ready.
class htcondor.FileLock
A context manager object created by the lock() function; upon exit from the context, it will release the lock.
3.1.5 Useful Enumerations
class htcondor.DaemonTypes
An enumeration of different types of daemons available to HTCondor.
Collector
Ads representing the condor_collector.
Negotiator
Ads representing the condor_negotiator.
Schedd
Ads representing the condor_schedd.
Startd
Ads representing the resources on a worker node.
HAD
Ads representing the high-availability daemons (condor_had).
Master
Ads representing the condor_master.
Generic
All other ads that are not categorized as above.
Any
Any type of daemon; useful when specifying queries where all matching daemons should be returned.
class htcondor.AdTypes
A list of different types of ads that may be kept in the condor_collector.
Any
Type representing any matching ad. Useful for queries that match everything in the collector.
Collector
Ads from the condor_collector daemon.
Generic
Generic ads, associated with no particular daemon.
Grid
Ads associated with the grid universe.

HAD
Ads produced by the condor_had.
License
License ads. These do not appear to be used by any modern HTCondor daemon.
Master
Master ads, produced by the condor_master daemon.
Negotiator
Negotiator ads, produced by the condor_negotiator daemon.
Schedd
Schedd ads, produced by the condor_schedd daemon.
Startd
Startd ads, produced by the condor_startd daemon. Represents the available slots managed by the
startd.
StartdPrivate
The “private” ads, containing the claim IDs associated with a particular slot. These require additional
authorization to read as the claim ID may be used to run jobs on the slot.
Submitter
Ads describing the submitters with available jobs to run; produced by the condor_schedd and read by
the condor_negotiator to determine which users need a new negotiation cycle.
class htcondor.JobAction
Different actions that may be performed on a job in queue.
Hold
Put a job on hold, vacating a running job if necessary. A job will stay in the hold state until explicitly acted
upon by the admin or owner.
Release
Release a job from the hold state, returning it to Idle.
Suspend
Suspend the processes of a running job (on Unix platforms, this triggers a SIGSTOP). The job’s processes
stay in memory but no longer get scheduled on the CPU.
Continue
Continue a suspended jobs (on Unix, SIGCONT). The processes in a previously suspended job will be
scheduled to get CPU time again.
Remove
Remove a job from the Schedd’s queue, cleaning it up first on the remote host (if running). This re-
quires the remote host to acknowledge it has successfully vacated the job, meaning Remove may not be
instantaneous.
RemoveX
Immediately remove a job from the schedd queue, even if it means the job is left running on the remote
resource.
Vacate
Cause a running job to be killed on the remote resource and return to idle state. With Vacate, jobs may
be given significant time to cleanly shut down.
VacateFast
Vacate a running job as quickly as possible, without providing time for the job to cleanly terminate.

class htcondor.DaemonCommands
Various state-changing commands that can be sent to to a HTCondor daemon using send_command().
DaemonOff
DaemonOffFast
DaemonOffPeaceful
DaemonsOff
DaemonsOffFast
DaemonsOffPeaceful
OffFast
OffForce
OffGraceful
OffPeaceful
Reconfig
Restart
RestartPeacful
SetForceShutdown
SetPeacefulShutdown
class htcondor.TransactionFlags
Flags affecting the characteristics of a transaction.
NonDurable
Non-durable transactions are changes that may be lost when the condor_schedd crashes.
NonDurable is used for performance, as it eliminates extra fsync() calls.
SetDirty
This marks the changed ClassAds as dirty, causing an update notification to be sent to the
condor_shadow and the condor_gridmanager, if they are managing the job.
ShouldLog
Causes any changes to the job queue to be logged in the relevant job event log.
class htcondor.QueryOpts
Flags sent to the condor_schedd during a query to alter its behavior.
Default
Queries should use all default behaviors.
AutoCluster
Instead of returning job ads, return an ad per auto-cluster.
class htcondor.BlockingMode
Controls the behavior of query iterators once they are out of data.
Blocking
Sets the iterator to block until more data is available.
NonBlocking
Sets the iterator to return immediately if additional data is not available.

class htcondor.DrainTypes
Draining policies that can be sent to a condor_startd.
Fast
Graceful
Quick
class htcondor.VacateTypes
Vacate policies that can be sent to a condor_startd.
Fast
Graceful
class htcondor.LockType
Lock policies that may be taken.
ReadLock
WriteLock
class htcondor.SubsystemType
An enumeration of known subsystem names.
Collector
Daemon
Dagman
GAHP
Job
Master
Negotiator
Schedd
Shadow
SharedPort
Startd
Starter
Submit
Tool
class htcondor.LogLevel
The log level attribute to use with log(). Note that HTCondor mixes both a class (debug, network, all) and the
header format (Timestamp, PID, NoHeader) within this enumeration.
Always
Audit
Config
DaemonCore
Error
FullDebug

Hostname
Job
Machine
Network
NoHeader
PID
Priv
Protocol
Security
Status
SubSecond
Terse
Timestamp
Verbose
class htcondor.JobEventType
The type event of a user log event; corresponds to ULogEventNumber in the C++ source.
SUBMIT
EXECUTE
EXECUTABLE_ERROR
CHECKPOINTED
JOB_EVICTED
JOB_TERMINATED
IMAGE_SIZE
SHADOW_EXCEPTION
GENERIC
JOB_ABORTED
JOB_SUSPENDED
JOB_UNSUSPENDED
JOB_HELD
JOB_RELEASED
NODE_EXECUTE
NODE_TERMINATED
POST_SCRIPT_TERMINATED
GLOBUS_SUBMIT
GLOBUS_SUBMIT_FAILED
GLOBUS_RESOURCE_UP

GLOBUS_RESOURCE_DOWN
REMOTE_ERROR
JOB_DISCONNECTED
JOB_RECONNECTED
JOB_RECONNECT_FAILED
GRID_RESOURCE_UP
GRID_RESOURCE_DOWN
GRID_SUBMIT
JOB_AD_INFORMATION
JOB_STATUS_UNKNOWN
JOB_STATUS_KNOWN
JOB_STAGE_IN
JOB_STAGE_OUT
ATTRIBUTE_UPDATE
PRESKIP
CLUSTER_SUBMIT
CLUSTER_REMOVE
FACTORY_PAUSED
FACTORY_RESUMED
NONE
FILE_TRANSFER
3.2 classad – ClassAd reference
This page is an exhaustive reference of the API exposed by the classad module. It is not meant to be a tutorial for
new users but rather a helpful guide for those who already understand the basic usage of the module.
This reference covers the following:
• Module-Level Functions: The module-level classad functions.
• ClassAd: Representation of a ClassAd.
• ExprTree: Representation of a ClassAd expression.
• Useful Enumerations: Useful enumerations.
3.2.1 Module-Level Functions
classad.quote(input)
Converts the Python string into a ClassAd string literal; this handles all the quoting rules for the ClassAd
language. For example:

>>> classad.quote('hello"world')
'"hello\\"world"'
This allows one to safely handle user-provided strings to build expressions. For example:
>>> classad.ExprTree("Foo =?= %s" % classad.quote('hello"world'))

Foo is "hello\"world"
Parameters input (str) – Input string to quote.

Returns The corresponding string literal as a Python string.
Return type str
classad.unquote(input)
Converts a ClassAd string literal, formatted as a string, back into a python string. This handles all the quoting
rules for the ClassAd language.
Parameters input (str) – Input string to unquote.
Returns The corresponding Python string for a string literal.
Return type str
classad.parseAds(input, parser=Auto)
Parse the input as a series of ClassAds.
Parameters
• input (str or file) – Serialized ClassAd input; may be a file-like object.
• parser (Parser) – Controls behavior of the ClassAd parser.
Returns An iterator that produces ClassAd.
classad.parseNext(input, parser=Auto)
Parse the next ClassAd in the input string. Advances the input to point after the consumed ClassAd.
Parameters
Returns An iterator that produces ClassAd.
classad.parseOne(input, parser=Auto)
Parse the entire input into a single ClassAd object.
In the presence of multiple ClassAds or blank lines in the input, continue to merge ClassAds together until the
entire input is consumed.
Parameters
Returns Corresponding ClassAd object.
Return type ClassAd
classad.version()
Return the version of the linked ClassAd library.
3.2. classad – ClassAd reference 43

classad.lastError()
Return the string representation of the last error to occur in the ClassAd library.
As the ClassAd language has no concept of an exception, this is the only mechanism to receive detailed error
messages from functions.
classad.Attribute(name)
Given an attribute name, construct an ExprTree object which is a reference to that attribute.
Note: This may be used to build ClassAd expressions easily from python. For example, the ClassAd expression
foo == 1 can be constructed by the python code Attribute("foo") == 1.
Parameters name (str) – Name of attribute to reference.

Returns Corresponding expression consisting of an attribute reference.
Return type ExprTree
classad.Function(name, arg1, arg2, ...)

Given function name name, and zero-or-more arguments, construct an ExprTree which is a function call
expression. The function is not evaluated.
For example, the ClassAd expression strcat("hello ", "world") can be constructed by the python
Function("strcat", "hello ", "world").
Returns Corresponding expression consisting of a function call.
classad.Literal(obj)
Convert a given python object to a ClassAd literal.
Python strings, floats, integers, and booleans have equivalent literals in the ClassAd language.
Parameters obj – Python object to convert to an expression.
Returns Corresponding expression consising of a literal.
classad.register(function, name=None)
Given the python function, register it as a function in the ClassAd language. This allows the invocation of the
python function from within a ClassAd evaluation context.
Parameters
• function – A callable object to register with the ClassAd runtime.
• name (str) – Provides an alternate name for the function within the ClassAd library. The
default, None, indicates to use the built in function name.
classad.registerLibrary(path)
Given a file system path, attempt to load it as a shared library of ClassAd functions. See the upstream docu-
mentation for configuration variable CLASSAD_USER_LIBS for more information about loadable libraries for
ClassAd functions.
Parameters path (str) – The library to load.

3.2.2 Module Classes
class classad.ClassAd
The ClassAd object is the python representation of a ClassAd. Where possible, the ClassAd attempts to
mimic a python dictionary. When attributes are referenced, they are converted to python values if possible;
otherwise, they are represented by a ExprTree object.
The ClassAd object is iterable (returning the attributes) and implements the dictionary protocol. The items,
keys, values, get, setdefault, and update methods have the same semantics as a dictionary.
__init__(ad)
Create a new ClassAd object; can be initialized via a string (which is parsed as an ad) or a dictionary-like
object.
Note: Where possible, we recommend using the dedicated parsing functions (parseOne(),
parseNext(), or parseAds()) instead of using the constructor.
Parameters ad (str or dict) – Initial values for this object.
eval(attr)
Evaluate an attribute to a python object. The result will not be an ExprTree but rather an built-in type
such as a string, integer, boolean, etc.
Parameters attr (str) – Attribute to evaluate.
Returns The Python object corresponding to the evaluated ClassAd attribute
Raises ValueError – if unable to evaluate the object.
lookup(attr)
Look up the ExprTree object associated with attribute.
No attempt will be made to convert to a Python object.
Parameters attr (str) – Attribute to evaluate.
Returns The ExprTree object referenced by attr.
printOld()
Serialize the ClassAd in the old ClassAd format.
Returns The “old ClassAd” representation of the ad.
Return type str
flatten(expression)
Given ExprTree object expression, perform a partial evaluation. All the attributes in expression and defined
in this ad are evaluated and expanded. Any constant expressions, such as 1 + 2, are evaluated; undefined
attributes are not evaluated.
Parameters expression (ExprTree) – The expression to evaluate in the context of this ad.
Returns The partially-evaluated expression.
matches(ad)
Lookup the Requirements attribute of given ad return True if the Requirements evaluate to True
in our context.
Parameters ad (ClassAd) – ClassAd whose Requirements we will evaluate.

Returns True if we satisfy ad’s requirements; False otherwise.

Return type bool
symmetricMatch(ad)
Check for two-way matching between given ad and ourselves.
Equivalent to self.matches(ad) and ad.matches(self).
Parameters ad (ClassAd) – ClassAd to check for matching.
Returns True if both ads’ requirements are satisfied.
Return type bool
externalRefs(expr)
Returns a python list of external references found in expr.
An external reference is any attribute in the expression which is not defined by the ClassAd object.
Parameters expr (ExprTree) – Expression to examine.
Returns A list of external attribute references.
Return type list[str]
internalRefs(expr)
Returns a python list of internal references found in expr.
An internal reference is any attribute in the expression which is defined by the ClassAd object.
Parameters expr (ExprTree) – Expression to examine.
Returns A list of internal attribute references.
Return type list[str]
class classad.ExprTree
The ExprTree class represents an expression in the ClassAd language.
As with typical ClassAd semantics, lazy-evaluation is used. So, the expression "foo" + 1 does not produce
an error until it is evaluated with a call to bool() or the ExprTree.eval() method.
Note: The python operators for ExprTree have been overloaded so, if e1 and e2 are ExprTree objects, then
e1 + e2 is also an :class:ExprTree object. However, Python short-circuit evaluation semantics for e1 &&
e2 cause e1 to be evaluated. In order to get the “logical and” of the two expressions without evaluating, use
e1.and_(e2). Similarly, e1.or_(e2) results in the “logical or”.
__init__(expr)
Parse the string expr as a ClassAd expression.
Parameters expr (str) – Initial expression, serialized as a string.
__str__()
Represent and return the ClassAd expression as a string.
Returns Expression represented as a string.
Return type str
__int__()
Converts expression to an integer (evaluating as necessary).
__float__()
Converts expression to a float (evaluating as necessary).

and_(expr2)
Return a new expression, formed by self && expr2.
Parameters expr2 (ExprTree) – Right-hand-side expression to “and”
Returns A new expression, defined to be self && expr2.
or_(expr2)
Return a new expression, formed by self || expr2.
Parameters expr2 (ExprTree) – Right-hand-side expression to “or”
Returns A new expression, defined to be self || expr2.
is_(expr2)
Logical comparison using the “meta-equals” operator.
Parameters expr2 (ExprTree) – Right-hand-side expression to =?= operator.
Returns A new expression, formed by self =?= expr2.
isnt_(expr2)
Logical comparison using the “meta-not-equals” operator.
Parameters expr2 (ExprTree) – Right-hand-side expression to =!= operator.
Returns A new expression, formed by self =!= expr2.
sameAs(expr2)
Returns True if given ExprTree is same as this one.
Parameters expr2 (ExprTree) – Expression to compare against.
Returns True if and only if expr2 is equivalent to this object.
Return type bool
eval()
Evaluate the expression and return as a ClassAd value, typically a Python object.
Returns The evaluated expression as a Python object.
3.2.3 Useful Enumerations
class classad.Parser
Controls the behavior of the ClassAd parser.
Auto
The parser should automatically determine the ClassAd representation.
Old
The parser should only accept the old ClassAd format.
New
The parser should only accept the new ClassAd format.

3.2.4 Deprecated Functions
The functions in this section are deprecated; new code should not use them and existing code should be rewritten to
use their replacements.
classad.parse(input)
This function is deprecated.
Parse input, in the new ClassAd format, into a ClassAd object.
Parameters input (str or file) – A string-like object or a file pointer.
Return type ClassAd
classad.parseOld(input)
This function is deprecated.
Parse input, in the old ClassAd format, into a ClassAd object.
Parameters input (str or file) – A string-like object or a file pointer.
Return type ClassAd
genindex

Python Module Index
c
classad (Unix, Windows, Mac OS X), 42
h
htcondor (Unix, Windows, Mac OS X), 23
49
50 Python Module Index

Index
Symbols CHECKPOINTED (htcondor.JobEventType attribute), 41

_Param (class in htcondor), 34 Claim (class in htcondor), 32
__float__() (classad.ExprTree method), 46 ClassAd (class in classad), 45
__init__() (classad.ClassAd method), 45 classad (module), 42
__init__() (classad.ExprTree method), 46 close() (htcondor.JobEventLog method), 34
__init__() (htcondor.Claim method), 33 cluster (htcondor.JobEvent attribute), 34
__init__() (htcondor.Collector method), 28 CLUSTER_REMOVE (htcondor.JobEventType attribute),
__init__() (htcondor.JobEventLog method), 34 42
__init__() (htcondor.Negotiator method), 30 CLUSTER_SUBMIT (htcondor.JobEventType attribute),
__init__() (htcondor.Schedd method), 24 42
__init__() (htcondor.SecMan method), 32 Collector (class in htcondor), 28
__init__() (htcondor.Startd method), 31 Collector (htcondor.AdTypes attribute), 37
__init__() (htcondor.Submit method), 29 Collector (htcondor.DaemonTypes attribute), 37
__int__() (classad.ExprTree method), 46 Collector (htcondor.SubsystemType attribute), 40
__str__() (classad.ExprTree method), 46 Config (htcondor.LogLevel attribute), 40
Continue (htcondor.JobAction attribute), 38
A
D
act() (htcondor.Schedd method), 26
activate() (htcondor.Claim method), 33 Daemon (htcondor.SubsystemType attribute), 40
AdTypes (class in htcondor), 37 DaemonCommands (class in htcondor), 38
advertise() (htcondor.Collector method), 29 DaemonCore (htcondor.LogLevel attribute), 40
Always (htcondor.LogLevel attribute), 40 DaemonOff (htcondor.DaemonCommands attribute), 39
and_() (classad.ExprTree method), 46 DaemonOffFast (htcondor.DaemonCommands at-
Any (htcondor.AdTypes attribute), 37 tribute), 39
Any (htcondor.DaemonTypes attribute), 37 DaemonOffPeaceful (htcondor.DaemonCommands
Attribute() (in module classad), 44 attribute), 39
ATTRIBUTE_UPDATE (htcondor.JobEventType at- DaemonsOff (htcondor.DaemonCommands attribute),
tribute), 42 39
Audit (htcondor.LogLevel attribute), 40 DaemonsOffFast (htcondor.DaemonCommands at-
Auto (classad.Parser attribute), 47 tribute), 39
AutoCluster (htcondor.QueryOpts attribute), 39 DaemonsOffPeaceful (htcondor.DaemonCommands
attribute), 39
B DaemonTypes (class in htcondor), 37
Dagman (htcondor.SubsystemType attribute), 40
Blocking (htcondor.BlockingMode attribute), 39
deactivate() (htcondor.Claim method), 33
BlockingMode (class in htcondor), 39
Default (htcondor.QueryOpts attribute), 39
BulkQueryIterator (class in htcondor), 37
delegateGSIProxy() (htcondor.Claim method), 33
deleteUser() (htcondor.Negotiator method), 30
C disconnect() (htcondor.ScheddNegotiate method),
cancelDrainJobs() (htcondor.Startd method), 31 34
51
done() (htcondor.QueryIterator method), 36 GRID_RESOURCE_DOWN (htcondor.JobEventType at-

DrainTypes (class in htcondor), 39 tribute), 42
GRID_RESOURCE_UP (htcondor.JobEventType at-
E tribute), 42
edit() (htcondor.Schedd method), 26 GRID_SUBMIT (htcondor.JobEventType attribute), 42
enable_debug() (in module htcondor), 23, 24
enable_log() (in module htcondor), 24 H
Error (htcondor.LogLevel attribute), 40 HAD (htcondor.AdTypes attribute), 37
eval() (classad.ClassAd method), 45 HAD (htcondor.DaemonTypes attribute), 37
eval() (classad.ExprTree method), 47 history() (htcondor.Schedd method), 26
events() (htcondor.JobEventLog method), 34 HistoryIterator (class in htcondor), 36
EXECUTABLE_ERROR (htcondor.JobEventType at- Hold (htcondor.JobAction attribute), 38
tribute), 41 Hostname (htcondor.LogLevel attribute), 40
EXECUTE (htcondor.JobEventType attribute), 41 htcondor (module), 23
expand() (htcondor.Submit method), 29
ExprTree (class in classad), 46 I
externalRefs() (classad.ClassAd method), 46 IMAGE_SIZE (htcondor.JobEventType attribute), 41
internalRefs() (classad.ClassAd method), 46
F invalidateAllSessions() (htcondor.SecMan
FACTORY_PAUSED (htcondor.JobEventType attribute), method), 32
42 is_() (classad.ExprTree method), 47
FACTORY_RESUMED (htcondor.JobEventType at- isnt_() (classad.ExprTree method), 47
tribute), 42
Fast (htcondor.DrainTypes attribute), 40 J
Fast (htcondor.VacateTypes attribute), 40 Job (htcondor.LogLevel attribute), 41
FILE_TRANSFER (htcondor.JobEventType attribute), Job (htcondor.SubsystemType attribute), 40
42 JOB_ABORTED (htcondor.JobEventType attribute), 41
FileLock (class in htcondor), 37 JOB_AD_INFORMATION (htcondor.JobEventType at-
flatten() (classad.ClassAd method), 45 tribute), 42
FullDebug (htcondor.LogLevel attribute), 40 JOB_DISCONNECTED (htcondor.JobEventType at-
Function() (in module classad), 44 tribute), 42
JOB_EVICTED (htcondor.JobEventType attribute), 41
G JOB_HELD (htcondor.JobEventType attribute), 41
GAHP (htcondor.SubsystemType attribute), 40 JOB_RECONNECT_FAILED (htcondor.JobEventType
Generic (htcondor.AdTypes attribute), 37 attribute), 42
Generic (htcondor.DaemonTypes attribute), 37 JOB_RECONNECTED (htcondor.JobEventType at-
GENERIC (htcondor.JobEventType attribute), 41 tribute), 42
getCommandString() (htcondor.SecMan method), JOB_RELEASED (htcondor.JobEventType attribute), 41
32 JOB_STAGE_IN (htcondor.JobEventType attribute), 42
getPriorities() (htcondor.Negotiator method), 30 JOB_STAGE_OUT (htcondor.JobEventType attribute),
getResourceUsage() (htcondor.Negotiator 42
method), 30 JOB_STATUS_KNOWN (htcondor.JobEventType at-
GLOBUS_RESOURCE_DOWN (htcondor.JobEventType tribute), 42
attribute), 41 JOB_STATUS_UNKNOWN (htcondor.JobEventType at-
GLOBUS_RESOURCE_UP (htcondor.JobEventType attribute), 42
tribute), 41 JOB_SUSPENDED (htcondor.JobEventType attribute),
GLOBUS_SUBMIT (htcondor.JobEventType attribute), 41
41 JOB_TERMINATED (htcondor.JobEventType attribute),
GLOBUS_SUBMIT_FAILED (htcondor.JobEventType 41
attribute), 41 JOB_UNSUSPENDED (htcondor.JobEventType at-
Graceful (htcondor.DrainTypes attribute), 40 tribute), 41
Graceful (htcondor.VacateTypes attribute), 40 JobAction (class in htcondor), 38
Grid (htcondor.AdTypes attribute), 37 JobEvent (class in htcondor), 34
JobEventLog (class in htcondor), 34
52 Index
JobEventType (class in htcondor), 41 parse() (in module classad), 48

parseAds() (in module classad), 43
L parseNext() (in module classad), 43
lastError() (in module classad), 43 parseOld() (in module classad), 48
License (htcondor.AdTypes attribute), 38 parseOne() (in module classad), 43
Literal() (in module classad), 44 Parser (class in classad), 47
locate() (htcondor.Collector method), 28 PID (htcondor.LogLevel attribute), 41
locateAll() (htcondor.Collector method), 29 ping() (htcondor.SecMan method), 32
lock() (in module htcondor), 35 platform() (in module htcondor), 23
LockType (class in htcondor), 40 poll() (in module htcondor), 24
log() (in module htcondor), 36 POST_SCRIPT_TERMINATED (htcon-
LogLevel (class in htcondor), 40 dor.JobEventType attribute), 41
lookup() (classad.ClassAd method), 45 PRESKIP (htcondor.JobEventType attribute), 42
printOld() (classad.ClassAd method), 45
M Priv (htcondor.LogLevel attribute), 41
Machine (htcondor.LogLevel attribute), 41 proc (htcondor.JobEvent attribute), 35
Master (htcondor.AdTypes attribute), 38 Protocol (htcondor.LogLevel attribute), 41
Master (htcondor.DaemonTypes attribute), 37
Master (htcondor.SubsystemType attribute), 40 Q
matches() (classad.ClassAd method), 45 query() (htcondor.Collector method), 29
query() (htcondor.Schedd method), 25
N QueryIterator (class in htcondor), 36
negotiate() (htcondor.Schedd method), 28 QueryOpts (class in htcondor), 39
Negotiator (class in htcondor), 30 queue() (htcondor.Submit method), 30
Negotiator (htcondor.AdTypes attribute), 38 Quick (htcondor.DrainTypes attribute), 40
Negotiator (htcondor.DaemonTypes attribute), 37 quote() (in module classad), 42
Negotiator (htcondor.SubsystemType attribute), 40
Network (htcondor.LogLevel attribute), 41 R
New (classad.Parser attribute), 47 read_events() (in module htcondor), 24
next() (htcondor.BulkQueryIterator method), 37 ReadLock (htcondor.LockType attribute), 40
next() (htcondor.HistoryIterator method), 36 Reconfig (htcondor.DaemonCommands attribute), 39
next() (htcondor.QueryIterator method), 36 refreshGSIProxy() (htcondor.Schedd method), 28
nextAdsNonBlocking() (htcondor.QueryIterator register() (in module classad), 44
method), 36 registerLibrary() (in module classad), 44
NODE_EXECUTE (htcondor.JobEventType attribute), 41
Release (htcondor.JobAction attribute), 38
NODE_TERMINATED (htcondor.JobEventType at-
release() (htcondor.Claim method), 33
tribute), 41 reload_config() (in module htcondor), 23
NoHeader (htcondor.LogLevel attribute), 41 REMOTE_ERROR (htcondor.JobEventType attribute), 42
NonBlocking (htcondor.BlockingMode attribute), 39Remove (htcondor.JobAction attribute), 38
NonDurable (htcondor.TransactionFlags attribute), 39
RemoveX (htcondor.JobAction attribute), 38
NONE (htcondor.JobEventType attribute), 42 renew() (htcondor.Claim method), 33
requestCOD() (htcondor.Claim method), 33
O reschedule() (htcondor.Schedd method), 28
OffFast (htcondor.DaemonCommands attribute), 39 resetAllUsage() (htcondor.Negotiator method), 30
OffForce (htcondor.DaemonCommands attribute), 39 resetUsage() (htcondor.Negotiator method), 30
OffGraceful (htcondor.DaemonCommands attribute), Restart (htcondor.DaemonCommands attribute), 39
39 RestartPeacful (htcondor.DaemonCommands at-
OffPeaceful (htcondor.DaemonCommands attribute), tribute), 39
39 resume() (htcondor.Claim method), 33
Old (classad.Parser attribute), 47 retrieve() (htcondor.Schedd method), 27
or_() (classad.ExprTree method), 47
S
P sameAs() (classad.ExprTree method), 47
param (in module htcondor), 24 Schedd (class in htcondor), 24
Index 53
Schedd (htcondor.AdTypes attribute), 38 Timestamp (htcondor.LogLevel attribute), 41

Schedd (htcondor.DaemonTypes attribute), 37 Tool (htcondor.SubsystemType attribute), 40
Schedd (htcondor.SubsystemType attribute), 40 transaction() (htcondor.Schedd method), 24
ScheddNegotiate (class in htcondor), 33 TransactionFlags (class in htcondor), 39
SecMan (class in htcondor), 31 type (htcondor.JobEvent attribute), 34
Security (htcondor.LogLevel attribute), 41
send_alive() (in module htcondor), 35 U
send_command() (in module htcondor), 35 unquote() (in module classad), 43
sendClaim() (htcondor.ScheddNegotiate method), 33
set_subsystem() (in module htcondor), 35 V
setBeginUsage() (htcondor.Negotiator method), 31 Vacate (htcondor.JobAction attribute), 38
setConfig() (htcondor.SecMan method), 32 VacateFast (htcondor.JobAction attribute), 38
SetDirty (htcondor.TransactionFlags attribute), 39 VacateTypes (class in htcondor), 40
setFactor() (htcondor.Negotiator method), 31 Verbose (htcondor.LogLevel attribute), 41
SetForceShutdown (htcondor.DaemonCommands version() (in module classad), 43
attribute), 39 version() (in module htcondor), 23
setGSICredential() (htcondor.SecMan method),
32 W
setLastUsage() (htcondor.Negotiator method), 31
watch() (htcondor.QueryIterator method), 36
SetPeacefulShutdown (htcon-
WriteLock (htcondor.LockType attribute), 40
dor.DaemonCommands attribute), 39
setPoolPassword() (htcondor.SecMan method), 32 X
setPriority() (htcondor.Negotiator method), 31
setTag() (htcondor.SecMan method), 32 xquery() (htcondor.Schedd method), 25
setUsage() (htcondor.Negotiator method), 31
Shadow (htcondor.SubsystemType attribute), 40
SHADOW_EXCEPTION (htcondor.JobEventType at-
tribute), 41
SharedPort (htcondor.SubsystemType attribute), 40
ShouldLog (htcondor.TransactionFlags attribute), 39
spool() (htcondor.Schedd method), 27
Startd (class in htcondor), 31
Startd (htcondor.AdTypes attribute), 38
Startd (htcondor.DaemonTypes attribute), 37
Startd (htcondor.SubsystemType attribute), 40
StartdPrivate (htcondor.AdTypes attribute), 38
Starter (htcondor.SubsystemType attribute), 40
Status (htcondor.LogLevel attribute), 41
Submit (class in htcondor), 29
SUBMIT (htcondor.JobEventType attribute), 41
Submit (htcondor.SubsystemType attribute), 40
submit() (htcondor.Schedd method), 26
submitMany() (htcondor.Schedd method), 27
Submitter (htcondor.AdTypes attribute), 38
SubSecond (htcondor.LogLevel attribute), 41
SubsystemType (class in htcondor), 40
Suspend (htcondor.JobAction attribute), 38
suspend() (htcondor.Claim method), 33
symmetricMatch() (classad.ClassAd method), 46
T
tag() (htcondor.QueryIterator method), 36
Terse (htcondor.LogLevel attribute), 41
timestamp (htcondor.JobEvent attribute), 35
54 Index

Htcondor Python

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Htcondor Python

Загружено:

Авторское право:

Доступные форматы

HTCondor Python Documentation

Mar 01, 2019

Python Module Index 49

1.1 ClassAds Introduction

>>> arith_expr = classad.ExprTree("1 + 4")

ClassAd arithemetic expression: 1 + 4 (of type <class 'classad.ExprTree'>)

>>> print classad.ExprTree("foo").eval()

And the “old ClassAd” format:

4 Chapter 1. Introductory Tutorial

>>> ad1 = classad.ClassAd("""[

>>> print ad1["a"]

>>> print ad1.eval("d")

>>> ad1["g"] = classad.ExprTree("b + 5")

1.1. ClassAds Introduction 5

1.2 HTCondor Introduction

Let’s start interacting with the HTCondor daemons!

>>> import htcondor

>>> coll = htcondor.Collector() # Create the object representing the collector.

6 Chapter 1. Introductory Tutorial

>>> coll.query(htcondor.AdTypes.Schedd, projection=["Name", "MyAddress",

[[ DaemonCoreDutyCycle = 1.439361064858868E-05; Name = "jovyan@eb4f00c8f1ca"; MyType

Some notes on the above:

Let’s try our hand at querying the schedd!

>>> schedd = htcondor.Schedd()

1.2. HTCondor Introduction 7

>>> sub = htcondor.Submit()

>>> for job in schedd.xquery(projection=['ClusterId', 'ProcId', 'JobStatus']):

[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 1;

[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 1;

[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 1;

[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 1;

[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 1;

[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 1;

[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 1;

[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 1;

[ TargetType = "Machine"; MyType = "Job"; ServerTime = 1482811230; JobStatus = 1;

>>> for job in schedd.xquery(requirements = 'ProcId >= 5', projection=['ProcId']):

8 Chapter 1. Introductory Tutorial

(continued from previous page)

>>> print coll.query(htcondor.AdTypes.Startd, projection=['Name', 'Status', 'Activity

1.3 Submitting and Managing Jobs

>>> import htcondor

1.3.1 Submitting Jobs

We will submit jobs utilizing the dedicated Submit object.

1.3. Submitting and Managing Jobs 9

>>> sub = htcondor.Submit({"foo": "1", "bar": "2", "baz": "$(foo)"})

>>> sub = htcondor.Submit({"executable": "/bin/sleep", "arguments": "5m"})

To go from a submit object to job in a schedd, one must do three things:

>>> schedd = htcondor.Schedd() # Create a schedd object using default

1.3.2 Managing Jobs

10 Chapter 1. Introductory Tutorial

>>> with schedd.transaction() as txn:

>>> for job in schedd.xquery(requirements="ClusterId == %d" % clusterId, projection=[

... print "%d: foo=%s, job_status = %d" % (job.get("ProcId"), job.get("foo",

... print "%d: foo=%s, job_status = %d" % (job.get("ProcId"), job.get("foo",

1.3. Submitting and Managing Jobs 11

12 Chapter 1. Introductory Tutorial

2.1 Advanced Schedd Interaction

2.1.1 Job and History Querying

>>> schedd = htcondor.Schedd()

>>> print sum(1 for _ in schedd.xquery(projection=["ProcID"], requirements=

Querying many Schedds

This is sub-optimal for two reasons:

>>> print master_param['UPDATE_INTERVAL'].repr() # Returns a string