8/7/2014

performance - Python string formatting: % vs. .format - Stack Overflow

sign up

Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no
registration required.

log in

tour

help

careers 2.0

Take the 2-minute tour

Python string formatting: % vs. .format

Python 2.6 introduced the string.format() method with a slightly different syntax from the existing %
operator. Which is better and for what situations?
1. The following uses each method and has the same outcome, so what is the difference?
#!/usr/bin/python
sub1 = "python string!"
sub2 = "an arg"
a = "i am a %s"%sub1
b = "i am a {0}".format(sub1)
c = "with %(kwarg)s!"%{'kwarg':sub2}
d = "with {kwarg}!".format(kwarg=sub2)
print a
print b
print c
print d
2. Furthermore when does string formatting occur in python? for example if my logging level is set to
HIGH will I still take a hit for performing the following % operation? And if so, is there a way to avoid
this?
log.debug("some debug info: %s" % some_info)
python

performance

logging

string-formatting

edited Aug 1 at 17:20

Gabriel
2,009 11 39

asked Feb 22 '11 at 18:46

NorthIsUp
2,372 5 13 21

similar to stackoverflow.com/questions/3691975/ carl Feb 22 '11 at 18:50

@S.Lott, the point is that there is is non zero work going on, if a logging statement is not printing I want zero
work done for the string formatting. NorthIsUp Feb 23 '11 at 18:58

add a comment

8 Answers
To answer your first question... .format just seems more sophisticated in many ways. You can do stuff
like re-use arguments, which you can't do with %. An annoying thing about % is also how it can either
take a variable or a tuple. You'd think the following would always work:
"hi there %s" % name
yet, if name happens to be (1, 2, 3), it will throw a TypeError. To guarantee that it always prints,
you'd need to do
"hi there %s" % (name,)

# supply the single argument as a single-item tuple

which is just ugly. .format doesn't have those issues. Also in the second example you gave, the
.format example is much cleaner looking.
Why would you not use it?
not knowing about it (me before reading this)
having to be compatible with Python 2.5

http://stackoverflow.com/questions/5082452/python-string-formatting-vs-format

1/6

8/7/2014

performance - Python string formatting: % vs. .format - Stack Overflow

To answer your second question, string formatting happens at the same time as any other operation when the string formatting expression is evaluated. And Python, not being a lazy language, evaluates
expressions before calling functions, so in your log.debug example, the expression "some debug
info: %s"%some_infowill first evaluate to, e.g. "some debug info: roflcopters are active",
then that string will be passed to log.debug().
edited Feb 8 '12 at 13:45
Community
1

answered Feb 22 '11 at 18:49

Claudiu
58.5k 55 228 396

47 what about "%(a)s, %(a)s" % {'a':'test'} ted Aug 23 '12 at 9:53

40 Note that you will waste time for log.debug("something: %s" % x) but not for
log.debug("something: %s", x) The string formatting will be handled in the method and you won't get
the performance hit if it won't be logged. As always, Python anticipates your needs =) darkfeline Dec 14
'12 at 23:13

11 ted: thats a worse-looking hack to do the same as '{0}, {0}'.format('test'). flying sheep Jan 30
'13 at 20:43

10 The point is: The one recurring argument that the new syntax allows reordering of items is a moot point:
You can do the same with the old syntax. Most people do not know that this is actually already defined in
the Ansi C99 Std! Check out a recent copy of man sprintf and learn about the $ notation inside %
placeholders cfi Feb 20 '13 at 12:42

@cfi: If you mean something like, printf("%2$d", 1, 3) to print out "3", that's specified in POSIX, not
C99. The very man page you referenced notes, "The C99 standard does not include the style using '$'".
Thanatos Mar 7 '13 at 23:55

show 7 more comments

Something that the modulo operator ( % ) can't do, afaik:

tu = (12,45,22222,103,6)
print '{0} {2} {1} {2} {3} {2} {4} {2}'.format(*tu)
result
12 22222 45 22222 103 22222 6 22222
Very useful.
Another point: format() being a function, it can be used as argument in other functions:
li = [12,45,78,784,2,69,1254,4785,984]
print map('the number is {}'.format,li)
print
from datetime import datetime,timedelta
once_upon_a_time = datetime(2010, 7, 1, 12, 0, 0)
delta = timedelta(days=13, hours=8, minutes=20)
gen =(once_upon_a_time +x*delta for x in xrange(20))
print '\n'.join(map('{:%Y-%m-%d %H:%M:%S}'.format, gen))
result

http://stackoverflow.com/questions/5082452/python-string-formatting-vs-format

2/6

8/7/2014

performance - Python string formatting: % vs. .format - Stack Overflow

['the number is 12', 'the number is 45', 'the number is 78', 'the number is 784', 'the number is 2', 'the number is 69'
2010-07-01 12:00:00
2010-07-14 20:20:00
2010-07-28 04:40:00
2010-08-10 13:00:00
2010-08-23 21:20:00
2010-09-06 05:40:00
2010-09-19 14:00:00
2010-10-02 22:20:00
2010-10-16 06:40:00
2010-10-29 15:00:00
2010-11-11 23:20:00
2010-11-25 07:40:00
2010-12-08 16:00:00
2010-12-22 00:20:00
2011-01-04 08:40:00
2011-01-17 17:00:00
2011-01-31 01:20:00
2011-02-13 09:40:00
2011-02-26 18:00:00
2011-03-12 02:20:00

edited Jul 18 at 7:17

todofixthis
4,206 4 24 50

answered Jun 13 '11 at 20:20

eyquem
9,842 1 13 25

You can use old style formatting in map just as easily as format.
map('some_format_string_%s'.__mod__, some_iterable) agf Nov 28 '12 at 5:49
This is wrong. See other answers, and have a look at the original definition for the old syntax. Most people do
not know that this is actually already defined in the Ansi C99 Std! Check out a recent copy of man sprintf
and learn about the $ notation inside % placeholders. So even if Python would not support this feature of
the original syntax, why introduce a new syntax? Why not just extend the interpreter to be able to
understand all of what's in the C99 std? cfi Feb 20 '13 at 12:44
@cfi: please prove you are right by rewriting the example above in C99 MarcH Feb 15 at 14:11

@MarcH: printf("%2$s %1$s\n", "One", "Two"); compiled with gcc -std=c99 test.c -o test,
the output is Two One. But I stand corrected: It is actually a POSIX extension and not C. I cannot find it
again in the C/C++ standard, where I thought I'd seen it. The code works even with 'c90' std flag. sprintf
man page. This does not list it, but allows libs to implement a superset. My original argument is still valid,
replacing C with Posix cfi Feb 15 at 15:18
My first comment here, does not apply to this answer. I regret the phrasing. In Python we cannot use the
modulo operator % for reordering placeholders. I'd still like to not delete that first comment for the sake of
comment consistency here. I apologize for having vented my anger here. It is directed against the often made
statement that the old syntax per se would not allow this. Instead of creating a completely new syntax we
could have introduced the std Posix extensions. We could have both. cfi Feb 15 at 15:25

show 1 more comment

Assuming you're using python's logging module, you can pass the string formatting arguments as
arguments to the .debug() method rather than doing the formatting yourself:
log.debug("some debug info: %s", some_info)
which avoids doing the formatting unless the logger actually logs something.
answered Feb 22 '11 at 19:21
Wooble
35.3k 8 51 76

This is some useful info that I just learned now. It's a pity it doesn't have it's own question as it seems
separate to the main question. Pity the OP didn't split his question in two separate questions. snth Nov 14
'12 at 7:36

You can use dict formatting like this: log.debug("some debug info: %(this)s and %(that)s",
dict(this='Tom', that='Jerry')) However, you can't use the new style .format() syntax here, not
even in Python 3.3, which is a shame. Cito Nov 25 '12 at 17:00

@Cito: See this: plumberjack.blogspot.co.uk/2010/10/ Vinay Sajip Jan 30 '13 at 19:56

The primary benefit of this is not performance (doing the string interpolation will be quick compared to
whatever you're doing with the output from logging, e.g displaying in a terminal, saving to disk) It is that if you
have a logging aggregator, it can tell you "you got 12 instances of this error message", even if they all had
different 'some_info' values. If the string formatting is done before passing the string to log.debug, then this is
impossible. The aggregator can only say "you had 12 different log messages" Jonathan Hartley Oct 10 '13
at 8:04

Oh! I see. I took your meaning backwards. Beg your pardon. Yep, I could imagine an aggregator that tried to
do that, but it seems silly. Hugs! Jonathan Hartley Oct 10 '13 at 12:26

http://stackoverflow.com/questions/5082452/python-string-formatting-vs-format

3/6

8/7/2014

performance - Python string formatting: % vs. .format - Stack Overflow

show 7 more comments

Also, PEP 3101 proposes the replacement of the % operator with the new, advanced string formatting in
python 3, where it would be the default.
answered Aug 1 '11 at 3:01
BrainStorm
1,008 7 17

+1 for referencing the PEP Yuri Prezument Aug 29 '12 at 8:54

Untrue: "Backwards compatibility can be maintained by leaving the existing mechanisms in place."; of
course, .format won't replace % string formatting. Tobias Jan 28 '13 at 23:32

add a comment

But please be careful, just now I've discovered one issue when trying to replace all % with .format in
existing code: '{}'.format(unicode_string) will try to encode unicode_string and will
probably fail
Just look at this python interactive session log:
Python 2.7.2 (default, Aug 27 2012, 19:52:55)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2
; s=''
; u=u''
;s
'\xd0\xb9'
;u
u'\u0439'
so s is just a string (called 'byte array' in python3) and u is a unicode string (called 'string' in python3)
; '%s' % s
'\xd0\xb9'
; '%s' % u
u'\u0439'
when you give a unicode object as a parameter to % operator it will produce a unicode string even if
original string wasn't unicode
; '{}'.format(s)
'\xd0\xb9'
; '{}'.format(u)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u0439' in position 0: ordinal not in range(256)
but .format function will raise UnicodeEncodeError
; u'{}'.format(s)
u'\xd0\xb9'
; u'{}'.format(u)
u'\u0439'
and it will work with unicode argument fine only if original string was a unicode
; '{}'.format(u'i')
'i'
or if argument string can be converted to a string (so called 'byte array')
answered Sep 3 '12 at 18:15
khrf
510 3 10

There is simply no reason to change working code unless the additional features of the new format method
are really needed ... Tobias Jan 28 '13 at 22:51
absolutely agree with you, Tobias, but sometimes it's needed when upgrading to newer versions of Python
khrf Jan 30 '13 at 13:17

For instance? AFAIK, it has never been needed; I don't consider it likely that the % string interpolation
would ever go away. Tobias Jan 31 '13 at 13:45

http://stackoverflow.com/questions/5082452/python-string-formatting-vs-format

4/6

8/7/2014

performance - Python string formatting: % vs. .format - Stack Overflow

1

I consider .format() function safer than % for strings. Often I see beginners' mistakes like this "p1=%s
p2=%d" % "abc", 2 or "p1=%s p2=%s" % (tuple_p1_p2,). You might think it's the coder's fault but I
think it's just weird faulty syntax that looks nice for the quicky-scriptie but is bad for production code. khrf
Jan 6 at 15:07
But I don't like the syntax of .format(), I'd be happier with good old %s, %02d like "p1=%s
p2=%02d".format("abc", 2). I blame those who invented and approved the curly braces formatting that
needs you to escape them like {{}} and looks ugly imho. khrf Jan 6 at 15:15

show 2 more comments

Using timeit I get that format is 1.17x faster (not much) than using the % operator. Here is my test:
>>>timeit( "%d and %.1f" % (4,2.2) )
1000000 loops, best of 3: 1.1 us per loop
>>>timeit( "{} and {}".format(4,2.2) )
1000000 loops, best of 3: 940 ns per loop
edited Apr 23 at 12:56
plok
315 2 12

answered Dec 23 '13 at 15:44

Cristian Garcia
805 2 14

The correct test for format is: timeit("{0} y {1:1}".format(4,2.2)) Mario Csar Apr 13 at 14:51

add a comment

"%" gives much better performance than "format" from my test.

"format" runs twice slower than "%"
edited Oct 16 '13 at 21:04

answered Jun 13 '11 at 18:43

lcltj
389 1 7

21 Instead, str.format gives more functionalities (especially type-specialized formatting e.g. '{0:%Y-%m%d}'.format(datetime.datetime.utcnow())). Performance cannot be the absolute requirement of all
jobs. Use the right tool for the job. minhee Sep 18 '11 at 17:25

11 "Premature optimization is the root of all evil" or so Donald Knuth once said... YatharthROCK Oct 17 '12
at 13:07

Sticking with a well-known formatting scheme (as long as it suits the needs, which it does in the vast
majority of cases), and which is twice as fast, is no "premature optimization" but simply reasonable. BTW,
the % operator allows to reuse printf knowledge; dictionary interpolation is a very simple extension of
the principle. Tobias Jan 28 '13 at 23:03

From my test there is also a huge difference between Python3 and Python 2.7. Where % is much more
efficient than format() in Python 3. The code that I used can be found here:
github.com/rasbt/python_efficiency_tweaks/blob/master/test_code/ and
github.com/rasbt/python_efficiency_tweaks/blob/master/test_code/ Sebastian Raschka Jan 24 at
14:15

add a comment

As I discovered today, the old way of formatting strings via % doesn't support Decimal, Python's
module for decimal fixed point & floating point arithmetics, out of the box.
Example (using Python 3.3.5):
#!/usr/bin/env python3
from decimal import *
getcontext().prec = 50
d = Decimal('3.12375239e-24') # no magic number, I rather produced it by banging my head on my keyboard
print('%.50f' % d)
print('{0:.50f}'.format(d))
Output:
0.00000000000000000000000312375239000000009907464850
0.00000000000000000000000312375239000000000000000000
There surely might be work-arounds but you still might consider using the format() method right
away.
answered May 13 at 17:10
balu
743 2 8 17

http://stackoverflow.com/questions/5082452/python-string-formatting-vs-format

5/6

8/7/2014

performance - Python string formatting: % vs. .format - Stack Overflow

add a comment

Not the answer you're looking for? Browse other questions tagged python
performance logging

string-formatting or ask your own question.

http://stackoverflow.com/questions/5082452/python-string-formatting-vs-format

6/6