Execution model

4. Execution model¶
4.1. Structure of a program¶
A Python program is constructed from code blocks.
A block is a piece of Python program text that is executed as a unit.
The following are blocks: a module, a function body, and a class definition.
Each command typed interactively is a block. A script file (a file given as
standard input to the interpreter or specified as a command line argument to the
interpreter) is a code block. A script command (a command specified on the
interpreter command line with the -c
option) is a code block.
A module run as a top level script (as module __main__
) from the command
line using a -m
argument is also a code block. The string
argument passed to the built-in functions eval()
and exec()
is a
code block.
A code block is executed in an execution frame. A frame contains some administrative information (used for debugging) and determines where and how execution continues after the code block’s execution has completed.
4.2. Naming and binding¶
4.2.1. Binding of names¶
Names refer to objects. Names are introduced by name binding operations.
The following constructs bind names:
formal parameters to functions,
class definitions,
function definitions,
assignment expressions,
targets that are identifiers if occurring in an assignment:
import
statements.type
statements.
The import
statement of the form from ... import *
binds all
names defined in the imported module, except those beginning with an underscore.
This form may only be used at the module level.
A target occurring in a del
statement is also considered bound for
this purpose (though the actual semantics are to unbind the name).
Each assignment or import statement occurs within a block defined by a class or function definition or at the module level (the top-level code block).
If a name is bound in a block, it is a local variable of that block, unless
declared as nonlocal
or global
. If a name is bound at
the module level, it is a global variable. (The variables of the module code
block are local and global.) If a variable is used in a code block but not
defined there, it is a free variable.
Each occurrence of a name in the program text refers to the binding of that name established by the following name resolution rules.
4.2.2. Resolution of names¶
A scope defines the visibility of a name within a block. If a local variable is defined in a block, its scope includes that block. If the definition occurs in a function block, the scope extends to any blocks contained within the defining one, unless a contained block introduces a different binding for the name.
When a name is used in a code block, it is resolved using the nearest enclosing scope. The set of all such scopes visible to a code block is called the block’s environment.
When a name is not found at all, a NameError
exception is raised.
If the current scope is a function scope, and the name refers to a local
variable that has not yet been bound to a value at the point where the name is
used, an UnboundLocalError
exception is raised.
UnboundLocalError
is a subclass of NameError
.
If a name binding operation occurs anywhere within a code block, all uses of the name within the block are treated as references to the current block. This can lead to errors when a name is used within a block before it is bound. This rule is subtle. Python lacks declarations and allows name binding operations to occur anywhere within a code block. The local variables of a code block can be determined by scanning the entire text of the block for name binding operations. See the FAQ entry on UnboundLocalError for examples.
If the global
statement occurs within a block, all uses of the names
specified in the statement refer to the bindings of those names in the top-level
namespace. Names are resolved in the top-level namespace by searching the
global namespace, i.e. the namespace of the module containing the code block,
and the builtins namespace, the namespace of the module builtins
. The
global namespace is searched first. If the names are not found there, the
builtins namespace is searched next. If the names are also not found in the
builtins namespace, new variables are created in the global namespace.
The global statement must precede all uses of the listed names.
The global
statement has the same scope as a name binding operation
in the same block. If the nearest enclosing scope for a free variable contains
a global statement, the free variable is treated as a global.
The nonlocal
statement causes corresponding names to refer
to previously bound variables in the nearest enclosing function scope.
SyntaxError
is raised at compile time if the given name does not
exist in any enclosing function scope. Type parameters
cannot be rebound with the nonlocal
statement.
The namespace for a module is automatically created the first time a module is
imported. The main module for a script is always called __main__
.
Class definition blocks and arguments to exec()
and eval()
are
special in the context of name resolution.
A class definition is an executable statement that may use and define names.
These references follow the normal rules for name resolution with an exception
that unbound local variables are looked up in the global namespace.
The namespace of the class definition becomes the attribute dictionary of
the class. The scope of names defined in a class block is limited to the
class block; it does not extend to the code blocks of methods. This includes
comprehensions and generator expressions, but it does not include
annotation scopes,
which have access to their enclosing class scopes.
This means that the following will fail:
class A:
a = 42
b = list(a + i for i in range(10))
However, the following will succeed:
class A:
type Alias = Nested
class Nested: pass
print(A.Alias.__value__) # <type 'A.Nested'>
4.2.3. Annotation scopes¶
Annotations, type parameter lists
and type
statements
introduce annotation scopes, which behave mostly like function scopes,
but with some exceptions discussed below.
Annotation scopes are used in the following contexts:
Type parameter lists for generic type aliases.
Type parameter lists for generic functions. A generic function’s annotations are executed within the annotation scope, but its defaults and decorators are not.
Type parameter lists for generic classes. A generic class’s base classes and keyword arguments are executed within the annotation scope, but its decorators are not.
The bounds, constraints, and default values for type parameters (lazily evaluated).
The value of type aliases (lazily evaluated).
Annotation scopes differ from function scopes in the following ways:
Annotation scopes have access to their enclosing class namespace. If an annotation scope is immediately within a class scope, or within another annotation scope that is immediately within a class scope, the code in the annotation scope can use names defined in the class scope as if it were executed directly within the class body. This contrasts with regular functions defined within classes, which cannot access names defined in the class scope.
Expressions in annotation scopes cannot contain
yield
,yield from
,await
, or:=
expressions. (These expressions are allowed in other scopes contained within the annotation scope.)Names defined in annotation scopes cannot be rebound with
nonlocal
statements in inner scopes. This includes only type parameters, as no other syntactic elements that can appear within annotation scopes can introduce new names.While annotation scopes have an internal name, that name is not reflected in the qualified name of objects defined within the scope. Instead, the
__qualname__
of such objects is as if the object were defined in the enclosing scope.
Added in version 3.12: Annotation scopes were introduced in Python 3.12 as part of PEP 695.
Changed in version 3.13: Annotation scopes are also used for type parameter defaults, as introduced by PEP 696.
4.2.4. Lazy evaluation¶
Most annotation scopes are lazily evaluated. This includes annotations,
the values of type aliases created through the type
statement, and
the bounds, constraints, and default values of type
variables created through the type parameter syntax.
This means that they are not evaluated when the type alias or type variable is
created, or when the object carrying annotations is created. Instead, they
are only evaluated when necessary, for example when the __value__
attribute on a type alias is accessed.
Example:
>>> type Alias = 1/0
>>> Alias.__value__
Traceback (most recent call last):
...
ZeroDivisionError: division by zero
>>> def func[T: 1/0](): pass
>>> T = func.__type_params__[0]
>>> T.__bound__
Traceback (most recent call last):
...
ZeroDivisionError: division by zero
Here the exception is raised only when the __value__
attribute
of the type alias or the __bound__
attribute of the type variable
is accessed.
This behavior is primarily useful for references to types that have not yet been defined when the type alias or type variable is created. For example, lazy evaluation enables creation of mutually recursive type aliases:
from typing import Literal
type SimpleExpr = int | Parenthesized
type Parenthesized = tuple[Literal["("], Expr, Literal[")"]]
type Expr = SimpleExpr | tuple[SimpleExpr, Literal["+", "-"], Expr]
Lazily evaluated values are evaluated in annotation scope, which means that names that appear inside the lazily evaluated value are looked up as if they were used in the immediately enclosing scope.
Added in version 3.12.
4.2.5. Builtins and restricted execution¶
CPython implementation detail: Users should not touch __builtins__
; it is strictly an implementation
detail. Users wanting to override values in the builtins namespace should
import
the builtins
module and modify its
attributes appropriately.
The builtins namespace associated with the execution of a code block
is actually found by looking up the name __builtins__
in its
global namespace; this should be a dictionary or a module (in the
latter case the module’s dictionary is used). By default, when in the
__main__
module, __builtins__
is the built-in module
builtins
; when in any other module, __builtins__
is an
alias for the dictionary of the builtins
module itself.
4.2.6. Interaction with dynamic features¶
Name resolution of free variables occurs at runtime, not at compile time. This means that the following code will print 42:
i = 10
def f():
print(i)
i = 42
f()
The eval()
and exec()
functions do not have access to the full
environment for resolving names. Names may be resolved in the local and global
namespaces of the caller. Free variables are not resolved in the nearest
enclosing namespace, but in the global namespace. [1] The exec()
and
eval()
functions have optional arguments to override the global and local
namespace. If only one namespace is specified, it is used for both.
4.3. Exceptions¶
Exceptions are a means of breaking out of the normal flow of control of a code block in order to handle errors or other exceptional conditions. An exception is raised at the point where the error is detected; it may be handled by the surrounding code block or by any code block that directly or indirectly invoked the code block where the error occurred.
The Python interpreter raises an exception when it detects a run-time error
(such as division by zero). A Python program can also explicitly raise an
exception with the raise
statement. Exception handlers are specified
with the try
… except
statement. The finally
clause of such a statement can be used to specify cleanup code which does not
handle the exception, but is executed whether an exception occurred or not in
the preceding code.
Python uses the “termination” model of error handling: an exception handler can find out what happened and continue execution at an outer level, but it cannot repair the cause of the error and retry the failing operation (except by re-entering the offending piece of code from the top).
When an exception is not handled at all, the interpreter terminates execution of
the program, or returns to its interactive main loop. In either case, it prints
a stack traceback, except when the exception is SystemExit
.
Exceptions are identified by class instances. The except
clause is
selected depending on the class of the instance: it must reference the class of
the instance or a non-virtual base class thereof.
The instance can be received by the handler and can carry additional information
about the exceptional condition.
Note
Exception messages are not part of the Python API. Their contents may change from one version of Python to the next without warning and should not be relied on by code which will run under multiple versions of the interpreter.
See also the description of the try
statement in section The try statement
and raise
statement in section The raise statement.
4.4. Runtime Components¶
4.4.1. General Computing Model¶
Python’s execution model does not operate in a vacuum. It runs on a host machine and through that host’s runtime environment, including its operating system (OS), if there is one. When a program runs, the conceptual layers of how it runs on the host look something like this:
host machineprocess (global resources)thread (runs machine code)
Each process represents a program running on the host. Think of each process itself as the data part of its program. Think of the process’ threads as the execution part of the program. This distinction will be important to understand the conceptual Python runtime.
The process, as the data part, is the execution context in which the program runs. It mostly consists of the set of resources assigned to the program by the host, including memory, signals, file handles, sockets, and environment variables.
Processes are isolated and independent from one another. (The same is true for hosts.) The host manages the process’ access to its assigned resources, in addition to coordinating between processes.
Each thread represents the actual execution of the program’s machine code, running relative to the resources assigned to the program’s process. It’s strictly up to the host how and when that execution takes place.
From the point of view of Python, a program always starts with exactly one thread. However, the program may grow to run in multiple simultaneous threads. Not all hosts support multiple threads per process, but most do. Unlike processes, threads in a process are not isolated and independent from one another. Specifically, all threads in a process share all of the process’ resources.
The fundamental point of threads is that each one does run independently, at the same time as the others. That may be only conceptually at the same time (“concurrently”) or physically (“in parallel”). Either way, the threads effectively run at a non-synchronized rate.
Note
That non-synchronized rate means none of the process’ memory is guaranteed to stay consistent for the code running in any given thread. Thus multi-threaded programs must take care to coordinate access to intentionally shared resources. Likewise, they must take care to be absolutely diligent about not accessing any other resources in multiple threads; otherwise two threads running at the same time might accidentally interfere with each other’s use of some shared data. All this is true for both Python programs and the Python runtime.
The cost of this broad, unstructured requirement is the tradeoff for the kind of raw concurrency that threads provide. The alternative to the required discipline generally means dealing with non-deterministic bugs and data corruption.
4.4.2. Python Runtime Model¶
The same conceptual layers apply to each Python program, with some extra data layers specific to Python:
host machineprocess (global resources)Python global runtime (state)Python interpreter (state)thread (runs Python bytecode and “C-API”)Python thread state
At the conceptual level: when a Python program starts, it looks exactly like that diagram, with one of each. The runtime may grow to include multiple interpreters, and each interpreter may grow to include multiple thread states.
Note
A Python implementation won’t necessarily implement the runtime
layers distinctly or even concretely. The only exception is places
where distinct layers are directly specified or exposed to users,
like through the threading
module.
Note
The initial interpreter is typically called the “main” interpreter. Some Python implementations, like CPython, assign special roles to the main interpreter.
Likewise, the host thread where the runtime was initialized is known as the “main” thread. It may be different from the process’ initial thread, though they are often the same. In some cases “main thread” may be even more specific and refer to the initial thread state. A Python runtime might assign specific responsibilities to the main thread, such as handling signals.
As a whole, the Python runtime consists of the global runtime state, interpreters, and thread states. The runtime ensures all that state stays consistent over its lifetime, particularly when used with multiple host threads.
The global runtime, at the conceptual level, is just a set of interpreters. While those interpreters are otherwise isolated and independent from one another, they may share some data or other resources. The runtime is responsible for managing these global resources safely. The actual nature and management of these resources is implementation-specific. Ultimately, the external utility of the global runtime is limited to managing interpreters.
In contrast, an “interpreter” is conceptually what we would normally think of as the (full-featured) “Python runtime”. When machine code executing in a host thread interacts with the Python runtime, it calls into Python in the context of a specific interpreter.
Note
The term “interpreter” here is not the same as the “bytecode interpreter”, which is what regularly runs in threads, executing compiled Python code.
In an ideal world, “Python runtime” would refer to what we currently call “interpreter”. However, it’s been called “interpreter” at least since introduced in 1997 (CPython:a027efa5b).
Each interpreter completely encapsulates all of the non-process-global,
non-thread-specific state needed for the Python runtime to work.
Notably, the interpreter’s state persists between uses. It includes
fundamental data like sys.modules
. The runtime ensures
multiple threads using the same interpreter will safely
share it between them.
A Python implementation may support using multiple interpreters at the
same time in the same process. They are independent and isolated from
one another. For example, each interpreter has its own
sys.modules
.
For thread-specific runtime state, each interpreter has a set of thread states, which it manages, in the same way the global runtime contains a set of interpreters. It can have thread states for as many host threads as it needs. It may even have multiple thread states for the same host thread, though that isn’t as common.
Each thread state, conceptually, has all the thread-specific runtime data an interpreter needs to operate in one host thread. The thread state includes the current raised exception and the thread’s Python call stack. It may include other thread-specific resources.
Note
The term “Python thread” can sometimes refer to a thread state, but
normally it means a thread created using the threading
module.
Each thread state, over its lifetime, is always tied to exactly one interpreter and exactly one host thread. It will only ever be used in that thread and with that interpreter.
Multiple thread states may be tied to the same host thread, whether for different interpreters or even the same interpreter. However, for any given host thread, only one of the thread states tied to it can be used by the thread at a time.
Thread states are isolated and independent from one another and don’t share any data, except for possibly sharing an interpreter and objects or other resources belonging to that interpreter.
Once a program is running, new Python threads can be created using the
threading
module (on platforms and Python implementations that
support threads). Additional processes can be created using the
os
, subprocess
, and multiprocessing
modules.
Interpreters can be created and used with the
interpreters
module. Coroutines (async) can
be run using asyncio
in each interpreter, typically only
in a single thread (often the main thread).
Footnotes


ossaudiodev
— Access to OSS-compatible audio devices¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
The last version of Python that provided the ossaudiodev
module was
Python 3.12.


— NNTP protocol client

nntplib
— NNTP protocol client¶
Deprecated since version 3.11, removed in version 3.13.
This module is no longer part of the Python standard library. It was removed in Python 3.13 after being deprecated in Python 3.11. The removal was decided in PEP 594.
The last version of Python that provided the nntplib
module was
Python 3.12.


Exceptions

Exceptions¶
Source code: Lib/asyncio/exceptions.py
- exception asyncio.TimeoutError¶
A deprecated alias of
TimeoutError
, raised when the operation has exceeded the given deadline.Changed in version 3.11: This class was made an alias of
TimeoutError
.
- exception asyncio.CancelledError¶
The operation has been cancelled.
This exception can be caught to perform custom operations when asyncio Tasks are cancelled. In almost all situations the exception must be re-raised.
Changed in version 3.8:
CancelledError
is now a subclass ofBaseException
rather thanException
.
- exception asyncio.InvalidStateError¶
Invalid internal state of
Task
orFuture
.Can be raised in situations like setting a result value for a Future object that already has a result value set.
- exception asyncio.SendfileNotAvailableError¶
The “sendfile” syscall is not available for the given socket or file type.
A subclass of
RuntimeError
.
- exception asyncio.IncompleteReadError¶
The requested read operation did not complete fully.
Raised by the asyncio stream APIs.
This exception is a subclass of
EOFError
.
- exception asyncio.LimitOverrunError¶
Reached the buffer size limit while looking for a separator.
Raised by the asyncio stream APIs.
- consumed¶
The total number of to be consumed bytes.


MemoryView objects

MemoryView objects¶
A memoryview
object exposes the C level buffer interface as a Python object which can then be passed around like
any other object.
-
PyTypeObject PyMemoryView_Type¶
- Part of the Stable ABI.
This instance of
PyTypeObject
represents the Python memoryview type. This is the same object asmemoryview
in the Python layer.
-
PyObject *PyMemoryView_FromObject(PyObject *obj)¶
- Return value: New reference. Part of the Stable ABI.
Create a memoryview object from an object that provides the buffer interface. If obj supports writable buffer exports, the memoryview object will be read/write, otherwise it may be either read-only or read/write at the discretion of the exporter.
-
PyBUF_READ¶
- Part of the Stable ABI since version 3.11.
Flag to request a readonly buffer.
-
PyBUF_WRITE¶
- Part of the Stable ABI since version 3.11.
Flag to request a writable buffer.
-
PyObject *PyMemoryView_FromMemory(char *mem, Py_ssize_t size, int flags)¶
- Return value: New reference. Part of the Stable ABI since version 3.7.
Create a memoryview object using mem as the underlying buffer. flags can be one of
PyBUF_READ
orPyBUF_WRITE
.Added in version 3.3.
-
PyObject *PyMemoryView_FromBuffer(const Py_buffer *view)¶
- Return value: New reference. Part of the Stable ABI since version 3.11.
Create a memoryview object wrapping the given buffer structure view. For simple byte buffers,
PyMemoryView_FromMemory()
is the preferred function.
-
PyObject *PyMemoryView_GetContiguous(PyObject *obj, int buffertype, char order)¶
- Return value: New reference. Part of the Stable ABI.
Create a memoryview object to a contiguous chunk of memory (in either ‘C’ or ‘F’ortran order) from an object that defines the buffer interface. If memory is contiguous, the memoryview object points to the original memory. Otherwise, a copy is made and the memoryview points to a new bytes object.
buffertype can be one of
PyBUF_READ
orPyBUF_WRITE
.
-
int PyMemoryView_Check(PyObject *obj)¶
Return true if the object obj is a memoryview object. It is not currently allowed to create subclasses of
memoryview
. This function always succeeds.
-
Py_buffer *PyMemoryView_GET_BUFFER(PyObject *mview)¶
Return a pointer to the memoryview’s private copy of the exporter’s buffer. mview must be a memoryview instance; this macro doesn’t check its type, you must do it yourself or you will risk crashes.
-
PyObject *PyMemoryView_GET_BASE(PyObject *mview)¶
Return either a pointer to the exporting object that the memoryview is based on or
NULL
if the memoryview has been created by one of the functionsPyMemoryView_FromMemory()
orPyMemoryView_FromBuffer()
. mview must be a memoryview instance.


: Miscellaneous utilities

email.utils
: Miscellaneous utilities¶
Source code: Lib/email/utils.py
There are a couple of useful utilities provided in the email.utils
module:
- email.utils.localtime(dt=None)¶
Return local time as an aware datetime object. If called without arguments, return current time. Otherwise dt argument should be a
datetime
instance, and it is converted to the local time zone according to the system time zone database. If dt is naive (that is,dt.tzinfo
isNone
), it is assumed to be in local time.Added in version 3.3.
Deprecated since version 3.12, removed in version 3.14: The isdst parameter.
- email.utils.make_msgid(idstring=None, domain=None)¶
Returns a string suitable for an RFC 2822-compliant Message-ID header. Optional idstring if given, is a string used to strengthen the uniqueness of the message id. Optional domain if given provides the portion of the msgid after the ‘@’. The default is the local hostname. It is not normally necessary to override this default, but may be useful certain cases, such as a constructing distributed system that uses a consistent domain name across multiple hosts.
Changed in version 3.2: Added the domain keyword.
The remaining functions are part of the legacy (Compat32
) email API. There
is no need to directly use these with the new API, since the parsing and
formatting they provide is done automatically by the header parsing machinery
of the new API.
- email.utils.quote(str)¶
Return a new string with backslashes in str replaced by two backslashes, and double quotes replaced by backslash-double quote.
- email.utils.unquote(str)¶
Return a new string which is an unquoted version of str. If str ends and begins with double quotes, they are stripped off. Likewise if str ends and begins with angle brackets, they are stripped off.
- email.utils.parseaddr(address, *, strict=True)¶
Parse address – which should be the value of some address-containing field such as To or Cc – into its constituent realname and email address parts. Returns a tuple of that information, unless the parse fails, in which case a 2-tuple of
('', '')
is returned.If strict is true, use a strict parser which rejects malformed inputs.
Changed in version 3.13: Add strict optional parameter and reject malformed inputs by default.
- email.utils.formataddr(pair, charset='utf-8')¶
The inverse of
parseaddr()
, this takes a 2-tuple of the form(realname, email_address)
and returns the string value suitable for a To or Cc header. If the first element of pair is false, then the second element is returned unmodified.Optional charset is the character set that will be used in the RFC 2047 encoding of the
realname
if therealname
contains non-ASCII characters. Can be an instance ofstr
or aCharset
. Defaults toutf-8
.Changed in version 3.3: Added the charset option.
- email.utils.getaddresses(fieldvalues, *, strict=True)¶
This method returns a list of 2-tuples of the form returned by
parseaddr()
. fieldvalues is a sequence of header field values as might be returned byMessage.get_all
.If strict is true, use a strict parser which rejects malformed inputs.
Here’s a simple example that gets all the recipients of a message:
from email.utils import getaddresses tos = msg.get_all('to', []) ccs = msg.get_all('cc', []) resent_tos = msg.get_all('resent-to', []) resent_ccs = msg.get_all('resent-cc', []) all_recipients = getaddresses(tos + ccs + resent_tos + resent_ccs)
Changed in version 3.13: Add strict optional parameter and reject malformed inputs by default.
- email.utils.parsedate(date)¶
Attempts to parse a date according to the rules in RFC 2822. however, some mailers don’t follow that format as specified, so
parsedate()
tries to guess correctly in such cases. date is a string containing an RFC 2822 date, such as"Mon, 20 Nov 1995 19:12:08 -0500"
. If it succeeds in parsing the date,parsedate()
returns a 9-tuple that can be passed directly totime.mktime()
; otherwiseNone
will be returned. Note that indexes 6, 7, and 8 of the result tuple are not usable.
- email.utils.parsedate_tz(date)¶
Performs the same function as
parsedate()
, but returns eitherNone
or a 10-tuple; the first 9 elements make up a tuple that can be passed directly totime.mktime()
, and the tenth is the offset of the date’s timezone from UTC (which is the official term for Greenwich Mean Time) [1]. If the input string has no timezone, the last element of the tuple returned is0
, which represents UTC. Note that indexes 6, 7, and 8 of the result tuple are not usable.
- email.utils.parsedate_to_datetime(date)¶
The inverse of
format_datetime()
. Performs the same function asparsedate()
, but on success returns adatetime
; otherwiseValueError
is raised if date contains an invalid value such as an hour greater than 23 or a timezone offset not between -24 and 24 hours. If the input date has a timezone of-0000
, thedatetime
will be a naivedatetime
, and if the date is conforming to the RFCs it will represent a time in UTC but with no indication of the actual source timezone of the message the date comes from. If the input date has any other valid timezone offset, thedatetime
will be an awaredatetime
with the corresponding atimezone
tzinfo
.Added in version 3.3.
- email.utils.mktime_tz(tuple)¶
Turn a 10-tuple as returned by
parsedate_tz()
into a UTC timestamp (seconds since the Epoch). If the timezone item in the tuple isNone
, assume local time.
- email.utils.formatdate(timeval=None, localtime=False, usegmt=False)¶
Returns a date string as per RFC 2822, e.g.:
Fri, 09 Nov 2001 01:08:47 -0000
Optional timeval if given is a floating-point time value as accepted by
time.gmtime()
andtime.localtime()
, otherwise the current time is used.Optional localtime is a flag that when
True
, interprets timeval, and returns a date relative to the local timezone instead of UTC, properly taking daylight savings time into account. The default isFalse
meaning UTC is used.Optional usegmt is a flag that when
True
, outputs a date string with the timezone as an ascii stringGMT
, rather than a numeric-0000
. This is needed for some protocols (such as HTTP). This only applies when localtime isFalse
. The default isFalse
.
- email.utils.format_datetime(dt, usegmt=False)¶
Like
formatdate
, but the input is adatetime
instance. If it is a naive datetime, it is assumed to be “UTC with no information about the source timezone”, and the conventional-0000
is used for the timezone. If it is an awaredatetime
, then the numeric timezone offset is used. If it is an aware timezone with offset zero, then usegmt may be set toTrue
, in which case the stringGMT
is used instead of the numeric timezone offset. This provides a way to generate standards conformant HTTP date headers.Added in version 3.3.
- email.utils.encode_rfc2231(s, charset=None, language=None)¶
Encode the string s according to RFC 2231. Optional charset and language, if given is the character set name and language name to use. If neither is given, s is returned as-is. If charset is given but language is not, the string is encoded using the empty string for language.
- email.utils.collapse_rfc2231_value(value, errors='replace', fallback_charset='us-ascii')¶
When a header parameter is encoded in RFC 2231 format,
Message.get_param
may return a 3-tuple containing the character set, language, and value.collapse_rfc2231_value()
turns this into a unicode string. Optional errors is passed to the errors argument ofstr
’sencode()
method; it defaults to'replace'
. Optional fallback_charset specifies the character set to use if the one in the RFC 2231 header is not known by Python; it defaults to'us-ascii'
.For convenience, if the value passed to
collapse_rfc2231_value()
is not a tuple, it should be a string and it is returned unquoted.
- email.utils.decode_params(params)¶
Decode parameters list according to RFC 2231. params is a sequence of 2-tuples containing elements of the form
(content-type, string-value)
.
Footnotes


Support for Perf Maps

Support for Perf Maps¶
On supported platforms (as of this writing, only Linux), the runtime can take
advantage of perf map files to make Python functions visible to an external
profiling tool (such as perf).
A running process may create a file in the /tmp
directory, which contains entries
that can map a section of executable code to a name. This interface is described in the
documentation of the Linux Perf tool.
In Python, these helper APIs can be used by libraries and features that rely on generating machine code on the fly.
Note that holding an attached thread state is not required for these APIs.
-
int PyUnstable_PerfMapState_Init(void)¶
- This is Unstable API. It may change without warning in minor releases.
Open the
/tmp/perf-$pid.map
file, unless it’s already opened, and create a lock to ensure thread-safe writes to the file (provided the writes are done throughPyUnstable_WritePerfMapEntry()
). Normally, there’s no need to call this explicitly; just usePyUnstable_WritePerfMapEntry()
and it will initialize the state on first call.Returns
0
on success,-1
on failure to create/open the perf map file, or-2
on failure to create a lock. Checkerrno
for more information about the cause of a failure.
-
int PyUnstable_WritePerfMapEntry(const void *code_addr, unsigned int code_size, const char *entry_name)¶
- This is Unstable API. It may change without warning in minor releases.
Write one single entry to the
/tmp/perf-$pid.map
file. This function is thread safe. Here is what an example entry looks like:# address size name 7f3529fcf759 b py::bar:/run/t.py
Will call
PyUnstable_PerfMapState_Init()
before writing the entry, if the perf map file is not already opened. Returns0
on success, or the same error codes asPyUnstable_PerfMapState_Init()
on failure.
-
void PyUnstable_PerfMapState_Fini(void)¶
- This is Unstable API. It may change without warning in minor releases.
Close the perf map file opened by
PyUnstable_PerfMapState_Init()
. This is called by the runtime itself during interpreter shut-down. In general, there shouldn’t be a reason to explicitly call this, except to handle specific scenarios such as forking.


PyTime C API

PyTime C API¶
Added in version 3.13.
The clock C API provides access to system clocks.
It is similar to the Python time
module.
For C API related to the datetime
module, see DateTime Objects.
Types¶
-
type PyTime_t¶
A timestamp or duration in nanoseconds, represented as a signed 64-bit integer.
The reference point for timestamps depends on the clock used. For example,
PyTime_Time()
returns timestamps relative to the UNIX epoch.The supported range is around [-292.3 years; +292.3 years]. Using the Unix epoch (January 1st, 1970) as reference, the supported date range is around [1677-09-21; 2262-04-11]. The exact limits are exposed as constants:
Clock Functions¶
The following functions take a pointer to a PyTime_t that they set to the value of a particular clock. Details of each clock are given in the documentation of the corresponding Python function.
The functions return 0
on success, or -1
(with an exception set)
on failure.
On integer overflow, they set the PyExc_OverflowError
exception and
set *result
to the value clamped to the [PyTime_MIN; PyTime_MAX]
range.
(On current systems, integer overflows are likely caused by misconfigured
system time.)
As any other C API (unless otherwise specified), the functions must be called with an attached thread state.
-
int PyTime_Monotonic(PyTime_t *result)¶
Read the monotonic clock. See
time.monotonic()
for important details on this clock.
-
int PyTime_PerfCounter(PyTime_t *result)¶
Read the performance counter. See
time.perf_counter()
for important details on this clock.
-
int PyTime_Time(PyTime_t *result)¶
Read the “wall clock” time. See
time.time()
for details important on this clock.
Raw Clock Functions¶
Similar to clock functions, but don’t set an exception on error and don’t require the caller to have an attached thread state.
On success, the functions return 0
.
On failure, they set *result
to 0
and return -1
, without setting
an exception. To get the cause of the error, attach a thread state,
and call the regular (non-Raw
) function. Note that the regular function may succeed after
the Raw
one failed.
-
int PyTime_MonotonicRaw(PyTime_t *result)¶
Similar to
PyTime_Monotonic()
, but don’t set an exception on error and don’t require an attached thread state.
-
int PyTime_PerfCounterRaw(PyTime_t *result)¶
Similar to
PyTime_PerfCounter()
, but don’t set an exception on error and don’t require an attached thread state.
-
int PyTime_TimeRaw(PyTime_t *result)¶
Similar to
PyTime_Time()
, but don’t set an exception on error and don’t require an attached thread state.


5. Building C and C++ Extensions on Windows¶
This chapter briefly explains how to create a Windows extension module for Python using Microsoft Visual C++, and follows with more detailed background information on how it works. The explanatory material is useful for both the Windows programmer learning to build Python extensions and the Unix programmer interested in producing software which can be successfully built on both Unix and Windows.
Module authors are encouraged to use the distutils approach for building extension modules, instead of the one described in this section. You will still need the C compiler that was used to build Python; typically Microsoft Visual C++.
Note
This chapter mentions a number of filenames that include an encoded Python
version number. These filenames are represented with the version number shown
as XY
; in practice, 'X'
will be the major version number and 'Y'
will be the minor version number of the Python release you’re working with. For
example, if you are using Python 2.2.1, XY
will actually be 22
.
5.1. A Cookbook Approach¶
There are two approaches to building extension modules on Windows, just as there
are on Unix: use the setuptools
package to control the build process, or
do things manually. The setuptools approach works well for most extensions;
documentation on using setuptools
to build and package extension modules
is available in Building C and C++ Extensions with setuptools. If you find you really need to do
things manually, it may be instructive to study the project file for the
winsound standard library module.
5.2. Differences Between Unix and Windows¶
Unix and Windows use completely different paradigms for run-time loading of code. Before you try to build a module that can be dynamically loaded, be aware of how your system works.
In Unix, a shared object (.so
) file contains code to be used by the
program, and also the names of functions and data that it expects to find in the
program. When the file is joined to the program, all references to those
functions and data in the file’s code are changed to point to the actual
locations in the program where the functions and data are placed in memory.
This is basically a link operation.
In Windows, a dynamic-link library (.dll
) file has no dangling
references. Instead, an access to functions or data goes through a lookup
table. So the DLL code does not have to be fixed up at runtime to refer to the
program’s memory; instead, the code already uses the DLL’s lookup table, and the
lookup table is modified at runtime to point to the functions and data.
In Unix, there is only one type of library file (.a
) which contains code
from several object files (.o
). During the link step to create a shared
object file (.so
), the linker may find that it doesn’t know where an
identifier is defined. The linker will look for it in the object files in the
libraries; if it finds it, it will include all the code from that object file.
In Windows, there are two types of library, a static library and an import
library (both called .lib
). A static library is like a Unix .a
file; it contains code to be included as necessary. An import library is
basically used only to reassure the linker that a certain identifier is legal,
and will be present in the program when the DLL is loaded. So the linker uses
the information from the import library to build the lookup table for using
identifiers that are not included in the DLL. When an application or a DLL is
linked, an import library may be generated, which will need to be used for all
future DLLs that depend on the symbols in the application or DLL.
Suppose you are building two dynamic-load modules, B and C, which should share
another block of code A. On Unix, you would not pass A.a
to the
linker for B.so
and C.so
; that would cause it to be included
twice, so that B and C would each have their own copy. In Windows, building
A.dll
will also build A.lib
. You do pass A.lib
to the
linker for B and C. A.lib
does not contain code; it just contains
information which will be used at runtime to access A’s code.
In Windows, using an import library is sort of like using import spam
; it
gives you access to spam’s names, but does not create a separate copy. On Unix,
linking with a library is more like from spam import *
; it does create a
separate copy.
-
Py_NO_LINK_LIB¶
Turn off the implicit,
#pragma
-based linkage with the Python library, performed inside CPython header files.Added in version 3.14.
5.3. Using DLLs in Practice¶
Windows Python is built in Microsoft Visual C++; using other compilers may or may not work. The rest of this section is MSVC++ specific.
When creating DLLs in Windows, you can use the CPython library in two ways:
By default, inclusion of
PC/pyconfig.h
directly or viaPython.h
triggers an implicit, configure-aware link with the library. The header file choosespythonXY_d.lib
for Debug,pythonXY.lib
for Release, andpythonX.lib
for Release with the Limited API enabled.To build two DLLs, spam and ni (which uses C functions found in spam), you could use these commands:
cl /LD /I/python/include spam.c cl /LD /I/python/include ni.c spam.lib
The first command created three files:
spam.obj
,spam.dll
andspam.lib
.Spam.dll
does not contain any Python functions (such asPyArg_ParseTuple()
), but it does know how to find the Python code thanks to the implicitly linkedpythonXY.lib
.The second command created
ni.dll
(and.obj
and.lib
), which knows how to find the necessary functions from spam, and also from the Python executable.Manually by defining
Py_NO_LINK_LIB
macro before includingPython.h
. You must passpythonXY.lib
to the linker.To build two DLLs, spam and ni (which uses C functions found in spam), you could use these commands:
cl /LD /DPy_NO_LINK_LIB /I/python/include spam.c ../libs/pythonXY.lib cl /LD /DPy_NO_LINK_LIB /I/python/include ni.c spam.lib ../libs/pythonXY.lib
The first command created three files:
spam.obj
,spam.dll
andspam.lib
.Spam.dll
does not contain any Python functions (such asPyArg_ParseTuple()
), but it does know how to find the Python code thanks topythonXY.lib
.The second command created
ni.dll
(and.obj
and.lib
), which knows how to find the necessary functions from spam, and also from the Python executable.
Not every identifier is exported to the lookup table. If you want any other
modules (including Python) to be able to see your identifiers, you have to say
_declspec(dllexport)
, as in void _declspec(dllexport) initspam(void)
or
PyObject _declspec(dllexport) *NiGetSpamData(void)
.
Developer Studio will throw in a lot of import libraries that you do not really
need, adding about 100K to your executable. To get rid of them, use the Project
Settings dialog, Link tab, to specify ignore default libraries. Add the
correct msvcrtxx.lib
to the list of libraries.


Capsules

Capsules¶
Refer to Providing a C API for an Extension Module for more information on using these objects.
Added in version 3.1.
-
type PyCapsule¶
This subtype of
PyObject
represents an opaque value, useful for C extension modules which need to pass an opaque value (as a void* pointer) through Python code to other C code. It is often used to make a C function pointer defined in one module available to other modules, so the regular import mechanism can be used to access C APIs defined in dynamically loaded modules.
-
PyTypeObject PyCapsule_Type¶
- Part of the Stable ABI.
The type object corresponding to capsule objects. This is the same object as
types.CapsuleType
in the Python layer.
-
type PyCapsule_Destructor¶
- Part of the Stable ABI.
The type of a destructor callback for a capsule. Defined as:
typedef void (*PyCapsule_Destructor)(PyObject *);
See
PyCapsule_New()
for the semantics of PyCapsule_Destructor callbacks.
-
int PyCapsule_CheckExact(PyObject *p)¶
Return true if its argument is a
PyCapsule
. This function always succeeds.
-
PyObject *PyCapsule_New(void *pointer, const char *name, PyCapsule_Destructor destructor)¶
- Return value: New reference. Part of the Stable ABI.
Create a
PyCapsule
encapsulating the pointer. The pointer argument may not beNULL
.On failure, set an exception and return
NULL
.The name string may either be
NULL
or a pointer to a valid C string. If non-NULL
, this string must outlive the capsule. (Though it is permitted to free it inside the destructor.)If the destructor argument is not
NULL
, it will be called with the capsule as its argument when it is destroyed.If this capsule will be stored as an attribute of a module, the name should be specified as
modulename.attributename
. This will enable other modules to import the capsule usingPyCapsule_Import()
.
-
void *PyCapsule_GetPointer(PyObject *capsule, const char *name)¶
- Part of the Stable ABI.
Retrieve the pointer stored in the capsule. On failure, set an exception and return
NULL
.The name parameter must compare exactly to the name stored in the capsule. If the name stored in the capsule is
NULL
, the name passed in must also beNULL
. Python uses the C functionstrcmp()
to compare capsule names.
-
PyCapsule_Destructor PyCapsule_GetDestructor(PyObject *capsule)¶
- Part of the Stable ABI.
Return the current destructor stored in the capsule. On failure, set an exception and return
NULL
.It is legal for a capsule to have a
NULL
destructor. This makes aNULL
return code somewhat ambiguous; usePyCapsule_IsValid()
orPyErr_Occurred()
to disambiguate.
-
void *PyCapsule_GetContext(PyObject *capsule)¶
- Part of the Stable ABI.
Return the current context stored in the capsule. On failure, set an exception and return
NULL
.It is legal for a capsule to have a
NULL
context. This makes aNULL
return code somewhat ambiguous; usePyCapsule_IsValid()
orPyErr_Occurred()
to disambiguate.
-
const char *PyCapsule_GetName(PyObject *capsule)¶
- Part of the Stable ABI.
Return the current name stored in the capsule. On failure, set an exception and return
NULL
.It is legal for a capsule to have a
NULL
name. This makes aNULL
return code somewhat ambiguous; usePyCapsule_IsValid()
orPyErr_Occurred()
to disambiguate.
-
void *PyCapsule_Import(const char *name, int no_block)¶
- Part of the Stable ABI.
Import a pointer to a C object from a capsule attribute in a module. The name parameter should specify the full name to the attribute, as in
module.attribute
. The name stored in the capsule must match this string exactly.This function splits name on the
.
character, and imports the first element. It then processes further elements using attribute lookups.Return the capsule’s internal pointer on success. On failure, set an exception and return
NULL
.Note
If name points to an attribute of some submodule or subpackage, this submodule or subpackage must be previously imported using other means (for example, by using
PyImport_ImportModule()
) for the attribute lookups to succeed.Changed in version 3.3: no_block has no effect anymore.
-
int PyCapsule_IsValid(PyObject *capsule, const char *name)¶
- Part of the Stable ABI.
Determines whether or not capsule is a valid capsule. A valid capsule is non-
NULL
, passesPyCapsule_CheckExact()
, has a non-NULL
pointer stored in it, and its internal name matches the name parameter. (SeePyCapsule_GetPointer()
for information on how capsule names are compared.)In other words, if
PyCapsule_IsValid()
returns a true value, calls to any of the accessors (any function starting withPyCapsule_Get
) are guaranteed to succeed.Return a nonzero value if the object is valid and matches the name passed in. Return
0
otherwise. This function will not fail.
-
int PyCapsule_SetContext(PyObject *capsule, void *context)¶
- Part of the Stable ABI.
Set the context pointer inside capsule to context.
Return
0
on success. Return nonzero and set an exception on failure.
-
int PyCapsule_SetDestructor(PyObject *capsule, PyCapsule_Destructor destructor)¶
- Part of the Stable ABI.
Set the destructor inside capsule to destructor.
Return
0
on success. Return nonzero and set an exception on failure.
-
int PyCapsule_SetName(PyObject *capsule, const char *name)¶
- Part of the Stable ABI.
Set the name inside capsule to name. If non-
NULL
, the name must outlive the capsule. If the previous name stored in the capsule was notNULL
, no attempt is made to free it.Return
0
on success. Return nonzero and set an exception on failure.
-
int PyCapsule_SetPointer(PyObject *capsule, void *pointer)¶
- Part of the Stable ABI.
Set the void pointer inside capsule to pointer. The pointer may not be
NULL
.Return
0
on success. Return nonzero and set an exception on failure.


Codec registry and support functions

Codec registry and support functions¶
-
int PyCodec_Register(PyObject *search_function)¶
- Part of the Stable ABI.
Register a new codec search function.
As a side effect, this tries to load the
encodings
package, if not yet done, to make sure that it is always first in the list of search functions.
-
int PyCodec_Unregister(PyObject *search_function)¶
- Part of the Stable ABI since version 3.10.
Unregister a codec search function and clear the registry’s cache. If the search function is not registered, do nothing. Return 0 on success. Raise an exception and return -1 on error.
Added in version 3.10.
-
int PyCodec_KnownEncoding(const char *encoding)¶
- Part of the Stable ABI.
Return
1
or0
depending on whether there is a registered codec for the given encoding. This function always succeeds.
-
PyObject *PyCodec_Encode(PyObject *object, const char *encoding, const char *errors)¶
- Return value: New reference. Part of the Stable ABI.
Generic codec based encoding API.
object is passed through the encoder function found for the given encoding using the error handling method defined by errors. errors may be
NULL
to use the default method defined for the codec. Raises aLookupError
if no encoder can be found.
-
PyObject *PyCodec_Decode(PyObject *object, const char *encoding, const char *errors)¶
- Return value: New reference. Part of the Stable ABI.
Generic codec based decoding API.
object is passed through the decoder function found for the given encoding using the error handling method defined by errors. errors may be
NULL
to use the default method defined for the codec. Raises aLookupError
if no decoder can be found.
Codec lookup API¶
In the following functions, the encoding string is looked up converted to all
lower-case characters, which makes encodings looked up through this mechanism
effectively case-insensitive. If no codec is found, a KeyError
is set
and NULL
returned.
-
PyObject *PyCodec_Encoder(const char *encoding)¶
- Return value: New reference. Part of the Stable ABI.
Get an encoder function for the given encoding.
-
PyObject *PyCodec_Decoder(const char *encoding)¶
- Return value: New reference. Part of the Stable ABI.
Get a decoder function for the given encoding.
-
PyObject *PyCodec_IncrementalEncoder(const char *encoding, const char *errors)¶
- Return value: New reference. Part of the Stable ABI.
Get an
IncrementalEncoder
object for the given encoding.
-
PyObject *PyCodec_IncrementalDecoder(const char *encoding, const char *errors)¶
- Return value: New reference. Part of the Stable ABI.
Get an
IncrementalDecoder
object for the given encoding.
-
PyObject *PyCodec_StreamReader(const char *encoding, PyObject *stream, const char *errors)¶
- Return value: New reference. Part of the Stable ABI.
Get a
StreamReader
factory function for the given encoding.
-
PyObject *PyCodec_StreamWriter(const char *encoding, PyObject *stream, const char *errors)¶
- Return value: New reference. Part of the Stable ABI.
Get a
StreamWriter
factory function for the given encoding.
Registry API for Unicode encoding error handlers¶
-
int PyCodec_RegisterError(const char *name, PyObject *error)¶
- Part of the Stable ABI.
Register the error handling callback function error under the given name. This callback function will be called by a codec when it encounters unencodable characters/undecodable bytes and name is specified as the error parameter in the call to the encode/decode function.
The callback gets a single argument, an instance of
UnicodeEncodeError
,UnicodeDecodeError
orUnicodeTranslateError
that holds information about the problematic sequence of characters or bytes and their offset in the original string (see Unicode Exception Objects for functions to extract this information). The callback must either raise the given exception, or return a two-item tuple containing the replacement for the problematic sequence, and an integer giving the offset in the original string at which encoding/decoding should be resumed.Return
0
on success,-1
on error.
-
PyObject *PyCodec_LookupError(const char *name)¶
- Return value: New reference. Part of the Stable ABI.
Lookup the error handling callback function registered under name. As a special case
NULL
can be passed, in which case the error handling callback for “strict” will be returned.
-
PyObject *PyCodec_StrictErrors(PyObject *exc)¶
- Return value: Always NULL. Part of the Stable ABI.
Raise exc as an exception.
-
PyObject *PyCodec_IgnoreErrors(PyObject *exc)¶
- Return value: New reference. Part of the Stable ABI.
Ignore the unicode error, skipping the faulty input.
-
PyObject *PyCodec_ReplaceErrors(PyObject *exc)¶
- Return value: New reference. Part of the Stable ABI.
Replace the unicode encode error with
?
orU+FFFD
.
-
PyObject *PyCodec_XMLCharRefReplaceErrors(PyObject *exc)¶
- Return value: New reference. Part of the Stable ABI.
Replace the unicode encode error with XML character references.
-
PyObject *PyCodec_BackslashReplaceErrors(PyObject *exc)¶
- Return value: New reference. Part of the Stable ABI.
Replace the unicode encode error with backslash escapes (
\x
,\u
and\U
).
-
PyObject *PyCodec_NameReplaceErrors(PyObject *exc)¶
- Return value: New reference. Part of the Stable ABI since version 3.7.
Replace the unicode encode error with
\N{...}
escapes.Added in version 3.5.
Codec utility variables¶
-
const char *Py_hexdigits¶
A string constant containing the lowercase hexadecimal digits:
"0123456789abcdef"
.Added in version 3.3.


Boolean Objects

Boolean Objects¶
Booleans in Python are implemented as a subclass of integers. There are only
two booleans, Py_False
and Py_True
. As such, the normal
creation and deletion functions don’t apply to booleans. The following macros
are available, however.
-
PyTypeObject PyBool_Type¶
- Part of the Stable ABI.
This instance of
PyTypeObject
represents the Python boolean type; it is the same object asbool
in the Python layer.
-
int PyBool_Check(PyObject *o)¶
Return true if o is of type
PyBool_Type
. This function always succeeds.
-
PyObject *PyBool_FromLong(long v)¶
- Return value: New reference. Part of the Stable ABI.
Return
Py_True
orPy_False
, depending on the truth value of v.


Curses C API

Curses C API¶
curses
exposes a small C interface for extension modules.
Consumers must include the header file py_curses.h
(which is not
included by default by Python.h
) and import_curses()
must
be invoked, usually as part of the module initialisation function, to populate
PyCurses_API
.
Warning
Neither the C API nor the pure Python curses
module are compatible
with subinterpreters.
-
import_curses()¶
Import the curses C API. The macro does not need a semi-colon to be called.
On success, populate the
PyCurses_API
pointer.On failure, set
PyCurses_API
to NULL and set an exception. The caller must check if an error occurred viaPyErr_Occurred()
:import_curses(); // semi-colon is optional but recommended if (PyErr_Occurred()) { /* cleanup */ }
-
void **PyCurses_API¶
Dynamically allocated object containing the curses C API. This variable is only available once
import_curses
succeeds.PyCurses_API[0]
corresponds toPyCursesWindow_Type
.PyCurses_API[1]
,PyCurses_API[2]
, andPyCurses_API[3]
are pointers to predicate functions of typeint (*)(void)
.When called, these predicates return whether
curses.setupterm()
,curses.initscr()
, andcurses.start_color()
have been called respectively.See also the convenience macros
PyCursesSetupTermCalled
,PyCursesInitialised
, andPyCursesInitialisedColor
.Note
The number of entries in this structure is subject to changes. Consider using
PyCurses_API_pointers
to check if new fields are available or not.
-
PyCurses_API_pointers¶
The number of accessible fields (
4
) inPyCurses_API
. This number is incremented whenever new fields are added.
-
PyTypeObject PyCursesWindow_Type¶
The heap type corresponding to
curses.window
.
-
int PyCursesWindow_Check(PyObject *op)¶
Return true if op is a
curses.window
instance, false otherwise.
The following macros are convenience macros expanding into C statements.
In particular, they can only be used as macro;
or macro
, but not
macro()
or macro();
.
-
PyCursesSetupTermCalled¶
Macro checking if
curses.setupterm()
has been called.The macro expansion is roughly equivalent to:
{ typedef int (*predicate_t)(void); predicate_t was_setupterm_called = (predicate_t)PyCurses_API[1]; if (!was_setupterm_called()) { return NULL; } }
-
PyCursesInitialised¶
Macro checking if
curses.initscr()
has been called.The macro expansion is roughly equivalent to:
{ typedef int (*predicate_t)(void); predicate_t was_initscr_called = (predicate_t)PyCurses_API[2]; if (!was_initscr_called()) { return NULL; } }
-
PyCursesInitialisedColor¶
Macro checking if
curses.start_color()
has been called.The macro expansion is roughly equivalent to:
{ typedef int (*predicate_t)(void); predicate_t was_start_color_called = (predicate_t)PyCurses_API[3]; if (!was_start_color_called()) { return NULL; } }
Internal data¶
The following objects are exposed by the C API but should be considered internal-only.
-
PyCurses_CAPSULE_NAME¶
Name of the curses capsule to pass to
PyCapsule_Import()
.Internal usage only. Use
import_curses
instead.


Objects for Type Hinting

Objects for Type Hinting¶
Various built-in types for type hinting are provided. Currently,
two types exist – GenericAlias and
Union. Only GenericAlias
is exposed to C.
-
PyObject *Py_GenericAlias(PyObject *origin, PyObject *args)¶
- Part of the Stable ABI since version 3.9.
Create a GenericAlias object. Equivalent to calling the Python class
types.GenericAlias
. The origin and args arguments set theGenericAlias
‘s__origin__
and__args__
attributes respectively. origin should be a PyTypeObject*, and args can be a PyTupleObject* or anyPyObject*
. If args passed is not a tuple, a 1-tuple is automatically constructed and__args__
is set to(args,)
. Minimal checking is done for the arguments, so the function will succeed even if origin is not a type. TheGenericAlias
‘s__parameters__
attribute is constructed lazily from__args__
. On failure, an exception is raised andNULL
is returned.Here’s an example of how to make an extension type generic:
... static PyMethodDef my_obj_methods[] = { // Other methods. ... {"__class_getitem__", Py_GenericAlias, METH_O|METH_CLASS, "See PEP 585"} ... }
See also
The data model method
__class_getitem__()
.Added in version 3.9.
-
PyTypeObject Py_GenericAliasType¶
- Part of the Stable ABI since version 3.9.
The C type of the object returned by
Py_GenericAlias()
. Equivalent totypes.GenericAlias
in Python.Added in version 3.9.


What’s New in Python 2.1

What’s New in Python 2.1¶
- Author:
A.M. Kuchling
Introduction¶
This article explains the new features in Python 2.1. While there aren’t as many changes in 2.1 as there were in Python 2.0, there are still some pleasant surprises in store. 2.1 is the first release to be steered through the use of Python Enhancement Proposals, or PEPs, so most of the sizable changes have accompanying PEPs that provide more complete documentation and a design rationale for the change. This article doesn’t attempt to document the new features completely, but simply provides an overview of the new features for Python programmers. Refer to the Python 2.1 documentation, or to the specific PEP, for more details about any new feature that particularly interests you.
One recent goal of the Python development team has been to accelerate the pace of new releases, with a new release coming every 6 to 9 months. 2.1 is the first release to come out at this faster pace, with the first alpha appearing in January, 3 months after the final version of 2.0 was released.
The final release of Python 2.1 was made on April 17, 2001.
PEP 227: Nested Scopes¶
The largest change in Python 2.1 is to Python’s scoping rules. In Python 2.0, at any given time there are at most three namespaces used to look up variable names: local, module-level, and the built-in namespace. This often surprised people because it didn’t match their intuitive expectations. For example, a nested recursive function definition doesn’t work:
def f():
...
def g(value):
...
return g(value-1) + 1
...
The function g()
will always raise a NameError
exception, because
the binding of the name g
isn’t in either its local namespace or in the
module-level namespace. This isn’t much of a problem in practice (how often do
you recursively define interior functions like this?), but this also made using
the lambda
expression clumsier, and this was a problem in practice.
In code which uses lambda
you can often find local variables being
copied by passing them as the default values of arguments.
def find(self, name):
"Return list of any entries equal to 'name'"
L = filter(lambda x, name=name: x == name,
self.list_attribute)
return L
The readability of Python code written in a strongly functional style suffers greatly as a result.
The most significant change to Python 2.1 is that static scoping has been added
to the language to fix this problem. As a first effect, the name=name
default argument is now unnecessary in the above example. Put simply, when a
given variable name is not assigned a value within a function (by an assignment,
or the def
, class
, or import
statements),
references to the variable will be looked up in the local namespace of the
enclosing scope. A more detailed explanation of the rules, and a dissection of
the implementation, can be found in the PEP.
This change may cause some compatibility problems for code where the same variable name is used both at the module level and as a local variable within a function that contains further function definitions. This seems rather unlikely though, since such code would have been pretty confusing to read in the first place.
One side effect of the change is that the from module import *
and
exec
statements have been made illegal inside a function scope under
certain conditions. The Python reference manual has said all along that from
module import *
is only legal at the top level of a module, but the CPython
interpreter has never enforced this before. As part of the implementation of
nested scopes, the compiler which turns Python source into bytecodes has to
generate different code to access variables in a containing scope. from
module import *
and exec
make it impossible for the compiler to
figure this out, because they add names to the local namespace that are
unknowable at compile time. Therefore, if a function contains function
definitions or lambda
expressions with free variables, the compiler
will flag this by raising a SyntaxError
exception.
To make the preceding explanation a bit clearer, here’s an example:
x = 1
def f():
# The next line is a syntax error
exec 'x=2'
def g():
return x
Line 4 containing the exec
statement is a syntax error, since
exec
would define a new local variable named x
whose value should
be accessed by g()
.
This shouldn’t be much of a limitation, since exec
is rarely used in
most Python code (and when it is used, it’s often a sign of a poor design
anyway).
Compatibility concerns have led to nested scopes being introduced gradually; in Python 2.1, they aren’t enabled by default, but can be turned on within a module by using a future statement as described in PEP 236. (See the following section for further discussion of PEP 236.) In Python 2.2, nested scopes will become the default and there will be no way to turn them off, but users will have had all of 2.1’s lifetime to fix any breakage resulting from their introduction.
See also
- PEP 227 - Statically Nested Scopes
Written and implemented by Jeremy Hylton.
PEP 236: __future__ Directives¶
The reaction to nested scopes was widespread concern about the dangers of breaking code with the 2.1 release, and it was strong enough to make the Pythoneers take a more conservative approach. This approach consists of introducing a convention for enabling optional functionality in release N that will become compulsory in release N+1.
The syntax uses a from...import
statement using the reserved module name
__future__
. Nested scopes can be enabled by the following statement:
from __future__ import nested_scopes
While it looks like a normal import
statement, it’s not; there are
strict rules on where such a future statement can be put. They can only be at
the top of a module, and must precede any Python code or regular
import
statements. This is because such statements can affect how
the Python bytecode compiler parses code and generates bytecode, so they must
precede any statement that will result in bytecodes being produced.
See also
- PEP 236 - Back to the
__future__
Written by Tim Peters, and primarily implemented by Jeremy Hylton.
PEP 207: Rich Comparisons¶
In earlier versions, Python’s support for implementing comparisons on user-defined
classes and extension types was quite simple. Classes could implement a
__cmp__()
method that was given two instances of a class, and could only
return 0 if they were equal or +1 or -1 if they weren’t; the method couldn’t
raise an exception or return anything other than a Boolean value. Users of
Numeric Python often found this model too weak and restrictive, because in the
number-crunching programs that numeric Python is used for, it would be more
useful to be able to perform elementwise comparisons of two matrices, returning
a matrix containing the results of a given comparison for each element. If the
two matrices are of different sizes, then the compare has to be able to raise an
exception to signal the error.
In Python 2.1, rich comparisons were added in order to support this need.
Python classes can now individually overload each of the <
, <=
, >
,
>=
, ==
, and !=
operations. The new magic method names are:
Operation |
Method name |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
(The magic methods are named after the corresponding Fortran operators .LT.
.
.LE.
, &c. Numeric programmers are almost certainly quite familiar with
these names and will find them easy to remember.)
Each of these magic methods is of the form method(self, other)
, where
self
will be the object on the left-hand side of the operator, while
other
will be the object on the right-hand side. For example, the
expression A < B
will cause A.__lt__(B)
to be called.
Each of these magic methods can return anything at all: a Boolean, a matrix, a list, or any other Python object. Alternatively they can raise an exception if the comparison is impossible, inconsistent, or otherwise meaningless.
The built-in cmp(A,B)
function can use the rich comparison machinery,
and now accepts an optional argument specifying which comparison operation to
use; this is given as one of the strings "<"
, "<="
, ">"
, ">="
,
"=="
, or "!="
. If called without the optional third argument,
cmp()
will only return -1, 0, or +1 as in previous versions of Python;
otherwise it will call the appropriate method and can return any Python object.
There are also corresponding changes of interest to C programmers; there’s a new
slot tp_richcmp
in type objects and an API for performing a given rich
comparison. I won’t cover the C API here, but will refer you to PEP 207, or to
2.1’s C API documentation, for the full list of related functions.
See also
- PEP 207 - Rich Comparisons
Written by Guido van Rossum, heavily based on earlier work by David Ascher, and implemented by Guido van Rossum.
PEP 230: Warning Framework¶
Over its 10 years of existence, Python has accumulated a certain number of obsolete modules and features along the way. It’s difficult to know when a feature is safe to remove, since there’s no way of knowing how much code uses it — perhaps no programs depend on the feature, or perhaps many do. To enable removing old features in a more structured way, a warning framework was added. When the Python developers want to get rid of a feature, it will first trigger a warning in the next version of Python. The following Python version can then drop the feature, and users will have had a full release cycle to remove uses of the old feature.
Python 2.1 adds the warning framework to be used in this scheme. It adds a
warnings
module that provide functions to issue warnings, and to filter
out warnings that you don’t want to be displayed. Third-party modules can also
use this framework to deprecate old features that they no longer wish to
support.
For example, in Python 2.1 the regex
module is deprecated, so importing
it causes a warning to be printed:
>>> import regex
__main__:1: DeprecationWarning: the regex module
is deprecated; please use the re module
>>>
Warnings can be issued by calling the warnings.warn()
function:
warnings.warn("feature X no longer supported")
The first parameter is the warning message; an additional optional parameters can be used to specify a particular warning category.
Filters can be added to disable certain warnings; a regular expression pattern
can be applied to the message or to the module name in order to suppress a
warning. For example, you may have a program that uses the regex
module
and not want to spare the time to convert it to use the re
module right
now. The warning can be suppressed by calling
import warnings
warnings.filterwarnings(action = 'ignore',
message='.*regex module is deprecated',
category=DeprecationWarning,
module = '__main__')
This adds a filter that will apply only to warnings of the class
DeprecationWarning
triggered in the __main__
module, and applies
a regular expression to only match the message about the regex
module
being deprecated, and will cause such warnings to be ignored. Warnings can also
be printed only once, printed every time the offending code is executed, or
turned into exceptions that will cause the program to stop (unless the
exceptions are caught in the usual way, of course).
Functions were also added to Python’s C API for issuing warnings; refer to PEP 230 or to Python’s API documentation for the details.
See also
- PEP 5 - Guidelines for Language Evolution
Written by Paul Prescod, to specify procedures to be followed when removing old features from Python. The policy described in this PEP hasn’t been officially adopted, but the eventual policy probably won’t be too different from Prescod’s proposal.
- PEP 230 - Warning Framework
Written and implemented by Guido van Rossum.
PEP 229: New Build System¶
When compiling Python, the user had to go in and edit the Modules/Setup
file in order to enable various additional modules; the default set is
relatively small and limited to modules that compile on most Unix platforms.
This means that on Unix platforms with many more features, most notably Linux,
Python installations often don’t contain all useful modules they could.
Python 2.0 added the Distutils, a set of modules for distributing and installing extensions. In Python 2.1, the Distutils are used to compile much of the standard library of extension modules, autodetecting which ones are supported on the current machine. It’s hoped that this will make Python installations easier and more featureful.
Instead of having to edit the Modules/Setup
file in order to enable
modules, a setup.py
script in the top directory of the Python source
distribution is run at build time, and attempts to discover which modules can be
enabled by examining the modules and header files on the system. If a module is
configured in Modules/Setup
, the setup.py
script won’t attempt
to compile that module and will defer to the Modules/Setup
file’s
contents. This provides a way to specific any strange command-line flags or
libraries that are required for a specific platform.
In another far-reaching change to the build mechanism, Neil Schemenauer
restructured things so Python now uses a single makefile that isn’t recursive,
instead of makefiles in the top directory and in each of the Python/
,
Parser/
, Objects/
, and Modules/
subdirectories. This
makes building Python faster and also makes hacking the Makefiles clearer and
simpler.
See also
- PEP 229 - Using Distutils to Build Python
Written and implemented by A.M. Kuchling.
PEP 205: Weak References¶
Weak references, available through the weakref
module, are a minor but
useful new data type in the Python programmer’s toolbox.
Storing a reference to an object (say, in a dictionary or a list) has the side effect of keeping that object alive forever. There are a few specific cases where this behaviour is undesirable, object caches being the most common one, and another being circular references in data structures such as trees.
For example, consider a memoizing function that caches the results of another
function f(x)
by storing the function’s argument and its result in a
dictionary:
_cache = {}
def memoize(x):
if _cache.has_key(x):
return _cache[x]
retval = f(x)
# Cache the returned object
_cache[x] = retval
return retval
This version works for simple things such as integers, but it has a side effect;
the _cache
dictionary holds a reference to the return values, so they’ll
never be deallocated until the Python process exits and cleans up. This isn’t
very noticeable for integers, but if f()
returns an object, or a data
structure that takes up a lot of memory, this can be a problem.
Weak references provide a way to implement a cache that won’t keep objects alive
beyond their time. If an object is only accessible through weak references, the
object will be deallocated and the weak references will now indicate that the
object it referred to no longer exists. A weak reference to an object obj is
created by calling wr = weakref.ref(obj)
. The object being referred to is
returned by calling the weak reference as if it were a function: wr()
. It
will return the referenced object, or None
if the object no longer exists.
This makes it possible to write a memoize()
function whose cache doesn’t
keep objects alive, by storing weak references in the cache.
_cache = {}
def memoize(x):
if _cache.has_key(x):
obj = _cache[x]()
# If weak reference object still exists,
# return it
if obj is not None: return obj
retval = f(x)
# Cache a weak reference
_cache[x] = weakref.ref(retval)
return retval
The weakref
module also allows creating proxy objects which behave like
weak references — an object referenced only by proxy objects is deallocated –
but instead of requiring an explicit call to retrieve the object, the proxy
transparently forwards all operations to the object as long as the object still
exists. If the object is deallocated, attempting to use a proxy will cause a
weakref.ReferenceError
exception to be raised.
proxy = weakref.proxy(obj)
proxy.attr # Equivalent to obj.attr
proxy.meth() # Equivalent to obj.meth()
del obj
proxy.attr # raises weakref.ReferenceError
See also
- PEP 205 - Weak References
Written and implemented by Fred L. Drake, Jr.
PEP 232: Function Attributes¶
In Python 2.1, functions can now have arbitrary information attached to them.
People were often using docstrings to hold information about functions and
methods, because the __doc__
attribute was the only way of
attaching any
information to a function. For example, in the Zope web application server,
functions are marked as safe for public access by having a docstring, and in
John Aycock’s SPARK parsing framework, docstrings hold parts of the BNF grammar
to be parsed. This overloading is unfortunate, since docstrings are really
intended to hold a function’s documentation; for example, it means you can’t
properly document functions intended for private use in Zope.
Arbitrary attributes can now be set and retrieved on functions using the regular Python syntax:
def f(): pass
f.publish = 1
f.secure = 1
f.grammar = "A ::= B (C D)*"
The dictionary containing attributes can be accessed as the function’s
__dict__
. Unlike the __dict__
attribute of class instances, in
functions you can actually assign a new dictionary to __dict__
, though
the new value is restricted to a regular Python dictionary; you can’t be
tricky and set it to a UserDict
instance, or any other random object
that behaves like a mapping.
See also
- PEP 232 - Function Attributes
Written and implemented by Barry Warsaw.
PEP 235: Importing Modules on Case-Insensitive Platforms¶
Some operating systems have filesystems that are case-insensitive, MacOS and
Windows being the primary examples; on these systems, it’s impossible to
distinguish the filenames FILE.PY
and file.py
, even though they do store
the file’s name in its original case (they’re case-preserving, too).
In Python 2.1, the import
statement will work to simulate case-sensitivity
on case-insensitive platforms. Python will now search for the first
case-sensitive match by default, raising an ImportError
if no such file
is found, so import file
will not import a module named FILE.PY
.
Case-insensitive matching can be requested by setting the PYTHONCASEOK
environment variable before starting the Python interpreter.
PEP 217: Interactive Display Hook¶
When using the Python interpreter interactively, the output of commands is
displayed using the built-in repr()
function. In Python 2.1, the variable
sys.displayhook()
can be set to a callable object which will be called
instead of repr()
. For example, you can set it to a special
pretty-printing function:
>>> # Create a recursive data structure
... L = [1,2,3]
>>> L.append(L)
>>> L # Show Python's default output
[1, 2, 3, [...]]
>>> # Use pprint.pprint() as the display function
... import sys, pprint
>>> sys.displayhook = pprint.pprint
>>> L
[1, 2, 3, <Recursion on list with id=135143996>]
>>>
See also
- PEP 217 - Display Hook for Interactive Use
Written and implemented by Moshe Zadka.
PEP 208: New Coercion Model¶
How numeric coercion is done at the C level was significantly modified. This will only affect the authors of C extensions to Python, allowing them more flexibility in writing extension types that support numeric operations.
Extension types can now set the type flag Py_TPFLAGS_CHECKTYPES
in their
PyTypeObject
structure to indicate that they support the new coercion model.
In such extension types, the numeric slot functions can no longer assume that
they’ll be passed two arguments of the same type; instead they may be passed two
arguments of differing types, and can then perform their own internal coercion.
If the slot function is passed a type it can’t handle, it can indicate the
failure by returning a reference to the Py_NotImplemented
singleton value.
The numeric functions of the other type will then be tried, and perhaps they can
handle the operation; if the other type also returns Py_NotImplemented
, then
a TypeError
will be raised. Numeric methods written in Python can also
return Py_NotImplemented
, causing the interpreter to act as if the method
did not exist (perhaps raising a TypeError
, perhaps trying another
object’s numeric methods).
See also
- PEP 208 - Reworking the Coercion Model
Written and implemented by Neil Schemenauer, heavily based upon earlier work by Marc-André Lemburg. Read this to understand the fine points of how numeric operations will now be processed at the C level.
PEP 241: Metadata in Python Packages¶
A common complaint from Python users is that there’s no single catalog of all
the Python modules in existence. T. Middleton’s Vaults of Parnassus at
www.vex.net/parnassus/
(retired in February 2009, available in the
Internet Archive Wayback Machine)
was the largest catalog of Python modules, but
registering software at the Vaults is optional, and many people did not bother.
As a first small step toward fixing the problem, Python software packaged using
the Distutils sdist command will include a file named
PKG-INFO
containing information about the package such as its name,
version, and author (metadata, in cataloguing terminology). PEP 241 contains
the full list of fields that can be present in the PKG-INFO
file. As
people began to package their software using Python 2.1, more and more packages
will include metadata, making it possible to build automated cataloguing systems
and experiment with them. With the result experience, perhaps it’ll be possible
to design a really good catalog and then build support for it into Python 2.2.
For example, the Distutils sdist and bdist_* commands
could support an upload
option that would automatically upload your
package to a catalog server.
You can start creating packages containing PKG-INFO
even if you’re not
using Python 2.1, since a new release of the Distutils will be made for users of
earlier Python versions. Version 1.0.2 of the Distutils includes the changes
described in PEP 241, as well as various bugfixes and enhancements. It will be
available from the Distutils SIG at https://www.python.org/community/sigs/current/distutils-sig/.
New and Improved Modules¶
Ka-Ping Yee contributed two new modules:
inspect.py
, a module for getting information about live Python code, andpydoc.py
, a module for interactively converting docstrings to HTML or text. As a bonus,Tools/scripts/pydoc
, which is now automatically installed, usespydoc.py
to display documentation given a Python module, package, or class name. For example,pydoc xml.dom
displays the following:Python Library Documentation: package xml.dom in xml NAME xml.dom - W3C Document Object Model implementation for Python. FILE /usr/local/lib/python2.1/xml/dom/__init__.pyc DESCRIPTION The Python mapping of the Document Object Model is documented in the Python Library Reference in the section on the xml.dom package. This package contains the following modules: ...
pydoc
also includes a Tk-based interactive help browser.pydoc
quickly becomes addictive; try it out!Two different modules for unit testing were added to the standard library. The
doctest
module, contributed by Tim Peters, provides a testing framework based on running embedded examples in docstrings and comparing the results against the expected output. PyUnit, contributed by Steve Purcell, is a unit testing framework inspired by JUnit, which was in turn an adaptation of Kent Beck’s Smalltalk testing framework. See https://pyunit.sourceforge.net/ for more information about PyUnit.The
difflib
module contains a class,SequenceMatcher
, which compares two sequences and computes the changes required to transform one sequence into the other. For example, this module can be used to write a tool similar to the Unix diff program, and in fact the sample programTools/scripts/ndiff.py
demonstrates how to write such a script.curses.panel
, a wrapper for the panel library, part of ncurses and of SYSV curses, was contributed by Thomas Gellekum. The panel library provides windows with the additional feature of depth. Windows can be moved higher or lower in the depth ordering, and the panel library figures out where panels overlap and which sections are visible.The PyXML package has gone through a few releases since Python 2.0, and Python 2.1 includes an updated version of the
xml
package. Some of the noteworthy changes include support for Expat 1.2 and later versions, the ability for Expat parsers to handle files in any encoding supported by Python, and various bugfixes for SAX, DOM, and theminidom
module.Ping also contributed another hook for handling uncaught exceptions.
sys.excepthook()
can be set to a callable object. When an exception isn’t caught by anytry
…except
blocks, the exception will be passed tosys.excepthook()
, which can then do whatever it likes. At the Ninth Python Conference, Ping demonstrated an application for this hook: printing an extended traceback that not only lists the stack frames, but also lists the function arguments and the local variables for each frame.Various functions in the
time
module, such asasctime()
andlocaltime()
, require a floating-point argument containing the time in seconds since the epoch. The most common use of these functions is to work with the current time, so the floating-point argument has been made optional; when a value isn’t provided, the current time will be used. For example, log file entries usually need a string containing the current time; in Python 2.1,time.asctime()
can be used, instead of the lengthiertime.asctime(time.localtime(time.time()))
that was previously required.This change was proposed and implemented by Thomas Wouters.
The
ftplib
module now defaults to retrieving files in passive mode, because passive mode is more likely to work from behind a firewall. This request came from the Debian bug tracking system, since other Debian packages useftplib
to retrieve files and then don’t work from behind a firewall. It’s deemed unlikely that this will cause problems for anyone, because Netscape defaults to passive mode and few people complain, but if passive mode is unsuitable for your application or network setup, callset_pasv(0)
on FTP objects to disable passive mode.Support for raw socket access has been added to the
socket
module, contributed by Grant Edwards.The
pstats
module now contains a simple interactive statistics browser for displaying timing profiles for Python programs, invoked when the module is run as a script. Contributed by Eric S. Raymond.A new implementation-dependent function,
sys._getframe([depth])
, has been added to return a given frame object from the current call stack.sys._getframe()
returns the frame at the top of the call stack; if the optional integer argument depth is supplied, the function returns the frame that is depth calls below the top of the stack. For example,sys._getframe(1)
returns the caller’s frame object.This function is only present in CPython, not in Jython or the .NET implementation. Use it for debugging, and resist the temptation to put it into production code.
Other Changes and Fixes¶
There were relatively few smaller changes made in Python 2.1 due to the shorter release cycle. A search through the CVS change logs turns up 117 patches applied, and 136 bugs fixed; both figures are likely to be underestimates. Some of the more notable changes are:
A specialized object allocator is now optionally available, that should be faster than the system
malloc()
and have less memory overhead. The allocator uses C’smalloc()
function to get large pools of memory, and then fulfills smaller memory requests from these pools. It can be enabled by providing the--with-pymalloc
option to the configure script; seeObjects/obmalloc.c
for the implementation details.Authors of C extension modules should test their code with the object allocator enabled, because some incorrect code may break, causing core dumps at runtime. There are a bunch of memory allocation functions in Python’s C API that have previously been just aliases for the C library’s
malloc()
andfree()
, meaning that if you accidentally called mismatched functions, the error wouldn’t be noticeable. When the object allocator is enabled, these functions aren’t aliases ofmalloc()
andfree()
any more, and calling the wrong function to free memory will get you a core dump. For example, if memory was allocated usingPyMem_New
, it has to be freed usingPyMem_Del()
, notfree()
. A few modules included with Python fell afoul of this and had to be fixed; doubtless there are more third-party modules that will have the same problem.The object allocator was contributed by Vladimir Marangozov.
The speed of line-oriented file I/O has been improved because people often complain about its lack of speed, and because it’s often been used as a naïve benchmark. The
readline()
method of file objects has therefore been rewritten to be much faster. The exact amount of the speedup will vary from platform to platform depending on how slow the C library’sgetc()
was, but is around 66%, and potentially much faster on some particular operating systems. Tim Peters did much of the benchmarking and coding for this change, motivated by a discussion in comp.lang.python.A new module and method for file objects was also added, contributed by Jeff Epler. The new method,
xreadlines()
, is similar to the existingxrange()
built-in.xreadlines()
returns an opaque sequence object that only supports being iterated over, reading a line on every iteration but not reading the entire file into memory as the existingreadlines()
method does. You’d use it like this:for line in sys.stdin.xreadlines(): # ... do something for each line ... ...
For a fuller discussion of the line I/O changes, see the python-dev summary for January 1–15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/.
A new method,
popitem()
, was added to dictionaries to enable destructively iterating through the contents of a dictionary; this can be faster for large dictionaries because there’s no need to construct a list containing all the keys or values.D.popitem()
removes a random(key, value)
pair from the dictionaryD
and returns it as a 2-tuple. This was implemented mostly by Tim Peters and Guido van Rossum, after a suggestion and preliminary patch by Moshe Zadka.Modules can now control which names are imported when
from module import *
is used, by defining an__all__
attribute containing a list of names that will be imported. One common complaint is that if the module imports other modules such assys
orstring
,from module import *
will add them to the importing module’s namespace. To fix this, simply list the public names in__all__
:# List public names __all__ = ['Database', 'open']
A stricter version of this patch was first suggested and implemented by Ben Wolfson, but after some python-dev discussion, a weaker final version was checked in.
Applying
repr()
to strings previously used octal escapes for non-printable characters; for example, a newline was'\012'
. This was a vestigial trace of Python’s C ancestry, but today octal is of very little practical use. Ka-Ping Yee suggested using hex escapes instead of octal ones, and using the\n
,\t
,\r
escapes for the appropriate characters, and implemented this new formatting.Syntax errors detected at compile-time can now raise exceptions containing the filename and line number of the error, a pleasant side effect of the compiler reorganization done by Jeremy Hylton.
C extensions which import other modules have been changed to use
PyImport_ImportModule()
, which means that they will use any import hooks that have been installed. This is also encouraged for third-party extensions that need to import some other module from C code.The size of the Unicode character database was shrunk by another 340K thanks to Fredrik Lundh.
Some new ports were contributed: MacOS X (by Steven Majewski), Cygwin (by Jason Tishler); RISCOS (by Dietmar Schwertberger); Unixware 7 (by Billy G. Allie).
And there’s the usual list of minor bugfixes, minor memory leaks, docstring edits, and other tweaks, too lengthy to be worth itemizing; see the CVS logs for the full details if you want them.
Acknowledgements¶
The author would like to thank the following people for offering suggestions on various drafts of this article: Graeme Cross, David Goodger, Jay Graves, Michael Hudson, Marc-André Lemburg, Fredrik Lundh, Neil Schemenauer, Thomas Wouters.