Sequence Protocol

Sequence Protocol¶
-
int PySequence_Check(PyObject *o)¶
- Part of the Stable ABI.
Return
1
if the object provides the sequence protocol, and0
otherwise. Note that it returns1
for Python classes with a__getitem__()
method, unless they aredict
subclasses, since in general it is impossible to determine what type of keys the class supports. This function always succeeds.
-
Py_ssize_t PySequence_Size(PyObject *o)¶
-
Py_ssize_t PySequence_Length(PyObject *o)¶
- Part of the Stable ABI.
Returns the number of objects in sequence o on success, and
-1
on failure. This is equivalent to the Python expressionlen(o)
.
-
PyObject *PySequence_Concat(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Return the concatenation of o1 and o2 on success, and
NULL
on failure. This is the equivalent of the Python expressiono1 + o2
.
-
PyObject *PySequence_Repeat(PyObject *o, Py_ssize_t count)¶
- Return value: New reference. Part of the Stable ABI.
Return the result of repeating sequence object o count times, or
NULL
on failure. This is the equivalent of the Python expressiono * count
.
-
PyObject *PySequence_InPlaceConcat(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Return the concatenation of o1 and o2 on success, and
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python expressiono1 += o2
.
-
PyObject *PySequence_InPlaceRepeat(PyObject *o, Py_ssize_t count)¶
- Return value: New reference. Part of the Stable ABI.
Return the result of repeating sequence object o count times, or
NULL
on failure. The operation is done in-place when o supports it. This is the equivalent of the Python expressiono *= count
.
-
PyObject *PySequence_GetItem(PyObject *o, Py_ssize_t i)¶
- Return value: New reference. Part of the Stable ABI.
Return the ith element of o, or
NULL
on failure. This is the equivalent of the Python expressiono[i]
.
-
PyObject *PySequence_GetSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2)¶
- Return value: New reference. Part of the Stable ABI.
Return the slice of sequence object o between i1 and i2, or
NULL
on failure. This is the equivalent of the Python expressiono[i1:i2]
.
-
int PySequence_SetItem(PyObject *o, Py_ssize_t i, PyObject *v)¶
- Part of the Stable ABI.
Assign object v to the ith element of o. Raise an exception and return
-1
on failure; return0
on success. This is the equivalent of the Python statemento[i] = v
. This function does not steal a reference to v.If v is
NULL
, the element is deleted, but this feature is deprecated in favour of usingPySequence_DelItem()
.
-
int PySequence_DelItem(PyObject *o, Py_ssize_t i)¶
- Part of the Stable ABI.
Delete the ith element of object o. Returns
-1
on failure. This is the equivalent of the Python statementdel o[i]
.
-
int PySequence_SetSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2, PyObject *v)¶
- Part of the Stable ABI.
Assign the sequence object v to the slice in sequence object o from i1 to i2. This is the equivalent of the Python statement
o[i1:i2] = v
.
-
int PySequence_DelSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2)¶
- Part of the Stable ABI.
Delete the slice in sequence object o from i1 to i2. Returns
-1
on failure. This is the equivalent of the Python statementdel o[i1:i2]
.
-
Py_ssize_t PySequence_Count(PyObject *o, PyObject *value)¶
- Part of the Stable ABI.
Return the number of occurrences of value in o, that is, return the number of keys for which
o[key] == value
. On failure, return-1
. This is equivalent to the Python expressiono.count(value)
.
-
int PySequence_Contains(PyObject *o, PyObject *value)¶
- Part of the Stable ABI.
Determine if o contains value. If an item in o is equal to value, return
1
, otherwise return0
. On error, return-1
. This is equivalent to the Python expressionvalue in o
.
-
int PySequence_In(PyObject *o, PyObject *value)¶
- Part of the Stable ABI.
Alias for
PySequence_Contains()
.Deprecated since version 3.14: The function is soft deprecated and should no longer be used to write new code.
-
Py_ssize_t PySequence_Index(PyObject *o, PyObject *value)¶
- Part of the Stable ABI.
Return the first index i for which
o[i] == value
. On error, return-1
. This is equivalent to the Python expressiono.index(value)
.
-
PyObject *PySequence_List(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Return a list object with the same contents as the sequence or iterable o, or
NULL
on failure. The returned list is guaranteed to be new. This is equivalent to the Python expressionlist(o)
.
-
PyObject *PySequence_Tuple(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Return a tuple object with the same contents as the sequence or iterable o, or
NULL
on failure. If o is a tuple, a new reference will be returned, otherwise a tuple will be constructed with the appropriate contents. This is equivalent to the Python expressiontuple(o)
.
-
PyObject *PySequence_Fast(PyObject *o, const char *m)¶
- Return value: New reference. Part of the Stable ABI.
Return the sequence or iterable o as an object usable by the other
PySequence_Fast*
family of functions. If the object is not a sequence or iterable, raisesTypeError
with m as the message text. ReturnsNULL
on failure.The
PySequence_Fast*
functions are thus named because they assume o is aPyTupleObject
or aPyListObject
and access the data fields of o directly.As a CPython implementation detail, if o is already a sequence or list, it will be returned.
-
Py_ssize_t PySequence_Fast_GET_SIZE(PyObject *o)¶
Returns the length of o, assuming that o was returned by
PySequence_Fast()
and that o is notNULL
. The size can also be retrieved by callingPySequence_Size()
on o, butPySequence_Fast_GET_SIZE()
is faster because it can assume o is a list or tuple.
-
PyObject *PySequence_Fast_GET_ITEM(PyObject *o, Py_ssize_t i)¶
- Return value: Borrowed reference.
Return the ith element of o, assuming that o was returned by
PySequence_Fast()
, o is notNULL
, and that i is within bounds.
-
PyObject **PySequence_Fast_ITEMS(PyObject *o)¶
Return the underlying array of PyObject pointers. Assumes that o was returned by
PySequence_Fast()
and o is notNULL
.Note, if a list gets resized, the reallocation may relocate the items array. So, only use the underlying array pointer in contexts where the sequence cannot change.
-
PyObject *PySequence_ITEM(PyObject *o, Py_ssize_t i)¶
- Return value: New reference.
Return the ith element of o or
NULL
on failure. Faster form ofPySequence_GetItem()
but without checking thatPySequence_Check()
on o is true and without adjustment for negative indices.


Bytes Objects

Bytes Objects¶
These functions raise TypeError
when expecting a bytes parameter and
called with a non-bytes parameter.
-
PyTypeObject PyBytes_Type¶
- Part of the Stable ABI.
This instance of
PyTypeObject
represents the Python bytes type; it is the same object asbytes
in the Python layer.
-
int PyBytes_Check(PyObject *o)¶
Return true if the object o is a bytes object or an instance of a subtype of the bytes type. This function always succeeds.
-
int PyBytes_CheckExact(PyObject *o)¶
Return true if the object o is a bytes object, but not an instance of a subtype of the bytes type. This function always succeeds.
-
PyObject *PyBytes_FromString(const char *v)¶
- Return value: New reference. Part of the Stable ABI.
Return a new bytes object with a copy of the string v as value on success, and
NULL
on failure. The parameter v must not beNULL
; it will not be checked.
-
PyObject *PyBytes_FromStringAndSize(const char *v, Py_ssize_t len)¶
- Return value: New reference. Part of the Stable ABI.
Return a new bytes object with a copy of the string v as value and length len on success, and
NULL
on failure. If v isNULL
, the contents of the bytes object are uninitialized.
-
PyObject *PyBytes_FromFormat(const char *format, ...)¶
- Return value: New reference. Part of the Stable ABI.
Take a C
printf()
-style format string and a variable number of arguments, calculate the size of the resulting Python bytes object and return a bytes object with the values formatted into it. The variable arguments must be C types and must correspond exactly to the format characters in the format string. The following format characters are allowed:Format Characters
Type
Comment
%%
n/a
The literal % character.
%c
int
A single byte, represented as a C int.
%d
int
Equivalent to
printf("%d")
. [1]%u
unsigned int
Equivalent to
printf("%u")
. [1]%ld
long
Equivalent to
printf("%ld")
. [1]%lu
unsigned long
Equivalent to
printf("%lu")
. [1]%zd
Equivalent to
printf("%zd")
. [1]%zu
size_t
Equivalent to
printf("%zu")
. [1]%i
int
Equivalent to
printf("%i")
. [1]%x
int
Equivalent to
printf("%x")
. [1]%s
const char*
A null-terminated C character array.
%p
const void*
The hex representation of a C pointer. Mostly equivalent to
printf("%p")
except that it is guaranteed to start with the literal0x
regardless of what the platform’sprintf
yields.An unrecognized format character causes all the rest of the format string to be copied as-is to the result object, and any extra arguments discarded.
-
PyObject *PyBytes_FromFormatV(const char *format, va_list vargs)¶
- Return value: New reference. Part of the Stable ABI.
Identical to
PyBytes_FromFormat()
except that it takes exactly two arguments.
-
PyObject *PyBytes_FromObject(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Return the bytes representation of object o that implements the buffer protocol.
-
Py_ssize_t PyBytes_Size(PyObject *o)¶
- Part of the Stable ABI.
Return the length of the bytes in bytes object o.
-
Py_ssize_t PyBytes_GET_SIZE(PyObject *o)¶
Similar to
PyBytes_Size()
, but without error checking.
-
char *PyBytes_AsString(PyObject *o)¶
- Part of the Stable ABI.
Return a pointer to the contents of o. The pointer refers to the internal buffer of o, which consists of
len(o) + 1
bytes. The last byte in the buffer is always null, regardless of whether there are any other null bytes. The data must not be modified in any way, unless the object was just created usingPyBytes_FromStringAndSize(NULL, size)
. It must not be deallocated. If o is not a bytes object at all,PyBytes_AsString()
returnsNULL
and raisesTypeError
.
-
char *PyBytes_AS_STRING(PyObject *string)¶
Similar to
PyBytes_AsString()
, but without error checking.
-
int PyBytes_AsStringAndSize(PyObject *obj, char **buffer, Py_ssize_t *length)¶
- Part of the Stable ABI.
Return the null-terminated contents of the object obj through the output variables buffer and length. Returns
0
on success.If length is
NULL
, the bytes object may not contain embedded null bytes; if it does, the function returns-1
and aValueError
is raised.The buffer refers to an internal buffer of obj, which includes an additional null byte at the end (not counted in length). The data must not be modified in any way, unless the object was just created using
PyBytes_FromStringAndSize(NULL, size)
. It must not be deallocated. If obj is not a bytes object at all,PyBytes_AsStringAndSize()
returns-1
and raisesTypeError
.Changed in version 3.5: Previously,
TypeError
was raised when embedded null bytes were encountered in the bytes object.
-
void PyBytes_Concat(PyObject **bytes, PyObject *newpart)¶
- Part of the Stable ABI.
Create a new bytes object in *bytes containing the contents of newpart appended to bytes; the caller will own the new reference. The reference to the old value of bytes will be stolen. If the new object cannot be created, the old reference to bytes will still be discarded and the value of *bytes will be set to
NULL
; the appropriate exception will be set.
-
void PyBytes_ConcatAndDel(PyObject **bytes, PyObject *newpart)¶
- Part of the Stable ABI.
Create a new bytes object in *bytes containing the contents of newpart appended to bytes. This version releases the strong reference to newpart (i.e. decrements its reference count).
-
PyObject *PyBytes_Join(PyObject *sep, PyObject *iterable)¶
Similar to
sep.join(iterable)
in Python.sep must be Python
bytes
object. (Note thatPyUnicode_Join()
acceptsNULL
separator and treats it as a space, whereasPyBytes_Join()
doesn’t acceptNULL
separator.)iterable must be an iterable object yielding objects that implement the buffer protocol.
On success, return a new
bytes
object. On error, set an exception and returnNULL
.Added in version 3.14.
-
int _PyBytes_Resize(PyObject **bytes, Py_ssize_t newsize)¶
Resize a bytes object. newsize will be the new length of the bytes object. You can think of it as creating a new bytes object and destroying the old one, only more efficiently. Pass the address of an existing bytes object as an lvalue (it may be written into), and the new size desired. On success, *bytes holds the resized bytes object and
0
is returned; the address in *bytes may differ from its input value. If the reallocation fails, the original bytes object at *bytes is deallocated, *bytes is set toNULL
,MemoryError
is set, and-1
is returned.
-
PyObject *PyBytes_Repr(PyObject *bytes, int smartquotes)¶
- Part of the Stable ABI.
Get the string representation of bytes. This function is currently used to implement
bytes.__repr__()
in Python.This function does not do type checking; it is undefined behavior to pass bytes as a non-bytes object or
NULL
.If smartquotes is true, the representation will use a double-quoted string instead of single-quoted string when single-quotes are present in bytes. For example, the byte string
'Python'
would be represented asb"'Python'"
when smartquotes is true, orb'\'Python\''
when it is false.On success, this function returns a strong reference to a
str
object containing the representation. On failure, this returnsNULL
with an exception set.
-
PyObject *PyBytes_DecodeEscape(const char *s, Py_ssize_t len, const char *errors, Py_ssize_t unicode, const char *recode_encoding)¶
- Part of the Stable ABI.
Unescape a backslash-escaped string s. s must not be
NULL
. len must be the size of s.errors must be one of
"strict"
,"replace"
, or"ignore"
. If errors isNULL
, then"strict"
is used by default.On success, this function returns a strong reference to a Python
bytes
object containing the unescaped string. On failure, this function returnsNULL
with an exception set.Changed in version 3.9: unicode and recode_encoding are now unused.


An introduction to the ipaddress module

An introduction to the ipaddress module¶
- author:
Peter Moody
- author:
Nick Coghlan
Creating Address/Network/Interface objects¶
Since ipaddress
is a module for inspecting and manipulating IP addresses,
the first thing you’ll want to do is create some objects. You can use
ipaddress
to create objects from strings and integers.
A Note on IP Versions¶
For readers that aren’t particularly familiar with IP addressing, it’s important to know that the Internet Protocol (IP) is currently in the process of moving from version 4 of the protocol to version 6. This transition is occurring largely because version 4 of the protocol doesn’t provide enough addresses to handle the needs of the whole world, especially given the increasing number of devices with direct connections to the internet.
Explaining the details of the differences between the two versions of the protocol is beyond the scope of this introduction, but readers need to at least be aware that these two versions exist, and it will sometimes be necessary to force the use of one version or the other.
IP Host Addresses¶
Addresses, often referred to as “host addresses” are the most basic unit
when working with IP addressing. The simplest way to create addresses is
to use the ipaddress.ip_address()
factory function, which automatically
determines whether to create an IPv4 or IPv6 address based on the passed in
value:
>>> ipaddress.ip_address('192.0.2.1')
IPv4Address('192.0.2.1')
>>> ipaddress.ip_address('2001:DB8::1')
IPv6Address('2001:db8::1')
Addresses can also be created directly from integers. Values that will fit within 32 bits are assumed to be IPv4 addresses:
>>> ipaddress.ip_address(3221225985)
IPv4Address('192.0.2.1')
>>> ipaddress.ip_address(42540766411282592856903984951653826561)
IPv6Address('2001:db8::1')
To force the use of IPv4 or IPv6 addresses, the relevant classes can be invoked directly. This is particularly useful to force creation of IPv6 addresses for small integers:
>>> ipaddress.ip_address(1)
IPv4Address('0.0.0.1')
>>> ipaddress.IPv4Address(1)
IPv4Address('0.0.0.1')
>>> ipaddress.IPv6Address(1)
IPv6Address('::1')
Defining Networks¶
Host addresses are usually grouped together into IP networks, so
ipaddress
provides a way to create, inspect and manipulate network
definitions. IP network objects are constructed from strings that define the
range of host addresses that are part of that network. The simplest form
for that information is a “network address/network prefix” pair, where the
prefix defines the number of leading bits that are compared to determine
whether or not an address is part of the network and the network address
defines the expected value of those bits.
As for addresses, a factory function is provided that determines the correct IP version automatically:
>>> ipaddress.ip_network('192.0.2.0/24')
IPv4Network('192.0.2.0/24')
>>> ipaddress.ip_network('2001:db8::0/96')
IPv6Network('2001:db8::/96')
Network objects cannot have any host bits set. The practical effect of this
is that 192.0.2.1/24
does not describe a network. Such definitions are
referred to as interface objects since the ip-on-a-network notation is
commonly used to describe network interfaces of a computer on a given network
and are described further in the next section.
By default, attempting to create a network object with host bits set will
result in ValueError
being raised. To request that the
additional bits instead be coerced to zero, the flag strict=False
can
be passed to the constructor:
>>> ipaddress.ip_network('192.0.2.1/24')
Traceback (most recent call last):
...
ValueError: 192.0.2.1/24 has host bits set
>>> ipaddress.ip_network('192.0.2.1/24', strict=False)
IPv4Network('192.0.2.0/24')
While the string form offers significantly more flexibility, networks can also be defined with integers, just like host addresses. In this case, the network is considered to contain only the single address identified by the integer, so the network prefix includes the entire network address:
>>> ipaddress.ip_network(3221225984)
IPv4Network('192.0.2.0/32')
>>> ipaddress.ip_network(42540766411282592856903984951653826560)
IPv6Network('2001:db8::/128')
As with addresses, creation of a particular kind of network can be forced by calling the class constructor directly instead of using the factory function.
Host Interfaces¶
As mentioned just above, if you need to describe an address on a particular
network, neither the address nor the network classes are sufficient.
Notation like 192.0.2.1/24
is commonly used by network engineers and the
people who write tools for firewalls and routers as shorthand for “the host
192.0.2.1
on the network 192.0.2.0/24
”, Accordingly, ipaddress
provides a set of hybrid classes that associate an address with a particular
network. The interface for creation is identical to that for defining network
objects, except that the address portion isn’t constrained to being a network
address.
>>> ipaddress.ip_interface('192.0.2.1/24')
IPv4Interface('192.0.2.1/24')
>>> ipaddress.ip_interface('2001:db8::1/96')
IPv6Interface('2001:db8::1/96')
Integer inputs are accepted (as with networks), and use of a particular IP version can be forced by calling the relevant constructor directly.
Inspecting Address/Network/Interface Objects¶
You’ve gone to the trouble of creating an IPv(4|6)(Address|Network|Interface)
object, so you probably want to get information about it. ipaddress
tries to make doing this easy and intuitive.
Extracting the IP version:
>>> addr4 = ipaddress.ip_address('192.0.2.1')
>>> addr6 = ipaddress.ip_address('2001:db8::1')
>>> addr6.version
6
>>> addr4.version
4
Obtaining the network from an interface:
>>> host4 = ipaddress.ip_interface('192.0.2.1/24')
>>> host4.network
IPv4Network('192.0.2.0/24')
>>> host6 = ipaddress.ip_interface('2001:db8::1/96')
>>> host6.network
IPv6Network('2001:db8::/96')
Finding out how many individual addresses are in a network:
>>> net4 = ipaddress.ip_network('192.0.2.0/24')
>>> net4.num_addresses
256
>>> net6 = ipaddress.ip_network('2001:db8::0/96')
>>> net6.num_addresses
4294967296
Iterating through the “usable” addresses on a network:
>>> net4 = ipaddress.ip_network('192.0.2.0/24')
>>> for x in net4.hosts():
... print(x)
192.0.2.1
192.0.2.2
192.0.2.3
192.0.2.4
...
192.0.2.252
192.0.2.253
192.0.2.254
Obtaining the netmask (i.e. set bits corresponding to the network prefix) or the hostmask (any bits that are not part of the netmask):
>>> net4 = ipaddress.ip_network('192.0.2.0/24')
>>> net4.netmask
IPv4Address('255.255.255.0')
>>> net4.hostmask
IPv4Address('0.0.0.255')
>>> net6 = ipaddress.ip_network('2001:db8::0/96')
>>> net6.netmask
IPv6Address('ffff:ffff:ffff:ffff:ffff:ffff::')
>>> net6.hostmask
IPv6Address('::ffff:ffff')
Exploding or compressing the address:
>>> addr6.exploded
'2001:0db8:0000:0000:0000:0000:0000:0001'
>>> addr6.compressed
'2001:db8::1'
>>> net6.exploded
'2001:0db8:0000:0000:0000:0000:0000:0000/96'
>>> net6.compressed
'2001:db8::/96'
While IPv4 doesn’t support explosion or compression, the associated objects still provide the relevant properties so that version neutral code can easily ensure the most concise or most verbose form is used for IPv6 addresses while still correctly handling IPv4 addresses.
Networks as lists of Addresses¶
It’s sometimes useful to treat networks as lists. This means it is possible to index them like this:
>>> net4[1]
IPv4Address('192.0.2.1')
>>> net4[-1]
IPv4Address('192.0.2.255')
>>> net6[1]
IPv6Address('2001:db8::1')
>>> net6[-1]
IPv6Address('2001:db8::ffff:ffff')
It also means that network objects lend themselves to using the list membership test syntax like this:
if address in network:
# do something
Containment testing is done efficiently based on the network prefix:
>>> addr4 = ipaddress.ip_address('192.0.2.1')
>>> addr4 in ipaddress.ip_network('192.0.2.0/24')
True
>>> addr4 in ipaddress.ip_network('192.0.3.0/24')
False
Comparisons¶
ipaddress
provides some simple, hopefully intuitive ways to compare
objects, where it makes sense:
>>> ipaddress.ip_address('192.0.2.1') < ipaddress.ip_address('192.0.2.2')
True
A TypeError
exception is raised if you try to compare objects of
different versions or different types.
Using IP Addresses with other modules¶
Other modules that use IP addresses (such as socket
) usually won’t
accept objects from this module directly. Instead, they must be coerced to
an integer or string that the other module will accept:
>>> addr4 = ipaddress.ip_address('192.0.2.1')
>>> str(addr4)
'192.0.2.1'
>>> int(addr4)
3221225985
Getting more detail when instance creation fails¶
When creating address/network/interface objects using the version-agnostic
factory functions, any errors will be reported as ValueError
with
a generic error message that simply says the passed in value was not
recognized as an object of that type. The lack of a specific error is
because it’s necessary to know whether the value is supposed to be IPv4
or IPv6 in order to provide more detail on why it has been rejected.
To support use cases where it is useful to have access to this additional
detail, the individual class constructors actually raise the
ValueError
subclasses ipaddress.AddressValueError
and
ipaddress.NetmaskValueError
to indicate exactly which part of
the definition failed to parse correctly.
The error messages are significantly more detailed when using the class constructors directly. For example:
>>> ipaddress.ip_address("192.168.0.256")
Traceback (most recent call last):
...
ValueError: '192.168.0.256' does not appear to be an IPv4 or IPv6 address
>>> ipaddress.IPv4Address("192.168.0.256")
Traceback (most recent call last):
...
ipaddress.AddressValueError: Octet 256 (> 255) not permitted in '192.168.0.256'
>>> ipaddress.ip_network("192.168.0.1/64")
Traceback (most recent call last):
...
ValueError: '192.168.0.1/64' does not appear to be an IPv4 or IPv6 network
>>> ipaddress.IPv4Network("192.168.0.1/64")
Traceback (most recent call last):
...
ipaddress.NetmaskValueError: '64' is not a valid netmask
However, both of the module specific exceptions have ValueError
as their
parent class, so if you’re not concerned with the particular type of error,
you can still write code like the following:
try:
network = ipaddress.IPv4Network(address)
except ValueError:
print('address/netmask is invalid for IPv4:', address)


List Objects

List Objects¶
-
PyTypeObject PyList_Type¶
- Part of the Stable ABI.
This instance of
PyTypeObject
represents the Python list type. This is the same object aslist
in the Python layer.
-
int PyList_Check(PyObject *p)¶
Return true if p is a list object or an instance of a subtype of the list type. This function always succeeds.
-
int PyList_CheckExact(PyObject *p)¶
Return true if p is a list object, but not an instance of a subtype of the list type. This function always succeeds.
-
PyObject *PyList_New(Py_ssize_t len)¶
- Return value: New reference. Part of the Stable ABI.
Return a new list of length len on success, or
NULL
on failure.Note
If len is greater than zero, the returned list object’s items are set to
NULL
. Thus you cannot use abstract API functions such asPySequence_SetItem()
or expose the object to Python code before setting all items to a real object withPyList_SetItem()
orPyList_SET_ITEM()
. The following APIs are safe APIs before the list is fully initialized:PyList_SetItem()
andPyList_SET_ITEM()
.
-
Py_ssize_t PyList_Size(PyObject *list)¶
- Part of the Stable ABI.
Return the length of the list object in list; this is equivalent to
len(list)
on a list object.
-
Py_ssize_t PyList_GET_SIZE(PyObject *list)¶
Similar to
PyList_Size()
, but without error checking.
-
PyObject *PyList_GetItemRef(PyObject *list, Py_ssize_t index)¶
- Return value: New reference. Part of the Stable ABI since version 3.13.
Return the object at position index in the list pointed to by list. The position must be non-negative; indexing from the end of the list is not supported. If index is out of bounds (
<0 or >=len(list)
), returnNULL
and set anIndexError
exception.Added in version 3.13.
-
PyObject *PyList_GetItem(PyObject *list, Py_ssize_t index)¶
- Return value: Borrowed reference. Part of the Stable ABI.
Like
PyList_GetItemRef()
, but returns a borrowed reference instead of a strong reference.
-
PyObject *PyList_GET_ITEM(PyObject *list, Py_ssize_t i)¶
- Return value: Borrowed reference.
Similar to
PyList_GetItem()
, but without error checking.
-
int PyList_SetItem(PyObject *list, Py_ssize_t index, PyObject *item)¶
- Part of the Stable ABI.
Set the item at index index in list to item. Return
0
on success. If index is out of bounds, return-1
and set anIndexError
exception.Note
This function “steals” a reference to item and discards a reference to an item already in the list at the affected position.
-
void PyList_SET_ITEM(PyObject *list, Py_ssize_t i, PyObject *o)¶
Macro form of
PyList_SetItem()
without error checking. This is normally only used to fill in new lists where there is no previous content.Bounds checking is performed as an assertion if Python is built in debug mode or
with assertions
.Note
This macro “steals” a reference to item, and, unlike
PyList_SetItem()
, does not discard a reference to any item that is being replaced; any reference in list at position i will be leaked.
-
int PyList_Insert(PyObject *list, Py_ssize_t index, PyObject *item)¶
- Part of the Stable ABI.
Insert the item item into list list in front of index index. Return
0
if successful; return-1
and set an exception if unsuccessful. Analogous tolist.insert(index, item)
.
-
int PyList_Append(PyObject *list, PyObject *item)¶
- Part of the Stable ABI.
Append the object item at the end of list list. Return
0
if successful; return-1
and set an exception if unsuccessful. Analogous tolist.append(item)
.
-
PyObject *PyList_GetSlice(PyObject *list, Py_ssize_t low, Py_ssize_t high)¶
- Return value: New reference. Part of the Stable ABI.
Return a list of the objects in list containing the objects between low and high. Return
NULL
and set an exception if unsuccessful. Analogous tolist[low:high]
. Indexing from the end of the list is not supported.
-
int PyList_SetSlice(PyObject *list, Py_ssize_t low, Py_ssize_t high, PyObject *itemlist)¶
- Part of the Stable ABI.
Set the slice of list between low and high to the contents of itemlist. Analogous to
list[low:high] = itemlist
. The itemlist may beNULL
, indicating the assignment of an empty list (slice deletion). Return0
on success,-1
on failure. Indexing from the end of the list is not supported.
-
int PyList_Extend(PyObject *list, PyObject *iterable)¶
Extend list with the contents of iterable. This is the same as
PyList_SetSlice(list, PY_SSIZE_T_MAX, PY_SSIZE_T_MAX, iterable)
and analogous tolist.extend(iterable)
orlist += iterable
.Raise an exception and return
-1
if list is not alist
object. Return 0 on success.Added in version 3.13.
-
int PyList_Clear(PyObject *list)¶
Remove all items from list. This is the same as
PyList_SetSlice(list, 0, PY_SSIZE_T_MAX, NULL)
and analogous tolist.clear()
ordel list[:]
.Raise an exception and return
-1
if list is not alist
object. Return 0 on success.Added in version 3.13.
-
int PyList_Sort(PyObject *list)¶
- Part of the Stable ABI.
Sort the items of list in place. Return
0
on success,-1
on failure. This is equivalent tolist.sort()
.
-
int PyList_Reverse(PyObject *list)¶
- Part of the Stable ABI.
Reverse the items of list in place. Return
0
on success,-1
on failure. This is the equivalent oflist.reverse()
.
-
PyObject *PyList_AsTuple(PyObject *list)¶
- Return value: New reference. Part of the Stable ABI.
Return a new tuple object containing the contents of list; equivalent to
tuple(list)
.


Tuple Objects

Tuple Objects¶
-
PyTypeObject PyTuple_Type¶
- Part of the Stable ABI.
This instance of
PyTypeObject
represents the Python tuple type; it is the same object astuple
in the Python layer.
-
int PyTuple_Check(PyObject *p)¶
Return true if p is a tuple object or an instance of a subtype of the tuple type. This function always succeeds.
-
int PyTuple_CheckExact(PyObject *p)¶
Return true if p is a tuple object, but not an instance of a subtype of the tuple type. This function always succeeds.
-
PyObject *PyTuple_New(Py_ssize_t len)¶
- Return value: New reference. Part of the Stable ABI.
Return a new tuple object of size len, or
NULL
with an exception set on failure.
-
PyObject *PyTuple_Pack(Py_ssize_t n, ...)¶
- Return value: New reference. Part of the Stable ABI.
Return a new tuple object of size n, or
NULL
with an exception set on failure. The tuple values are initialized to the subsequent n C arguments pointing to Python objects.PyTuple_Pack(2, a, b)
is equivalent toPy_BuildValue("(OO)", a, b)
.
-
Py_ssize_t PyTuple_Size(PyObject *p)¶
- Part of the Stable ABI.
Take a pointer to a tuple object, and return the size of that tuple. On error, return
-1
with an exception set.
-
Py_ssize_t PyTuple_GET_SIZE(PyObject *p)¶
Like
PyTuple_Size()
, but without error checking.
-
PyObject *PyTuple_GetItem(PyObject *p, Py_ssize_t pos)¶
- Return value: Borrowed reference. Part of the Stable ABI.
Return the object at position pos in the tuple pointed to by p. If pos is negative or out of bounds, return
NULL
and set anIndexError
exception.The returned reference is borrowed from the tuple p (that is: it is only valid as long as you hold a reference to p). To get a strong reference, use
Py_NewRef(PyTuple_GetItem(...))
orPySequence_GetItem()
.
-
PyObject *PyTuple_GET_ITEM(PyObject *p, Py_ssize_t pos)¶
- Return value: Borrowed reference.
Like
PyTuple_GetItem()
, but does no checking of its arguments.
-
PyObject *PyTuple_GetSlice(PyObject *p, Py_ssize_t low, Py_ssize_t high)¶
- Return value: New reference. Part of the Stable ABI.
Return the slice of the tuple pointed to by p between low and high, or
NULL
with an exception set on failure.This is the equivalent of the Python expression
p[low:high]
. Indexing from the end of the tuple is not supported.
-
int PyTuple_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o)¶
- Part of the Stable ABI.
Insert a reference to object o at position pos of the tuple pointed to by p. Return
0
on success. If pos is out of bounds, return-1
and set anIndexError
exception.Note
This function “steals” a reference to o and discards a reference to an item already in the tuple at the affected position.
-
void PyTuple_SET_ITEM(PyObject *p, Py_ssize_t pos, PyObject *o)¶
Like
PyTuple_SetItem()
, but does no error checking, and should only be used to fill in brand new tuples.Bounds checking is performed as an assertion if Python is built in debug mode or
with assertions
.Note
This function “steals” a reference to o, and, unlike
PyTuple_SetItem()
, does not discard a reference to any item that is being replaced; any reference in the tuple at position pos will be leaked.Warning
This macro should only be used on tuples that are newly created. Using this macro on a tuple that is already in use (or in other words, has a refcount > 1) could lead to undefined behavior.
-
int _PyTuple_Resize(PyObject **p, Py_ssize_t newsize)¶
Can be used to resize a tuple. newsize will be the new length of the tuple. Because tuples are supposed to be immutable, this should only be used if there is only one reference to the object. Do not use this if the tuple may already be known to some other part of the code. The tuple will always grow or shrink at the end. Think of this as destroying the old tuple and creating a new one, only more efficiently. Returns
0
on success. Client code should never assume that the resulting value of*p
will be the same as before calling this function. If the object referenced by*p
is replaced, the original*p
is destroyed. On failure, returns-1
and sets*p
toNULL
, and raisesMemoryError
orSystemError
.
Struct Sequence Objects¶
Struct sequence objects are the C equivalent of namedtuple()
objects, i.e. a sequence whose items can also be accessed through attributes.
To create a struct sequence, you first have to create a specific struct sequence
type.
-
PyTypeObject *PyStructSequence_NewType(PyStructSequence_Desc *desc)¶
- Return value: New reference. Part of the Stable ABI.
Create a new struct sequence type from the data in desc, described below. Instances of the resulting type can be created with
PyStructSequence_New()
.Return
NULL
with an exception set on failure.
-
void PyStructSequence_InitType(PyTypeObject *type, PyStructSequence_Desc *desc)¶
Initializes a struct sequence type type from desc in place.
-
int PyStructSequence_InitType2(PyTypeObject *type, PyStructSequence_Desc *desc)¶
Like
PyStructSequence_InitType()
, but returns0
on success and-1
with an exception set on failure.Added in version 3.4.
-
type PyStructSequence_Desc¶
- Part of the Stable ABI (including all members).
Contains the meta information of a struct sequence type to create.
-
const char *name¶
Fully qualified name of the type; null-terminated UTF-8 encoded. The name must contain the module name.
-
const char *doc¶
Pointer to docstring for the type or
NULL
to omit.
-
PyStructSequence_Field *fields¶
Pointer to
NULL
-terminated array with field names of the new type.
-
int n_in_sequence¶
Number of fields visible to the Python side (if used as tuple).
-
const char *name¶
-
type PyStructSequence_Field¶
- Part of the Stable ABI (including all members).
Describes a field of a struct sequence. As a struct sequence is modeled as a tuple, all fields are typed as PyObject*. The index in the
fields
array of thePyStructSequence_Desc
determines which field of the struct sequence is described.-
const char *name¶
Name for the field or
NULL
to end the list of named fields, set toPyStructSequence_UnnamedField
to leave unnamed.
-
const char *doc¶
Field docstring or
NULL
to omit.
-
const char *name¶
-
const char *const PyStructSequence_UnnamedField¶
- Part of the Stable ABI since version 3.11.
Special value for a field name to leave it unnamed.
Changed in version 3.9: The type was changed from
char *
.
-
PyObject *PyStructSequence_New(PyTypeObject *type)¶
- Return value: New reference. Part of the Stable ABI.
Creates an instance of type, which must have been created with
PyStructSequence_NewType()
.Return
NULL
with an exception set on failure.
-
PyObject *PyStructSequence_GetItem(PyObject *p, Py_ssize_t pos)¶
- Return value: Borrowed reference. Part of the Stable ABI.
Return the object at position pos in the struct sequence pointed to by p.
Bounds checking is performed as an assertion if Python is built in debug mode or
with assertions
.
-
PyObject *PyStructSequence_GET_ITEM(PyObject *p, Py_ssize_t pos)¶
- Return value: Borrowed reference.
Alias to
PyStructSequence_GetItem()
.Changed in version 3.13: Now implemented as an alias to
PyStructSequence_GetItem()
.
-
void PyStructSequence_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o)¶
- Part of the Stable ABI.
Sets the field at index pos of the struct sequence p to value o. Like
PyTuple_SET_ITEM()
, this should only be used to fill in brand new instances.Bounds checking is performed as an assertion if Python is built in debug mode or
with assertions
.Note
This function “steals” a reference to o.
-
void PyStructSequence_SET_ITEM(PyObject *p, Py_ssize_t *pos, PyObject *o)¶
Alias to
PyStructSequence_SetItem()
.Changed in version 3.13: Now implemented as an alias to
PyStructSequence_SetItem()
.


The Very High Level Layer

The Very High Level Layer¶
The functions in this chapter will let you execute Python source code given in a file or a buffer, but they will not let you interact in a more detailed way with the interpreter.
Several of these functions accept a start symbol from the grammar as a
parameter. The available start symbols are Py_eval_input
,
Py_file_input
, Py_single_input
, and
Py_func_type_input
. These are described following the functions
which accept them as parameters.
Note also that several of these functions take FILE* parameters. One
particular issue which needs to be handled carefully is that the FILE
structure for different C libraries can be different and incompatible. Under
Windows (at least), it is possible for dynamically linked extensions to actually
use different libraries, so care should be taken that FILE* parameters
are only passed to these functions if it is certain that they were created by
the same library that the Python runtime is using.
-
int PyRun_AnyFile(FILE *fp, const char *filename)¶
This is a simplified interface to
PyRun_AnyFileExFlags()
below, leaving closeit set to0
and flags set toNULL
.
-
int PyRun_AnyFileFlags(FILE *fp, const char *filename, PyCompilerFlags *flags)¶
This is a simplified interface to
PyRun_AnyFileExFlags()
below, leaving the closeit argument set to0
.
-
int PyRun_AnyFileEx(FILE *fp, const char *filename, int closeit)¶
This is a simplified interface to
PyRun_AnyFileExFlags()
below, leaving the flags argument set toNULL
.
-
int PyRun_AnyFileExFlags(FILE *fp, const char *filename, int closeit, PyCompilerFlags *flags)¶
If fp refers to a file associated with an interactive device (console or terminal input or Unix pseudo-terminal), return the value of
PyRun_InteractiveLoop()
, otherwise return the result ofPyRun_SimpleFile()
. filename is decoded from the filesystem encoding (sys.getfilesystemencoding()
). If filename isNULL
, this function uses"???"
as the filename. If closeit is true, the file is closed beforePyRun_SimpleFileExFlags()
returns.
-
int PyRun_SimpleString(const char *command)¶
This is a simplified interface to
PyRun_SimpleStringFlags()
below, leaving thePyCompilerFlags
* argument set toNULL
.
-
int PyRun_SimpleStringFlags(const char *command, PyCompilerFlags *flags)¶
Executes the Python source code from command in the
__main__
module according to the flags argument. If__main__
does not already exist, it is created. Returns0
on success or-1
if an exception was raised. If there was an error, there is no way to get the exception information. For the meaning of flags, see below.Note that if an otherwise unhandled
SystemExit
is raised, this function will not return-1
, but exit the process, as long asPyConfig.inspect
is zero.
-
int PyRun_SimpleFile(FILE *fp, const char *filename)¶
This is a simplified interface to
PyRun_SimpleFileExFlags()
below, leaving closeit set to0
and flags set toNULL
.
-
int PyRun_SimpleFileEx(FILE *fp, const char *filename, int closeit)¶
This is a simplified interface to
PyRun_SimpleFileExFlags()
below, leaving flags set toNULL
.
-
int PyRun_SimpleFileExFlags(FILE *fp, const char *filename, int closeit, PyCompilerFlags *flags)¶
Similar to
PyRun_SimpleStringFlags()
, but the Python source code is read from fp instead of an in-memory string. filename should be the name of the file, it is decoded from filesystem encoding and error handler. If closeit is true, the file is closed beforePyRun_SimpleFileExFlags()
returns.Note
On Windows, fp should be opened as binary mode (e.g.
fopen(filename, "rb")
). Otherwise, Python may not handle script file with LF line ending correctly.
-
int PyRun_InteractiveOneObject(FILE *fp, PyObject *filename, PyCompilerFlags *flags)¶
Read and execute a single statement from a file associated with an interactive device according to the flags argument. The user will be prompted using
sys.ps1
andsys.ps2
. filename must be a Pythonstr
object.Returns
0
when the input was executed successfully,-1
if there was an exception, or an error code from theerrcode.h
include file distributed as part of Python if there was a parse error. (Note thaterrcode.h
is not included byPython.h
, so must be included specifically if needed.)
-
int PyRun_InteractiveOne(FILE *fp, const char *filename)¶
This is a simplified interface to
PyRun_InteractiveOneFlags()
below, leaving flags set toNULL
.
-
int PyRun_InteractiveOneFlags(FILE *fp, const char *filename, PyCompilerFlags *flags)¶
Similar to
PyRun_InteractiveOneObject()
, but filename is a const char*, which is decoded from the filesystem encoding and error handler.
-
int PyRun_InteractiveLoop(FILE *fp, const char *filename)¶
This is a simplified interface to
PyRun_InteractiveLoopFlags()
below, leaving flags set toNULL
.
-
int PyRun_InteractiveLoopFlags(FILE *fp, const char *filename, PyCompilerFlags *flags)¶
Read and execute statements from a file associated with an interactive device until EOF is reached. The user will be prompted using
sys.ps1
andsys.ps2
. filename is decoded from the filesystem encoding and error handler. Returns0
at EOF or a negative number upon failure.
-
int (*PyOS_InputHook)(void)¶
- Part of the Stable ABI.
Can be set to point to a function with the prototype
int func(void)
. The function will be called when Python’s interpreter prompt is about to become idle and wait for user input from the terminal. The return value is ignored. Overriding this hook can be used to integrate the interpreter’s prompt with other event loops, as done inModules/_tkinter.c
in the Python source code.Changed in version 3.12: This function is only called from the main interpreter.
-
char *(*PyOS_ReadlineFunctionPointer)(FILE*, FILE*, const char*)¶
Can be set to point to a function with the prototype
char *func(FILE *stdin, FILE *stdout, char *prompt)
, overriding the default function used to read a single line of input at the interpreter’s prompt. The function is expected to output the string prompt if it’s notNULL
, and then read a line of input from the provided standard input file, returning the resulting string. For example, Thereadline
module sets this hook to provide line-editing and tab-completion features.The result must be a string allocated by
PyMem_RawMalloc()
orPyMem_RawRealloc()
, orNULL
if an error occurred.Changed in version 3.4: The result must be allocated by
PyMem_RawMalloc()
orPyMem_RawRealloc()
, instead of being allocated byPyMem_Malloc()
orPyMem_Realloc()
.Changed in version 3.12: This function is only called from the main interpreter.
-
PyObject *PyRun_String(const char *str, int start, PyObject *globals, PyObject *locals)¶
- Return value: New reference.
This is a simplified interface to
PyRun_StringFlags()
below, leaving flags set toNULL
.
-
PyObject *PyRun_StringFlags(const char *str, int start, PyObject *globals, PyObject *locals, PyCompilerFlags *flags)¶
- Return value: New reference.
Execute Python source code from str in the context specified by the objects globals and locals with the compiler flags specified by flags. globals must be a dictionary; locals can be any object that implements the mapping protocol. The parameter start specifies the start symbol and must be one of the available start symbols.
Returns the result of executing the code as a Python object, or
NULL
if an exception was raised.
-
PyObject *PyRun_File(FILE *fp, const char *filename, int start, PyObject *globals, PyObject *locals)¶
- Return value: New reference.
This is a simplified interface to
PyRun_FileExFlags()
below, leaving closeit set to0
and flags set toNULL
.
-
PyObject *PyRun_FileEx(FILE *fp, const char *filename, int start, PyObject *globals, PyObject *locals, int closeit)¶
- Return value: New reference.
This is a simplified interface to
PyRun_FileExFlags()
below, leaving flags set toNULL
.
-
PyObject *PyRun_FileFlags(FILE *fp, const char *filename, int start, PyObject *globals, PyObject *locals, PyCompilerFlags *flags)¶
- Return value: New reference.
This is a simplified interface to
PyRun_FileExFlags()
below, leaving closeit set to0
.
-
PyObject *PyRun_FileExFlags(FILE *fp, const char *filename, int start, PyObject *globals, PyObject *locals, int closeit, PyCompilerFlags *flags)¶
- Return value: New reference.
Similar to
PyRun_StringFlags()
, but the Python source code is read from fp instead of an in-memory string. filename should be the name of the file, it is decoded from the filesystem encoding and error handler. If closeit is true, the file is closed beforePyRun_FileExFlags()
returns.
-
PyObject *Py_CompileString(const char *str, const char *filename, int start)¶
- Return value: New reference. Part of the Stable ABI.
This is a simplified interface to
Py_CompileStringFlags()
below, leaving flags set toNULL
.
-
PyObject *Py_CompileStringFlags(const char *str, const char *filename, int start, PyCompilerFlags *flags)¶
- Return value: New reference.
This is a simplified interface to
Py_CompileStringExFlags()
below, with optimize set to-1
.
-
PyObject *Py_CompileStringObject(const char *str, PyObject *filename, int start, PyCompilerFlags *flags, int optimize)¶
- Return value: New reference.
Parse and compile the Python source code in str, returning the resulting code object. The start symbol is given by start; this can be used to constrain the code which can be compiled and should be available start symbols. The filename specified by filename is used to construct the code object and may appear in tracebacks or
SyntaxError
exception messages. This returnsNULL
if the code cannot be parsed or compiled.The integer optimize specifies the optimization level of the compiler; a value of
-1
selects the optimization level of the interpreter as given by-O
options. Explicit levels are0
(no optimization;__debug__
is true),1
(asserts are removed,__debug__
is false) or2
(docstrings are removed too).Added in version 3.4.
-
PyObject *Py_CompileStringExFlags(const char *str, const char *filename, int start, PyCompilerFlags *flags, int optimize)¶
- Return value: New reference.
Like
Py_CompileStringObject()
, but filename is a byte string decoded from the filesystem encoding and error handler.Added in version 3.2.
-
PyObject *PyEval_EvalCode(PyObject *co, PyObject *globals, PyObject *locals)¶
- Return value: New reference. Part of the Stable ABI.
This is a simplified interface to
PyEval_EvalCodeEx()
, with just the code object, and global and local variables. The other arguments are set toNULL
.
-
PyObject *PyEval_EvalCodeEx(PyObject *co, PyObject *globals, PyObject *locals, PyObject *const *args, int argcount, PyObject *const *kws, int kwcount, PyObject *const *defs, int defcount, PyObject *kwdefs, PyObject *closure)¶
- Return value: New reference. Part of the Stable ABI.
Evaluate a precompiled code object, given a particular environment for its evaluation. This environment consists of a dictionary of global variables, a mapping object of local variables, arrays of arguments, keywords and defaults, a dictionary of default values for keyword-only arguments and a closure tuple of cells.
-
PyObject *PyEval_EvalFrame(PyFrameObject *f)¶
- Return value: New reference. Part of the Stable ABI.
Evaluate an execution frame. This is a simplified interface to
PyEval_EvalFrameEx()
, for backward compatibility.
-
PyObject *PyEval_EvalFrameEx(PyFrameObject *f, int throwflag)¶
- Return value: New reference. Part of the Stable ABI.
This is the main, unvarnished function of Python interpretation. The code object associated with the execution frame f is executed, interpreting bytecode and executing calls as needed. The additional throwflag parameter can mostly be ignored - if true, then it causes an exception to immediately be thrown; this is used for the
throw()
methods of generator objects.Changed in version 3.4: This function now includes a debug assertion to help ensure that it does not silently discard an active exception.
-
int PyEval_MergeCompilerFlags(PyCompilerFlags *cf)¶
This function changes the flags of the current evaluation frame, and returns true on success, false on failure.
-
struct PyCompilerFlags¶
This is the structure used to hold compiler flags. In cases where code is only being compiled, it is passed as
int flags
, and in cases where code is being executed, it is passed asPyCompilerFlags *flags
. In this case,from __future__ import
can modify flags.Whenever
PyCompilerFlags *flags
isNULL
,cf_flags
is treated as equal to0
, and any modification due tofrom __future__ import
is discarded.-
int cf_flags¶
Compiler flags.
-
int cf_feature_version¶
cf_feature_version is the minor Python version. It should be initialized to
PY_MINOR_VERSION
.The field is ignored by default, it is used if and only if
PyCF_ONLY_AST
flag is set incf_flags
.
Changed in version 3.8: Added cf_feature_version field.
The available compiler flags are accessible as macros:
-
PyCF_ALLOW_TOP_LEVEL_AWAIT¶
-
PyCF_ONLY_AST¶
-
PyCF_OPTIMIZED_AST¶
-
PyCF_TYPE_COMMENTS¶
See compiler flags in documentation of the
ast
Python module, which exports these constants under the same names.
The “
PyCF
” flags above can be combined with “CO_FUTURE
” flags such asCO_FUTURE_ANNOTATIONS
to enable features normally selectable using future statements. See Code Object Flags for a complete list.-
int cf_flags¶
Available start symbols¶
-
int Py_eval_input¶
The start symbol from the Python grammar for isolated expressions; for use with
Py_CompileString()
.
-
int Py_file_input¶
The start symbol from the Python grammar for sequences of statements as read from a file or other source; for use with
Py_CompileString()
. This is the symbol to use when compiling arbitrarily long Python source code.
-
int Py_single_input¶
The start symbol from the Python grammar for a single statement; for use with
Py_CompileString()
. This is the symbol used for the interactive interpreter loop.
-
int Py_func_type_input¶
The start symbol from the Python grammar for a function type; for use with
Py_CompileString()
. This is used to parse “signature type comments” from PEP 484.This requires the
PyCF_ONLY_AST
flag to be set.See also
Added in version 3.8.
Stack Effects¶
See also
-
PY_INVALID_STACK_EFFECT¶
Sentinel value representing an invalid stack effect.
This is currently equivalent to
INT_MAX
.Added in version 3.8.
-
int PyCompile_OpcodeStackEffect(int opcode, int oparg)¶
Compute the stack effect of opcode with argument oparg.
On success, this function returns the stack effect; on failure, this returns
PY_INVALID_STACK_EFFECT
.Added in version 3.4.
-
int PyCompile_OpcodeStackEffectWithJump(int opcode, int oparg, int jump)¶
Similar to
PyCompile_OpcodeStackEffect()
, but don’t include the stack effect of jumping if jump is zero.If jump is
0
, this will not include the stack effect of jumping, but if jump is1
or-1
, this will include it.On success, this function returns the stack effect; on failure, this returns
PY_INVALID_STACK_EFFECT
.Added in version 3.8.


Introduction

Introduction¶
The Application Programmer’s Interface to Python gives C and C++ programmers access to the Python interpreter at a variety of levels. The API is equally usable from C++, but for brevity it is generally referred to as the Python/C API. There are two fundamentally different reasons for using the Python/C API. The first reason is to write extension modules for specific purposes; these are C modules that extend the Python interpreter. This is probably the most common use. The second reason is to use Python as a component in a larger application; this technique is generally referred to as embedding Python in an application.
Writing an extension module is a relatively well-understood process, where a “cookbook” approach works well. There are several tools that automate the process to some extent. While people have embedded Python in other applications since its early existence, the process of embedding Python is less straightforward than writing an extension.
Many API functions are useful independent of whether you’re embedding or extending Python; moreover, most applications that embed Python will need to provide a custom extension as well, so it’s probably a good idea to become familiar with writing an extension before attempting to embed Python in a real application.
Language version compatibility¶
Python’s C API is compatible with C11 and C++11 versions of C and C++.
This is a lower limit: the C API does not require features from later C/C++ versions. You do not need to enable your compiler’s “c11 mode”.
Coding standards¶
If you’re writing C code for inclusion in CPython, you must follow the guidelines and standards defined in PEP 7. These guidelines apply regardless of the version of Python you are contributing to. Following these conventions is not necessary for your own third party extension modules, unless you eventually expect to contribute them to Python.
Include Files¶
All function, type and macro definitions needed to use the Python/C API are included in your code by the following line:
#define PY_SSIZE_T_CLEAN
#include <Python.h>
This implies inclusion of the following standard headers: <stdio.h>
,
<string.h>
, <errno.h>
, <limits.h>
, <assert.h>
and <stdlib.h>
(if available).
Note
Since Python may define some pre-processor definitions which affect the standard
headers on some systems, you must include Python.h
before any standard
headers are included.
It is recommended to always define PY_SSIZE_T_CLEAN
before including
Python.h
. See Parsing arguments and building values for a description of this macro.
All user visible names defined by Python.h (except those defined by the included
standard headers) have one of the prefixes Py
or _Py
. Names beginning
with _Py
are for internal use by the Python implementation and should not be
used by extension writers. Structure member names do not have a reserved prefix.
Note
User code should never define names that begin with Py
or _Py
. This
confuses the reader, and jeopardizes the portability of the user code to
future Python versions, which may define additional names beginning with one
of these prefixes.
The header files are typically installed with Python. On Unix, these are
located in the directories prefix/include/pythonversion/
and
exec_prefix/include/pythonversion/
, where prefix
and
exec_prefix
are defined by the corresponding parameters to Python’s
configure script and version is
'%d.%d' % sys.version_info[:2]
. On Windows, the headers are installed
in prefix/include
, where prefix
is the installation
directory specified to the installer.
To include the headers, place both directories (if different) on your compiler’s
search path for includes. Do not place the parent directories on the search
path and then use #include <pythonX.Y/Python.h>
; this will break on
multi-platform builds since the platform independent headers under
prefix
include the platform specific headers from
exec_prefix
.
C++ users should note that although the API is defined entirely using C, the
header files properly declare the entry points to be extern "C"
. As a result,
there is no need to do anything special to use the API from C++.
Useful macros¶
Several useful macros are defined in the Python header files. Many are
defined closer to where they are useful (for example, Py_RETURN_NONE
,
PyMODINIT_FUNC
).
Others of a more general utility are defined here. This is not necessarily a
complete listing.
-
Py_CAN_START_THREADS¶
If this macro is defined, then the current system is able to start threads.
Currently, all systems supported by CPython (per PEP 11), with the exception of some WebAssembly platforms, support starting threads.
Added in version 3.13.
-
Py_GETENV(s)¶
Like
getenv(s)
, but returnsNULL
if-E
was passed on the command line (seePyConfig.use_environment
).
Docstring macros¶
-
PyDoc_STRVAR(name, str)¶
Creates a variable with name name that can be used in docstrings. If Python is built without docstrings (
--without-doc-strings
), the value will be an empty string.Example:
PyDoc_STRVAR(pop_doc, "Remove and return the rightmost element."); static PyMethodDef deque_methods[] = { // ... {"pop", (PyCFunction)deque_pop, METH_NOARGS, pop_doc}, // ... }
Expands to
PyDoc_VAR(name) = PyDoc_STR(str)
.
-
PyDoc_STR(str)¶
Expands to the given input string, or an empty string if docstrings are disabled (
--without-doc-strings
).Example:
static PyMethodDef pysqlite_row_methods[] = { {"keys", (PyCFunction)pysqlite_row_keys, METH_NOARGS, PyDoc_STR("Returns the keys of the row.")}, {NULL, NULL} };
-
PyDoc_VAR(name)¶
Declares a static character array variable with the given name. Expands to
static const char name[]
For example:
PyDoc_VAR(python_doc) = PyDoc_STR( "A genus of constricting snakes in the Pythonidae family native " "to the tropics and subtropics of the Eastern Hemisphere.");
General utility macros¶
The following macros are for common tasks not specific to Python.
-
Py_UNUSED(arg)¶
Use this for unused arguments in a function definition to silence compiler warnings. Example:
int func(int a, int Py_UNUSED(b)) { return a; }
.Added in version 3.4.
-
Py_GCC_ATTRIBUTE(name)¶
Use a GCC attribute name, hiding it from compilers that don’t support GCC attributes (such as MSVC).
This expands to
__attribute__((name))
on a GCC compiler, and expands to nothing on compilers that don’t support GCC attributes.
Numeric utilities¶
-
Py_ABS(x)¶
Return the absolute value of
x
.The argument may be evaluated more than once. Consequently, do not pass an expression with side-effects directly to this macro.
If the result cannot be represented (for example, if
x
hasINT_MIN
value for int type), the behavior is undefined.Corresponds roughly to
((x) < 0 ? -(x) : (x))
Added in version 3.3.
-
Py_MAX(x, y)¶
-
Py_MIN(x, y)¶
Return the larger or smaller of the arguments, respectively.
Any arguments may be evaluated more than once. Consequently, do not pass an expression with side-effects directly to this macro.
Py_MAX
corresponds roughly to(((x) > (y)) ? (x) : (y))
.Added in version 3.3.
-
Py_ARITHMETIC_RIGHT_SHIFT(type, integer, positions)¶
Similar to
integer >> positions
, but forces sign extension, as the C standard does not define whether a right-shift of a signed integer will perform sign extension or a zero-fill.integer should be any signed integer type. positions is the number of positions to shift to the right.
Both integer and positions can be evaluated more than once; consequently, avoid directly passing a function call or some other operation with side-effects to this macro. Instead, store the result as a variable and then pass it.
type is unused and only kept for backwards compatibility. Historically, type was used to cast integer.
Changed in version 3.1: This macro is now valid for all signed integer types, not just those for which
unsigned type
is legal. As a result, type is no longer used.
-
Py_CHARMASK(c)¶
Argument must be a character or an integer in the range [-128, 127] or [0, 255]. This macro returns
c
cast to anunsigned char
.
Assertion utilities¶
-
Py_UNREACHABLE()¶
Use this when you have a code path that cannot be reached by design. For example, in the
default:
clause in aswitch
statement for which all possible values are covered incase
statements. Use this in places where you might be tempted to put anassert(0)
orabort()
call.In release mode, the macro helps the compiler to optimize the code, and avoids a warning about unreachable code. For example, the macro is implemented with
__builtin_unreachable()
on GCC in release mode.In debug mode, and on unsupported compilers, the macro expands to a call to
Py_FatalError()
.A use for
Py_UNREACHABLE()
is following a call to a function that never returns but that is not declared_Noreturn
.If a code path is very unlikely code but can be reached under exceptional case, this macro must not be used. For example, under low memory condition or if a system call returns a value out of the expected range. In this case, it’s better to report the error to the caller. If the error cannot be reported to caller,
Py_FatalError()
can be used.Added in version 3.7.
-
Py_SAFE_DOWNCAST(value, larger, smaller)¶
Cast value to type smaller from type larger, validating that no information was lost.
On release builds of Python, this is roughly equivalent to
((smaller) value)
(in C++,static_cast<smaller>(value)
will be used instead).On debug builds (implying that
Py_DEBUG
is defined), this asserts that no information was lost with the cast from larger to smaller.value, larger, and smaller may all be evaluated more than once in the expression; consequently, do not pass an expression with side-effects directly to this macro.
-
Py_BUILD_ASSERT(cond)¶
Asserts a compile-time condition cond, as a statement. The build will fail if the condition is false or cannot be evaluated at compile time.
Corresponds roughly to
static_assert(cond)
on C23 and above.For example:
Py_BUILD_ASSERT(sizeof(PyTime_t) == sizeof(int64_t));
Added in version 3.3.
-
Py_BUILD_ASSERT_EXPR(cond)¶
Asserts a compile-time condition cond, as an expression that evaluates to
0
. The build will fail if the condition is false or cannot be evaluated at compile time.For example:
#define foo_to_char(foo) \ ((char *)(foo) + Py_BUILD_ASSERT_EXPR(offsetof(struct foo, string) == 0))
Added in version 3.3.
Type size utilities¶
-
Py_ARRAY_LENGTH(array)¶
Compute the length of a statically allocated C array at compile time.
The array argument must be a C array with a size known at compile time. Passing an array with an unknown size, such as a heap-allocated array, will result in a compilation error on some compilers, or otherwise produce incorrect results.
This is roughly equivalent to:
sizeof(array) / sizeof((array)[0])
-
Py_MEMBER_SIZE(type, member)¶
Return the size of a structure (type) member in bytes.
Corresponds roughly to
sizeof(((type *)NULL)->member)
.Added in version 3.6.
Macro definition utilities¶
-
Py_FORCE_EXPANSION(X)¶
This is equivalent to
X
, which is useful for token-pasting in macros, as macro expansions in X are forcefully evaluated by the preprocessor.
-
Py_STRINGIFY(x)¶
Convert
x
to a C string. For example,Py_STRINGIFY(123)
returns"123"
.Added in version 3.4.
Declaration utilities¶
The following macros can be used in declarations. They are most useful for defining the C API itself, and have limited use for extension authors. Most of them expand to compiler-specific spellings of common extensions to the C language.
-
Py_ALWAYS_INLINE¶
Ask the compiler to always inline a static inline function. The compiler can ignore it and decide to not inline the function.
Corresponds to
always_inline
attribute in GCC and__forceinline
in MSVC.It can be used to inline performance critical static inline functions when building Python in debug mode with function inlining disabled. For example, MSC disables function inlining when building in debug mode.
Marking blindly a static inline function with Py_ALWAYS_INLINE can result in worse performances (due to increased code size for example). The compiler is usually smarter than the developer for the cost/benefit analysis.
If Python is built in debug mode (if the
Py_DEBUG
macro is defined), thePy_ALWAYS_INLINE
macro does nothing.It must be specified before the function return type. Usage:
static inline Py_ALWAYS_INLINE int random(void) { return 4; }
Added in version 3.11.
-
Py_NO_INLINE¶
Disable inlining on a function. For example, it reduces the C stack consumption: useful on LTO+PGO builds which heavily inline code (see bpo-33720).
Corresponds to the
noinline
attribute/specification on GCC and MSVC.Usage:
Py_NO_INLINE static int random(void) { return 4; }
Added in version 3.11.
-
Py_DEPRECATED(version)¶
Use this to declare APIs that were deprecated in a specific CPython version. The macro must be placed before the symbol name.
Example:
Py_DEPRECATED(3.8) PyAPI_FUNC(int) Py_OldFunction(void);
Changed in version 3.8: MSVC support was added.
-
Py_LOCAL(type)¶
Declare a function returning the specified type using a fast-calling qualifier for functions that are local to the current file. Semantically, this is equivalent to
static type
.
-
Py_LOCAL_SYMBOL¶
Macro used to declare a symbol as local to the shared library (hidden). On supported platforms, it ensures the symbol is not exported.
On compatible versions of GCC/Clang, it expands to
__attribute__((visibility("hidden")))
.
-
Py_EXPORTED_SYMBOL¶
Macro used to declare a symbol (function or data) as exported. On Windows, this expands to
__declspec(dllexport)
. On compatible versions of GCC/Clang, it expands to__attribute__((visibility("default")))
. This macro is for defining the C API itself; extension modules should not use it.
-
Py_IMPORTED_SYMBOL¶
Macro used to declare a symbol as imported. On Windows, this expands to
__declspec(dllimport)
. This macro is for defining the C API itself; extension modules should not use it.
-
PyAPI_FUNC(type)¶
Macro used by CPython to declare a function as part of the C API. Its expansion depends on the platform and build configuration. This macro is intended for defining CPython’s C API itself; extension modules should not use it for their own symbols.
-
PyAPI_DATA(type)¶
Macro used by CPython to declare a public global variable as part of the C API. Its expansion depends on the platform and build configuration. This macro is intended for defining CPython’s C API itself; extension modules should not use it for their own symbols.
Outdated macros¶
The following macros have been used to features that have been standardized in C11.
-
Py_ALIGNED(num)¶
Specify alignment to num bytes on compilers that support it.
Consider using the C11 standard
_Alignas
specifier over this macro.
-
Py_LL(number)¶
-
Py_ULL(number)¶
Use number as a
long long
orunsigned long long
integer literal, respectively.Expands to number followed by
LL
orLLU
, respectively, but will expand to some compiler-specific suffixes on some older compilers.Consider using the C99 standard suffixes
LL
andLLU
directly.
-
Py_MEMCPY(dest, src, n)¶
This is a soft deprecated alias to
memcpy()
. Usememcpy()
directly instead.Deprecated since version 3.14: The macro is soft deprecated.
-
Py_VA_COPY¶
This is a soft deprecated alias to the C99-standard
va_copy
function.Historically, this would use a compiler-specific method to copy a
va_list
.Changed in version 3.6: This is now an alias to
va_copy
.
Objects, Types and Reference Counts¶
Most Python/C API functions have one or more arguments as well as a return value
of type PyObject*. This type is a pointer to an opaque data type
representing an arbitrary Python object. Since all Python object types are
treated the same way by the Python language in most situations (e.g.,
assignments, scope rules, and argument passing), it is only fitting that they
should be represented by a single C type. Almost all Python objects live on the
heap: you never declare an automatic or static variable of type
PyObject
, only pointer variables of type PyObject* can be
declared. The sole exception are the type objects; since these must never be
deallocated, they are typically static PyTypeObject
objects.
All Python objects (even Python integers) have a type and a
reference count. An object’s type determines what kind of object it is
(e.g., an integer, a list, or a user-defined function; there are many more as
explained in The standard type hierarchy). For each of the well-known types there is a macro
to check whether an object is of that type; for instance, PyList_Check(a)
is
true if (and only if) the object pointed to by a is a Python list.
Reference Counts¶
The reference count is important because today’s computers have a finite (and often severely limited) memory size; it counts how many different places there are that have a strong reference to an object. Such a place could be another object, or a global (or static) C variable, or a local variable in some C function. When the last strong reference to an object is released (i.e. its reference count becomes zero), the object is deallocated. If it contains references to other objects, those references are released. Those other objects may be deallocated in turn, if there are no more references to them, and so on. (There’s an obvious problem with objects that reference each other here; for now, the solution is “don’t do that.”)
Reference counts are always manipulated explicitly. The normal way is
to use the macro Py_INCREF()
to take a new reference to an
object (i.e. increment its reference count by one),
and Py_DECREF()
to release that reference (i.e. decrement the
reference count by one). The Py_DECREF()
macro
is considerably more complex than the incref one, since it must check whether
the reference count becomes zero and then cause the object’s deallocator to be
called. The deallocator is a function pointer contained in the object’s type
structure. The type-specific deallocator takes care of releasing references
for other objects contained in the object if this is a compound
object type, such as a list, as well as performing any additional finalization
that’s needed. There’s no chance that the reference count can overflow; at
least as many bits are used to hold the reference count as there are distinct
memory locations in virtual memory (assuming sizeof(Py_ssize_t) >= sizeof(void*)
).
Thus, the reference count increment is a simple operation.
It is not necessary to hold a strong reference (i.e. increment the reference count) for every local variable that contains a pointer to an object. In theory, the object’s reference count goes up by one when the variable is made to point to it and it goes down by one when the variable goes out of scope. However, these two cancel each other out, so at the end the reference count hasn’t changed. The only real reason to use the reference count is to prevent the object from being deallocated as long as our variable is pointing to it. If we know that there is at least one other reference to the object that lives at least as long as our variable, there is no need to take a new strong reference (i.e. increment the reference count) temporarily. An important situation where this arises is in objects that are passed as arguments to C functions in an extension module that are called from Python; the call mechanism guarantees to hold a reference to every argument for the duration of the call.
However, a common pitfall is to extract an object from a list and hold on to it
for a while without taking a new reference. Some other operation might
conceivably remove the object from the list, releasing that reference,
and possibly deallocating it. The real danger is that innocent-looking
operations may invoke arbitrary Python code which could do this; there is a code
path which allows control to flow back to the user from a Py_DECREF()
, so
almost any operation is potentially dangerous.
A safe approach is to always use the generic operations (functions whose name
begins with PyObject_
, PyNumber_
, PySequence_
or PyMapping_
).
These operations always create a new strong reference
(i.e. increment the reference count) of the object they return.
This leaves the caller with the responsibility to call Py_DECREF()
when
they are done with the result; this soon becomes second nature.
Reference Count Details¶
The reference count behavior of functions in the Python/C API is best explained
in terms of ownership of references. Ownership pertains to references, never
to objects (objects are not owned: they are always shared). “Owning a
reference” means being responsible for calling Py_DECREF on it when the
reference is no longer needed. Ownership can also be transferred, meaning that
the code that receives ownership of the reference then becomes responsible for
eventually releasing it by calling Py_DECREF()
or Py_XDECREF()
when it’s no longer needed—or passing on this responsibility (usually to its
caller). When a function passes ownership of a reference on to its caller, the
caller is said to receive a new reference. When no ownership is transferred,
the caller is said to borrow the reference. Nothing needs to be done for a
borrowed reference.
Conversely, when a calling function passes in a reference to an object, there are two possibilities: the function steals a reference to the object, or it does not. Stealing a reference means that when you pass a reference to a function, that function assumes that it now owns that reference, and you are not responsible for it any longer.
Few functions steal references; the two notable exceptions are
PyList_SetItem()
and PyTuple_SetItem()
, which steal a reference
to the item (but not to the tuple or list into which the item is put!). These
functions were designed to steal a reference because of a common idiom for
populating a tuple or list with newly created objects; for example, the code to
create the tuple (1, 2, "three")
could look like this (forgetting about
error handling for the moment; a better way to code this is shown below):
PyObject *t;
t = PyTuple_New(3);
PyTuple_SetItem(t, 0, PyLong_FromLong(1L));
PyTuple_SetItem(t, 1, PyLong_FromLong(2L));
PyTuple_SetItem(t, 2, PyUnicode_FromString("three"));
Here, PyLong_FromLong()
returns a new reference which is immediately
stolen by PyTuple_SetItem()
. When you want to keep using an object
although the reference to it will be stolen, use Py_INCREF()
to grab
another reference before calling the reference-stealing function.
Incidentally, PyTuple_SetItem()
is the only way to set tuple items;
PySequence_SetItem()
and PyObject_SetItem()
refuse to do this
since tuples are an immutable data type. You should only use
PyTuple_SetItem()
for tuples that you are creating yourself.
Equivalent code for populating a list can be written using PyList_New()
and PyList_SetItem()
.
However, in practice, you will rarely use these ways of creating and populating
a tuple or list. There’s a generic function, Py_BuildValue()
, that can
create most common objects from C values, directed by a format string.
For example, the above two blocks of code could be replaced by the following
(which also takes care of the error checking):
PyObject *tuple, *list;
tuple = Py_BuildValue("(iis)", 1, 2, "three");
list = Py_BuildValue("[iis]", 1, 2, "three");
It is much more common to use PyObject_SetItem()
and friends with items
whose references you are only borrowing, like arguments that were passed in to
the function you are writing. In that case, their behaviour regarding references
is much saner, since you don’t have to take a new reference just so you
can give that reference away (“have it be stolen”). For example, this function
sets all items of a list (actually, any mutable sequence) to a given item:
int
set_all(PyObject *target, PyObject *item)
{
Py_ssize_t i, n;
n = PyObject_Length(target);
if (n < 0)
return -1;
for (i = 0; i < n; i++) {
PyObject *index = PyLong_FromSsize_t(i);
if (!index)
return -1;
if (PyObject_SetItem(target, index, item) < 0) {
Py_DECREF(index);
return -1;
}
Py_DECREF(index);
}
return 0;
}
The situation is slightly different for function return values. While passing
a reference to most functions does not change your ownership responsibilities
for that reference, many functions that return a reference to an object give
you ownership of the reference. The reason is simple: in many cases, the
returned object is created on the fly, and the reference you get is the only
reference to the object. Therefore, the generic functions that return object
references, like PyObject_GetItem()
and PySequence_GetItem()
,
always return a new reference (the caller becomes the owner of the reference).
It is important to realize that whether you own a reference returned by a
function depends on which function you call only — the plumage (the type of
the object passed as an argument to the function) doesn’t enter into it!
Thus, if you extract an item from a list using PyList_GetItem()
, you
don’t own the reference — but if you obtain the same item from the same list
using PySequence_GetItem()
(which happens to take exactly the same
arguments), you do own a reference to the returned object.
Here is an example of how you could write a function that computes the sum of
the items in a list of integers; once using PyList_GetItem()
, and once
using PySequence_GetItem()
.
long
sum_list(PyObject *list)
{
Py_ssize_t i, n;
long total = 0, value;
PyObject *item;
n = PyList_Size(list);
if (n < 0)
return -1; /* Not a list */
for (i = 0; i < n; i++) {
item = PyList_GetItem(list, i); /* Can't fail */
if (!PyLong_Check(item)) continue; /* Skip non-integers */
value = PyLong_AsLong(item);
if (value == -1 && PyErr_Occurred())
/* Integer too big to fit in a C long, bail out */
return -1;
total += value;
}
return total;
}
long
sum_sequence(PyObject *sequence)
{
Py_ssize_t i, n;
long total = 0, value;
PyObject *item;
n = PySequence_Length(sequence);
if (n < 0)
return -1; /* Has no length */
for (i = 0; i < n; i++) {
item = PySequence_GetItem(sequence, i);
if (item == NULL)
return -1; /* Not a sequence, or other failure */
if (PyLong_Check(item)) {
value = PyLong_AsLong(item);
Py_DECREF(item);
if (value == -1 && PyErr_Occurred())
/* Integer too big to fit in a C long, bail out */
return -1;
total += value;
}
else {
Py_DECREF(item); /* Discard reference ownership */
}
}
return total;
}
Types¶
There are few other data types that play a significant role in the Python/C API; most are simple C types such as int, long, double and char*. A few structure types are used to describe static tables used to list the functions exported by a module or the data attributes of a new object type, and another is used to describe the value of a complex number. These will be discussed together with the functions that use them.
-
type Py_ssize_t¶
- Part of the Stable ABI.
A signed integral type such that
sizeof(Py_ssize_t) == sizeof(size_t)
. C99 doesn’t define such a thing directly (size_t is an unsigned integral type). See PEP 353 for details.PY_SSIZE_T_MAX
is the largest positive value of typePy_ssize_t
.
Exceptions¶
The Python programmer only needs to deal with exceptions if specific error handling is required; unhandled exceptions are automatically propagated to the caller, then to the caller’s caller, and so on, until they reach the top-level interpreter, where they are reported to the user accompanied by a stack traceback.
For C programmers, however, error checking always has to be explicit. All
functions in the Python/C API can raise exceptions, unless an explicit claim is
made otherwise in a function’s documentation. In general, when a function
encounters an error, it sets an exception, discards any object references that
it owns, and returns an error indicator. If not documented otherwise, this
indicator is either NULL
or -1
, depending on the function’s return type.
A few functions return a Boolean true/false result, with false indicating an
error. Very few functions return no explicit error indicator or have an
ambiguous return value, and require explicit testing for errors with
PyErr_Occurred()
. These exceptions are always explicitly documented.
Exception state is maintained in per-thread storage (this is equivalent to
using global storage in an unthreaded application). A thread can be in one of
two states: an exception has occurred, or not. The function
PyErr_Occurred()
can be used to check for this: it returns a borrowed
reference to the exception type object when an exception has occurred, and
NULL
otherwise. There are a number of functions to set the exception state:
PyErr_SetString()
is the most common (though not the most general)
function to set the exception state, and PyErr_Clear()
clears the
exception state.
The full exception state consists of three objects (all of which can be
NULL
): the exception type, the corresponding exception value, and the
traceback. These have the same meanings as the Python result of
sys.exc_info()
; however, they are not the same: the Python objects represent
the last exception being handled by a Python try
…
except
statement, while the C level exception state only exists while
an exception is being passed on between C functions until it reaches the Python
bytecode interpreter’s main loop, which takes care of transferring it to
sys.exc_info()
and friends.
Note that starting with Python 1.5, the preferred, thread-safe way to access the
exception state from Python code is to call the function sys.exc_info()
,
which returns the per-thread exception state for Python code. Also, the
semantics of both ways to access the exception state have changed so that a
function which catches an exception will save and restore its thread’s exception
state so as to preserve the exception state of its caller. This prevents common
bugs in exception handling code caused by an innocent-looking function
overwriting the exception being handled; it also reduces the often unwanted
lifetime extension for objects that are referenced by the stack frames in the
traceback.
As a general principle, a function that calls another function to perform some task should check whether the called function raised an exception, and if so, pass the exception state on to its caller. It should discard any object references that it owns, and return an error indicator, but it should not set another exception — that would overwrite the exception that was just raised, and lose important information about the exact cause of the error.
A simple example of detecting exceptions and passing them on is shown in the
sum_sequence()
example above. It so happens that this example doesn’t
need to clean up any owned references when it detects an error. The following
example function shows some error cleanup. First, to remind you why you like
Python, we show the equivalent Python code:
def incr_item(dict, key):
try:
item = dict[key]
except KeyError:
item = 0
dict[key] = item + 1
Here is the corresponding C code, in all its glory:
int
incr_item(PyObject *dict, PyObject *key)
{
/* Objects all initialized to NULL for Py_XDECREF */
PyObject *item = NULL, *const_one = NULL, *incremented_item = NULL;
int rv = -1; /* Return value initialized to -1 (failure) */
item = PyObject_GetItem(dict, key);
if (item == NULL) {
/* Handle KeyError only: */
if (!PyErr_ExceptionMatches(PyExc_KeyError))
goto error;
/* Clear the error and use zero: */
PyErr_Clear();
item = PyLong_FromLong(0L);
if (item == NULL)
goto error;
}
const_one = PyLong_FromLong(1L);
if (const_one == NULL)
goto error;
incremented_item = PyNumber_Add(item, const_one);
if (incremented_item == NULL)
goto error;
if (PyObject_SetItem(dict, key, incremented_item) < 0)
goto error;
rv = 0; /* Success */
/* Continue with cleanup code */
error:
/* Cleanup code, shared by success and failure path */
/* Use Py_XDECREF() to ignore NULL references */
Py_XDECREF(item);
Py_XDECREF(const_one);
Py_XDECREF(incremented_item);
return rv; /* -1 for error, 0 for success */
}
This example represents an endorsed use of the goto
statement in C!
It illustrates the use of PyErr_ExceptionMatches()
and
PyErr_Clear()
to handle specific exceptions, and the use of
Py_XDECREF()
to dispose of owned references that may be NULL
(note the
'X'
in the name; Py_DECREF()
would crash when confronted with a
NULL
reference). It is important that the variables used to hold owned
references are initialized to NULL
for this to work; likewise, the proposed
return value is initialized to -1
(failure) and only set to success after
the final call made is successful.
Embedding Python¶
The one important task that only embedders (as opposed to extension writers) of the Python interpreter have to worry about is the initialization, and possibly the finalization, of the Python interpreter. Most functionality of the interpreter can only be used after the interpreter has been initialized.
The basic initialization function is Py_Initialize()
. This initializes
the table of loaded modules, and creates the fundamental modules
builtins
, __main__
, and sys
. It also
initializes the module search path (sys.path
).
Py_Initialize()
does not set the “script argument list” (sys.argv
).
If this variable is needed by Python code that will be executed later, setting
PyConfig.argv
and PyConfig.parse_argv
must be set: see
Python Initialization Configuration.
On most systems (in particular, on Unix and Windows, although the details are
slightly different), Py_Initialize()
calculates the module search path
based upon its best guess for the location of the standard Python interpreter
executable, assuming that the Python library is found in a fixed location
relative to the Python interpreter executable. In particular, it looks for a
directory named lib/pythonX.Y
relative to the parent directory
where the executable named python
is found on the shell command search
path (the environment variable PATH
).
For instance, if the Python executable is found in
/usr/local/bin/python
, it will assume that the libraries are in
/usr/local/lib/pythonX.Y
. (In fact, this particular path is also
the “fallback” location, used when no executable file named python
is
found along PATH
.) The user can override this behavior by setting the
environment variable PYTHONHOME
, or insert additional directories in
front of the standard path by setting PYTHONPATH
.
The embedding application can steer the search by setting
PyConfig.program_name
before calling
Py_InitializeFromConfig()
. Note that
PYTHONHOME
still overrides this and PYTHONPATH
is still
inserted in front of the standard path. An application that requires total
control has to provide its own implementation of Py_GetPath()
,
Py_GetPrefix()
, Py_GetExecPrefix()
, and
Py_GetProgramFullPath()
(all defined in Modules/getpath.c
).
Sometimes, it is desirable to “uninitialize” Python. For instance, the
application may want to start over (make another call to
Py_Initialize()
) or the application is simply done with its use of
Python and wants to free memory allocated by Python. This can be accomplished
by calling Py_FinalizeEx()
. The function Py_IsInitialized()
returns
true if Python is currently in the initialized state. More information about
these functions is given in a later chapter. Notice that Py_FinalizeEx()
does not free all memory allocated by the Python interpreter, e.g. memory
allocated by extension modules currently cannot be released.
Debugging Builds¶
Python can be built with several macros to enable extra checks of the interpreter and extension modules. These checks tend to add a large amount of overhead to the runtime so they are not enabled by default.
A full list of the various types of debugging builds is in the file
Misc/SpecialBuilds.txt
in the Python source distribution. Builds are
available that support tracing of reference counts, debugging the memory
allocator, or low-level profiling of the main interpreter loop. Only the most
frequently used builds will be described in the remainder of this section.
-
Py_DEBUG¶
Compiling the interpreter with the Py_DEBUG
macro defined produces
what is generally meant by a debug build of Python.
Py_DEBUG
is enabled in the Unix build by adding
--with-pydebug
to the ./configure
command.
It is also implied by the presence of the
not-Python-specific _DEBUG
macro. When Py_DEBUG
is enabled
in the Unix build, compiler optimization is disabled.
In addition to the reference count debugging described below, extra checks are performed, see Python Debug Build.
Defining Py_TRACE_REFS
enables reference tracing
(see the configure --with-trace-refs option
).
When defined, a circular doubly linked list of active objects is maintained by adding two extra
fields to every PyObject
. Total allocations are tracked as well. Upon
exit, all existing references are printed. (In interactive mode this happens
after every statement run by the interpreter.)
Please refer to Misc/SpecialBuilds.txt
in the Python source distribution
for more detailed information.
Recommended third party tools¶
The following third party tools offer both simpler and more sophisticated approaches to creating C, C++ and Rust extensions for Python:
Using tools such as these can help avoid writing code that is tightly bound to a particular version of CPython, avoid reference counting errors, and focus more on your own code than on using the CPython API. In general, new versions of Python can be supported by updating the tool, and your code will often use newer and more efficient APIs automatically. Some tools also support compiling for other implementations of Python from a single set of sources.
These projects are not supported by the same people who maintain Python, and issues need to be raised with the projects directly. Remember to check that the project is still maintained and supported, as the list above may become outdated.
See also
- Python Packaging User Guide: Binary Extensions
The Python Packaging User Guide not only covers several available tools that simplify the creation of binary extensions, but also discusses the various reasons why creating an extension module may be desirable in the first place.