Object Protocol

Object Protocol¶
-
PyObject *Py_GetConstant(unsigned int constant_id)¶
- Part of the Stable ABI since version 3.13.
Get a strong reference to a constant.
Set an exception and return
NULL
if constant_id is invalid.constant_id must be one of these constant identifiers:
Constant Identifier
Value
Returned object
-
Py_CONSTANT_NONE¶
0
-
Py_CONSTANT_FALSE¶
1
-
Py_CONSTANT_TRUE¶
2
-
Py_CONSTANT_ELLIPSIS¶
3
-
Py_CONSTANT_NOT_IMPLEMENTED¶
4
-
Py_CONSTANT_ZERO¶
5
0
-
Py_CONSTANT_ONE¶
6
1
-
Py_CONSTANT_EMPTY_STR¶
7
''
-
Py_CONSTANT_EMPTY_BYTES¶
8
b''
-
Py_CONSTANT_EMPTY_TUPLE¶
9
()
Numeric values are only given for projects which cannot use the constant identifiers.
Added in version 3.13.
CPython implementation detail: In CPython, all of these constants are immortal.
-
Py_CONSTANT_NONE¶
-
PyObject *Py_GetConstantBorrowed(unsigned int constant_id)¶
- Part of the Stable ABI since version 3.13.
Similar to
Py_GetConstant()
, but return a borrowed reference.This function is primarily intended for backwards compatibility: using
Py_GetConstant()
is recommended for new code.The reference is borrowed from the interpreter, and is valid until the interpreter finalization.
Added in version 3.13.
-
PyObject *Py_NotImplemented¶
The
NotImplemented
singleton, used to signal that an operation is not implemented for the given type combination.
-
Py_RETURN_NOTIMPLEMENTED¶
Properly handle returning
Py_NotImplemented
from within a C function (that is, create a new strong reference toNotImplemented
and return it).
-
Py_PRINT_RAW¶
Flag to be used with multiple functions that print the object (like
PyObject_Print()
andPyFile_WriteObject()
). If passed, these functions use thestr()
of the object instead of therepr()
.
-
int PyObject_Print(PyObject *o, FILE *fp, int flags)¶
Print an object o, on file fp. Returns
-1
on error. The flags argument is used to enable certain printing options. The only option currently supported isPy_PRINT_RAW
; if given, thestr()
of the object is written instead of therepr()
.
-
int PyObject_HasAttrWithError(PyObject *o, PyObject *attr_name)¶
- Part of the Stable ABI since version 3.13.
Returns
1
if o has the attribute attr_name, and0
otherwise. This is equivalent to the Python expressionhasattr(o, attr_name)
. On failure, return-1
.Added in version 3.13.
-
int PyObject_HasAttrStringWithError(PyObject *o, const char *attr_name)¶
- Part of the Stable ABI since version 3.13.
This is the same as
PyObject_HasAttrWithError()
, but attr_name is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.Added in version 3.13.
-
int PyObject_HasAttr(PyObject *o, PyObject *attr_name)¶
- Part of the Stable ABI.
Returns
1
if o has the attribute attr_name, and0
otherwise. This function always succeeds.Note
Exceptions that occur when this calls
__getattr__()
and__getattribute__()
methods aren’t propagated, but instead given tosys.unraisablehook()
. For proper error handling, usePyObject_HasAttrWithError()
,PyObject_GetOptionalAttr()
orPyObject_GetAttr()
instead.
-
int PyObject_HasAttrString(PyObject *o, const char *attr_name)¶
- Part of the Stable ABI.
This is the same as
PyObject_HasAttr()
, but attr_name is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.Note
Exceptions that occur when this calls
__getattr__()
and__getattribute__()
methods or while creating the temporarystr
object are silently ignored. For proper error handling, usePyObject_HasAttrStringWithError()
,PyObject_GetOptionalAttrString()
orPyObject_GetAttrString()
instead.
-
PyObject *PyObject_GetAttr(PyObject *o, PyObject *attr_name)¶
- Return value: New reference. Part of the Stable ABI.
Retrieve an attribute named attr_name from object o. Returns the attribute value on success, or
NULL
on failure. This is the equivalent of the Python expressiono.attr_name
.If the missing attribute should not be treated as a failure, you can use
PyObject_GetOptionalAttr()
instead.
-
PyObject *PyObject_GetAttrString(PyObject *o, const char *attr_name)¶
- Return value: New reference. Part of the Stable ABI.
This is the same as
PyObject_GetAttr()
, but attr_name is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.If the missing attribute should not be treated as a failure, you can use
PyObject_GetOptionalAttrString()
instead.
-
int PyObject_GetOptionalAttr(PyObject *obj, PyObject *attr_name, PyObject **result);¶
- Part of the Stable ABI since version 3.13.
Variant of
PyObject_GetAttr()
which doesn’t raiseAttributeError
if the attribute is not found.If the attribute is found, return
1
and set *result to a new strong reference to the attribute. If the attribute is not found, return0
and set *result toNULL
; theAttributeError
is silenced. If an error other thanAttributeError
is raised, return-1
and set *result toNULL
.Added in version 3.13.
-
int PyObject_GetOptionalAttrString(PyObject *obj, const char *attr_name, PyObject **result);¶
- Part of the Stable ABI since version 3.13.
This is the same as
PyObject_GetOptionalAttr()
, but attr_name is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.Added in version 3.13.
-
PyObject *PyObject_GenericGetAttr(PyObject *o, PyObject *name)¶
- Return value: New reference. Part of the Stable ABI.
Generic attribute getter function that is meant to be put into a type object’s
tp_getattro
slot. It looks for a descriptor in the dictionary of classes in the object’s MRO as well as an attribute in the object’s__dict__
(if present). As outlined in Implementing Descriptors, data descriptors take preference over instance attributes, while non-data descriptors don’t. Otherwise, anAttributeError
is raised.
-
int PyObject_SetAttr(PyObject *o, PyObject *attr_name, PyObject *v)¶
- Part of the Stable ABI.
Set the value of the attribute named attr_name, for object o, to the value v. Raise an exception and return
-1
on failure; return0
on success. This is the equivalent of the Python statemento.attr_name = v
.If v is
NULL
, the attribute is deleted. This behaviour is deprecated in favour of usingPyObject_DelAttr()
, but there are currently no plans to remove it.
-
int PyObject_SetAttrString(PyObject *o, const char *attr_name, PyObject *v)¶
- Part of the Stable ABI.
This is the same as
PyObject_SetAttr()
, but attr_name is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.If v is
NULL
, the attribute is deleted, but this feature is deprecated in favour of usingPyObject_DelAttrString()
.The number of different attribute names passed to this function should be kept small, usually by using a statically allocated string as attr_name. For attribute names that aren’t known at compile time, prefer calling
PyUnicode_FromString()
andPyObject_SetAttr()
directly. For more details, seePyUnicode_InternFromString()
, which may be used internally to create a key object.
-
int PyObject_GenericSetAttr(PyObject *o, PyObject *name, PyObject *value)¶
- Part of the Stable ABI.
Generic attribute setter and deleter function that is meant to be put into a type object’s
tp_setattro
slot. It looks for a data descriptor in the dictionary of classes in the object’s MRO, and if found it takes preference over setting or deleting the attribute in the instance dictionary. Otherwise, the attribute is set or deleted in the object’s__dict__
(if present). On success,0
is returned, otherwise anAttributeError
is raised and-1
is returned.
-
int PyObject_DelAttr(PyObject *o, PyObject *attr_name)¶
- Part of the Stable ABI since version 3.13.
Delete attribute named attr_name, for object o. Returns
-1
on failure. This is the equivalent of the Python statementdel o.attr_name
.
-
int PyObject_DelAttrString(PyObject *o, const char *attr_name)¶
- Part of the Stable ABI since version 3.13.
This is the same as
PyObject_DelAttr()
, but attr_name is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.The number of different attribute names passed to this function should be kept small, usually by using a statically allocated string as attr_name. For attribute names that aren’t known at compile time, prefer calling
PyUnicode_FromString()
andPyObject_DelAttr()
directly. For more details, seePyUnicode_InternFromString()
, which may be used internally to create a key object for lookup.
-
PyObject *PyObject_GenericGetDict(PyObject *o, void *context)¶
- Return value: New reference. Part of the Stable ABI since version 3.10.
A generic implementation for the getter of a
__dict__
descriptor. It creates the dictionary if necessary.This function may also be called to get the
__dict__
of the object o. PassNULL
for context when calling it. Since this function may need to allocate memory for the dictionary, it may be more efficient to callPyObject_GetAttr()
when accessing an attribute on the object.On failure, returns
NULL
with an exception set.Added in version 3.3.
-
int PyObject_GenericSetDict(PyObject *o, PyObject *value, void *context)¶
- Part of the Stable ABI since version 3.7.
A generic implementation for the setter of a
__dict__
descriptor. This implementation does not allow the dictionary to be deleted.Added in version 3.3.
-
PyObject **_PyObject_GetDictPtr(PyObject *obj)¶
Return a pointer to
__dict__
of the object obj. If there is no__dict__
, returnNULL
without setting an exception.This function may need to allocate memory for the dictionary, so it may be more efficient to call
PyObject_GetAttr()
when accessing an attribute on the object.
-
PyObject *PyObject_RichCompare(PyObject *o1, PyObject *o2, int opid)¶
- Return value: New reference. Part of the Stable ABI.
Compare the values of o1 and o2 using the operation specified by opid, which must be one of
Py_LT
,Py_LE
,Py_EQ
,Py_NE
,Py_GT
, orPy_GE
, corresponding to<
,<=
,==
,!=
,>
, or>=
respectively. This is the equivalent of the Python expressiono1 op o2
, whereop
is the operator corresponding to opid. Returns the value of the comparison on success, orNULL
on failure.
-
int PyObject_RichCompareBool(PyObject *o1, PyObject *o2, int opid)¶
- Part of the Stable ABI.
Compare the values of o1 and o2 using the operation specified by opid, like
PyObject_RichCompare()
, but returns-1
on error,0
if the result is false,1
otherwise.
Note
If o1 and o2 are the same object, PyObject_RichCompareBool()
will always return 1
for Py_EQ
and 0
for Py_NE
.
-
PyObject *PyObject_Format(PyObject *obj, PyObject *format_spec)¶
- Part of the Stable ABI.
Format obj using format_spec. This is equivalent to the Python expression
format(obj, format_spec)
.format_spec may be
NULL
. In this case the call is equivalent toformat(obj)
. Returns the formatted string on success,NULL
on failure.
-
PyObject *PyObject_Repr(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Compute a string representation of object o. Returns the string representation on success,
NULL
on failure. This is the equivalent of the Python expressionrepr(o)
. Called by therepr()
built-in function.Changed in version 3.4: This function now includes a debug assertion to help ensure that it does not silently discard an active exception.
-
PyObject *PyObject_ASCII(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
As
PyObject_Repr()
, compute a string representation of object o, but escape the non-ASCII characters in the string returned byPyObject_Repr()
with\x
,\u
or\U
escapes. This generates a string similar to that returned byPyObject_Repr()
in Python 2. Called by theascii()
built-in function.
-
PyObject *PyObject_Str(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Compute a string representation of object o. Returns the string representation on success,
NULL
on failure. This is the equivalent of the Python expressionstr(o)
. Called by thestr()
built-in function and, therefore, by theprint()
function.Changed in version 3.4: This function now includes a debug assertion to help ensure that it does not silently discard an active exception.
-
PyObject *PyObject_Bytes(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Compute a bytes representation of object o.
NULL
is returned on failure and a bytes object on success. This is equivalent to the Python expressionbytes(o)
, when o is not an integer. Unlikebytes(o)
, a TypeError is raised when o is an integer instead of a zero-initialized bytes object.
-
int PyObject_IsSubclass(PyObject *derived, PyObject *cls)¶
- Part of the Stable ABI.
Return
1
if the class derived is identical to or derived from the class cls, otherwise return0
. In case of an error, return-1
.If cls is a tuple, the check will be done against every entry in cls. The result will be
1
when at least one of the checks returns1
, otherwise it will be0
.If cls has a
__subclasscheck__()
method, it will be called to determine the subclass status as described in PEP 3119. Otherwise, derived is a subclass of cls if it is a direct or indirect subclass, i.e. contained incls.__mro__
.Normally only class objects, i.e. instances of
type
or a derived class, are considered classes. However, objects can override this by having a__bases__
attribute (which must be a tuple of base classes).
-
int PyObject_IsInstance(PyObject *inst, PyObject *cls)¶
- Part of the Stable ABI.
Return
1
if inst is an instance of the class cls or a subclass of cls, or0
if not. On error, returns-1
and sets an exception.If cls is a tuple, the check will be done against every entry in cls. The result will be
1
when at least one of the checks returns1
, otherwise it will be0
.If cls has a
__instancecheck__()
method, it will be called to determine the subclass status as described in PEP 3119. Otherwise, inst is an instance of cls if its class is a subclass of cls.An instance inst can override what is considered its class by having a
__class__
attribute.An object cls can override if it is considered a class, and what its base classes are, by having a
__bases__
attribute (which must be a tuple of base classes).
-
Py_hash_t PyObject_Hash(PyObject *o)¶
- Part of the Stable ABI.
Compute and return the hash value of an object o. On failure, return
-1
. This is the equivalent of the Python expressionhash(o)
.Changed in version 3.2: The return type is now Py_hash_t. This is a signed integer the same size as
Py_ssize_t
.
-
Py_hash_t PyObject_HashNotImplemented(PyObject *o)¶
- Part of the Stable ABI.
Set a
TypeError
indicating thattype(o)
is not hashable and return-1
. This function receives special treatment when stored in atp_hash
slot, allowing a type to explicitly indicate to the interpreter that it is not hashable.
-
int PyObject_IsTrue(PyObject *o)¶
- Part of the Stable ABI.
Returns
1
if the object o is considered to be true, and0
otherwise. This is equivalent to the Python expressionnot not o
. On failure, return-1
.
-
int PyObject_Not(PyObject *o)¶
- Part of the Stable ABI.
Returns
0
if the object o is considered to be true, and1
otherwise. This is equivalent to the Python expressionnot o
. On failure, return-1
.
-
PyObject *PyObject_Type(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
When o is non-
NULL
, returns a type object corresponding to the object type of object o. On failure, raisesSystemError
and returnsNULL
. This is equivalent to the Python expressiontype(o)
. This function creates a new strong reference to the return value. There’s really no reason to use this function instead of thePy_TYPE()
function, which returns a pointer of type PyTypeObject*, except when a new strong reference is needed.
-
int PyObject_TypeCheck(PyObject *o, PyTypeObject *type)¶
Return non-zero if the object o is of type type or a subtype of type, and
0
otherwise. Both parameters must be non-NULL
.
-
Py_ssize_t PyObject_Size(PyObject *o)¶
-
Py_ssize_t PyObject_Length(PyObject *o)¶
- Part of the Stable ABI.
Return the length of object o. If the object o provides either the sequence and mapping protocols, the sequence length is returned. On error,
-1
is returned. This is the equivalent to the Python expressionlen(o)
.
-
Py_ssize_t PyObject_LengthHint(PyObject *o, Py_ssize_t defaultvalue)¶
Return an estimated length for the object o. First try to return its actual length, then an estimate using
__length_hint__()
, and finally return the default value. On error return-1
. This is the equivalent to the Python expressionoperator.length_hint(o, defaultvalue)
.Added in version 3.4.
-
PyObject *PyObject_GetItem(PyObject *o, PyObject *key)¶
- Return value: New reference. Part of the Stable ABI.
Return element of o corresponding to the object key or
NULL
on failure. This is the equivalent of the Python expressiono[key]
.
-
int PyObject_SetItem(PyObject *o, PyObject *key, PyObject *v)¶
- Part of the Stable ABI.
Map the object key to the value v. Raise an exception and return
-1
on failure; return0
on success. This is the equivalent of the Python statemento[key] = v
. This function does not steal a reference to v.
-
int PyObject_DelItem(PyObject *o, PyObject *key)¶
- Part of the Stable ABI.
Remove the mapping for the object key from the object o. Return
-1
on failure. This is equivalent to the Python statementdel o[key]
.
-
int PyObject_DelItemString(PyObject *o, const char *key)¶
- Part of the Stable ABI.
This is the same as
PyObject_DelItem()
, but key is specified as a const char* UTF-8 encoded bytes string, rather than a PyObject*.
-
PyObject *PyObject_Dir(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
This is equivalent to the Python expression
dir(o)
, returning a (possibly empty) list of strings appropriate for the object argument, orNULL
if there was an error. If the argument isNULL
, this is like the Pythondir()
, returning the names of the current locals; in this case, if no execution frame is active thenNULL
is returned butPyErr_Occurred()
will return false.
-
PyObject *PyObject_GetIter(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
This is equivalent to the Python expression
iter(o)
. It returns a new iterator for the object argument, or the object itself if the object is already an iterator. RaisesTypeError
and returnsNULL
if the object cannot be iterated.
-
PyObject *PyObject_SelfIter(PyObject *obj)¶
- Return value: New reference. Part of the Stable ABI.
This is equivalent to the Python
__iter__(self): return self
method. It is intended for iterator types, to be used in thePyTypeObject.tp_iter
slot.
-
PyObject *PyObject_GetAIter(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI since version 3.10.
This is the equivalent to the Python expression
aiter(o)
. Takes anAsyncIterable
object and returns anAsyncIterator
for it. This is typically a new iterator but if the argument is anAsyncIterator
, this returns itself. RaisesTypeError
and returnsNULL
if the object cannot be iterated.Added in version 3.10.
-
void *PyObject_GetTypeData(PyObject *o, PyTypeObject *cls)¶
- Part of the Stable ABI since version 3.12.
Get a pointer to subclass-specific data reserved for cls.
The object o must be an instance of cls, and cls must have been created using negative
PyType_Spec.basicsize
. Python does not check this.On error, set an exception and return
NULL
.Added in version 3.12.
-
Py_ssize_t PyType_GetTypeDataSize(PyTypeObject *cls)¶
- Part of the Stable ABI since version 3.12.
Return the size of the instance memory space reserved for cls, i.e. the size of the memory
PyObject_GetTypeData()
returns.This may be larger than requested using
-PyType_Spec.basicsize
; it is safe to use this larger size (e.g. withmemset()
).The type cls must have been created using negative
PyType_Spec.basicsize
. Python does not check this.On error, set an exception and return a negative value.
Added in version 3.12.
-
void *PyObject_GetItemData(PyObject *o)¶
Get a pointer to per-item data for a class with
Py_TPFLAGS_ITEMS_AT_END
.On error, set an exception and return
NULL
.TypeError
is raised if o does not havePy_TPFLAGS_ITEMS_AT_END
set.Added in version 3.12.
-
int PyObject_VisitManagedDict(PyObject *obj, visitproc visit, void *arg)¶
Visit the managed dictionary of obj.
This function must only be called in a traverse function of the type which has the
Py_TPFLAGS_MANAGED_DICT
flag set.Added in version 3.13.
-
void PyObject_ClearManagedDict(PyObject *obj)¶
Clear the managed dictionary of obj.
This function must only be called in a clear function of the type which has the
Py_TPFLAGS_MANAGED_DICT
flag set.Added in version 3.13.
-
int PyUnstable_Object_EnableDeferredRefcount(PyObject *obj)¶
- This is Unstable API. It may change without warning in minor releases.
Enable deferred reference counting on obj, if supported by the runtime. In the free-threaded build, this allows the interpreter to avoid reference count adjustments to obj, which may improve multi-threaded performance. The tradeoff is that obj will only be deallocated by the tracing garbage collector, and not when the interpreter no longer has any references to it.
This function returns
1
if deferred reference counting is enabled on obj, and0
if deferred reference counting is not supported or if the hint was ignored by the interpreter, such as when deferred reference counting is already enabled on obj. This function is thread-safe, and cannot fail.This function does nothing on builds with the GIL enabled, which do not support deferred reference counting. This also does nothing if obj is not an object tracked by the garbage collector (see
gc.is_tracked()
andPyObject_GC_IsTracked()
).This function is intended to be used soon after obj is created, by the code that creates it, such as in the object’s
tp_new
slot.Added in version 3.14.
-
int PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *obj)¶
- This is Unstable API. It may change without warning in minor releases.
Check if obj is a unique temporary object. Returns
1
if obj is known to be a unique temporary object, and0
otherwise. This function cannot fail, but the check is conservative, and may return0
in some cases even if obj is a unique temporary object.If an object is a unique temporary, it is guaranteed that the current code has the only reference to the object. For arguments to C functions, this should be used instead of checking if the reference count is
1
. Starting with Python 3.14, the interpreter internally avoids some reference count modifications when loading objects onto the operands stack by borrowing references when possible, which means that a reference count of1
by itself does not guarantee that a function argument uniquely referenced.In the example below,
my_func
is called with a unique temporary object as its argument:my_func([1, 2, 3])
In the example below,
my_func
is not called with a unique temporary object as its argument, even if its refcount is1
:my_list = [1, 2, 3] my_func(my_list)
See also the function
Py_REFCNT()
.Added in version 3.14.
-
int PyUnstable_IsImmortal(PyObject *obj)¶
- This is Unstable API. It may change without warning in minor releases.
This function returns non-zero if obj is immortal, and zero otherwise. This function cannot fail.
Note
Objects that are immortal in one CPython version are not guaranteed to be immortal in another.
Added in version 3.14.
-
int PyUnstable_TryIncRef(PyObject *obj)¶
- This is Unstable API. It may change without warning in minor releases.
Increments the reference count of obj if it is not zero. Returns
1
if the object’s reference count was successfully incremented. Otherwise, this function returns0
.PyUnstable_EnableTryIncRef()
must have been called earlier on obj or this function may spuriously return0
in the free-threaded build.This function is logically equivalent to the following C code, except that it behaves atomically in the free-threaded build:
if (Py_REFCNT(op) > 0) { Py_INCREF(op); return 1; } return 0;
This is intended as a building block for managing weak references without the overhead of a Python weak reference object.
Typically, correct use of this function requires support from obj’s deallocator (
tp_dealloc
). For example, the following sketch could be adapted to implement a “weakmap” that works like aWeakValueDictionary
for a specific type:PyMutex mutex; PyObject * add_entry(weakmap_key_type *key, PyObject *value) { PyUnstable_EnableTryIncRef(value); weakmap_type weakmap = ...; PyMutex_Lock(&mutex); weakmap_add_entry(weakmap, key, value); PyMutex_Unlock(&mutex); Py_RETURN_NONE; } PyObject * get_value(weakmap_key_type *key) { weakmap_type weakmap = ...; PyMutex_Lock(&mutex); PyObject *result = weakmap_find(weakmap, key); if (PyUnstable_TryIncRef(result)) { // `result` is safe to use PyMutex_Unlock(&mutex); return result; } // if we get here, `result` is starting to be garbage-collected, // but has not been removed from the weakmap yet PyMutex_Unlock(&mutex); return NULL; } // tp_dealloc function for weakmap values void value_dealloc(PyObject *value) { weakmap_type weakmap = ...; PyMutex_Lock(&mutex); weakmap_remove_value(weakmap, value); ... PyMutex_Unlock(&mutex); }
Added in version 3.14.
-
void PyUnstable_EnableTryIncRef(PyObject *obj)¶
- This is Unstable API. It may change without warning in minor releases.
Enables subsequent uses of
PyUnstable_TryIncRef()
on obj. The caller must hold a strong reference to obj when calling this.Added in version 3.14.
-
int PyUnstable_Object_IsUniquelyReferenced(PyObject *op)¶
- This is Unstable API. It may change without warning in minor releases.
Determine if op only has one reference.
On GIL-enabled builds, this function is equivalent to Py_REFCNT(op) == 1.
On a free-threaded build, this checks if op’s reference count is equal to one and additionally checks if op is only used by this thread. Py_REFCNT(op) == 1 is not thread-safe on free-threaded builds; prefer this function.
The caller must hold an attached thread state, despite the fact that this function doesn’t call into the Python interpreter. This function cannot fail.
Added in version 3.14.


Reference Counting

Reference Counting¶
The functions and macros in this section are used for managing reference counts of Python objects.
-
Py_ssize_t Py_REFCNT(PyObject *o)¶
- Part of the Stable ABI since version 3.14.
Get the reference count of the Python object o.
Note that the returned value may not actually reflect how many references to the object are actually held. For example, some objects are immortal and have a very high refcount that does not reflect the actual number of references. Consequently, do not rely on the returned value to be accurate, other than a value of 0 or 1.
Use the
Py_SET_REFCNT()
function to set an object reference count.Note
On free-threaded builds of Python, returning 1 isn’t sufficient to determine if it’s safe to treat o as having no access by other threads. Use
PyUnstable_Object_IsUniquelyReferenced()
for that instead.See also the function
PyUnstable_Object_IsUniqueReferencedTemporary()
.Changed in version 3.10:
Py_REFCNT()
is changed to the inline static function.Changed in version 3.11: The parameter type is no longer const PyObject*.
-
void Py_SET_REFCNT(PyObject *o, Py_ssize_t refcnt)¶
Set the object o reference counter to refcnt.
On Python build with Free Threading, if refcnt is larger than
UINT32_MAX
, the object is made immortal.This function has no effect on immortal objects.
Added in version 3.9.
Changed in version 3.12: Immortal objects are not modified.
-
void Py_INCREF(PyObject *o)¶
Indicate taking a new strong reference to object o, indicating it is in use and should not be destroyed.
This function has no effect on immortal objects.
This function is usually used to convert a borrowed reference to a strong reference in-place. The
Py_NewRef()
function can be used to create a new strong reference.When done using the object, release is by calling
Py_DECREF()
.The object must not be
NULL
; if you aren’t sure that it isn’tNULL
, usePy_XINCREF()
.Do not expect this function to actually modify o in any way. For at least some objects, this function has no effect.
Changed in version 3.12: Immortal objects are not modified.
-
void Py_XINCREF(PyObject *o)¶
Similar to
Py_INCREF()
, but the object o can beNULL
, in which case this has no effect.See also
Py_XNewRef()
.
-
PyObject *Py_NewRef(PyObject *o)¶
- Part of the Stable ABI since version 3.10.
Create a new strong reference to an object: call
Py_INCREF()
on o and return the object o.When the strong reference is no longer needed,
Py_DECREF()
should be called on it to release the reference.The object o must not be
NULL
; usePy_XNewRef()
if o can beNULL
.For example:
Py_INCREF(obj); self->attr = obj;
can be written as:
self->attr = Py_NewRef(obj);
See also
Py_INCREF()
.Added in version 3.10.
-
PyObject *Py_XNewRef(PyObject *o)¶
- Part of the Stable ABI since version 3.10.
Similar to
Py_NewRef()
, but the object o can be NULL.If the object o is
NULL
, the function just returnsNULL
.Added in version 3.10.
-
void Py_DECREF(PyObject *o)¶
Release a strong reference to object o, indicating the reference is no longer used.
This function has no effect on immortal objects.
Once the last strong reference is released (i.e. the object’s reference count reaches 0), the object’s type’s deallocation function (which must not be
NULL
) is invoked.This function is usually used to delete a strong reference before exiting its scope.
The object must not be
NULL
; if you aren’t sure that it isn’tNULL
, usePy_XDECREF()
.Do not expect this function to actually modify o in any way. For at least some objects, this function has no effect.
Warning
The deallocation function can cause arbitrary Python code to be invoked (e.g. when a class instance with a
__del__()
method is deallocated). While exceptions in such code are not propagated, the executed code has free access to all Python global variables. This means that any object that is reachable from a global variable should be in a consistent state beforePy_DECREF()
is invoked. For example, code to delete an object from a list should copy a reference to the deleted object in a temporary variable, update the list data structure, and then callPy_DECREF()
for the temporary variable.Changed in version 3.12: Immortal objects are not modified.
-
void Py_XDECREF(PyObject *o)¶
Similar to
Py_DECREF()
, but the object o can beNULL
, in which case this has no effect. The same warning fromPy_DECREF()
applies here as well.
-
void Py_CLEAR(PyObject *o)¶
Release a strong reference for object o. The object may be
NULL
, in which case the macro has no effect; otherwise the effect is the same as forPy_DECREF()
, except that the argument is also set toNULL
. The warning forPy_DECREF()
does not apply with respect to the object passed because the macro carefully uses a temporary variable and sets the argument toNULL
before releasing the reference.It is a good idea to use this macro whenever releasing a reference to an object that might be traversed during garbage collection.
Changed in version 3.12: The macro argument is now only evaluated once. If the argument has side effects, these are no longer duplicated.
-
void Py_IncRef(PyObject *o)¶
- Part of the Stable ABI.
Indicate taking a new strong reference to object o. A function version of
Py_XINCREF()
. It can be used for runtime dynamic embedding of Python.
-
void Py_DecRef(PyObject *o)¶
- Part of the Stable ABI.
Release a strong reference to object o. A function version of
Py_XDECREF()
. It can be used for runtime dynamic embedding of Python.
-
Py_SETREF(dst, src)¶
Macro safely releasing a strong reference to object dst and setting dst to src.
As in case of
Py_CLEAR()
, “the obvious” code can be deadly:Py_DECREF(dst); dst = src;
The safe way is:
Py_SETREF(dst, src);
That arranges to set dst to src before releasing the reference to the old value of dst, so that any code triggered as a side-effect of dst getting torn down no longer believes dst points to a valid object.
Added in version 3.6.
Changed in version 3.12: The macro arguments are now only evaluated once. If an argument has side effects, these are no longer duplicated.
-
Py_XSETREF(dst, src)¶
Variant of
Py_SETREF
macro that usesPy_XDECREF()
instead ofPy_DECREF()
.Added in version 3.6.
Changed in version 3.12: The macro arguments are now only evaluated once. If an argument has side effects, these are no longer duplicated.


Parsing arguments and building values

Parsing arguments and building values¶
These functions are useful when creating your own extension functions and methods. Additional information and examples are available in Extending and Embedding the Python Interpreter.
The first three of these functions described, PyArg_ParseTuple()
,
PyArg_ParseTupleAndKeywords()
, and PyArg_Parse()
, all use format
strings which are used to tell the function about the expected arguments. The
format strings use the same syntax for each of these functions.
Parsing arguments¶
A format string consists of zero or more “format units.” A format unit describes one Python object; it is usually a single character or a parenthesized sequence of format units. With a few exceptions, a format unit that is not a parenthesized sequence normally corresponds to a single address argument to these functions. In the following description, the quoted form is the format unit; the entry in (round) parentheses is the Python object type that matches the format unit; and the entry in [square] brackets is the type of the C variable(s) whose address should be passed.
Strings and buffers¶
Note
On Python 3.12 and older, the macro PY_SSIZE_T_CLEAN
must be
defined before including Python.h
to use all #
variants of
formats (s#
, y#
, etc.) explained below.
This is not necessary on Python 3.13 and later.
These formats allow accessing an object as a contiguous chunk of memory. You don’t have to provide raw storage for the returned unicode or bytes area.
Unless otherwise stated, buffers are not NUL-terminated.
There are three ways strings and buffers can be converted to C:
Formats such as
y*
ands*
fill aPy_buffer
structure. This locks the underlying buffer so that the caller can subsequently use the buffer even inside aPy_BEGIN_ALLOW_THREADS
block without the risk of mutable data being resized or destroyed. As a result, you have to callPyBuffer_Release()
after you have finished processing the data (or in any early abort case).The
es
,es#
,et
andet#
formats allocate the result buffer. You have to callPyMem_Free()
after you have finished processing the data (or in any early abort case).Other formats take a
str
or a read-only bytes-like object, such asbytes
, and provide aconst char *
pointer to its buffer. In this case the buffer is “borrowed”: it is managed by the corresponding Python object, and shares the lifetime of this object. You won’t have to release any memory yourself.To ensure that the underlying buffer may be safely borrowed, the object’s
PyBufferProcs.bf_releasebuffer
field must beNULL
. This disallows common mutable objects such asbytearray
, but also some read-only objects such asmemoryview
ofbytes
.Besides this
bf_releasebuffer
requirement, there is no check to verify whether the input object is immutable (e.g. whether it would honor a request for a writable buffer, or whether another thread can mutate the data).
s
(str
) [const char *]Convert a Unicode object to a C pointer to a character string. A pointer to an existing string is stored in the character pointer variable whose address you pass. The C string is NUL-terminated. The Python string must not contain embedded null code points; if it does, a
ValueError
exception is raised. Unicode objects are converted to C strings using'utf-8'
encoding. If this conversion fails, aUnicodeError
is raised.Note
This format does not accept bytes-like objects. If you want to accept filesystem paths and convert them to C character strings, it is preferable to use the
O&
format withPyUnicode_FSConverter()
as converter.Changed in version 3.5: Previously,
TypeError
was raised when embedded null code points were encountered in the Python string.s*
(str
or bytes-like object) [Py_buffer]This format accepts Unicode objects as well as bytes-like objects. It fills a
Py_buffer
structure provided by the caller. In this case the resulting C string may contain embedded NUL bytes. Unicode objects are converted to C strings using'utf-8'
encoding.s#
(str
, read-only bytes-like object) [const char *,Py_ssize_t
]Like
s*
, except that it provides a borrowed buffer. The result is stored into two C variables, the first one a pointer to a C string, the second one its length. The string may contain embedded null bytes. Unicode objects are converted to C strings using'utf-8'
encoding.z
(str
orNone
) [const char *]Like
s
, but the Python object may also beNone
, in which case the C pointer is set toNULL
.z*
(str
, bytes-like object orNone
) [Py_buffer]Like
s*
, but the Python object may also beNone
, in which case thebuf
member of thePy_buffer
structure is set toNULL
.z#
(str
, read-only bytes-like object orNone
) [const char *,Py_ssize_t
]Like
s#
, but the Python object may also beNone
, in which case the C pointer is set toNULL
.y
(read-only bytes-like object) [const char *]This format converts a bytes-like object to a C pointer to a borrowed character string; it does not accept Unicode objects. The bytes buffer must not contain embedded null bytes; if it does, a
ValueError
exception is raised.Changed in version 3.5: Previously,
TypeError
was raised when embedded null bytes were encountered in the bytes buffer.y*
(bytes-like object) [Py_buffer]This variant on
s*
doesn’t accept Unicode objects, only bytes-like objects. This is the recommended way to accept binary data.y#
(read-only bytes-like object) [const char *,Py_ssize_t
]This variant on
s#
doesn’t accept Unicode objects, only bytes-like objects.S
(bytes
) [PyBytesObject *]Requires that the Python object is a
bytes
object, without attempting any conversion. RaisesTypeError
if the object is not a bytes object. The C variable may also be declared as PyObject*.Y
(bytearray
) [PyByteArrayObject *]Requires that the Python object is a
bytearray
object, without attempting any conversion. RaisesTypeError
if the object is not abytearray
object. The C variable may also be declared as PyObject*.U
(str
) [PyObject *]Requires that the Python object is a Unicode object, without attempting any conversion. Raises
TypeError
if the object is not a Unicode object. The C variable may also be declared as PyObject*.w*
(read-write bytes-like object) [Py_buffer]This format accepts any object which implements the read-write buffer interface. It fills a
Py_buffer
structure provided by the caller. The buffer may contain embedded null bytes. The caller has to callPyBuffer_Release()
when it is done with the buffer.es
(str
) [const char *encoding, char **buffer]This variant on
s
is used for encoding Unicode into a character buffer. It only works for encoded data without embedded NUL bytes.This format requires two arguments. The first is only used as input, and must be a const char* which points to the name of an encoding as a NUL-terminated string, or
NULL
, in which case'utf-8'
encoding is used. An exception is raised if the named encoding is not known to Python. The second argument must be a char**; the value of the pointer it references will be set to a buffer with the contents of the argument text. The text will be encoded in the encoding specified by the first argument.PyArg_ParseTuple()
will allocate a buffer of the needed size, copy the encoded data into this buffer and adjust *buffer to reference the newly allocated storage. The caller is responsible for callingPyMem_Free()
to free the allocated buffer after use.et
(str
,bytes
orbytearray
) [const char *encoding, char **buffer]Same as
es
except that byte string objects are passed through without recoding them. Instead, the implementation assumes that the byte string object uses the encoding passed in as parameter.es#
(str
) [const char *encoding, char **buffer,Py_ssize_t
*buffer_length]This variant on
s#
is used for encoding Unicode into a character buffer. Unlike thees
format, this variant allows input data which contains NUL characters.It requires three arguments. The first is only used as input, and must be a const char* which points to the name of an encoding as a NUL-terminated string, or
NULL
, in which case'utf-8'
encoding is used. An exception is raised if the named encoding is not known to Python. The second argument must be a char**; the value of the pointer it references will be set to a buffer with the contents of the argument text. The text will be encoded in the encoding specified by the first argument. The third argument must be a pointer to an integer; the referenced integer will be set to the number of bytes in the output buffer.There are two modes of operation:
If *buffer points a
NULL
pointer, the function will allocate a buffer of the needed size, copy the encoded data into this buffer and set *buffer to reference the newly allocated storage. The caller is responsible for callingPyMem_Free()
to free the allocated buffer after usage.If *buffer points to a non-
NULL
pointer (an already allocated buffer),PyArg_ParseTuple()
will use this location as the buffer and interpret the initial value of *buffer_length as the buffer size. It will then copy the encoded data into the buffer and NUL-terminate it. If the buffer is not large enough, aValueError
will be set.In both cases, *buffer_length is set to the length of the encoded data without the trailing NUL byte.
et#
(str
,bytes
orbytearray
) [const char *encoding, char **buffer,Py_ssize_t
*buffer_length]Same as
es#
except that byte string objects are passed through without recoding them. Instead, the implementation assumes that the byte string object uses the encoding passed in as parameter.
Changed in version 3.12: u
, u#
, Z
, and Z#
are removed because they used a legacy
Py_UNICODE*
representation.
Numbers¶
These formats allow representing Python numbers or single characters as C numbers.
Formats that require int
, float
or complex
can
also use the corresponding special methods __index__()
,
__float__()
or __complex__()
to convert
the Python object to the required type.
For signed integer formats, OverflowError
is raised if the value
is out of range for the C type.
For unsigned integer formats, no range checking is done — the
most significant bits are silently truncated when the receiving field is too
small to receive the value.
b
(int
) [unsigned char]Convert a nonnegative Python integer to an unsigned tiny integer, stored in a C unsigned char.
B
(int
) [unsigned char]Convert a Python integer to a tiny integer without overflow checking, stored in a C unsigned char.
h
(int
) [short int]Convert a Python integer to a C short int.
H
(int
) [unsigned short int]Convert a Python integer to a C unsigned short int, without overflow checking.
i
(int
) [int]Convert a Python integer to a plain C int.
I
(int
) [unsigned int]Convert a Python integer to a C unsigned int, without overflow checking.
l
(int
) [long int]Convert a Python integer to a C long int.
k
(int
) [unsigned long]Convert a Python integer to a C unsigned long without overflow checking.
Changed in version 3.14: Use
__index__()
if available.L
(int
) [long long]Convert a Python integer to a C long long.
K
(int
) [unsigned long long]Convert a Python integer to a C unsigned long long without overflow checking.
Changed in version 3.14: Use
__index__()
if available.n
(int
) [Py_ssize_t
]Convert a Python integer to a C
Py_ssize_t
.c
(bytes
orbytearray
of length 1) [char]Convert a Python byte, represented as a
bytes
orbytearray
object of length 1, to a C char.Changed in version 3.3: Allow
bytearray
objects.C
(str
of length 1) [int]Convert a Python character, represented as a
str
object of length 1, to a C int.f
(float
) [float]Convert a Python floating-point number to a C float.
d
(float
) [double]Convert a Python floating-point number to a C double.
D
(complex
) [Py_complex]Convert a Python complex number to a C
Py_complex
structure.
Other objects¶
O
(object) [PyObject *]Store a Python object (without any conversion) in a C object pointer. The C program thus receives the actual object that was passed. A new strong reference to the object is not created (i.e. its reference count is not increased). The pointer stored is not
NULL
.O!
(object) [typeobject, PyObject *]Store a Python object in a C object pointer. This is similar to
O
, but takes two C arguments: the first is the address of a Python type object, the second is the address of the C variable (of type PyObject*) into which the object pointer is stored. If the Python object does not have the required type,TypeError
is raised.
O&
(object) [converter, address]Convert a Python object to a C variable through a converter function. This takes two arguments: the first is a function, the second is the address of a C variable (of arbitrary type), converted to void*. The converter function in turn is called as follows:
status = converter(object, address);
where object is the Python object to be converted and address is the void* argument that was passed to the
PyArg_Parse*
function. The returned status should be1
for a successful conversion and0
if the conversion has failed. When the conversion fails, the converter function should raise an exception and leave the content of address unmodified.If the converter returns
Py_CLEANUP_SUPPORTED
, it may get called a second time if the argument parsing eventually fails, giving the converter a chance to release any memory that it had already allocated. In this second call, the object parameter will beNULL
; address will have the same value as in the original call.Examples of converters:
PyUnicode_FSConverter()
andPyUnicode_FSDecoder()
.Changed in version 3.1:
Py_CLEANUP_SUPPORTED
was added.p
(bool
) [int]Tests the value passed in for truth (a boolean predicate) and converts the result to its equivalent C true/false integer value. Sets the int to
1
if the expression was true and0
if it was false. This accepts any valid Python value. See Truth Value Testing for more information about how Python tests values for truth.Added in version 3.3.
(items)
(sequence) [matching-items]The object must be a Python sequence (except
str
,bytes
orbytearray
) whose length is the number of format units in items. The C arguments must correspond to the individual format units in items. Format units for sequences may be nested.If items contains format units which store a borrowed buffer (
s
,s#
,z
,z#
,y
, ory#
) or a borrowed reference (S
,Y
,U
,O
, orO!
), the object must be a Python tuple. The converter for theO&
format unit in items must not store a borrowed buffer or a borrowed reference.Deprecated since version 3.14: Non-tuple sequences are deprecated if items contains format units which store a borrowed buffer or a borrowed reference.
A few other characters have a meaning in a format string. These may not occur inside nested parentheses. They are:
|
Indicates that the remaining arguments in the Python argument list are optional. The C variables corresponding to optional arguments should be initialized to their default value — when an optional argument is not specified,
PyArg_ParseTuple()
does not touch the contents of the corresponding C variable(s).$
PyArg_ParseTupleAndKeywords()
only: Indicates that the remaining arguments in the Python argument list are keyword-only. Currently, all keyword-only arguments must also be optional arguments, so|
must always be specified before$
in the format string.Added in version 3.3.
:
The list of format units ends here; the string after the colon is used as the function name in error messages (the “associated value” of the exception that
PyArg_ParseTuple()
raises).;
The list of format units ends here; the string after the semicolon is used as the error message instead of the default error message.
:
and;
mutually exclude each other.
Note that any Python object references which are provided to the caller are borrowed references; do not release them (i.e. do not decrement their reference count)!
Additional arguments passed to these functions must be addresses of variables whose type is determined by the format string; these are used to store values from the input tuple. There are a few cases, as described in the list of format units above, where these parameters are used as input values; they should match what is specified for the corresponding format unit in that case.
For the conversion to succeed, the arg object must match the format
and the format must be exhausted. On success, the
PyArg_Parse*
functions return true, otherwise they return
false and raise an appropriate exception. When the
PyArg_Parse*
functions fail due to conversion failure in one
of the format units, the variables at the addresses corresponding to that
and the following format units are left untouched.
API Functions¶
-
int PyArg_ParseTuple(PyObject *args, const char *format, ...)¶
- Part of the Stable ABI.
Parse the parameters of a function that takes only positional parameters into local variables. Returns true on success; on failure, it returns false and raises the appropriate exception.
-
int PyArg_VaParse(PyObject *args, const char *format, va_list vargs)¶
- Part of the Stable ABI.
Identical to
PyArg_ParseTuple()
, except that it accepts a va_list rather than a variable number of arguments.
-
int PyArg_ParseTupleAndKeywords(PyObject *args, PyObject *kw, const char *format, char *const *keywords, ...)¶
- Part of the Stable ABI.
Parse the parameters of a function that takes both positional and keyword parameters into local variables. The keywords argument is a
NULL
-terminated array of keyword parameter names specified as null-terminated ASCII or UTF-8 encoded C strings. Empty names denote positional-only parameters. Returns true on success; on failure, it returns false and raises the appropriate exception.Note
The keywords parameter declaration is char *const* in C and const char *const* in C++. This can be overridden with the
PY_CXX_CONST
macro.Changed in version 3.6: Added support for positional-only parameters.
Changed in version 3.13: The keywords parameter has now type char *const* in C and const char *const* in C++, instead of char**. Added support for non-ASCII keyword parameter names.
-
int PyArg_VaParseTupleAndKeywords(PyObject *args, PyObject *kw, const char *format, char *const *keywords, va_list vargs)¶
- Part of the Stable ABI.
Identical to
PyArg_ParseTupleAndKeywords()
, except that it accepts a va_list rather than a variable number of arguments.
-
int PyArg_ValidateKeywordArguments(PyObject*)¶
- Part of the Stable ABI.
Ensure that the keys in the keywords argument dictionary are strings. This is only needed if
PyArg_ParseTupleAndKeywords()
is not used, since the latter already does this check.Added in version 3.2.
-
int PyArg_Parse(PyObject *args, const char *format, ...)¶
- Part of the Stable ABI.
Parse the parameter of a function that takes a single positional parameter into a local variable. Returns true on success; on failure, it returns false and raises the appropriate exception.
Example:
// Function using METH_O calling convention static PyObject* my_function(PyObject *module, PyObject *arg) { int value; if (!PyArg_Parse(arg, "i:my_function", &value)) { return NULL; } // ... use value ... }
-
int PyArg_UnpackTuple(PyObject *args, const char *name, Py_ssize_t min, Py_ssize_t max, ...)¶
- Part of the Stable ABI.
A simpler form of parameter retrieval which does not use a format string to specify the types of the arguments. Functions which use this method to retrieve their parameters should be declared as
METH_VARARGS
in function or method tables. The tuple containing the actual parameters should be passed as args; it must actually be a tuple. The length of the tuple must be at least min and no more than max; min and max may be equal. Additional arguments must be passed to the function, each of which should be a pointer to a PyObject* variable; these will be filled in with the values from args; they will contain borrowed references. The variables which correspond to optional parameters not given by args will not be filled in; these should be initialized by the caller. This function returns true on success and false if args is not a tuple or contains the wrong number of elements; an exception will be set if there was a failure.This is an example of the use of this function, taken from the sources for the
_weakref
helper module for weak references:static PyObject * weakref_ref(PyObject *self, PyObject *args) { PyObject *object; PyObject *callback = NULL; PyObject *result = NULL; if (PyArg_UnpackTuple(args, "ref", 1, 2, &object, &callback)) { result = PyWeakref_NewRef(object, callback); } return result; }
The call to
PyArg_UnpackTuple()
in this example is entirely equivalent to this call toPyArg_ParseTuple()
:PyArg_ParseTuple(args, "O|O:ref", &object, &callback)
-
PY_CXX_CONST¶
The value to be inserted, if any, before char *const* in the keywords parameter declaration of
PyArg_ParseTupleAndKeywords()
andPyArg_VaParseTupleAndKeywords()
. Default empty for C andconst
for C++ (const char *const*). To override, define it to the desired value before includingPython.h
.Added in version 3.13.
Building values¶
-
PyObject *Py_BuildValue(const char *format, ...)¶
- Return value: New reference. Part of the Stable ABI.
Create a new value based on a format string similar to those accepted by the
PyArg_Parse*
family of functions and a sequence of values. Returns the value orNULL
in the case of an error; an exception will be raised ifNULL
is returned.Py_BuildValue()
does not always build a tuple. It builds a tuple only if its format string contains two or more format units. If the format string is empty, it returnsNone
; if it contains exactly one format unit, it returns whatever object is described by that format unit. To force it to return a tuple of size 0 or one, parenthesize the format string.When memory buffers are passed as parameters to supply data to build objects, as for the
s
ands#
formats, the required data is copied. Buffers provided by the caller are never referenced by the objects created byPy_BuildValue()
. In other words, if your code invokesmalloc()
and passes the allocated memory toPy_BuildValue()
, your code is responsible for callingfree()
for that memory oncePy_BuildValue()
returns.In the following description, the quoted form is the format unit; the entry in (round) parentheses is the Python object type that the format unit will return; and the entry in [square] brackets is the type of the C value(s) to be passed.
The characters space, tab, colon and comma are ignored in format strings (but not within format units such as
s#
). This can be used to make long format strings a tad more readable.s
(str
orNone
) [const char *]Convert a null-terminated C string to a Python
str
object using'utf-8'
encoding. If the C string pointer isNULL
,None
is used.s#
(str
orNone
) [const char *,Py_ssize_t
]Convert a C string and its length to a Python
str
object using'utf-8'
encoding. If the C string pointer isNULL
, the length is ignored andNone
is returned.y
(bytes
) [const char *]This converts a C string to a Python
bytes
object. If the C string pointer isNULL
,None
is returned.y#
(bytes
) [const char *,Py_ssize_t
]This converts a C string and its lengths to a Python object. If the C string pointer is
NULL
,None
is returned.z
(str
orNone
) [const char *]Same as
s
.z#
(str
orNone
) [const char *,Py_ssize_t
]Same as
s#
.u
(str
) [const wchar_t *]Convert a null-terminated
wchar_t
buffer of Unicode (UTF-16 or UCS-4) data to a Python Unicode object. If the Unicode buffer pointer isNULL
,None
is returned.u#
(str
) [const wchar_t *,Py_ssize_t
]Convert a Unicode (UTF-16 or UCS-4) data buffer and its length to a Python Unicode object. If the Unicode buffer pointer is
NULL
, the length is ignored andNone
is returned.U
(str
orNone
) [const char *]Same as
s
.U#
(str
orNone
) [const char *,Py_ssize_t
]Same as
s#
.i
(int
) [int]Convert a plain C int to a Python integer object.
b
(int
) [char]Convert a plain C char to a Python integer object.
h
(int
) [short int]Convert a plain C short int to a Python integer object.
l
(int
) [long int]Convert a C long int to a Python integer object.
B
(int
) [unsigned char]Convert a C unsigned char to a Python integer object.
H
(int
) [unsigned short int]Convert a C unsigned short int to a Python integer object.
I
(int
) [unsigned int]Convert a C unsigned int to a Python integer object.
k
(int
) [unsigned long]Convert a C unsigned long to a Python integer object.
L
(int
) [long long]Convert a C long long to a Python integer object.
K
(int
) [unsigned long long]Convert a C unsigned long long to a Python integer object.
n
(int
) [Py_ssize_t
]Convert a C
Py_ssize_t
to a Python integer.p
(bool
) [int]Convert a C int to a Python
bool
object.Be aware that this format requires an
int
argument. Unlike most other contexts in C, variadic arguments are not coerced to a suitable type automatically. You can convert another type (for example, a pointer or a float) to a suitableint
value using(x) ? 1 : 0
or!!x
.Added in version 3.14.
c
(bytes
of length 1) [char]Convert a C int representing a byte to a Python
bytes
object of length 1.C
(str
of length 1) [int]Convert a C int representing a character to Python
str
object of length 1.d
(float
) [double]Convert a C double to a Python floating-point number.
f
(float
) [float]Convert a C float to a Python floating-point number.
D
(complex
) [Py_complex *]Convert a C
Py_complex
structure to a Python complex number.O
(object) [PyObject *]Pass a Python object untouched but create a new strong reference to it (i.e. its reference count is incremented by one). If the object passed in is a
NULL
pointer, it is assumed that this was caused because the call producing the argument found an error and set an exception. Therefore,Py_BuildValue()
will returnNULL
but won’t raise an exception. If no exception has been raised yet,SystemError
is set.S
(object) [PyObject *]Same as
O
.N
(object) [PyObject *]Same as
O
, except it doesn’t create a new strong reference. Useful when the object is created by a call to an object constructor in the argument list.O&
(object) [converter, anything]Convert anything to a Python object through a converter function. The function is called with anything (which should be compatible with void*) as its argument and should return a “new” Python object, or
NULL
if an error occurred.(items)
(tuple
) [matching-items]Convert a sequence of C values to a Python tuple with the same number of items.
[items]
(list
) [matching-items]Convert a sequence of C values to a Python list with the same number of items.
{items}
(dict
) [matching-items]Convert a sequence of C values to a Python dictionary. Each pair of consecutive C values adds one item to the dictionary, serving as key and value, respectively.
If there is an error in the format string, the
SystemError
exception is set andNULL
returned.
-
PyObject *Py_VaBuildValue(const char *format, va_list vargs)¶
- Return value: New reference. Part of the Stable ABI.
Identical to
Py_BuildValue()
, except that it accepts a va_list rather than a variable number of arguments.


Call Protocol

Call Protocol¶
CPython supports two different calling protocols: tp_call and vectorcall.
The tp_call Protocol¶
Instances of classes that set tp_call
are callable.
The signature of the slot is:
PyObject *tp_call(PyObject *callable, PyObject *args, PyObject *kwargs);
A call is made using a tuple for the positional arguments
and a dict for the keyword arguments, similarly to
callable(*args, **kwargs)
in Python code.
args must be non-NULL (use an empty tuple if there are no arguments)
but kwargs may be NULL if there are no keyword arguments.
This convention is not only used by tp_call:
tp_new
and tp_init
also pass arguments this way.
To call an object, use PyObject_Call()
or another
call API.
The Vectorcall Protocol¶
Added in version 3.9.
The vectorcall protocol was introduced in PEP 590 as an additional protocol for making calls more efficient.
As rule of thumb, CPython will prefer the vectorcall for internal calls
if the callable supports it. However, this is not a hard rule.
Additionally, some third-party extensions use tp_call directly
(rather than using PyObject_Call()
).
Therefore, a class supporting vectorcall must also implement
tp_call
.
Moreover, the callable must behave the same
regardless of which protocol is used.
The recommended way to achieve this is by setting
tp_call
to PyVectorcall_Call()
.
This bears repeating:
Warning
A class supporting vectorcall must also implement
tp_call
with the same semantics.
Changed in version 3.12: The Py_TPFLAGS_HAVE_VECTORCALL
flag is now removed from a class
when the class’s __call__()
method is reassigned.
(This internally sets tp_call
only, and thus
may make it behave differently than the vectorcall function.)
In earlier Python versions, vectorcall should only be used with
immutable
or static types.
A class should not implement vectorcall if that would be slower than tp_call. For example, if the callee needs to convert the arguments to an args tuple and kwargs dict anyway, then there is no point in implementing vectorcall.
Classes can implement the vectorcall protocol by enabling the
Py_TPFLAGS_HAVE_VECTORCALL
flag and setting
tp_vectorcall_offset
to the offset inside the
object structure where a vectorcallfunc appears.
This is a pointer to a function with the following signature:
-
typedef PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)¶
- Part of the Stable ABI since version 3.12.
callable is the object being called.
- args is a C array consisting of the positional arguments followed by the
values of the keyword arguments. This can be NULL if there are no arguments.
- nargsf is the number of positional arguments plus possibly the
PY_VECTORCALL_ARGUMENTS_OFFSET
flag. To get the actual number of positional arguments from nargsf, usePyVectorcall_NARGS()
.
- kwnames is a tuple containing the names of the keyword arguments;
in other words, the keys of the kwargs dict. These names must be strings (instances of
str
or a subclass) and they must be unique. If there are no keyword arguments, then kwnames can instead be NULL.
-
PY_VECTORCALL_ARGUMENTS_OFFSET¶
- Part of the Stable ABI since version 3.12.
If this flag is set in a vectorcall nargsf argument, the callee is allowed to temporarily change
args[-1]
. In other words, args points to argument 1 (not 0) in the allocated vector. The callee must restore the value ofargs[-1]
before returning.For
PyObject_VectorcallMethod()
, this flag means instead thatargs[0]
may be changed.Whenever they can do so cheaply (without additional allocation), callers are encouraged to use
PY_VECTORCALL_ARGUMENTS_OFFSET
. Doing so will allow callables such as bound methods to make their onward calls (which include a prepended self argument) very efficiently.Added in version 3.8.
To call an object that implements vectorcall, use a call API
function as with any other callable.
PyObject_Vectorcall()
will usually be most efficient.
Recursion Control¶
When using tp_call, callees do not need to worry about
recursion: CPython uses
Py_EnterRecursiveCall()
and Py_LeaveRecursiveCall()
for calls made using tp_call.
For efficiency, this is not the case for calls done using vectorcall: the callee should use Py_EnterRecursiveCall and Py_LeaveRecursiveCall if needed.
Vectorcall Support API¶
-
Py_ssize_t PyVectorcall_NARGS(size_t nargsf)¶
- Part of the Stable ABI since version 3.12.
Given a vectorcall nargsf argument, return the actual number of arguments. Currently equivalent to:
(Py_ssize_t)(nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET)
However, the function
PyVectorcall_NARGS
should be used to allow for future extensions.Added in version 3.8.
-
vectorcallfunc PyVectorcall_Function(PyObject *op)¶
If op does not support the vectorcall protocol (either because the type does not or because the specific instance does not), return NULL. Otherwise, return the vectorcall function pointer stored in op. This function never raises an exception.
This is mostly useful to check whether or not op supports vectorcall, which can be done by checking
PyVectorcall_Function(op) != NULL
.Added in version 3.9.
-
PyObject *PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict)¶
- Part of the Stable ABI since version 3.12.
Call callable’s
vectorcallfunc
with positional and keyword arguments given in a tuple and dict, respectively.This is a specialized function, intended to be put in the
tp_call
slot or be used in an implementation oftp_call
. It does not check thePy_TPFLAGS_HAVE_VECTORCALL
flag and it does not fall back totp_call
.Added in version 3.8.
Object Calling API¶
Various functions are available for calling a Python object. Each converts its arguments to a convention supported by the called object – either tp_call or vectorcall. In order to do as little conversion as possible, pick one that best fits the format of data you have available.
The following table summarizes the available functions; please see individual documentation for details.
Function |
callable |
args |
kwargs |
|---|---|---|---|
|
tuple |
dict/ |
|
|
— |
— |
|
|
1 object |
— |
|
|
tuple/ |
— |
|
|
format |
— |
|
obj + |
format |
— |
|
|
variadic |
— |
|
obj + name |
variadic |
— |
|
obj + name |
— |
— |
|
obj + name |
1 object |
— |
|
|
vectorcall |
vectorcall |
|
|
vectorcall |
dict/ |
|
arg + name |
vectorcall |
vectorcall |
-
PyObject *PyObject_Call(PyObject *callable, PyObject *args, PyObject *kwargs)¶
- Return value: New reference. Part of the Stable ABI.
Call a callable Python object callable, with arguments given by the tuple args, and named arguments given by the dictionary kwargs.
args must not be NULL; use an empty tuple if no arguments are needed. If no named arguments are needed, kwargs can be NULL.
Return the result of the call on success, or raise an exception and return NULL on failure.
This is the equivalent of the Python expression:
callable(*args, **kwargs)
.
-
PyObject *PyObject_CallNoArgs(PyObject *callable)¶
- Return value: New reference. Part of the Stable ABI since version 3.10.
Call a callable Python object callable without any arguments. It is the most efficient way to call a callable Python object without any argument.
Return the result of the call on success, or raise an exception and return NULL on failure.
Added in version 3.9.
-
PyObject *PyObject_CallOneArg(PyObject *callable, PyObject *arg)¶
- Return value: New reference.
Call a callable Python object callable with exactly 1 positional argument arg and no keyword arguments.
Return the result of the call on success, or raise an exception and return NULL on failure.
Added in version 3.9.
-
PyObject *PyObject_CallObject(PyObject *callable, PyObject *args)¶
- Return value: New reference. Part of the Stable ABI.
Call a callable Python object callable, with arguments given by the tuple args. If no arguments are needed, then args can be NULL.
Return the result of the call on success, or raise an exception and return NULL on failure.
This is the equivalent of the Python expression:
callable(*args)
.
-
PyObject *PyObject_CallFunction(PyObject *callable, const char *format, ...)¶
- Return value: New reference. Part of the Stable ABI.
Call a callable Python object callable, with a variable number of C arguments. The C arguments are described using a
Py_BuildValue()
style format string. The format can be NULL, indicating that no arguments are provided.Return the result of the call on success, or raise an exception and return NULL on failure.
This is the equivalent of the Python expression:
callable(*args)
.Note that if you only pass PyObject* args,
PyObject_CallFunctionObjArgs()
is a faster alternative.Changed in version 3.4: The type of format was changed from
char *
.
-
PyObject *PyObject_CallMethod(PyObject *obj, const char *name, const char *format, ...)¶
- Return value: New reference. Part of the Stable ABI.
Call the method named name of object obj with a variable number of C arguments. The C arguments are described by a
Py_BuildValue()
format string that should produce a tuple.The format can be NULL, indicating that no arguments are provided.
Return the result of the call on success, or raise an exception and return NULL on failure.
This is the equivalent of the Python expression:
obj.name(arg1, arg2, ...)
.Note that if you only pass PyObject* args,
PyObject_CallMethodObjArgs()
is a faster alternative.Changed in version 3.4: The types of name and format were changed from
char *
.
-
PyObject *PyObject_CallFunctionObjArgs(PyObject *callable, ...)¶
- Return value: New reference. Part of the Stable ABI.
Call a callable Python object callable, with a variable number of PyObject* arguments. The arguments are provided as a variable number of parameters followed by NULL.
Return the result of the call on success, or raise an exception and return NULL on failure.
This is the equivalent of the Python expression:
callable(arg1, arg2, ...)
.
-
PyObject *PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ...)¶
- Return value: New reference. Part of the Stable ABI.
Call a method of the Python object obj, where the name of the method is given as a Python string object in name. It is called with a variable number of PyObject* arguments. The arguments are provided as a variable number of parameters followed by NULL.
Return the result of the call on success, or raise an exception and return NULL on failure.
-
PyObject *PyObject_CallMethodNoArgs(PyObject *obj, PyObject *name)¶
Call a method of the Python object obj without arguments, where the name of the method is given as a Python string object in name.
Return the result of the call on success, or raise an exception and return NULL on failure.
Added in version 3.9.
-
PyObject *PyObject_CallMethodOneArg(PyObject *obj, PyObject *name, PyObject *arg)¶
Call a method of the Python object obj with a single positional argument arg, where the name of the method is given as a Python string object in name.
Return the result of the call on success, or raise an exception and return NULL on failure.
Added in version 3.9.
-
PyObject *PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)¶
- Part of the Stable ABI since version 3.12.
Call a callable Python object callable. The arguments are the same as for
vectorcallfunc
. If callable supports vectorcall, this directly calls the vectorcall function stored in callable.Return the result of the call on success, or raise an exception and return NULL on failure.
Added in version 3.8: as
_PyObject_Vectorcall
Changed in version 3.9: Renamed to the current name, without the leading underscore. The old provisional name is soft deprecated.
-
PyObject *PyObject_VectorcallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict)¶
Call callable with positional arguments passed exactly as in the vectorcall protocol, but with keyword arguments passed as a dictionary kwdict. The args array contains only the positional arguments.
Regardless of which protocol is used internally, a conversion of arguments needs to be done. Therefore, this function should only be used if the caller already has a dictionary ready to use for the keyword arguments, but not a tuple for the positional arguments.
Added in version 3.9.
-
PyObject *PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames)¶
- Part of the Stable ABI since version 3.12.
Call a method using the vectorcall calling convention. The name of the method is given as a Python string name. The object whose method is called is args[0], and the args array starting at args[1] represents the arguments of the call. There must be at least one positional argument. nargsf is the number of positional arguments including args[0], plus
PY_VECTORCALL_ARGUMENTS_OFFSET
if the value ofargs[0]
may temporarily be changed. Keyword arguments can be passed just like inPyObject_Vectorcall()
.If the object has the
Py_TPFLAGS_METHOD_DESCRIPTOR
feature, this will call the unbound method object with the full args vector as arguments.Return the result of the call on success, or raise an exception and return NULL on failure.
Added in version 3.9.
Call Support API¶
-
int PyCallable_Check(PyObject *o)¶
- Part of the Stable ABI.
Determine if the object o is callable. Return
1
if the object is callable and0
otherwise. This function always succeeds.


Sorting Techniques

Sorting Techniques¶
- Author:
Andrew Dalke and Raymond Hettinger
Python lists have a built-in list.sort()
method that modifies the list
in-place. There is also a sorted()
built-in function that builds a new
sorted list from an iterable.
In this document, we explore the various techniques for sorting data using Python.
Sorting Basics¶
A simple ascending sort is very easy: just call the sorted()
function. It
returns a new sorted list:
>>> sorted([5, 2, 3, 1, 4])
[1, 2, 3, 4, 5]
You can also use the list.sort()
method. It modifies the list
in-place (and returns None
to avoid confusion). Usually it’s less convenient
than sorted()
- but if you don’t need the original list, it’s slightly
more efficient.
>>> a = [5, 2, 3, 1, 4]
>>> a.sort()
>>> a
[1, 2, 3, 4, 5]
Another difference is that the list.sort()
method is only defined for
lists. In contrast, the sorted()
function accepts any iterable.
>>> sorted({1: 'D', 2: 'B', 3: 'B', 4: 'E', 5: 'A'})
[1, 2, 3, 4, 5]
Key Functions¶
The list.sort()
method and the functions sorted()
,
min()
, max()
, heapq.nsmallest()
, and
heapq.nlargest()
have a key parameter to specify a function (or
other callable) to be called on each list element prior to making
comparisons.
For example, here’s a case-insensitive string comparison using
str.casefold()
:
>>> sorted("This is a test string from Andrew".split(), key=str.casefold)
['a', 'Andrew', 'from', 'is', 'string', 'test', 'This']
The value of the key parameter should be a function (or other callable) that takes a single argument and returns a key to use for sorting purposes. This technique is fast because the key function is called exactly once for each input record.
A common pattern is to sort complex objects using some of the object’s indices as keys. For example:
>>> student_tuples = [
... ('john', 'A', 15),
... ('jane', 'B', 12),
... ('dave', 'B', 10),
... ]
>>> sorted(student_tuples, key=lambda student: student[2]) # sort by age
[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]
The same technique works for objects with named attributes. For example:
>>> class Student:
... def __init__(self, name, grade, age):
... self.name = name
... self.grade = grade
... self.age = age
... def __repr__(self):
... return repr((self.name, self.grade, self.age))
>>> student_objects = [
... Student('john', 'A', 15),
... Student('jane', 'B', 12),
... Student('dave', 'B', 10),
... ]
>>> sorted(student_objects, key=lambda student: student.age) # sort by age
[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]
Objects with named attributes can be made by a regular class as shown
above, or they can be instances of dataclass
or
a named tuple.
Operator Module Functions and Partial Function Evaluation¶
The key function patterns shown above are very common, so Python provides
convenience functions to make accessor functions easier and faster. The
operator
module has itemgetter()
,
attrgetter()
, and a methodcaller()
function.
Using those functions, the above examples become simpler and faster:
>>> from operator import itemgetter, attrgetter
>>> sorted(student_tuples, key=itemgetter(2))
[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]
>>> sorted(student_objects, key=attrgetter('age'))
[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]
The operator module functions allow multiple levels of sorting. For example, to sort by grade then by age:
>>> sorted(student_tuples, key=itemgetter(1,2))
[('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)]
>>> sorted(student_objects, key=attrgetter('grade', 'age'))
[('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)]
The functools
module provides another helpful tool for making
key-functions. The partial()
function can reduce the
arity of a multi-argument
function making it suitable for use as a key-function.
>>> from functools import partial
>>> from unicodedata import normalize
>>> names = 'Zoë Åbjørn Núñez Élana Zeke Abe Nubia Eloise'.split()
>>> sorted(names, key=partial(normalize, 'NFD'))
['Abe', 'Åbjørn', 'Eloise', 'Élana', 'Nubia', 'Núñez', 'Zeke', 'Zoë']
>>> sorted(names, key=partial(normalize, 'NFC'))
['Abe', 'Eloise', 'Nubia', 'Núñez', 'Zeke', 'Zoë', 'Åbjørn', 'Élana']
Ascending and Descending¶
Both list.sort()
and sorted()
accept a reverse parameter with a
boolean value. This is used to flag descending sorts. For example, to get the
student data in reverse age order:
>>> sorted(student_tuples, key=itemgetter(2), reverse=True)
[('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]
>>> sorted(student_objects, key=attrgetter('age'), reverse=True)
[('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]
Sort Stability and Complex Sorts¶
Sorts are guaranteed to be stable. That means that when multiple records have the same key, their original order is preserved.
>>> data = [('red', 1), ('blue', 1), ('red', 2), ('blue', 2)]
>>> sorted(data, key=itemgetter(0))
[('blue', 1), ('blue', 2), ('red', 1), ('red', 2)]
Notice how the two records for blue retain their original order so that
('blue', 1)
is guaranteed to precede ('blue', 2)
.
This wonderful property lets you build complex sorts in a series of sorting steps. For example, to sort the student data by descending grade and then ascending age, do the age sort first and then sort again using grade:
>>> s = sorted(student_objects, key=attrgetter('age')) # sort on secondary key
>>> sorted(s, key=attrgetter('grade'), reverse=True) # now sort on primary key, descending
[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]
This can be abstracted out into a wrapper function that can take a list and tuples of field and order to sort them on multiple passes.
>>> def multisort(xs, specs):
... for key, reverse in reversed(specs):
... xs.sort(key=attrgetter(key), reverse=reverse)
... return xs
>>> multisort(list(student_objects), (('grade', True), ('age', False)))
[('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]
The Timsort algorithm used in Python does multiple sorts efficiently because it can take advantage of any ordering already present in a dataset.
Decorate-Sort-Undecorate¶
This idiom is called Decorate-Sort-Undecorate after its three steps:
First, the initial list is decorated with new values that control the sort order.
Second, the decorated list is sorted.
Finally, the decorations are removed, creating a list that contains only the initial values in the new order.
For example, to sort the student data by grade using the DSU approach:
>>> decorated = [(student.grade, i, student) for i, student in enumerate(student_objects)]
>>> decorated.sort()
>>> [student for grade, i, student in decorated] # undecorate
[('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]
This idiom works because tuples are compared lexicographically; the first items are compared; if they are the same then the second items are compared, and so on.
It is not strictly necessary in all cases to include the index i in the decorated list, but including it gives two benefits:
The sort is stable – if two items have the same key, their order will be preserved in the sorted list.
The original items do not have to be comparable because the ordering of the decorated tuples will be determined by at most the first two items. So for example the original list could contain complex numbers which cannot be sorted directly.
Another name for this idiom is Schwartzian transform, after Randal L. Schwartz, who popularized it among Perl programmers.
Now that Python sorting provides key-functions, this technique is not often needed.
Comparison Functions¶
Unlike key functions that return an absolute value for sorting, a comparison function computes the relative ordering for two inputs.
For example, a balance scale
compares two samples giving a relative ordering: lighter, equal, or heavier.
Likewise, a comparison function such as cmp(a, b)
will return a negative
value for less-than, zero if the inputs are equal, or a positive value for
greater-than.
It is common to encounter comparison functions when translating algorithms from
other languages. Also, some libraries provide comparison functions as part of
their API. For example, locale.strcoll()
is a comparison function.
To accommodate those situations, Python provides
functools.cmp_to_key
to wrap the comparison function
to make it usable as a key function:
sorted(words, key=cmp_to_key(strcoll)) # locale-aware sort order
Strategies For Unorderable Types and Values¶
A number of type and value issues can arise when sorting. Here are some strategies that can help:
Convert non-comparable input types to strings prior to sorting:
>>> data = ['twelve', '11', 10]
>>> sorted(map(str, data))
['10', '11', 'twelve']
This is needed because most cross-type comparisons raise a
TypeError
.
Remove special values prior to sorting:
>>> from math import isnan
>>> from itertools import filterfalse
>>> data = [3.3, float('nan'), 1.1, 2.2]
>>> sorted(filterfalse(isnan, data))
[1.1, 2.2, 3.3]
This is needed because the IEEE-754 standard specifies that, “Every NaN shall compare unordered with everything, including itself.”
Likewise, None
can be stripped from datasets as well:
>>> data = [3.3, None, 1.1, 2.2]
>>> sorted(x for x in data if x is not None)
[1.1, 2.2, 3.3]
This is needed because None
is not comparable to other types.
Convert mapping types into sorted item lists before sorting:
>>> data = [{'a': 1}, {'b': 2}]
>>> sorted(data, key=lambda d: sorted(d.items()))
[{'a': 1}, {'b': 2}]
This is needed because dict-to-dict comparisons raise a
TypeError
.
Convert set types into sorted lists before sorting:
>>> data = [{'a', 'b', 'c'}, {'b', 'c', 'd'}]
>>> sorted(map(sorted, data))
[['a', 'b', 'c'], ['b', 'c', 'd']]
This is needed because the elements contained in set types do not have a
deterministic order. For example, list({'a', 'b'})
may produce
either ['a', 'b']
or ['b', 'a']
.
Odds and Ends¶
For locale aware sorting, use
locale.strxfrm()
for a key function orlocale.strcoll()
for a comparison function. This is necessary because “alphabetical” sort orderings can vary across cultures even if the underlying alphabet is the same.The reverse parameter still maintains sort stability (so that records with equal keys retain the original order). Interestingly, that effect can be simulated without the parameter by using the builtin
reversed()
function twice:>>> data = [('red', 1), ('blue', 1), ('red', 2), ('blue', 2)] >>> standard_way = sorted(data, key=itemgetter(0), reverse=True) >>> double_reversed = list(reversed(sorted(reversed(data), key=itemgetter(0)))) >>> assert standard_way == double_reversed >>> standard_way [('red', 1), ('red', 2), ('blue', 1), ('blue', 2)]
The sort routines use
<
when making comparisons between two objects. So, it is easy to add a standard sort order to a class by defining an__lt__()
method:>>> Student.__lt__ = lambda self, other: self.age < other.age >>> sorted(student_objects) [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)]
However, note that
<
can fall back to using__gt__()
if__lt__()
is not implemented (seeobject.__lt__()
for details on the mechanics). To avoid surprises, PEP 8 recommends that all six comparison methods be implemented. Thetotal_ordering()
decorator is provided to make that task easier.Key functions need not depend directly on the objects being sorted. A key function can also access external resources. For instance, if the student grades are stored in a dictionary, they can be used to sort a separate list of student names:
>>> students = ['dave', 'john', 'jane'] >>> newgrades = {'john': 'F', 'jane':'A', 'dave': 'C'} >>> sorted(students, key=newgrades.__getitem__) ['jane', 'dave', 'john']
Partial Sorts¶
Some applications require only some of the data to be ordered. The standard library provides several tools that do less work than a full sort:
min()
andmax()
return the smallest and largest values, respectively. These functions make a single pass over the input data and require almost no auxiliary memory.heapq.nsmallest()
andheapq.nlargest()
return the n smallest and largest values, respectively. These functions make a single pass over the data keeping only n elements in memory at a time. For values of n that are small relative to the number of inputs, these functions make far fewer comparisons than a full sort.heapq.heappush()
andheapq.heappop()
create and maintain a partially sorted arrangement of data that keeps the smallest element at position0
. These functions are suitable for implementing priority queues which are commonly used for task scheduling.


Number Protocol

Number Protocol¶
-
int PyNumber_Check(PyObject *o)¶
- Part of the Stable ABI.
Returns
1
if the object o provides numeric protocols, and false otherwise. This function always succeeds.Changed in version 3.8: Returns
1
if o is an index integer.
-
PyObject *PyNumber_Add(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of adding o1 and o2, or
NULL
on failure. This is the equivalent of the Python expressiono1 + o2
.
-
PyObject *PyNumber_Subtract(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of subtracting o2 from o1, or
NULL
on failure. This is the equivalent of the Python expressiono1 - o2
.
-
PyObject *PyNumber_Multiply(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of multiplying o1 and o2, or
NULL
on failure. This is the equivalent of the Python expressiono1 * o2
.
-
PyObject *PyNumber_MatrixMultiply(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI since version 3.7.
Returns the result of matrix multiplication on o1 and o2, or
NULL
on failure. This is the equivalent of the Python expressiono1 @ o2
.Added in version 3.5.
-
PyObject *PyNumber_FloorDivide(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Return the floor of o1 divided by o2, or
NULL
on failure. This is the equivalent of the Python expressiono1 // o2
.
-
PyObject *PyNumber_TrueDivide(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Return a reasonable approximation for the mathematical value of o1 divided by o2, or
NULL
on failure. The return value is “approximate” because binary floating-point numbers are approximate; it is not possible to represent all real numbers in base two. This function can return a floating-point value when passed two integers. This is the equivalent of the Python expressiono1 / o2
.
-
PyObject *PyNumber_Remainder(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the remainder of dividing o1 by o2, or
NULL
on failure. This is the equivalent of the Python expressiono1 % o2
.
-
PyObject *PyNumber_Divmod(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
See the built-in function
divmod()
. ReturnsNULL
on failure. This is the equivalent of the Python expressiondivmod(o1, o2)
.
-
PyObject *PyNumber_Power(PyObject *o1, PyObject *o2, PyObject *o3)¶
- Return value: New reference. Part of the Stable ABI.
See the built-in function
pow()
. ReturnsNULL
on failure. This is the equivalent of the Python expressionpow(o1, o2, o3)
, where o3 is optional. If o3 is to be ignored, passPy_None
in its place (passingNULL
for o3 would cause an illegal memory access).
-
PyObject *PyNumber_Negative(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Returns the negation of o on success, or
NULL
on failure. This is the equivalent of the Python expression-o
.
-
PyObject *PyNumber_Positive(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Returns o on success, or
NULL
on failure. This is the equivalent of the Python expression+o
.
-
PyObject *PyNumber_Absolute(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Returns the absolute value of o, or
NULL
on failure. This is the equivalent of the Python expressionabs(o)
.
-
PyObject *PyNumber_Invert(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Returns the bitwise negation of o on success, or
NULL
on failure. This is the equivalent of the Python expression~o
.
-
PyObject *PyNumber_Lshift(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of left shifting o1 by o2 on success, or
NULL
on failure. This is the equivalent of the Python expressiono1 << o2
.
-
PyObject *PyNumber_Rshift(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of right shifting o1 by o2 on success, or
NULL
on failure. This is the equivalent of the Python expressiono1 >> o2
.
-
PyObject *PyNumber_And(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the “bitwise and” of o1 and o2 on success and
NULL
on failure. This is the equivalent of the Python expressiono1 & o2
.
-
PyObject *PyNumber_Xor(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the “bitwise exclusive or” of o1 by o2 on success, or
NULL
on failure. This is the equivalent of the Python expressiono1 ^ o2
.
-
PyObject *PyNumber_Or(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the “bitwise or” of o1 and o2 on success, or
NULL
on failure. This is the equivalent of the Python expressiono1 | o2
.
-
PyObject *PyNumber_InPlaceAdd(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of adding o1 and o2, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 += o2
.
-
PyObject *PyNumber_InPlaceSubtract(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of subtracting o2 from o1, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 -= o2
.
-
PyObject *PyNumber_InPlaceMultiply(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of multiplying o1 and o2, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 *= o2
.
-
PyObject *PyNumber_InPlaceMatrixMultiply(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI since version 3.7.
Returns the result of matrix multiplication on o1 and o2, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 @= o2
.Added in version 3.5.
-
PyObject *PyNumber_InPlaceFloorDivide(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the mathematical floor of dividing o1 by o2, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 //= o2
.
-
PyObject *PyNumber_InPlaceTrueDivide(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Return a reasonable approximation for the mathematical value of o1 divided by o2, or
NULL
on failure. The return value is “approximate” because binary floating-point numbers are approximate; it is not possible to represent all real numbers in base two. This function can return a floating-point value when passed two integers. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 /= o2
.
-
PyObject *PyNumber_InPlaceRemainder(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the remainder of dividing o1 by o2, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 %= o2
.
-
PyObject *PyNumber_InPlacePower(PyObject *o1, PyObject *o2, PyObject *o3)¶
- Return value: New reference. Part of the Stable ABI.
See the built-in function
pow()
. ReturnsNULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 **= o2
when o3 isPy_None
, or an in-place variant ofpow(o1, o2, o3)
otherwise. If o3 is to be ignored, passPy_None
in its place (passingNULL
for o3 would cause an illegal memory access).
-
PyObject *PyNumber_InPlaceLshift(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of left shifting o1 by o2 on success, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 <<= o2
.
-
PyObject *PyNumber_InPlaceRshift(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the result of right shifting o1 by o2 on success, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 >>= o2
.
-
PyObject *PyNumber_InPlaceAnd(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the “bitwise and” of o1 and o2 on success and
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 &= o2
.
-
PyObject *PyNumber_InPlaceXor(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the “bitwise exclusive or” of o1 by o2 on success, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 ^= o2
.
-
PyObject *PyNumber_InPlaceOr(PyObject *o1, PyObject *o2)¶
- Return value: New reference. Part of the Stable ABI.
Returns the “bitwise or” of o1 and o2 on success, or
NULL
on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statemento1 |= o2
.
-
PyObject *PyNumber_Long(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Returns the o converted to an integer object on success, or
NULL
on failure. This is the equivalent of the Python expressionint(o)
.
-
PyObject *PyNumber_Float(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Returns the o converted to a float object on success, or
NULL
on failure. This is the equivalent of the Python expressionfloat(o)
.
-
PyObject *PyNumber_Index(PyObject *o)¶
- Return value: New reference. Part of the Stable ABI.
Returns the o converted to a Python int on success or
NULL
with aTypeError
exception raised on failure.Changed in version 3.10: The result always has exact type
int
. Previously, the result could have been an instance of a subclass ofint
.
-
PyObject *PyNumber_ToBase(PyObject *n, int base)¶
- Return value: New reference. Part of the Stable ABI.
Returns the integer n converted to base base as a string. The base argument must be one of 2, 8, 10, or 16. For base 2, 8, or 16, the returned string is prefixed with a base marker of
'0b'
,'0o'
, or'0x'
, respectively. If n is not a Python int, it is converted withPyNumber_Index()
first.
-
Py_ssize_t PyNumber_AsSsize_t(PyObject *o, PyObject *exc)¶
- Part of the Stable ABI.
Returns o converted to a
Py_ssize_t
value if o can be interpreted as an integer. If the call fails, an exception is raised and-1
is returned.If o can be converted to a Python int but the attempt to convert to a
Py_ssize_t
value would raise anOverflowError
, then the exc argument is the type of exception that will be raised (usuallyIndexError
orOverflowError
). If exc isNULL
, then the exception is cleared and the value is clipped toPY_SSIZE_T_MIN
for a negative integer orPY_SSIZE_T_MAX
for a positive integer.
-
int PyIndex_Check(PyObject *o)¶
- Part of the Stable ABI since version 3.8.
Returns
1
if o is an index integer (has thenb_index
slot of thetp_as_number
structure filled in), and0
otherwise. This function always succeeds.