Subprocesses

Subprocesses¶
Source code: Lib/asyncio/subprocess.py, Lib/asyncio/base_subprocess.py
This section describes high-level async/await asyncio APIs to create and manage subprocesses.
Here’s an example of how asyncio can run a shell command and obtain its result:
import asyncio
async def run(cmd):
proc = await asyncio.create_subprocess_shell(
cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE)
stdout, stderr = await proc.communicate()
print(f'[{cmd!r} exited with {proc.returncode}]')
if stdout:
print(f'[stdout]\n{stdout.decode()}')
if stderr:
print(f'[stderr]\n{stderr.decode()}')
asyncio.run(run('ls /zzz'))
will print:
['ls /zzz' exited with 1]
[stderr]
ls: /zzz: No such file or directory
Because all asyncio subprocess functions are asynchronous and asyncio provides many tools to work with such functions, it is easy to execute and monitor multiple subprocesses in parallel. It is indeed trivial to modify the above example to run several commands simultaneously:
async def main():
await asyncio.gather(
run('ls /zzz'),
run('sleep 1; echo "hello"'))
asyncio.run(main())
See also the Examples subsection.
Creating Subprocesses¶
- async asyncio.create_subprocess_exec(program, *args, stdin=None, stdout=None, stderr=None, limit=None, **kwds)¶
Create a subprocess.
The limit argument sets the buffer limit for
StreamReader
wrappers forstdout
andstderr
(ifsubprocess.PIPE
is passed to stdout and stderr arguments).Return a
Process
instance.See the documentation of
loop.subprocess_exec()
for other parameters.If the process object is garbage collected while the process is still running, the child process will be killed.
Changed in version 3.10: Removed the loop parameter.
- async asyncio.create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, limit=None, **kwds)¶
Run the cmd shell command.
The limit argument sets the buffer limit for
StreamReader
wrappers forstdout
andstderr
(ifsubprocess.PIPE
is passed to stdout and stderr arguments).Return a
Process
instance.See the documentation of
loop.subprocess_shell()
for other parameters.If the process object is garbage collected while the process is still running, the child process will be killed.
Important
It is the application’s responsibility to ensure that all whitespace and special characters are quoted appropriately to avoid shell injection vulnerabilities. The
shlex.quote()
function can be used to properly escape whitespace and special shell characters in strings that are going to be used to construct shell commands.Changed in version 3.10: Removed the loop parameter.
Note
Subprocesses are available for Windows if a ProactorEventLoop
is
used. See Subprocess Support on Windows
for details.
See also
asyncio also has the following low-level APIs to work with subprocesses:
loop.subprocess_exec()
, loop.subprocess_shell()
,
loop.connect_read_pipe()
, loop.connect_write_pipe()
,
as well as the Subprocess Transports
and Subprocess Protocols.
Constants¶
- asyncio.subprocess.PIPE¶
Can be passed to the stdin, stdout or stderr parameters.
If PIPE is passed to stdin argument, the
Process.stdin
attribute will point to aStreamWriter
instance.If PIPE is passed to stdout or stderr arguments, the
Process.stdout
andProcess.stderr
attributes will point toStreamReader
instances.
- asyncio.subprocess.STDOUT¶
Special value that can be used as the stderr argument and indicates that standard error should be redirected into standard output.
- asyncio.subprocess.DEVNULL¶
Special value that can be used as the stdin, stdout or stderr argument to process creation functions. It indicates that the special file
os.devnull
will be used for the corresponding subprocess stream.
Interacting with Subprocesses¶
Both create_subprocess_exec()
and create_subprocess_shell()
functions return instances of the Process class. Process is a high-level
wrapper that allows communicating with subprocesses and watching for
their completion.
- class asyncio.subprocess.Process¶
An object that wraps OS processes created by the
create_subprocess_exec()
andcreate_subprocess_shell()
functions.This class is designed to have a similar API to the
subprocess.Popen
class, but there are some notable differences:unlike Popen, Process instances do not have an equivalent to the
poll()
method;the
communicate()
andwait()
methods don’t have a timeout parameter: use thewait_for()
function;the
Process.wait()
method is asynchronous, whereassubprocess.Popen.wait()
method is implemented as a blocking busy loop;the universal_newlines parameter is not supported.
This class is not thread safe.
See also the Subprocess and Threads section.
- async wait()¶
Wait for the child process to terminate.
Set and return the
returncode
attribute.Note
This method can deadlock when using
stdout=PIPE
orstderr=PIPE
and the child process generates so much output that it blocks waiting for the OS pipe buffer to accept more data. Use thecommunicate()
method when using pipes to avoid this condition.
- async communicate(input=None)¶
Interact with process:
send data to stdin (if input is not
None
);closes stdin;
read data from stdout and stderr, until EOF is reached;
wait for process to terminate.
The optional input argument is the data (
bytes
object) that will be sent to the child process.Return a tuple
(stdout_data, stderr_data)
.If either
BrokenPipeError
orConnectionResetError
exception is raised when writing input into stdin, the exception is ignored. This condition occurs when the process exits before all data are written into stdin.If it is desired to send data to the process’ stdin, the process needs to be created with
stdin=PIPE
. Similarly, to get anything other thanNone
in the result tuple, the process has to be created withstdout=PIPE
and/orstderr=PIPE
arguments.Note, that the data read is buffered in memory, so do not use this method if the data size is large or unlimited.
Changed in version 3.12: stdin gets closed when
input=None
too.
- send_signal(signal)¶
Sends the signal signal to the child process.
Note
On Windows,
SIGTERM
is an alias forterminate()
.CTRL_C_EVENT
andCTRL_BREAK_EVENT
can be sent to processes started with a creationflags parameter which includesCREATE_NEW_PROCESS_GROUP
.
- terminate()¶
Stop the child process.
On POSIX systems this method sends
SIGTERM
to the child process.On Windows the Win32 API function
TerminateProcess()
is called to stop the child process.
- kill()¶
Kill the child process.
On POSIX systems this method sends
SIGKILL
to the child process.On Windows this method is an alias for
terminate()
.
- stdin¶
Standard input stream (
StreamWriter
) orNone
if the process was created withstdin=None
.
- stdout¶
Standard output stream (
StreamReader
) orNone
if the process was created withstdout=None
.
- stderr¶
Standard error stream (
StreamReader
) orNone
if the process was created withstderr=None
.
Warning
Use the
communicate()
method rather thanprocess.stdin.write()
,await process.stdout.read()
orawait process.stderr.read()
. This avoids deadlocks due to streams pausing reading or writing and blocking the child process.- pid¶
Process identification number (PID).
Note that for processes created by the
create_subprocess_shell()
function, this attribute is the PID of the spawned shell.
- returncode¶
Return code of the process when it exits.
A
None
value indicates that the process has not terminated yet.A negative value
-N
indicates that the child was terminated by signalN
(POSIX only).
Subprocess and Threads¶
Standard asyncio event loop supports running subprocesses from different threads by default.
On Windows subprocesses are provided by ProactorEventLoop
only (default),
SelectorEventLoop
has no subprocess support.
Note that alternative event loop implementations might have own limitations; please refer to their documentation.
See also
The Concurrency and multithreading in asyncio section.
Examples¶
An example using the Process
class to
control a subprocess and the StreamReader
class to read from
its standard output.
The subprocess is created by the create_subprocess_exec()
function:
import asyncio
import sys
async def get_date():
code = 'import datetime; print(datetime.datetime.now())'
# Create the subprocess; redirect the standard output
# into a pipe.
proc = await asyncio.create_subprocess_exec(
sys.executable, '-c', code,
stdout=asyncio.subprocess.PIPE)
# Read one line of output.
data = await proc.stdout.readline()
line = data.decode('ascii').rstrip()
# Wait for the subprocess exit.
await proc.wait()
return line
date = asyncio.run(get_date())
print(f"Current date: {date}")
See also the same example written using low-level APIs.


Coroutine Objects

Coroutine Objects¶
Added in version 3.5.
Coroutine objects are what functions declared with an async
keyword
return.
-
type PyCoroObject¶
The C structure used for coroutine objects.
-
PyTypeObject PyCoro_Type¶
The type object corresponding to coroutine objects.
-
int PyCoro_CheckExact(PyObject *ob)¶
Return true if ob’s type is
PyCoro_Type
; ob must not beNULL
. This function always succeeds.
-
PyObject *PyCoro_New(PyFrameObject *frame, PyObject *name, PyObject *qualname)¶
- Return value: New reference.
Create and return a new coroutine object based on the frame object, with
__name__
and__qualname__
set to name and qualname. A reference to frame is stolen by this function. The frame argument must not beNULL
.


Migrating

Migrating optparse
code to argparse
¶
The argparse
module offers several higher level features not natively
provided by the optparse
module, including:
Handling positional arguments.
Supporting subcommands.
Allowing alternative option prefixes like
+
and/
.Handling zero-or-more and one-or-more style arguments.
Producing more informative usage messages.
Providing a much simpler interface for custom
type
andaction
.
Originally, the argparse
module attempted to maintain compatibility
with optparse
. However, the fundamental design differences between
supporting declarative command line option processing (while leaving positional
argument processing to application code), and supporting both named options
and positional arguments in the declarative interface mean that the
API has diverged from that of optparse
over time.
As described in Choosing an argument parsing library, applications that are
currently using optparse
and are happy with the way it works can
just continue to use optparse
.
Application developers that are considering migrating should also review the list of intrinsic behavioural differences described in that section before deciding whether or not migration is desirable.
For applications that do choose to migrate from optparse
to argparse
,
the following suggestions should be helpful:
Replace all
optparse.OptionParser.add_option()
calls withArgumentParser.add_argument()
calls.Replace
(options, args) = parser.parse_args()
withargs = parser.parse_args()
and add additionalArgumentParser.add_argument()
calls for the positional arguments. Keep in mind that what was previously calledoptions
, now in theargparse
context is calledargs
.Replace
optparse.OptionParser.disable_interspersed_args()
by usingparse_intermixed_args()
instead ofparse_args()
.Replace callback actions and the
callback_*
keyword arguments withtype
oraction
arguments.Replace string names for
type
keyword arguments with the corresponding type objects (e.g. int, float, complex, etc).Replace
optparse.Values
withNamespace
andoptparse.OptionError
andoptparse.OptionValueError
withArgumentError
.Replace strings with implicit arguments such as
%default
or%prog
with the standard Python syntax to use dictionaries to format strings, that is,%(default)s
and%(prog)s
.Replace the OptionParser constructor
version
argument with a call toparser.add_argument('--version', action='version', version='<the version>')
.


How to port Python 2 Code to Python 3

How to port Python 2 Code to Python 3¶
- author:
Brett Cannon
Python 2 reached its official end-of-life at the start of 2020. This means that no new bug reports, fixes, or changes will be made to Python 2 - it’s no longer supported: see PEP 373 and status of Python versions.
If you are looking to port an extension module instead of pure Python code, please see Porting Extension Modules to Python 3.
The archived python-porting mailing list may contain some useful guidance.
Since Python 3.11 the original porting guide was discontinued. You can find the old guide in the archive.
Third-party guides¶
There are also multiple third-party guides that might be useful:


Generator Objects

Generator Objects¶
Generator objects are what Python uses to implement generator iterators. They
are normally created by iterating over a function that yields values, rather
than explicitly calling PyGen_New()
or PyGen_NewWithQualName()
.
-
type PyGenObject¶
The C structure used for generator objects.
-
PyTypeObject PyGen_Type¶
The type object corresponding to generator objects.
-
int PyGen_Check(PyObject *ob)¶
Return true if ob is a generator object; ob must not be
NULL
. This function always succeeds.
-
int PyGen_CheckExact(PyObject *ob)¶
Return true if ob’s type is
PyGen_Type
; ob must not beNULL
. This function always succeeds.
-
PyObject *PyGen_New(PyFrameObject *frame)¶
- Return value: New reference.
Create and return a new generator object based on the frame object. A reference to frame is stolen by this function. The argument must not be
NULL
.
-
PyObject *PyGen_NewWithQualName(PyFrameObject *frame, PyObject *name, PyObject *qualname)¶
- Return value: New reference.
Create and return a new generator object based on the frame object, with
__name__
and__qualname__
set to name and qualname. A reference to frame is stolen by this function. The frame argument must not beNULL
.
-
PyCodeObject *PyGen_GetCode(PyGenObject *gen)¶
Return a new strong reference to the code object wrapped by gen. This function always succeeds.
Asynchronous Generator Objects¶
See also
-
PyTypeObject PyAsyncGen_Type¶
The type object corresponding to asynchronous generator objects. This is available as
types.AsyncGeneratorType
in the Python layer.Added in version 3.6.
-
PyObject *PyAsyncGen_New(PyFrameObject *frame, PyObject *name, PyObject *qualname)¶
Create a new asynchronous generator wrapping frame, with
__name__
and__qualname__
set to name and qualname. frame is stolen by this function and must not beNULL
.On success, this function returns a strong reference to the new asynchronous generator. On failure, this function returns
NULL
with an exception set.Added in version 3.6.
Deprecated API¶
-
PyAsyncGenASend_CheckExact(op)¶
This is a soft deprecated API that was included in Python’s C API by mistake.
It is solely here for completeness; do not use this API.


Pickle buffer objects

Pickle buffer objects¶
Added in version 3.8.
A pickle.PickleBuffer
object wraps a buffer-providing object for out-of-band data transfer with the pickle
module.
-
PyTypeObject PyPickleBuffer_Type¶
This instance of
PyTypeObject
represents the Python pickle buffer type. This is the same object aspickle.PickleBuffer
in the Python layer.
-
int PyPickleBuffer_Check(PyObject *op)¶
Return true if op is a pickle buffer instance. This function always succeeds.
-
PyObject *PyPickleBuffer_FromObject(PyObject *obj)¶
Create a pickle buffer from the object obj.
This function will fail if obj doesn’t support the buffer protocol.
On success, return a new pickle buffer instance. On failure, set an exception and return
NULL
.Analogous to calling
pickle.PickleBuffer
with obj in Python.
-
const Py_buffer *PyPickleBuffer_GetBuffer(PyObject *picklebuf)¶
Get a pointer to the underlying
Py_buffer
that the pickle buffer wraps.The returned pointer is valid as long as picklebuf is alive and has not been released. The caller must not modify or free the returned
Py_buffer
. If the pickle buffer has been released, raiseValueError
.On success, return a pointer to the buffer view. On failure, set an exception and return
NULL
.
-
int PyPickleBuffer_Release(PyObject *picklebuf)¶
Release the underlying buffer held by the pickle buffer.
Return
0
on success. On failure, set an exception and return-1
.Analogous to calling
pickle.PickleBuffer.release()
in Python.


Slice Objects

Slice Objects¶
-
PyTypeObject PySlice_Type¶
- Part of the Stable ABI.
The type object for slice objects. This is the same as
slice
in the Python layer.
-
int PySlice_Check(PyObject *ob)¶
Return true if ob is a slice object; ob must not be
NULL
. This function always succeeds.
-
PyObject *PySlice_New(PyObject *start, PyObject *stop, PyObject *step)¶
- Return value: New reference. Part of the Stable ABI.
Return a new slice object with the given values. The start, stop, and step parameters are used as the values of the slice object attributes of the same names. Any of the values may be
NULL
, in which case theNone
will be used for the corresponding attribute.Return
NULL
with an exception set if the new object could not be allocated.
-
int PySlice_GetIndices(PyObject *slice, Py_ssize_t length, Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step)¶
- Part of the Stable ABI.
Retrieve the start, stop and step indices from the slice object slice, assuming a sequence of length length. Treats indices greater than length as errors.
Returns
0
on success and-1
on error with no exception set (unless one of the indices was notNone
and failed to be converted to an integer, in which case-1
is returned with an exception set).You probably do not want to use this function.
Changed in version 3.2: The parameter type for the slice parameter was
PySliceObject*
before.
-
int PySlice_GetIndicesEx(PyObject *slice, Py_ssize_t length, Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step, Py_ssize_t *slicelength)¶
- Part of the Stable ABI.
Usable replacement for
PySlice_GetIndices()
. Retrieve the start, stop, and step indices from the slice object slice assuming a sequence of length length, and store the length of the slice in slicelength. Out of bounds indices are clipped in a manner consistent with the handling of normal slices.Return
0
on success and-1
on error with an exception set.Note
This function is considered not safe for resizable sequences. Its invocation should be replaced by a combination of
PySlice_Unpack()
andPySlice_AdjustIndices()
whereif (PySlice_GetIndicesEx(slice, length, &start, &stop, &step, &slicelength) < 0) { // return error }
is replaced by
if (PySlice_Unpack(slice, &start, &stop, &step) < 0) { // return error } slicelength = PySlice_AdjustIndices(length, &start, &stop, step);
Changed in version 3.2: The parameter type for the slice parameter was
PySliceObject*
before.Changed in version 3.6.1: If
Py_LIMITED_API
is not set or set to the value between0x03050400
and0x03060000
(not including) or0x03060100
or higherPySlice_GetIndicesEx()
is implemented as a macro usingPySlice_Unpack()
andPySlice_AdjustIndices()
. Arguments start, stop and step are evaluated more than once.Deprecated since version 3.6.1: If
Py_LIMITED_API
is set to the value less than0x03050400
or between0x03060000
and0x03060100
(not including)PySlice_GetIndicesEx()
is a deprecated function.
-
int PySlice_Unpack(PyObject *slice, Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step)¶
- Part of the Stable ABI since version 3.7.
Extract the start, stop and step data members from a slice object as C integers. Silently reduce values larger than
PY_SSIZE_T_MAX
toPY_SSIZE_T_MAX
, silently boost the start and stop values less thanPY_SSIZE_T_MIN
toPY_SSIZE_T_MIN
, and silently boost the step values less than-PY_SSIZE_T_MAX
to-PY_SSIZE_T_MAX
.Return
-1
with an exception set on error,0
on success.Added in version 3.6.1.
-
Py_ssize_t PySlice_AdjustIndices(Py_ssize_t length, Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t step)¶
- Part of the Stable ABI since version 3.7.
Adjust start/end slice indices assuming a sequence of the specified length. Out of bounds indices are clipped in a manner consistent with the handling of normal slices.
Return the length of the slice. Always successful. Doesn’t call Python code.
Added in version 3.6.1.
Ellipsis Object¶
-
PyTypeObject PyEllipsis_Type¶
- Part of the Stable ABI.
The type of Python
Ellipsis
object. Same astypes.EllipsisType
in the Python layer.
-
PyObject *Py_Ellipsis¶
The Python
Ellipsis
object. This object has no methods. LikePy_None
, it is an immortal singleton object.Changed in version 3.12:
Py_Ellipsis
is immortal.


Descriptor Objects

Descriptor Objects¶
“Descriptors” are objects that describe some attribute of an object. They are found in the dictionary of type objects.
-
PyTypeObject PyProperty_Type¶
- Part of the Stable ABI.
The type object for the built-in descriptor types.
-
PyObject *PyDescr_NewGetSet(PyTypeObject *type, struct PyGetSetDef *getset)¶
- Return value: New reference. Part of the Stable ABI.
-
PyObject *PyDescr_NewMember(PyTypeObject *type, struct PyMemberDef *meth)¶
- Return value: New reference. Part of the Stable ABI.
-
PyTypeObject PyMemberDescr_Type¶
- Part of the Stable ABI.
The type object for member descriptor objects created from
PyMemberDef
structures. These descriptors expose fields of a C struct as attributes on a type, and correspond totypes.MemberDescriptorType
objects in Python.
-
PyTypeObject PyGetSetDescr_Type¶
- Part of the Stable ABI.
The type object for get/set descriptor objects created from
PyGetSetDef
structures. These descriptors implement attributes whose value is computed by C getter and setter functions, and are used for many built-in type attributes.
-
PyObject *PyDescr_NewMethod(PyTypeObject *type, struct PyMethodDef *meth)¶
- Return value: New reference. Part of the Stable ABI.
-
PyTypeObject PyMethodDescr_Type¶
- Part of the Stable ABI.
The type object for method descriptor objects created from
PyMethodDef
structures. These descriptors expose C functions as methods on a type, and correspond totypes.MemberDescriptorType
objects in Python.
-
PyObject *PyDescr_NewWrapper(PyTypeObject *type, struct wrapperbase *wrapper, void *wrapped)¶
- Return value: New reference.
-
PyTypeObject PyWrapperDescr_Type¶
- Part of the Stable ABI.
The type object for wrapper descriptor objects created by
PyDescr_NewWrapper()
andPyWrapper_New()
. Wrapper descriptors are used internally to expose special methods implemented via wrapper structures, and appear in Python astypes.WrapperDescriptorType
objects.
-
PyObject *PyDescr_NewClassMethod(PyTypeObject *type, PyMethodDef *method)¶
- Return value: New reference. Part of the Stable ABI.
-
int PyDescr_IsData(PyObject *descr)¶
Return non-zero if the descriptor object descr describes a data attribute, or
0
if it describes a method. descr must be a descriptor object; there is no error checking.
-
PyObject *PyWrapper_New(PyObject*, PyObject*)¶
- Return value: New reference. Part of the Stable ABI.
Built-in descriptors¶
-
PyTypeObject PySuper_Type¶
- Part of the Stable ABI.
The type object for super objects. This is the same object as
super
in the Python layer.
-
PyTypeObject PyClassMethod_Type¶
The type of class method objects. This is the same object as
classmethod
in the Python layer.
-
PyTypeObject PyClassMethodDescr_Type¶
- Part of the Stable ABI.
The type object for C-level class method descriptor objects. This is the type of the descriptors created for
classmethod()
defined in C extension types, and is the same object asclassmethod
in Python.
-
PyObject *PyClassMethod_New(PyObject *callable)¶
Create a new
classmethod
object wrapping callable. callable must be a callable object and must not beNULL
.On success, this function returns a strong reference to a new class method descriptor. On failure, this function returns
NULL
with an exception set.
-
PyTypeObject PyStaticMethod_Type¶
The type of static method objects. This is the same object as
staticmethod
in the Python layer.
-
PyObject *PyStaticMethod_New(PyObject *callable)¶
Create a new
staticmethod
object wrapping callable. callable must be a callable object and must not beNULL
.On success, this function returns a strong reference to a new static method descriptor. On failure, this function returns
NULL
with an exception set.


Iterator Objects

Iterator Objects¶
Python provides two general-purpose iterator objects. The first, a sequence
iterator, works with an arbitrary sequence supporting the __getitem__()
method. The second works with a callable object and a sentinel value, calling
the callable for each item in the sequence, and ending the iteration when the
sentinel value is returned.
-
PyTypeObject PySeqIter_Type¶
- Part of the Stable ABI.
Type object for iterator objects returned by
PySeqIter_New()
and the one-argument form of theiter()
built-in function for built-in sequence types.
-
int PySeqIter_Check(PyObject *op)¶
Return true if the type of op is
PySeqIter_Type
. This function always succeeds.
-
PyObject *PySeqIter_New(PyObject *seq)¶
- Return value: New reference. Part of the Stable ABI.
Return an iterator that works with a general sequence object, seq. The iteration ends when the sequence raises
IndexError
for the subscripting operation.
-
PyTypeObject PyCallIter_Type¶
- Part of the Stable ABI.
Type object for iterator objects returned by
PyCallIter_New()
and the two-argument form of theiter()
built-in function.
-
int PyCallIter_Check(PyObject *op)¶
Return true if the type of op is
PyCallIter_Type
. This function always succeeds.
-
PyObject *PyCallIter_New(PyObject *callable, PyObject *sentinel)¶
- Return value: New reference. Part of the Stable ABI.
Return a new iterator. The first parameter, callable, can be any Python callable object that can be called with no parameters; each call to it should return the next item in the iteration. When callable returns a value equal to sentinel, the iteration will be terminated.
Range Objects¶
-
PyTypeObject PyRange_Type¶
- Part of the Stable ABI.
The type object for
range
objects.
Builtin Iterator Types¶
These are built-in iteration types that are included in Python’s C API, but provide no additional functions. They are here for completeness.
C type |
Python type |
|---|---|
|
|
|
|
|
|
|
|
|
Other Iterator Objects¶
-
PyTypeObject PyByteArrayIter_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyBytesIter_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyListIter_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyListRevIter_Type¶
- Part of the Stable ABI.
-
PyTypeObject PySetIter_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyTupleIter_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyRangeIter_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyLongRangeIter_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyDictIterKey_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyDictRevIterKey_Type¶
- Part of the Stable ABI since version 3.8.
-
PyTypeObject PyDictIterValue_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyDictRevIterValue_Type¶
- Part of the Stable ABI since version 3.8.
-
PyTypeObject PyDictIterItem_Type¶
- Part of the Stable ABI.
-
PyTypeObject PyDictRevIterItem_Type¶
- Part of the Stable ABI since version 3.8.
-
PyTypeObject PyODictIter_Type¶
Type objects for iterators of various built-in objects.
Do not create instances of these directly; prefer calling
PyObject_GetIter()
instead.Note that there is no guarantee that a given built-in type uses a given iterator type. For example, iterating over
range
will use one of two iterator types depending on the size of the range. Other types may start using a similar scheme in the future, without warning.


: Generating MIME documents

email.generator
: Generating MIME documents¶
Source code: Lib/email/generator.py
One of the most common tasks is to generate the flat (serialized) version of
the email message represented by a message object structure. You will need to
do this if you want to send your message via smtplib.SMTP.sendmail()
,
or print the message on the console. Taking a
message object structure and producing a serialized representation is the job
of the generator classes.
As with the email.parser
module, you aren’t limited to the functionality
of the bundled generator; you could write one from scratch yourself. However
the bundled generator knows how to generate most email in a standards-compliant
way, should handle MIME and non-MIME email messages just fine, and is designed
so that the bytes-oriented parsing and generation operations are inverses,
assuming the same non-transforming policy
is used for both. That
is, parsing the serialized byte stream via the
BytesParser
class and then regenerating the serialized
byte stream using BytesGenerator
should produce output identical to
the input [1]. (On the other hand, using the generator on an
EmailMessage
constructed by program may result in
changes to the EmailMessage
object as defaults are
filled in.)
The Generator
class can be used to flatten a message into a text (as
opposed to binary) serialized representation, but since Unicode cannot
represent binary data directly, the message is of necessity transformed into
something that contains only ASCII characters, using the standard email RFC
Content Transfer Encoding techniques for encoding email messages for transport
over channels that are not “8 bit clean”.
To accommodate reproducible processing of SMIME-signed messages
Generator
disables header folding for message parts of type
multipart/signed
and all subparts.
- class email.generator.BytesGenerator(outfp, mangle_from_=None, maxheaderlen=None, *, policy=None)¶
Return a
BytesGenerator
object that will write any message provided to theflatten()
method, or any surrogateescape encoded text provided to thewrite()
method, to the file-like object outfp. outfp must support awrite
method that accepts binary data.If optional mangle_from_ is
True
, put a>
character in front of any line in the body that starts with the exact string"From "
, that isFrom
followed by a space at the beginning of a line. mangle_from_ defaults to the value of themangle_from_
setting of the policy (which isTrue
for thecompat32
policy andFalse
for all others). mangle_from_ is intended for use when messages are stored in Unix mbox format (seemailbox
and WHY THE CONTENT-LENGTH FORMAT IS BAD).If maxheaderlen is not
None
, refold any header lines that are longer than maxheaderlen, or if0
, do not rewrap any headers. If manheaderlen isNone
(the default), wrap headers and other message lines according to the policy settings.If policy is specified, use that policy to control message generation. If policy is
None
(the default), use the policy associated with theMessage
orEmailMessage
object passed toflatten
to control the message generation. Seeemail.policy
for details on what policy controls.Added in version 3.2.
Changed in version 3.3: Added the policy keyword.
Changed in version 3.6: The default behavior of the mangle_from_ and maxheaderlen parameters is to follow the policy.
- flatten(msg, unixfrom=False, linesep=None)¶
Print the textual representation of the message object structure rooted at msg to the output file specified when the
BytesGenerator
instance was created.If the
policy
optioncte_type
is8bit
(the default), copy any headers in the original parsed message that have not been modified to the output with any bytes with the high bit set reproduced as in the original, and preserve the non-ASCII Content-Transfer-Encoding of any body parts that have them. Ifcte_type
is7bit
, convert the bytes with the high bit set as needed using an ASCII-compatible Content-Transfer-Encoding. That is, transform parts with non-ASCII Content-Transfer-Encoding (Content-Transfer-Encoding: 8bit) to an ASCII compatible Content-Transfer-Encoding, and encode RFC-invalid non-ASCII bytes in headers using the MIMEunknown-8bit
character set, thus rendering them RFC-compliant.If unixfrom is
True
, print the envelope header delimiter used by the Unix mailbox format (seemailbox
) before the first of the RFC 5322 headers of the root message object. If the root object has no envelope header, craft a standard one. The default isFalse
. Note that for subparts, no envelope header is ever printed.If linesep is not
None
, use it as the separator character between all the lines of the flattened message. If linesep isNone
(the default), use the value specified in the policy.
- clone(fp)¶
Return an independent clone of this
BytesGenerator
instance with the exact same option settings, and fp as the new outfp.
- write(s)¶
Encode s using the
ASCII
codec and thesurrogateescape
error handler, and pass it to the write method of the outfp passed to theBytesGenerator
’s constructor.
As a convenience, EmailMessage
provides the methods
as_bytes()
and bytes(aMessage)
(a.k.a.
__bytes__()
), which simplify the generation of
a serialized binary representation of a message object. For more detail, see
email.message
.
Because strings cannot represent binary data, the Generator
class must
convert any binary data in any message it flattens to an ASCII compatible
format, by converting them to an ASCII compatible
Content-Transfer_Encoding. Using the terminology of the email
RFCs, you can think of this as Generator
serializing to an I/O stream
that is not “8 bit clean”. In other words, most applications will want
to be using BytesGenerator
, and not Generator
.
- class email.generator.Generator(outfp, mangle_from_=None, maxheaderlen=None, *, policy=None)¶
Return a
Generator
object that will write any message provided to theflatten()
method, or any text provided to thewrite()
method, to the file-like object outfp. outfp must support awrite
method that accepts string data.If optional mangle_from_ is
True
, put a>
character in front of any line in the body that starts with the exact string"From "
, that isFrom
followed by a space at the beginning of a line. mangle_from_ defaults to the value of themangle_from_
setting of the policy (which isTrue
for thecompat32
policy andFalse
for all others). mangle_from_ is intended for use when messages are stored in Unix mbox format (seemailbox
and WHY THE CONTENT-LENGTH FORMAT IS BAD).If maxheaderlen is not
None
, refold any header lines that are longer than maxheaderlen, or if0
, do not rewrap any headers. If manheaderlen isNone
(the default), wrap headers and other message lines according to the policy settings.If policy is specified, use that policy to control message generation. If policy is
None
(the default), use the policy associated with theMessage
orEmailMessage
object passed toflatten
to control the message generation. Seeemail.policy
for details on what policy controls.Changed in version 3.3: Added the policy keyword.
Changed in version 3.6: The default behavior of the mangle_from_ and maxheaderlen parameters is to follow the policy.
- flatten(msg, unixfrom=False, linesep=None)¶
Print the textual representation of the message object structure rooted at msg to the output file specified when the
Generator
instance was created.If the
policy
optioncte_type
is8bit
, generate the message as if the option were set to7bit
. (This is required because strings cannot represent non-ASCII bytes.) Convert any bytes with the high bit set as needed using an ASCII-compatible Content-Transfer-Encoding. That is, transform parts with non-ASCII Content-Transfer-Encoding (Content-Transfer-Encoding: 8bit) to an ASCII compatible Content-Transfer-Encoding, and encode RFC-invalid non-ASCII bytes in headers using the MIMEunknown-8bit
character set, thus rendering them RFC-compliant.If unixfrom is
True
, print the envelope header delimiter used by the Unix mailbox format (seemailbox
) before the first of the RFC 5322 headers of the root message object. If the root object has no envelope header, craft a standard one. The default isFalse
. Note that for subparts, no envelope header is ever printed.If linesep is not
None
, use it as the separator character between all the lines of the flattened message. If linesep isNone
(the default), use the value specified in the policy.Changed in version 3.2: Added support for re-encoding
8bit
message bodies, and the linesep argument.
As a convenience, EmailMessage
provides the methods
as_string()
and str(aMessage)
(a.k.a.
__str__()
), which simplify the generation of
a formatted string representation of a message object. For more detail, see
email.message
.
The email.generator
module also provides a derived class,
DecodedGenerator
, which is like the Generator
base class,
except that non-text parts are not serialized, but are instead
represented in the output stream by a string derived from a template filled
in with information about the part.
- class email.generator.DecodedGenerator(outfp, mangle_from_=None, maxheaderlen=None, fmt=None, *, policy=None)¶
Act like
Generator
, except that for any subpart of the message passed toGenerator.flatten()
, if the subpart is of main type text, print the decoded payload of the subpart, and if the main type is not text, instead of printing it fill in the string fmt using information from the part and print the resulting filled-in string.To fill in fmt, execute
fmt % part_info
, wherepart_info
is a dictionary composed of the following keys and values:type
– Full MIME type of the non-text partmaintype
– Main MIME type of the non-text partsubtype
– Sub-MIME type of the non-text partfilename
– Filename of the non-text partdescription
– Description associated with the non-text partencoding
– Content transfer encoding of the non-text part
If fmt is
None
, use the following default fmt:“[Non-text (%(type)s) part of message omitted, filename %(filename)s]”
Optional _mangle_from_ and maxheaderlen are as with the
Generator
base class.
Footnotes


email.message.Message
: Representing an email message using the compat32
API¶
The Message
class is very similar to the
EmailMessage
class, without the methods added by that
class, and with the default behavior of certain other methods being slightly
different. We also document here some methods that, while supported by the
EmailMessage
class, are not recommended unless you are
dealing with legacy code.
The philosophy and structure of the two classes is otherwise the same.
This document describes the behavior under the default (for Message
)
policy Compat32
. If you are going to use another policy,
you should be using the EmailMessage
class instead.
An email message consists of headers and a payload. Headers must be RFC 5322 style names and values, where the field name and value are separated by a colon. The colon is not part of either the field name or the field value. The payload may be a simple text message, or a binary object, or a structured sequence of sub-messages each with their own set of headers and their own payload. The latter type of payload is indicated by the message having a MIME type such as multipart/* or message/rfc822.
The conceptual model provided by a Message
object is that of an
ordered dictionary of headers with additional methods for accessing both
specialized information from the headers, for accessing the payload, for
generating a serialized version of the message, and for recursively walking
over the object tree. Note that duplicate headers are supported but special
methods must be used to access them.
The Message
pseudo-dictionary is indexed by the header names, which
must be ASCII values. The values of the dictionary are strings that are
supposed to contain only ASCII characters; there is some special handling for
non-ASCII input, but it doesn’t always produce the correct results. Headers
are stored and returned in case-preserving form, but field names are matched
case-insensitively. There may also be a single envelope header, also known as
the Unix-From header or the From_
header. The payload is either a
string or bytes, in the case of simple message objects, or a list of
Message
objects, for MIME container documents (e.g.
multipart/* and message/rfc822).
Here are the methods of the Message
class:
- class email.message.Message(policy=compat32)¶
If policy is specified (it must be an instance of a
policy
class) use the rules it specifies to update and serialize the representation of the message. If policy is not set, use thecompat32
policy, which maintains backward compatibility with the Python 3.2 version of the email package. For more information see thepolicy
documentation.Changed in version 3.3: The policy keyword argument was added.
- as_string(unixfrom=False, maxheaderlen=0, policy=None)¶
Return the entire message flattened as a string. When optional unixfrom is true, the envelope header is included in the returned string. unixfrom defaults to
False
. For backward compatibility reasons, maxheaderlen defaults to0
, so if you want a different value you must override it explicitly (the value specified for max_line_length in the policy will be ignored by this method). The policy argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified policy will be passed to theGenerator
.Flattening the message may trigger changes to the
Message
if defaults need to be filled in to complete the transformation to a string (for example, MIME boundaries may be generated or modified).Note that this method is provided as a convenience and may not always format the message the way you want. For example, by default it does not do the mangling of lines that begin with
From
that is required by the Unix mbox format. For more flexibility, instantiate aGenerator
instance and use itsflatten()
method directly. For example:from io import StringIO from email.generator import Generator fp = StringIO() g = Generator(fp, mangle_from_=True, maxheaderlen=60) g.flatten(msg) text = fp.getvalue()
If the message object contains binary data that is not encoded according to RFC standards, the non-compliant data will be replaced by unicode “unknown character” code points. (See also
as_bytes()
andBytesGenerator
.)Changed in version 3.4: the policy keyword argument was added.
- __str__()¶
Equivalent to
as_string()
. Allowsstr(msg)
to produce a string containing the formatted message.
- as_bytes(unixfrom=False, policy=None)¶
Return the entire message flattened as a bytes object. When optional unixfrom is true, the envelope header is included in the returned string. unixfrom defaults to
False
. The policy argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified policy will be passed to theBytesGenerator
.Flattening the message may trigger changes to the
Message
if defaults need to be filled in to complete the transformation to a string (for example, MIME boundaries may be generated or modified).Note that this method is provided as a convenience and may not always format the message the way you want. For example, by default it does not do the mangling of lines that begin with
From
that is required by the Unix mbox format. For more flexibility, instantiate aBytesGenerator
instance and use itsflatten()
method directly. For example:from io import BytesIO from email.generator import BytesGenerator fp = BytesIO() g = BytesGenerator(fp, mangle_from_=True, maxheaderlen=60) g.flatten(msg) text = fp.getvalue()
Added in version 3.4.
- __bytes__()¶
Equivalent to
as_bytes()
. Allowsbytes(msg)
to produce a bytes object containing the formatted message.Added in version 3.4.
- is_multipart()¶
Return
True
if the message’s payload is a list of sub-Message
objects, otherwise returnFalse
. Whenis_multipart()
returnsFalse
, the payload should be a string object (which might be a CTE encoded binary payload). (Note thatis_multipart()
returningTrue
does not necessarily mean that “msg.get_content_maintype() == ‘multipart’” will return theTrue
. For example,is_multipart
will returnTrue
when theMessage
is of typemessage/rfc822
.)
- set_unixfrom(unixfrom)¶
Set the message’s envelope header to unixfrom, which should be a string.
- get_unixfrom()¶
Return the message’s envelope header. Defaults to
None
if the envelope header was never set.
- attach(payload)¶
Add the given payload to the current payload, which must be
None
or a list ofMessage
objects before the call. After the call, the payload will always be a list ofMessage
objects. If you want to set the payload to a scalar object (e.g. a string), useset_payload()
instead.This is a legacy method. On the
EmailMessage
class its functionality is replaced byset_content()
and the relatedmake
andadd
methods.
- get_payload(i=None, decode=False)¶
Return the current payload, which will be a list of
Message
objects whenis_multipart()
isTrue
, or a string whenis_multipart()
isFalse
. If the payload is a list and you mutate the list object, you modify the message’s payload in place.With optional argument i,
get_payload()
will return the i-th element of the payload, counting from zero, ifis_multipart()
isTrue
. AnIndexError
will be raised if i is less than 0 or greater than or equal to the number of items in the payload. If the payload is a string (i.e.is_multipart()
isFalse
) and i is given, aTypeError
is raised.Optional decode is a flag indicating whether the payload should be decoded or not, according to the Content-Transfer-Encoding header. When
True
and the message is not a multipart, the payload will be decoded if this header’s value isquoted-printable
orbase64
. If some other encoding is used, or Content-Transfer-Encoding header is missing, the payload is returned as-is (undecoded). In all cases the returned value is binary data. If the message is a multipart and the decode flag isTrue
, thenNone
is returned. If the payload is base64 and it was not perfectly formed (missing padding, characters outside the base64 alphabet), then an appropriate defect will be added to the message’s defect property (InvalidBase64PaddingDefect
orInvalidBase64CharactersDefect
, respectively).When decode is
False
(the default) the body is returned as a string without decoding the Content-Transfer-Encoding. However, for a Content-Transfer-Encoding of 8bit, an attempt is made to decode the original bytes using thecharset
specified by the Content-Type header, using thereplace
error handler. If nocharset
is specified, or if thecharset
given is not recognized by the email package, the body is decoded using the default ASCII charset.This is a legacy method. On the
EmailMessage
class its functionality is replaced byget_content()
anditer_parts()
.
- set_payload(payload, charset=None)¶
Set the entire message object’s payload to payload. It is the client’s responsibility to ensure the payload invariants. Optional charset sets the message’s default character set; see
set_charset()
for details.This is a legacy method. On the
EmailMessage
class its functionality is replaced byset_content()
.
- set_charset(charset)¶
Set the character set of the payload to charset, which can either be a
Charset
instance (seeemail.charset
), a string naming a character set, orNone
. If it is a string, it will be converted to aCharset
instance. If charset isNone
, thecharset
parameter will be removed from the Content-Type header (the message will not be otherwise modified). Anything else will generate aTypeError
.If there is no existing MIME-Version header one will be added. If there is no existing Content-Type header, one will be added with a value of text/plain. Whether the Content-Type header already exists or not, its
charset
parameter will be set to charset.output_charset. If charset.input_charset and charset.output_charset differ, the payload will be re-encoded to the output_charset. If there is no existing Content-Transfer-Encoding header, then the payload will be transfer-encoded, if needed, using the specifiedCharset
, and a header with the appropriate value will be added. If a Content-Transfer-Encoding header already exists, the payload is assumed to already be correctly encoded using that Content-Transfer-Encoding and is not modified.This is a legacy method. On the
EmailMessage
class its functionality is replaced by the charset parameter of theemail.message.EmailMessage.set_content()
method.
- get_charset()¶
Return the
Charset
instance associated with the message’s payload.This is a legacy method. On the
EmailMessage
class it always returnsNone
.
The following methods implement a mapping-like interface for accessing the message’s RFC 2822 headers. Note that there are some semantic differences between these methods and a normal mapping (i.e. dictionary) interface. For example, in a dictionary there are no duplicate keys, but here there may be duplicate message headers. Also, in dictionaries there is no guaranteed order to the keys returned by
keys()
, but in aMessage
object, headers are always returned in the order they appeared in the original message, or were added to the message later. Any header deleted and then re-added are always appended to the end of the header list.These semantic differences are intentional and are biased toward maximal convenience.
Note that in all cases, any envelope header present in the message is not included in the mapping interface.
In a model generated from bytes, any header values that (in contravention of the RFCs) contain non-ASCII bytes will, when retrieved through this interface, be represented as
Header
objects with a charset ofunknown-8bit
.- __len__()¶
Return the total number of headers, including duplicates.
- __contains__(name)¶
Return
True
if the message object has a field named name. Matching is done case-insensitively and name should not include the trailing colon. Used for thein
operator, e.g.:if 'message-id' in myMessage: print('Message-ID:', myMessage['message-id'])
- __getitem__(name)¶
Return the value of the named header field. name should not include the colon field separator. If the header is missing,
None
is returned; aKeyError
is never raised.Note that if the named field appears more than once in the message’s headers, exactly which of those field values will be returned is undefined. Use the
get_all()
method to get the values of all the extant named headers.
- __setitem__(name, val)¶
Add a header to the message with field name name and value val. The field is appended to the end of the message’s existing fields.
Note that this does not overwrite or delete any existing header with the same name. If you want to ensure that the new header is the only one present in the message with field name name, delete the field first, e.g.:
del msg['subject'] msg['subject'] = 'Python roolz!'
- __delitem__(name)¶
Delete all occurrences of the field with name name from the message’s headers. No exception is raised if the named field isn’t present in the headers.
- keys()¶
Return a list of all the message’s header field names.
- values()¶
Return a list of all the message’s field values.
- items()¶
Return a list of 2-tuples containing all the message’s field headers and values.
- get(name, failobj=None)¶
Return the value of the named header field. This is identical to
__getitem__()
except that optional failobj is returned if the named header is missing (defaults toNone
).
Here are some additional useful methods:
- get_all(name, failobj=None)¶
Return a list of all the values for the field named name. If there are no such named headers in the message, failobj is returned (defaults to
None
).
- add_header(_name, _value, **_params)¶
Extended header setting. This method is similar to
__setitem__()
except that additional header parameters can be provided as keyword arguments. _name is the header field to add and _value is the primary value for the header.For each item in the keyword argument dictionary _params, the key is taken as the parameter name, with underscores converted to dashes (since dashes are illegal in Python identifiers). Normally, the parameter will be added as
key="value"
unless the value isNone
, in which case only the key will be added. If the value contains non-ASCII characters, it can be specified as a three tuple in the format(CHARSET, LANGUAGE, VALUE)
, whereCHARSET
is a string naming the charset to be used to encode the value,LANGUAGE
can usually be set toNone
or the empty string (see RFC 2231 for other possibilities), andVALUE
is the string value containing non-ASCII code points. If a three tuple is not passed and the value contains non-ASCII characters, it is automatically encoded in RFC 2231 format using aCHARSET
ofutf-8
and aLANGUAGE
ofNone
.Here’s an example:
msg.add_header('Content-Disposition', 'attachment', filename='bud.gif')
This will add a header that looks like
Content-Disposition: attachment; filename="bud.gif"
An example with non-ASCII characters:
msg.add_header('Content-Disposition', 'attachment', filename=('iso-8859-1', '', 'Fußballer.ppt'))
Which produces
Content-Disposition: attachment; filename*="iso-8859-1''Fu%DFballer.ppt"
- replace_header(_name, _value)¶
Replace a header. Replace the first header found in the message that matches _name, retaining header order and field name case. If no matching header was found, a
KeyError
is raised.
- get_content_type()¶
Return the message’s content type. The returned string is coerced to lower case of the form maintype/subtype. If there was no Content-Type header in the message the default type as given by
get_default_type()
will be returned. Since according to RFC 2045, messages always have a default type,get_content_type()
will always return a value.RFC 2045 defines a message’s default type to be text/plain unless it appears inside a multipart/digest container, in which case it would be message/rfc822. If the Content-Type header has an invalid type specification, RFC 2045 mandates that the default type be text/plain.
- get_content_maintype()¶
Return the message’s main content type. This is the maintype part of the string returned by
get_content_type()
.
- get_content_subtype()¶
Return the message’s sub-content type. This is the subtype part of the string returned by
get_content_type()
.
- get_default_type()¶
Return the default content type. Most messages have a default content type of text/plain, except for messages that are subparts of multipart/digest containers. Such subparts have a default content type of message/rfc822.
- set_default_type(ctype)¶
Set the default content type. ctype should either be text/plain or message/rfc822, although this is not enforced. The default content type is not stored in the Content-Type header.
- get_params(failobj=None, header='content-type', unquote=True)¶
Return the message’s Content-Type parameters, as a list. The elements of the returned list are 2-tuples of key/value pairs, as split on the
'='
sign. The left hand side of the'='
is the key, while the right hand side is the value. If there is no'='
sign in the parameter the value is the empty string, otherwise the value is as described inget_param()
and is unquoted if optional unquote isTrue
(the default).Optional failobj is the object to return if there is no Content-Type header. Optional header is the header to search instead of Content-Type.
This is a legacy method. On the
EmailMessage
class its functionality is replaced by the params property of the individual header objects returned by the header access methods.
- get_param(param, failobj=None, header='content-type', unquote=True)¶
Return the value of the Content-Type header’s parameter param as a string. If the message has no Content-Type header or if there is no such parameter, then failobj is returned (defaults to
None
).Optional header if given, specifies the message header to use instead of Content-Type.
Parameter keys are always compared case insensitively. The return value can either be a string, or a 3-tuple if the parameter was RFC 2231 encoded. When it’s a 3-tuple, the elements of the value are of the form
(CHARSET, LANGUAGE, VALUE)
. Note that bothCHARSET
andLANGUAGE
can beNone
, in which case you should considerVALUE
to be encoded in theus-ascii
charset. You can usually ignoreLANGUAGE
.If your application doesn’t care whether the parameter was encoded as in RFC 2231, you can collapse the parameter value by calling
email.utils.collapse_rfc2231_value()
, passing in the return value fromget_param()
. This will return a suitably decoded Unicode string when the value is a tuple, or the original string unquoted if it isn’t. For example:rawparam = msg.get_param('foo') param = email.utils.collapse_rfc2231_value(rawparam)
In any case, the parameter value (either the returned string, or the
VALUE
item in the 3-tuple) is always unquoted, unless unquote is set toFalse
.This is a legacy method. On the
EmailMessage
class its functionality is replaced by the params property of the individual header objects returned by the header access methods.
- set_param(param, value, header='Content-Type', requote=True, charset=None, language='', replace=False)¶
Set a parameter in the Content-Type header. If the parameter already exists in the header, its value will be replaced with value. If the Content-Type header as not yet been defined for this message, it will be set to text/plain and the new parameter value will be appended as per RFC 2045.
Optional header specifies an alternative header to Content-Type, and all parameters will be quoted as necessary unless optional requote is
False
(the default isTrue
).If optional charset is specified, the parameter will be encoded according to RFC 2231. Optional language specifies the RFC 2231 language, defaulting to the empty string. Both charset and language should be strings.
If replace is
False
(the default) the header is moved to the end of the list of headers. If replace isTrue
, the header will be updated in place.Changed in version 3.4:
replace
keyword was added.
- del_param(param, header='content-type', requote=True)¶
Remove the given parameter completely from the Content-Type header. The header will be re-written in place without the parameter or its value. All values will be quoted as necessary unless requote is
False
(the default isTrue
). Optional header specifies an alternative to Content-Type.
- set_type(type, header='Content-Type', requote=True)¶
Set the main type and subtype for the Content-Type header. type must be a string in the form maintype/subtype, otherwise a
ValueError
is raised.This method replaces the Content-Type header, keeping all the parameters in place. If requote is
False
, this leaves the existing header’s quoting as is, otherwise the parameters will be quoted (the default).An alternative header can be specified in the header argument. When the Content-Type header is set a MIME-Version header is also added.
This is a legacy method. On the
EmailMessage
class its functionality is replaced by themake_
andadd_
methods.
- get_filename(failobj=None)¶
Return the value of the
filename
parameter of the Content-Disposition header of the message. If the header does not have afilename
parameter, this method falls back to looking for thename
parameter on the Content-Type header. If neither is found, or the header is missing, then failobj is returned. The returned string will always be unquoted as peremail.utils.unquote()
.
- get_boundary(failobj=None)¶
Return the value of the
boundary
parameter of the Content-Type header of the message, or failobj if either the header is missing, or has noboundary
parameter. The returned string will always be unquoted as peremail.utils.unquote()
.
- set_boundary(boundary)¶
Set the
boundary
parameter of the Content-Type header to boundary.set_boundary()
will always quote boundary if necessary. AHeaderParseError
is raised if the message object has no Content-Type header.Note that using this method is subtly different than deleting the old Content-Type header and adding a new one with the new boundary via
add_header()
, becauseset_boundary()
preserves the order of the Content-Type header in the list of headers. However, it does not preserve any continuation lines which may have been present in the original Content-Type header.
- get_content_charset(failobj=None)¶
Return the
charset
parameter of the Content-Type header, coerced to lower case. If there is no Content-Type header, or if that header has nocharset
parameter, failobj is returned.Note that this method differs from
get_charset()
which returns theCharset
instance for the default encoding of the message body.
- get_charsets(failobj=None)¶
Return a list containing the character set names in the message. If the message is a multipart, then the list will contain one element for each subpart in the payload, otherwise, it will be a list of length 1.
Each item in the list will be a string which is the value of the
charset
parameter in the Content-Type header for the represented subpart. However, if the subpart has no Content-Type header, nocharset
parameter, or is not of the text main MIME type, then that item in the returned list will be failobj.
- get_content_disposition()¶
Return the lowercased value (without parameters) of the message’s Content-Disposition header if it has one, or
None
. The possible values for this method are inline, attachment orNone
if the message follows RFC 2183.Added in version 3.5.
- walk()¶
The
walk()
method is an all-purpose generator which can be used to iterate over all the parts and subparts of a message object tree, in depth-first traversal order. You will typically usewalk()
as the iterator in afor
loop; each iteration returns the next subpart.Here’s an example that prints the MIME type of every part of a multipart message structure:
>>> for part in msg.walk(): ... print(part.get_content_type()) multipart/report text/plain message/delivery-status text/plain text/plain message/rfc822 text/plain
walk
iterates over the subparts of any part whereis_multipart()
returnsTrue
, even thoughmsg.get_content_maintype() == 'multipart'
may returnFalse
. We can see this in our example by making use of the_structure
debug helper function:>>> for part in msg.walk(): ... print(part.get_content_maintype() == 'multipart', ... part.is_multipart()) True True False False False True False False False False False True False False >>> _structure(msg) multipart/report text/plain message/delivery-status text/plain text/plain message/rfc822 text/plain
Here the
message
parts are notmultiparts
, but they do contain subparts.is_multipart()
returnsTrue
andwalk
descends into the subparts.
Message
objects can also optionally contain two instance attributes, which can be used when generating the plain text of a MIME message.- preamble¶
The format of a MIME document allows for some text between the blank line following the headers, and the first multipart boundary string. Normally, this text is never visible in a MIME-aware mail reader because it falls outside the standard MIME armor. However, when viewing the raw text of the message, or when viewing the message in a non-MIME aware reader, this text can become visible.
The preamble attribute contains this leading extra-armor text for MIME documents. When the
Parser
discovers some text after the headers but before the first boundary string, it assigns this text to the message’s preamble attribute. When theGenerator
is writing out the plain text representation of a MIME message, and it finds the message has a preamble attribute, it will write this text in the area between the headers and the first boundary. Seeemail.parser
andemail.generator
for details.Note that if the message object has no preamble, the preamble attribute will be
None
.
- epilogue¶
The epilogue attribute acts the same way as the preamble attribute, except that it contains text that appears between the last boundary and the end of the message.
You do not need to set the epilogue to the empty string in order for the
Generator
to print a newline at the end of the file.
- defects¶
The defects attribute contains a list of all the problems found when parsing this message. See
email.errors
for a detailed description of the possible parsing defects.


: Parsing email messages

email.parser
: Parsing email messages¶
Source code: Lib/email/parser.py
Message object structures can be created in one of two ways: they can be
created from whole cloth by creating an EmailMessage
object, adding headers using the dictionary interface, and adding payload(s)
using set_content()
and related methods, or
they can be created by parsing a serialized representation of the email
message.
The email
package provides a standard parser that understands most email
document structures, including MIME documents. You can pass the parser a
bytes, string or file object, and the parser will return to you the root
EmailMessage
instance of the object structure. For
simple, non-MIME messages the payload of this root object will likely be a
string containing the text of the message. For MIME messages, the root object
will return True
from its is_multipart()
method, and the subparts can be accessed via the payload manipulation methods,
such as get_body()
,
iter_parts()
, and
walk()
.
There are actually two parser interfaces available for use, the Parser
API and the incremental FeedParser
API. The Parser
API is
most useful if you have the entire text of the message in memory, or if the
entire message lives in a file on the file system. FeedParser
is more
appropriate when you are reading the message from a stream which might block
waiting for more input (such as reading an email message from a socket). The
FeedParser
can consume and parse the message incrementally, and only
returns the root object when you close the parser.
Note that the parser can be extended in limited ways, and of course you can
implement your own parser completely from scratch. All of the logic that
connects the email
package’s bundled parser and the
EmailMessage
class is embodied in the Policy
class, so a custom parser can create message object trees any way it finds
necessary by implementing custom versions of the appropriate Policy
methods.
FeedParser API¶
The BytesFeedParser
, imported from the email.feedparser
module,
provides an API that is conducive to incremental parsing of email messages,
such as would be necessary when reading the text of an email message from a
source that can block (such as a socket). The BytesFeedParser
can of
course be used to parse an email message fully contained in a bytes-like
object, string, or file, but the BytesParser
API may be more
convenient for such use cases. The semantics and results of the two parser
APIs are identical.
The BytesFeedParser
’s API is simple; you create an instance, feed it a
bunch of bytes until there’s no more to feed it, then close the parser to
retrieve the root message object. The BytesFeedParser
is extremely
accurate when parsing standards-compliant messages, and it does a very good job
of parsing non-compliant messages, providing information about how a message
was deemed broken. It will populate a message object’s
defects
attribute with a list of any
problems it found in a message. See the email.errors
module for the
list of defects that it can find.
Here is the API for the BytesFeedParser
:
- class email.parser.BytesFeedParser(_factory=None, *, policy=policy.compat32)¶
Create a
BytesFeedParser
instance. Optional _factory is a no-argument callable; if not specified use themessage_factory
from the policy. Call _factory whenever a new message object is needed.If policy is specified use the rules it specifies to update the representation of the message. If policy is not set, use the
compat32
policy, which maintains backward compatibility with the Python 3.2 version of the email package and providesMessage
as the default factory. All other policies provideEmailMessage
as the default _factory. For more information on what else policy controls, see thepolicy
documentation.Note: The policy keyword should always be specified; The default will change to
email.policy.default
in a future version of Python.Added in version 3.2.
Changed in version 3.3: Added the policy keyword.
Changed in version 3.6: _factory defaults to the policy
message_factory
.- feed(data)¶
Feed the parser some more data. data should be a bytes-like object containing one or more lines. The lines can be partial and the parser will stitch such partial lines together properly. The lines can have any of the three common line endings: carriage return, newline, or carriage return and newline (they can even be mixed).
- class email.parser.FeedParser(_factory=None, *, policy=policy.compat32)¶
Works like
BytesFeedParser
except that the input to thefeed()
method must be a string. This is of limited utility, since the only way for such a message to be valid is for it to contain only ASCII text or, ifutf8
isTrue
, no binary attachments.Changed in version 3.3: Added the policy keyword.
Parser API¶
The BytesParser
class, imported from the email.parser
module,
provides an API that can be used to parse a message when the complete contents
of the message are available in a bytes-like object or file. The
email.parser
module also provides Parser
for parsing strings,
and header-only parsers, BytesHeaderParser
and
HeaderParser
, which can be used if you’re only interested in the
headers of the message. BytesHeaderParser
and HeaderParser
can be much faster in these situations, since they do not attempt to parse the
message body, instead setting the payload to the raw body.
- class email.parser.BytesParser(_class=None, *, policy=policy.compat32)¶
Create a
BytesParser
instance. The _class and policy arguments have the same meaning and semantics as the _factory and policy arguments ofBytesFeedParser
.Note: The policy keyword should always be specified; The default will change to
email.policy.default
in a future version of Python.Changed in version 3.3: Removed the strict argument that was deprecated in 2.4. Added the policy keyword.
Changed in version 3.6: _class defaults to the policy
message_factory
.- parse(fp, headersonly=False)¶
Read all the data from the binary file-like object fp, parse the resulting bytes, and return the message object. fp must support both the
readline()
and theread()
methods.The bytes contained in fp must be formatted as a block of RFC 5322 (or, if
utf8
isTrue
, RFC 6532) style headers and header continuation lines, optionally preceded by an envelope header. The header block is terminated either by the end of the data or by a blank line. Following the header block is the body of the message (which may contain MIME-encoded subparts, including subparts with a Content-Transfer-Encoding of8bit
).Optional headersonly is a flag specifying whether to stop parsing after reading the headers or not. The default is
False
, meaning it parses the entire contents of the file.
- parsebytes(bytes, headersonly=False)¶
Similar to the
parse()
method, except it takes a bytes-like object instead of a file-like object. Calling this method on a bytes-like object is equivalent to wrapping bytes in aBytesIO
instance first and callingparse()
.Optional headersonly is as with the
parse()
method.
Added in version 3.2.
- class email.parser.BytesHeaderParser(_class=None, *, policy=policy.compat32)¶
Exactly like
BytesParser
, except that headersonly defaults toTrue
.Added in version 3.3.
- class email.parser.Parser(_class=None, *, policy=policy.compat32)¶
This class is parallel to
BytesParser
, but handles string input.Changed in version 3.3: Removed the strict argument. Added the policy keyword.
Changed in version 3.6: _class defaults to the policy
message_factory
.- parse(fp, headersonly=False)¶
Read all the data from the text-mode file-like object fp, parse the resulting text, and return the root message object. fp must support both the
readline()
and theread()
methods on file-like objects.Other than the text mode requirement, this method operates like
BytesParser.parse()
.
- class email.parser.HeaderParser(_class=None, *, policy=policy.compat32)¶
Exactly like
Parser
, except that headersonly defaults toTrue
.
Since creating a message object structure from a string or a file object is such
a common task, four functions are provided as a convenience. They are available
in the top-level email
package namespace.
- email.message_from_bytes(s, _class=None, *, policy=policy.compat32)¶
Return a message object structure from a bytes-like object. This is equivalent to
BytesParser().parsebytes(s)
. Optional _class and policy are interpreted as with theBytesParser
class constructor.Added in version 3.2.
Changed in version 3.3: Removed the strict argument. Added the policy keyword.
- email.message_from_binary_file(fp, _class=None, *, policy=policy.compat32)¶
Return a message object structure tree from an open binary file object. This is equivalent to
BytesParser().parse(fp)
. _class and policy are interpreted as with theBytesParser
class constructor.Added in version 3.2.
Changed in version 3.3: Removed the strict argument. Added the policy keyword.
- email.message_from_string(s, _class=None, *, policy=policy.compat32)¶
Return a message object structure from a string. This is equivalent to
Parser().parsestr(s)
. _class and policy are interpreted as with theParser
class constructor.Changed in version 3.3: Removed the strict argument. Added the policy keyword.
- email.message_from_file(fp, _class=None, *, policy=policy.compat32)¶
Return a message object structure tree from an open file object. This is equivalent to
Parser().parse(fp)
. _class and policy are interpreted as with theParser
class constructor.Changed in version 3.3: Removed the strict argument. Added the policy keyword.
Changed in version 3.6: _class defaults to the policy
message_factory
.
Here’s an example of how you might use message_from_bytes()
at an
interactive Python prompt:
>>> import email
>>> msg = email.message_from_bytes(myBytes)
Additional notes¶
Here are some notes on the parsing semantics:
Most non-multipart type messages are parsed as a single message object with a string payload. These objects will return
False
foris_multipart()
, anditer_parts()
will yield an empty list.All multipart type messages will be parsed as a container message object with a list of sub-message objects for their payload. The outer container message will return
True
foris_multipart()
, anditer_parts()
will yield a list of subparts.Most messages with a content type of message/* (such as message/delivery-status and message/rfc822) will also be parsed as container object containing a list payload of length 1. Their
is_multipart()
method will returnTrue
. The single element yielded byiter_parts()
will be a sub-message object.Some non-standards-compliant messages may not be internally consistent about their multipart-edness. Such messages may have a Content-Type header of type multipart, but their
is_multipart()
method may returnFalse
. If such messages were parsed with theFeedParser
, they will have an instance of theMultipartInvariantViolationDefect
class in their defects attribute list. Seeemail.errors
for details.


: Policy Objects

email.policy
: Policy Objects¶
Added in version 3.3.
Source code: Lib/email/policy.py
The email
package’s prime focus is the handling of email messages as
described by the various email and MIME RFCs. However, the general format of
email messages (a block of header fields each consisting of a name followed by
a colon followed by a value, the whole block followed by a blank line and an
arbitrary ‘body’), is a format that has found utility outside of the realm of
email. Some of these uses conform fairly closely to the main email RFCs, some
do not. Even when working with email, there are times when it is desirable to
break strict compliance with the RFCs, such as generating emails that
interoperate with email servers that do not themselves follow the standards, or
that implement extensions you want to use in ways that violate the
standards.
Policy objects give the email package the flexibility to handle all these disparate use cases.
A Policy
object encapsulates a set of attributes and methods that
control the behavior of various components of the email package during use.
Policy
instances can be passed to various classes and methods in the
email package to alter the default behavior. The settable values and their
defaults are described below.
There is a default policy used by all classes in the email package. For all of
the parser
classes and the related convenience functions, and for
the Message
class, this is the Compat32
policy, via its corresponding pre-defined instance compat32
. This
policy provides for complete backward compatibility (in some cases, including
bug compatibility) with the pre-Python3.3 version of the email package.
This default value for the policy keyword to
EmailMessage
is the EmailPolicy
policy, via
its pre-defined instance default
.
When a Message
or EmailMessage
object is created, it acquires a policy. If the message is created by a
parser
, a policy passed to the parser will be the policy used by
the message it creates. If the message is created by the program, then the
policy can be specified when it is created. When a message is passed to a
generator
, the generator uses the policy from the message by
default, but you can also pass a specific policy to the generator that will
override the one stored on the message object.
The default value for the policy keyword for the email.parser
classes
and the parser convenience functions will be changing in a future version of
Python. Therefore you should always specify explicitly which policy you want
to use when calling any of the classes and functions described in the
parser
module.
The first part of this documentation covers the features of Policy
, an
abstract base class that defines the features that are common to all
policy objects, including compat32
. This includes certain hook
methods that are called internally by the email package, which a custom policy
could override to obtain different behavior. The second part describes the
concrete classes EmailPolicy
and Compat32
, which implement
the hooks that provide the standard behavior and the backward compatible
behavior and features, respectively.
Policy
instances are immutable, but they can be cloned, accepting the
same keyword arguments as the class constructor and returning a new
Policy
instance that is a copy of the original but with the specified
attributes values changed.
As an example, the following code could be used to read an email message from a
file on disk and pass it to the system sendmail
program on a Unix system:
>>> from email import message_from_binary_file
>>> from email.generator import BytesGenerator
>>> from email import policy
>>> from subprocess import Popen, PIPE
>>> with open('mymsg.txt', 'rb') as f:
... msg = message_from_binary_file(f, policy=policy.default)
...
>>> p = Popen(['sendmail', msg['To'].addresses[0]], stdin=PIPE)
>>> g = BytesGenerator(p.stdin, policy=msg.policy.clone(linesep='\r\n'))
>>> g.flatten(msg)
>>> p.stdin.close()
>>> rc = p.wait()
Here we are telling BytesGenerator
to use the RFC
correct line separator characters when creating the binary string to feed into
sendmail's
stdin
, where the default policy would use \n
line
separators.
Some email package methods accept a policy keyword argument, allowing the
policy to be overridden for that method. For example, the following code uses
the as_bytes()
method of the msg object from
the previous example and writes the message to a file using the native line
separators for the platform on which it is running:
>>> import os
>>> with open('converted.txt', 'wb') as f:
... f.write(msg.as_bytes(policy=msg.policy.clone(linesep=os.linesep)))
17
Policy objects can also be combined using the addition operator, producing a policy object whose settings are a combination of the non-default values of the summed objects:
>>> compat_SMTP = policy.compat32.clone(linesep='\r\n')
>>> compat_strict = policy.compat32.clone(raise_on_defect=True)
>>> compat_strict_SMTP = compat_SMTP + compat_strict
This operation is not commutative; that is, the order in which the objects are added matters. To illustrate:
>>> policy100 = policy.compat32.clone(max_line_length=100)
>>> policy80 = policy.compat32.clone(max_line_length=80)
>>> apolicy = policy100 + policy80
>>> apolicy.max_line_length
80
>>> apolicy = policy80 + policy100
>>> apolicy.max_line_length
100
- class email.policy.Policy(**kw)¶
This is the abstract base class for all policy classes. It provides default implementations for a couple of trivial methods, as well as the implementation of the immutability property, the
clone()
method, and the constructor semantics.The constructor of a policy class can be passed various keyword arguments. The arguments that may be specified are any non-method properties on this class, plus any additional non-method properties on the concrete class. A value specified in the constructor will override the default value for the corresponding attribute.
This class defines the following properties, and thus values for the following may be passed in the constructor of any policy class:
- max_line_length¶
The maximum length of any line in the serialized output, not counting the end of line character(s). Default is 78, per RFC 5322. A value of
0
orNone
indicates that no line wrapping should be done at all.
- linesep¶
The string to be used to terminate lines in serialized output. The default is
\n
because that’s the internal end-of-line discipline used by Python, though\r\n
is required by the RFCs.
- cte_type¶
Controls the type of Content Transfer Encodings that may be or are required to be used. The possible values are:
7bit
all data must be “7 bit clean” (ASCII-only). This means that where necessary data will be encoded using either quoted-printable or base64 encoding.
8bit
data is not constrained to be 7 bit clean. Data in headers is still required to be ASCII-only and so will be encoded (see
fold_binary()
andutf8
below for exceptions), but body parts may use the8bit
CTE.A
cte_type
value of8bit
only works withBytesGenerator
, notGenerator
, because strings cannot contain binary data. If aGenerator
is operating under a policy that specifiescte_type=8bit
, it will act as ifcte_type
is7bit
.
- raise_on_defect¶
If
True
, any defects encountered will be raised as errors. IfFalse
(the default), defects will be passed to theregister_defect()
method.
- mangle_from_¶
If
True
, lines starting with “From “ in the body are escaped by putting a>
in front of them. This parameter is used when the message is being serialized by a generator. Default:False
.Added in version 3.5.
- message_factory¶
A factory function for constructing a new empty message object. Used by the parser when building messages. Defaults to
None
, in which caseMessage
is used.Added in version 3.6.
- verify_generated_headers¶
If
True
(the default), the generator will raiseHeaderWriteError
instead of writing a header that is improperly folded or delimited, such that it would be parsed as multiple headers or joined with adjacent data. Such headers can be generated by custom header classes or bugs in theemail
module.As it’s a security feature, this defaults to
True
even in theCompat32
policy. For backwards compatible, but unsafe, behavior, it must be set toFalse
explicitly.Added in version 3.13.
The following
Policy
method is intended to be called by code using the email library to create policy instances with custom settings:- clone(**kw)¶
Return a new
Policy
instance whose attributes have the same values as the current instance, except where those attributes are given new values by the keyword arguments.
The remaining
Policy
methods are called by the email package code, and are not intended to be called by an application using the email package. A custom policy must implement all of these methods.- handle_defect(obj, defect)¶
Handle a defect found on obj. When the email package calls this method, defect will always be a subclass of
MessageDefect
.The default implementation checks the
raise_on_defect
flag. If it isTrue
, defect is raised as an exception. If it isFalse
(the default), obj and defect are passed toregister_defect()
.
- register_defect(obj, defect)¶
Register a defect on obj. In the email package, defect will always be a subclass of
MessageDefect
.The default implementation calls the
append
method of thedefects
attribute of obj. When the email package callshandle_defect
, obj will normally have adefects
attribute that has anappend
method. Custom object types used with the email package (for example, customMessage
objects) should also provide such an attribute, otherwise defects in parsed messages will raise unexpected errors.
- header_max_count(name)¶
Return the maximum allowed number of headers named name.
Called when a header is added to an
EmailMessage
orMessage
object. If the returned value is not0
orNone
, and there are already a number of headers with the name name greater than or equal to the value returned, aValueError
is raised.Because the default behavior of
Message.__setitem__
is to append the value to the list of headers, it is easy to create duplicate headers without realizing it. This method allows certain headers to be limited in the number of instances of that header that may be added to aMessage
programmatically. (The limit is not observed by the parser, which will faithfully produce as many headers as exist in the message being parsed.)The default implementation returns
None
for all header names.
- header_source_parse(sourcelines)¶
The email package calls this method with a list of strings, each string ending with the line separation characters found in the source being parsed. The first line includes the field header name and separator. All whitespace in the source is preserved. The method should return the
(name, value)
tuple that is to be stored in theMessage
to represent the parsed header.If an implementation wishes to retain compatibility with the existing email package policies, name should be the case preserved name (all characters up to the ‘
:
’ separator), while value should be the unfolded value (all line separator characters removed, but whitespace kept intact), stripped of leading whitespace.sourcelines may contain surrogateescaped binary data.
There is no default implementation
- header_store_parse(name, value)¶
The email package calls this method with the name and value provided by the application program when the application program is modifying a
Message
programmatically (as opposed to aMessage
created by a parser). The method should return the(name, value)
tuple that is to be stored in theMessage
to represent the header.If an implementation wishes to retain compatibility with the existing email package policies, the name and value should be strings or string subclasses that do not change the content of the passed in arguments.
There is no default implementation
- header_fetch_parse(name, value)¶
The email package calls this method with the name and value currently stored in the
Message
when that header is requested by the application program, and whatever the method returns is what is passed back to the application as the value of the header being retrieved. Note that there may be more than one header with the same name stored in theMessage
; the method is passed the specific name and value of the header destined to be returned to the application.value may contain surrogateescaped binary data. There should be no surrogateescaped binary data in the value returned by the method.
There is no default implementation
- fold(name, value)¶
The email package calls this method with the name and value currently stored in the
Message
for a given header. The method should return a string that represents that header “folded” correctly (according to the policy settings) by composing the name with the value and insertinglinesep
characters at the appropriate places. See RFC 5322 for a discussion of the rules for folding email headers.value may contain surrogateescaped binary data. There should be no surrogateescaped binary data in the string returned by the method.
- class email.policy.EmailPolicy(**kw)¶
This concrete
Policy
provides behavior that is intended to be fully compliant with the current email RFCs. These include (but are not limited to) RFC 5322, RFC 2047, and the current MIME RFCs.This policy adds new header parsing and folding algorithms. Instead of simple strings, headers are
str
subclasses with attributes that depend on the type of the field. The parsing and folding algorithm fully implement RFC 2047 and RFC 5322.The default value for the
message_factory
attribute isEmailMessage
.In addition to the settable attributes listed above that apply to all policies, this policy adds the following additional attributes:
Added in version 3.6: [1]
- utf8¶
If
False
, follow RFC 5322, supporting non-ASCII characters in headers by encoding them as “encoded words”. IfTrue
, follow RFC 6532 and useutf-8
encoding for headers. Messages formatted in this way may be passed to SMTP servers that support theSMTPUTF8
extension (RFC 6531).
- refold_source¶
If the value for a header in the
Message
object originated from aparser
(as opposed to being set by a program), this attribute indicates whether or not a generator should refold that value when transforming the message back into serialized form. The possible values are:none
all source values use original folding
long
source values that have any line that is longer than
max_line_length
will be refoldedall
all values are refolded.
The default is
long
.
- header_factory¶
A callable that takes two arguments,
name
andvalue
, wherename
is a header field name andvalue
is an unfolded header field value, and returns a string subclass that represents that header. A defaultheader_factory
(seeheaderregistry
) is provided that supports custom parsing for the various address and date RFC 5322 header field types, and the major MIME header field stypes. Support for additional custom parsing will be added in the future.
- content_manager¶
An object with at least two methods: get_content and set_content. When the
get_content()
orset_content()
method of anEmailMessage
object is called, it calls the corresponding method of this object, passing it the message object as its first argument, and any arguments or keywords that were passed to it as additional arguments. By defaultcontent_manager
is set toraw_data_manager
.Added in version 3.4.
The class provides the following concrete implementations of the abstract methods of
Policy
:- header_max_count(name)¶
Returns the value of the
max_count
attribute of the specialized class used to represent the header with the given name.
- header_source_parse(sourcelines)¶
The name is parsed as everything up to the ‘
:
’ and returned unmodified. The value is determined by stripping leading whitespace off the remainder of the first line, joining all subsequent lines together, and stripping any trailing carriage return or linefeed characters.
- header_store_parse(name, value)¶
The name is returned unchanged. If the input value has a
name
attribute and it matches name ignoring case, the value is returned unchanged. Otherwise the name and value are passed toheader_factory
, and the resulting header object is returned as the value. In this case aValueError
is raised if the input value contains CR or LF characters.
- header_fetch_parse(name, value)¶
If the value has a
name
attribute, it is returned to unmodified. Otherwise the name, and the value with any CR or LF characters removed, are passed to theheader_factory
, and the resulting header object is returned. Any surrogateescaped bytes get turned into the unicode unknown-character glyph.
- fold(name, value)¶
Header folding is controlled by the
refold_source
policy setting. A value is considered to be a ‘source value’ if and only if it does not have aname
attribute (having aname
attribute means it is a header object of some sort). If a source value needs to be refolded according to the policy, it is converted into a header object by passing the name and the value with any CR and LF characters removed to theheader_factory
. Folding of a header object is done by calling itsfold
method with the current policy.Source values are split into lines using
splitlines()
. If the value is not to be refolded, the lines are rejoined using thelinesep
from the policy and returned. The exception is lines containing non-ascii binary data. In that case the value is refolded regardless of therefold_source
setting, which causes the binary data to be CTE encoded using theunknown-8bit
charset.
- fold_binary(name, value)¶
The same as
fold()
ifcte_type
is7bit
, except that the returned value is bytes.If
cte_type
is8bit
, non-ASCII binary data is converted back into bytes. Headers with binary data are not refolded, regardless of therefold_header
setting, since there is no way to know whether the binary data consists of single byte characters or multibyte characters.
The following instances of EmailPolicy
provide defaults suitable for
specific application domains. Note that in the future the behavior of these
instances (in particular the HTTP
instance) may be adjusted to conform even
more closely to the RFCs relevant to their domains.
- email.policy.default¶
An instance of
EmailPolicy
with all defaults unchanged. This policy uses the standard Python\n
line endings rather than the RFC-correct\r\n
.
- email.policy.SMTP¶
Suitable for serializing messages in conformance with the email RFCs. Like
default
, but withlinesep
set to\r\n
, which is RFC compliant.
- email.policy.SMTPUTF8¶
The same as
SMTP
except thatutf8
isTrue
. Useful for serializing messages to a message store without using encoded words in the headers. Should only be used for SMTP transmission if the sender or recipient addresses have non-ASCII characters (thesmtplib.SMTP.send_message()
method handles this automatically).
- email.policy.HTTP¶
Suitable for serializing headers with for use in HTTP traffic. Like
SMTP
except thatmax_line_length
is set toNone
(unlimited).
- email.policy.strict¶
Convenience instance. The same as
default
except thatraise_on_defect
is set toTrue
. This allows any policy to be made strict by writing:somepolicy + policy.strict
With all of these EmailPolicies
, the effective API of
the email package is changed from the Python 3.2 API in the following ways:
Setting a header on a
Message
results in that header being parsed and a header object created.Fetching a header value from a
Message
results in that header being parsed and a header object created and returned.Any header object, or any header that is refolded due to the policy settings, is folded using an algorithm that fully implements the RFC folding algorithms, including knowing where encoded words are required and allowed.
From the application view, this means that any header obtained through the
EmailMessage
is a header object with extra
attributes, whose string value is the fully decoded unicode value of the
header. Likewise, a header may be assigned a new value, or a new header
created, using a unicode string, and the policy will take care of converting
the unicode string into the correct RFC encoded form.
The header objects and their attributes are described in
headerregistry
.
- class email.policy.Compat32(**kw)¶
This concrete
Policy
is the backward compatibility policy. It replicates the behavior of the email package in Python 3.2. Thepolicy
module also defines an instance of this class,compat32
, that is used as the default policy. Thus the default behavior of the email package is to maintain compatibility with Python 3.2.The following attributes have values that are different from the
Policy
default:- mangle_from_¶
The default is
True
.
The class provides the following concrete implementations of the abstract methods of
Policy
:- header_source_parse(sourcelines)¶
The name is parsed as everything up to the ‘
:
’ and returned unmodified. The value is determined by stripping leading whitespace off the remainder of the first line, joining all subsequent lines together, and stripping any trailing carriage return or linefeed characters.
- header_store_parse(name, value)¶
The name and value are returned unmodified.
- header_fetch_parse(name, value)¶
If the value contains binary data, it is converted into a
Header
object using theunknown-8bit
charset. Otherwise it is returned unmodified.
- fold(name, value)¶
Headers are folded using the
Header
folding algorithm, which preserves existing line breaks in the value, and wraps each resulting line to themax_line_length
. Non-ASCII binary data are CTE encoded using theunknown-8bit
charset.
- fold_binary(name, value)¶
Headers are folded using the
Header
folding algorithm, which preserves existing line breaks in the value, and wraps each resulting line to themax_line_length
. Ifcte_type
is7bit
, non-ascii binary data is CTE encoded using theunknown-8bit
charset. Otherwise the original source header is used, with its existing line breaks and any (RFC invalid) binary data it may contain.
- email.policy.compat32¶
An instance of
Compat32
, providing backward compatibility with the behavior of the email package in Python 3.2.Note
The
compat32
policy should not be used as a policy forEmailMessage
objects, and should only be used to serialize messages that were created using thecompat32
policy.
Footnotes