PEP 594 – Inserting off wearisome batteries from the accepted library

22
PEP
594
Title
Inserting off wearisome batteries from the accepted library
Creator
Christian Heimes ,
Brett Cannon
Discussions-To
https://discuss about.python.org/t/13508
Position
Permitted
Kind
Requirements Be aware
Created
20-May well-2019
Python-Model
3.11
Put up-Historical past
21-May well-2019, 04-Feb-2022
Resolution
https://discuss about.python.org/t/13508/22

Contents

Abstract

This PEP proposed a checklist of accepted library modules to be removed from the
accepted library. The modules are largely historical files codecs (e.g. Commodore
and SUN file codecs), APIs and dealing programs which had been superseded a
very lengthy time ago (e.g. Mac OS 9), or modules that hang security implications and
higher decisions (e.g. password and login).

The PEP follows within the footsteps of alternative PEPS relish PEP 3108. The
Long-established Library Reorganization proposal removed a bunch of modules from
Python 3.0. In 2007, the PEP referred to upkeep burden as:

“Through the years, sure modules hang become a heavy burden upon python-dev
to care for up. In situations relish this, it’s some distance higher for the module to be
given to the community to care for as a lot as free python-dev to point of curiosity extra on
language make stronger and other modules within the accepted library that produce no longer draw shut
up an undue amount of time and effort.”

The withdrawn PEP 206 from 2000 expresses factors with the Python accepted
library in an unvarnished and forthright system:

“[…] the accepted library modules aren’t always the totally decisions for a
job. Some library modules had been immediate hacks (e.g. calendar,
commands), some had been designed poorly and are now shut to-no longer attainable to
repair (cgi), and a few had been rendered oldschool by other, extra entire
modules […].”

Rationale

Support within the early days of Python, the interpreter came with a immense effect of
priceless modules. This used to be on the entire known as “batteries integrated”
philosophy and used to be one of the important cornerstones to Python’s success fable.
Customers didn’t hang to set up out tips on how to download and set up separate
packages in direct to write down a truly easy net server or parse electronic mail.

Instances hang modified. The introduction of PyPI (née Cheeseshop), setuptools,
and later pip, it turned easy and easy to download and set up
packages. On this closing date Python has a neatly off and brilliant ecosystem of third-obtain collectively
packages. It’s enough mighty accepted to both set up packages from PyPI or
exhaust one of the important a gargantuan collection of Python or Linux distributions.

On the replace hand, Python’s accepted library is piling up with cruft, pointless
duplication of functionality, and dispensable aspects. That is undesirable
for a couple of reasons.

  • Any further module increases the maintenance rate for the Python core
    constructing crew. The crew has little sources, diminished upkeep rate
    frees constructing time for other improvements.
  • Modules within the accepted library are on the entire appreciated and seen because the
    de facto resolution for a scenario. A majority of customers totally grasp third-obtain collectively
    modules to interchange a stdlib module, when they’ve a compelling design, e.g.
    lxml rather than xml. The removal of an unmaintained stdlib module
    increases the percentages of a community-contributed module to become broadly
    feeble.
  • A lean and mean accepted library advantages platforms with little sources
    relish devices with supreme a couple of hundred kilobyte of storage (e.g. BBC
    Micro:bit). Python on cell platforms relish BeeWare or WebAssembly
    (e.g. pyodide) additionally hang the encourage of diminished download dimension.

The modules on this PEP had been selected for deprecation because their
removal is both least controversial or most priceless. Shall we embrace,
least controversial are 30-one year-outdated multimedia codecs relish the sunau
audio format, which used to be feeble on SPARC and NeXT workstations within the slack
1980s. The crypt module has fundamental flaws which may perchance be higher solved
open air the accepted library.

This PEP additionally designates some modules as no longer scheduled for removal. Some
modules had been deprecated for a couple of releases or appear pointless at
first stumble on. Then all once more it’s some distance priceless to care for up the modules within the accepted
library, largely for environments the place installing a equipment from PyPI is no longer
an choice. This can also just additionally be corporate environments or classrooms the place exterior
code is no longer authorized without appropriate approval.

  • The usage of FTP is declining, but some files are aloof equipped over
    the FTP protocol or hosters supply FTP to be able to add bellow material. Subsequently,
    ftplib goes to pause.
  • The optparse and getopt modules are broadly feeble. They are gentle
    modules with very low upkeep overhead.
  • Based mostly totally on David Beazley [5] the wave module is straightforward to coach to
    youth and can build crazy sounds. Making a laptop generate sounds is a
    essential and extremely motivating impart for a 9-one year-outdated aspiring developer.
    It’s a fun battery to care for up.

Deprecation schedule

3.11

Starting with Python 3.11, deprecated modules will commence issuing
DeprecationWarning. The estimated EOL of Python 3.10, the final
version without the warning, is October 2026.

3.12

There needs to be no particular trade in contrast to Python 3.11.
That is the final version of Python with the deprecated modules,
with an estimated EOL of October 2028.

3.13

All modules deprecated by this PEP are removed from the major branch
of the CPython repository and are no longer dispensed as section of Python.

Deprecated modules

The modules are grouped as files encoding, multimedia, network, OS interface,
and misc modules. The majority of modules are for outdated files codecs or
outdated APIs. Some others are rarely priceless and hang higher replacements on
PyPI, e.g. Pillow for describe processing or NumPy-essentially based totally initiatives to address
audio processing.

Desk 1: Proposed modules deprecations
Module Deprecated in To be removed Added in Has maintainer? Replacement
aifc 3.11 (3.0*) 3.13 1993 yes (lazy)
asynchat 3.6 (3.0*) 3.12 1999 yes asyncio
asyncore 3.6 (3.0*) 3.12 1999 yes asyncio
audioop 3.11 (3.0*) 3.13 1992 yes
cgi 3.11 (2.0 3.13 1995 no
cgitb 3.11 (2.0 3.13 1995 no
chunk 3.11 3.13 1999 no
crypt 3.11 3.13 1994 yes (lazy) legacycrypt, bcrypt, argon2-cffi, hashlib, passlib
imghdr 3.11 3.13 1992 no filetype, puremagic, python-magic
msilib 3.11 3.13 2006 no
nntplib 3.11 3.13 1992 no
nis 3.8 (3.0*) 3.13 1992 no
ossaudiodev 3.11 3.13 2002 no
pipes 3.11 3.13 1992 no subprocess
smtpd 3.4.7, 3.5.4 3.12 2001 yes aiosmtpd
sndhdr 3.11 3.13 1994 no filetype, puremagic, python-magic
spwd 3.11 3.13 2005 no python-pam
sunau 3.11 (3.0*) 3.13 1993 no
telnetlib 3.11 (3.0*) 3.13 1997 no telnetlib3, Exscript
uu 3.11 3.13 1994 no
xdrlib 3.11 3.13 1992/1996 no

Some module deprecations proposed by PEP 3108 for 3.0 and PEP 206 for
2.0. The added in column illustrates, when a module used to be within the muse designed
and added to the accepted library. The has maintainer column refers to the
knowledgeable index, a checklist of enviornment
specialists and maintainers within the DevGuide.

Files encoding modules

uu and the uu encoding

The uu module provides
uuencode format, an outdated binary encoding format for electronic mail from 1980. The uu
format has been modified by MIME. The uu codec is equipped by the binascii
module. There’s additionally encodings/uu_codec.py which is a codec for the
same encoding; it will hang to aloof additionally be deprecated.

xdrlib

The xdrlib module helps
the Solar Exterior Files Representation Long-established. XDR is an outdated binary
serialization format from 1987. In the intervening time it’s rarely feeble open air
truly supreme domains relish NFS.

Multimedia modules

aifc

The aifc module provides
make stronger for reading and writing AIFF and AIFF-C files. The Audio Interchange
File Structure is an outdated audio format from 1988 in accordance with Amiga IFF. It used to be most
typically feeble on the Apple Macintosh. In the intervening time totally few truly supreme
utility exhaust AIFF.

A user disclosed [6] that the put up manufacturing movie industry makes heavy
exhaust of the AIFC file format. The usage of the aifc module in closed supply
and inner application used to be unknown ahead of the fundamental posting of this PEP. This
is known as a compelling argument to care for up the aifc module within the accepted
library. The file format is acquire and the module does no longer require mighty
upkeep. The strategic advantages for Python can also just outmatch the burden.

audioop

The audioop module
incorporates helper capabilities to manipulate uncooked audio files and adaptive
differential pulse-code modulated audio files. The module is implemented in
C without any further dependencies. The aifc, sunau, and wave
modules rely on audioop for some operations.

The byteswap operation within the wave module can also just additionally be substituted with little
further work. In case aifc is no longer deprecated as neatly, a diminished version of
the audioop module is converted into a non-public implementation detail,
e.g. _audioop with byteswap, alaw2lin, ulaw2lin, lin2alaw,
lin2ulaw, and lin2adpcm.

chunk

The chunk module provides
make stronger for reading and writing Electronic Arts’ Interchange File Structure.
IFF is an outdated audio file format within the muse launched for Commodore and
Amiga. The format is no longer associated.

imghdr

The imghdr module is a
easy tool to bet the describe file format from the fundamental 32 bytes
of a file or buffer. It helps totally a little collection of codecs and
neither returns resolution nor coloration depth.

ossaudiodev

The ossaudiodev
module provides make stronger for Launch Sound Procedure, an interface to sound
playback and obtain devices. OSS used to be at the muse free application, but later
make stronger for newer sound devices and enhancements had been proprietary. Linux
community abandoned OSS in want of ALSA [1]. Some working programs relish
OpenBSD and NetBSD present an incomplete [2] emulation of OSS.

To totally of my files, FreeBSD is the totally standard working system
that makes exhaust of Launch Sound Procedure as of this day. The ossaudiodev hasn’t seen any
improvements or fresh aspects since 2003. All commits since 2003 are
challenge-extensive code cleanups and a couple of worm fixes. It’d be priceless
for both FreeBSD community and core constructing, if the module may perchance perchance well presumably be
maintained and dispensed by of us who love it and exhaust it.

The accepted library feeble to hang extra audio-associated modules. The replace
audio application interfaces (audiodev, linuxaudiodev, sunaudiodev)
had been removed in 2007 as section of the PEP 3108 stdlib re-group.

sndhdr

The sndhdr module is
equivalent to the imghdr module but for audio codecs. It guesses file
format, channels, frame rate, and sample widths from the fundamental 512 bytes of
a file or buffer. The module totally helps AU, AIFF, HCOM, VOC, WAV, and
other extinct codecs.

sunau

The sunau module provides
make stronger for Solar AU sound format. It’s but one other outdated, oldschool file format.

Networking modules

asynchat

The asynchat module
is constructed on high of asyncore and has been deprecated since Python 3.6.

asyncore

The asyncore module used to be
the fundamental module for asynchronous socket service customers and servers. It
has been modified by asyncio and is deprecated since Python 3.6.

The asyncore module is additionally feeble in stdlib checks. The checks for
ftplib, logging, smptd, smtplib, and ssl are partly
in accordance with asyncore. These checks needs to be up up to now to exhaust asyncio or
threading.

cgi

The cgi module is a make stronger
module for General Gateway Interface (CGI) scripts. CGI is deemed as
inefficient because every incoming set up a question to is handled in a brand fresh direction of.
PEP 206 considers the module as:

“[…] designed poorly and are now shut to-no longer attainable to repair (cgi) […]”

Replacements for the a gargantuan collection of substances of cgi which may perchance be no longer straight
associated to executing code are:

  • parse with urllib.parse.parse_qs (parse is supreme a wrapper)
  • parse_header with electronic mail.message.Message (survey example below)
  • parse_multipart with electronic mail.message.Message (same MIME RFCs)
  • FieldStorage/MiniFieldStorage has no direct replace, but can
    on the entire be modified by the usage of multipart (for POST and PUT
    requests) or urllib.parse.parse_qsl (for GET and HEAD
    requests)
  • valid_boundary (undocumented) with re.collect("^[ -~]{0,200}[!-~]$")

As an yell example of how shut parse_header and
electronic mail.message.Message are:

>>> from cgi import parse_header
>>> from electronic mail.message import Message
>>> parse_header(h)
('utility/json', {'charset': 'utf8'})
>>> m = Message()
>>> m['content-type'] = h
>>> m.get_params()
[('application/json', ''), ('charset', 'utf8')]
>>> m.get_param('charset')
'utf8'

cgitb

The cgitb module is a
helper for the cgi module for configurable tracebacks.

The cgitb module is no longer feeble by any fundamental Python net framework (Django,
Pyramid, Plone, Flask, CherryPy, or Bottle). Handiest Paste makes exhaust of it in an
elective debugging middleware.

smtpd

The smtpd module provides
a truly easy implementation of a SMTP mail server. The module documentation
marks the module as deprecated and recommends aiosmtpd as a replace. The
deprecation message used to be added in releases 3.4.7, 3.5.4, and 3.6.1.

nntplib

The nntplib module
implements the consumer facet of the Network Data Switch Protocol (nntp). Data
groups feeble to be a dominant platform for on-line discussions. Over the final
twenty years, files has been slowly but progressively modified with mailing lists
and net-essentially based totally dialogue platforms. Twisted is additionally
planning to deprecate NNTP
make stronger and pynntp hasn’t seen any
impart since 2014. That is a legitimate indicator that the public passion in
NNTP make stronger is declining.

The nntplib checks had been the design within the encourage of further work within the most modern
past. Python totally incorporates the consumer facet of NNTP, so the checks hook up with
exterior files servers. The servers are typically unavailable, too slack, or produce
no longer work accurately over IPv6. The venture causes flaky take a look at runs on
buildbots.

telnetlib

The telnetlib module
provides a Telnet class that implements the Telnet protocol.

Working system interface

crypt

The crypt module implements
password hashing in accordance with the crypt(3) aim from libcrypt or
libxcrypt on Unix-relish platforms. The algorithms are largely outdated, of sad
good and timid. Customers are heart-broken from the usage of them.

  • The module is no longer on hand on Dwelling windows. Corrupt-platform applications need
    some other implementation anyway.
  • Handiest DES encryption is assured to be on hand. DES has an especially
    little key intention of 256.
  • MD5, salted SHA256, salted SHA512, and Blowfish are elective extensions.
    SSHA256 and SSHA512 are glibc extensions. Blowfish (bcrypt) is the totally
    algorithm that is aloof acquire. Then all once more it’s in glibc and therefore no longer
    typically on hand on Linux.
  • Reckoning on the platform, the crypt module is no longer thread acquire. Handiest
    implementations with crypt_r(3) are thread acquire.
  • The module used to be by no draw priceless to engage with system user and password
    databases. On BSD, macOS, and Linux, all user authentication and
    password modification operations need to plow thru PAM (pluggable
    authentication module); survey the spwd deprecation.

nis

The nis module provides
NIS/YP make stronger. Network Files Service / Yellow Pages is an outdated and
deprecated itemizing service protocol developed by Solar Microsystems. Its
designed successor NIS+ from 1992 by no draw took off. For a truly very lengthy time, libc’s
Name Service Swap, LDAP, and Kerberos/GSSAPI had been view a couple of extra essential
and further acquire replace for NIS.

spwd

The spwd module provides
direct obtain admission to to Unix shadow password database the usage of non-accepted APIs.

In classic, it’s a accelerate belief to exhaust spwd. It circumvents system
security insurance policies, does no longer exhaust the PAM stack, and is totally relish minded
with native user accounts, because it ignores NSS. The exhaust of the spwd
module for obtain admission to care for an eye on needs to be view a couple of security worm, as it bypasses
PAM’s obtain admission to care for an eye on.

Moreover, the spwd module makes exhaust of the
shadow(3) APIs.
Functions relish getspnam(3) obtain admission to the /and plenty of others/shadow file straight. This
is bad and even forbidden for confined services on programs with a
security engine relish SELinux or AppArmor.

Misc modules

msilib

The msilib equipment is a
Dwelling windows-totally equipment. It helps the creation of Microsoft Installers (MSI).
The equipment additionally exposes further APIs to create cabinet files (CAB). The
module is feeble to facilitate distutils to create MSI installers with the
bdist_msi clarify. Previously it used to be feeble to create CPython’s legitimate
Dwelling windows installer, too.

Microsoft is slowly transferring away from MSI in want of Dwelling windows 10 Apps (AppX)
as a brand fresh deployment model [3].

pipes

The pipes module provides
helpers to pipe the input of 1 clarify into the output of 1 other clarify.
The module is constructed on high of os.popen. Customers are impressed to exhaust
the subprocess module as a replace.

Modules to care for up

Some modules had been within the muse proposed for deprecation but are no longer
listed as such on this PEP.

Desk 2: Withdrawn deprecations
Module Deprecated in Replacement
colorsys colormath, coloration, colorspacious, Pillow
fileinput argparse
getopt argparse, optparse
optparse 3.2 argparse
wave

colorsys

The colorsys module
defines coloration conversion capabilities between RGB, YIQ, HSL, and HSV coordinate
programs.

Walter Dörwald, Petr Viktorin, and others requested to care for up colorsys. The
module is priceless to transform CSS colours between coordinate programs. The
implementation is straightforward, gentle, and does no longer impose upkeep overhead
on core constructing.

The PyPI packages colormath, coloration, and colorspacious present extra and
improved aspects. The Pillow library is higher fine to transform photos
between coloration programs.

fileinput

The fileinput module
implements helpers to iterate over a checklist of files from sys.argv. The
module predates the optparse and argparse modules. The same functionality
can also just additionally be implemented with the argparse module.

Plenty of core builders expressed their passion to care for up the module within the
accepted library, as it’s some distance at hand for immediate scripts.

getopt

The getopt module mimics
C’s getopt() choice parser.

Even supposing customers are impressed to exhaust argparse as a replace, the getopt module is
aloof broadly feeble. The module is dinky, easy, and at hand for C builders
to write down easy Python scripts.

optparse

The optparse module is
the predecessor of the argparse module.

Even supposing it has been deprecated for a couple of years, it’s aloof too broadly feeble
to amass it.

wave

The wave module provides
make stronger for the WAV sound format.

The module is no longer deprecated, for the explanation that WAV format is aloof associated these
days. The wave module is additionally feeble in education, e.g. to indicate youth how
to build noise with a laptop.

The module makes exhaust of 1 easy aim from the audioop module to form
byte swapping between little and tall endian codecs. Earlier than 24 bit WAV
make stronger used to be added, byte swap feeble to be implemented with the array
module. To amass wave’s dependency on audioop, the byte swap
aim can be both be moved to 1 other module (e.g. operator) or
the array module may perchance perchance well presumably effect make stronger for 24-bit (3-byte) arrays.

Discussions

  • Elana Hashman and Sever Coghlan suggested to care for up the getopt module.
  • Berker Peksag proposed to deprecate and hang msilib.
  • Brett Cannon urged to lengthen sharp deprecation warnings and removal
    of modules relish imp except Python 3.10. Model 3.8 can be released
    almost at present sooner than Python 2 reaches stay-of-life. A lengthen diminished churn for
    customers that migrate from Python 2 to three.8.
  • At one point, distutils used to be talked about within the identical sentence as this PEP.
    To care for away from lengthy dialogue and lengthen of the PEP, I made up my mind against dealing
    with distutils. Deprecation of the distutils equipment can be handled by
    one other PEP.
  • Multiple of us (Gregory P. Smith, David Beazley, Sever Coghlan, …)
    ecstatic me to care for up the wave module. [4]
  • Gregory P. Smith proposed to deprecate nntplib. [4]
  • Andrew Svetlov talked about the socketserver module is questionable.
    Then all once more it’s feeble to implement http.server and xmlrpc.server. The
    stdlib doesn’t hang a replace for the servers, but.

Rejected tips

Increasing/sustaining a separate repo for the deprecated modules

It used to be beforehand proposed to create a separate repository containing the
deprecated modules packaged for set up. Knowing to be one of the important PEP authors went up to now
as to create a demo repository . In the
stay, even though, it used to be made up our minds that the added workload to create and care for such
a repo officially wasn’t justified, because the availability code will continue to be
on hand within the CPython repository for of us to vendor as wanted. Identical
work has additionally no longer been completed when outdated modules had been deprecated and removed,
and it reputedly wasn’t an undue burden on the community.

Update history

Update 1

  • Deprecate parser module
  • Wait on fileinput module
  • Clarify why crypt and spwd are bad and accelerate
  • Give a raise to sections for cgitb, colorsys, nntplib, and smtpd modules
  • The colorsys, crypt, imghdr, sndhdr, and spwd sections now
    checklist factual substitutions
  • Level to that socketserver goes to pause for http.server and
    xmlrpc.server
  • The future upkeep piece now states that the deprecated modules
    can be adopted by Python community members

Update 2

  • Wait on colorsys module
  • Add specialists
  • Redirect discussions to discuss about.python.org
  • Deprecate telnetlib
  • Deprecate compat32 coverage of electronic mail equipment
  • Add creation one year to overview table
  • Level to PEP 206 and PEP 3108
  • Update sections for aifc, audioop, cgi, and wave.

Update 3

  • Wait on the legacy electronic mail API modules. Interior deprecations can be
    handled one at a time.

Update 4

  • Add Brett as a co-author.
  • Retarget the PEP for Python 3.11.
  • Examples of tips on how to interchange the associated substances of cgi
    (thanks Martijn Pieters).

References

Read More

Knowasiak
WRITTEN BY

Knowasiak

Hey! look, i give tutorials to all my users and i help them!