tauthon/tauthon2.7
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Import Upstream version 2.7.18

Browse Source
This commit is contained in:
geos_one
2025-08-15 16:28:06 +02:00
commit ba1f69ab39
4521 changed files with 1778434 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

135
Doc/extending/building.rst Normal file
View File

@@ -0,0 +1,135 @@
.. highlightlang:: c
.. _building:
********************************************
Building C and C++ Extensions with distutils
********************************************
.. sectionauthor:: Martin v. Löwis <martin@v.loewis.de>
Starting in Python 1.4, Python provides, on Unix, a special make file for
building make files for building dynamically-linked extensions and custom
interpreters. Starting with Python 2.0, this mechanism (known as related to
Makefile.pre.in, and Setup files) is no longer supported. Building custom
interpreters was rarely used, and extension modules can be built using
distutils.
Building an extension module using distutils requires that distutils is
installed on the build machine, which is included in Python 2.x and available
separately for Python 1.5. Since distutils also supports creation of binary
packages, users don't necessarily need a compiler and distutils to install the
extension.
A distutils package contains a driver script, :file:`setup.py`. This is a plain
Python file, which, in the most simple case, could look like this:
.. code-block:: python
from distutils.core import setup, Extension
module1 = Extension('demo',
sources = ['demo.c'])
setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
ext_modules = [module1])
With this :file:`setup.py`, and a file :file:`demo.c`, running ::
python setup.py build
will compile :file:`demo.c`, and produce an extension module named ``demo`` in
the :file:`build` directory. Depending on the system, the module file will end
up in a subdirectory :file:`build/lib.system`, and may have a name like
:file:`demo.so` or :file:`demo.pyd`.
In the :file:`setup.py`, all execution is performed by calling the ``setup``
function. This takes a variable number of keyword arguments, of which the
example above uses only a subset. Specifically, the example specifies
meta-information to build packages, and it specifies the contents of the
package. Normally, a package will contain of addition modules, like Python
source modules, documentation, subpackages, etc. Please refer to the distutils
documentation in :ref:`distutils-index` to learn more about the features of
distutils; this section explains building extension modules only.
It is common to pre-compute arguments to :func:`setup`, to better structure the
driver script. In the example above, the ``ext_modules`` argument to
:func:`setup` is a list of extension modules, each of which is an instance of
the :class:`~distutils.extension.Extension`. In the example, the instance
defines an extension named ``demo`` which is build by compiling a single source
file, :file:`demo.c`.
In many cases, building an extension is more complex, since additional
preprocessor defines and libraries may be needed. This is demonstrated in the
example below.
.. code-block:: python
from distutils.core import setup, Extension
module1 = Extension('demo',
define_macros = [('MAJOR_VERSION', '1'),
('MINOR_VERSION', '0')],
include_dirs = ['/usr/local/include'],
libraries = ['tcl83'],
library_dirs = ['/usr/local/lib'],
sources = ['demo.c'])
setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
author = 'Martin v. Loewis',
author_email = 'martin@v.loewis.de',
url = 'https://docs.python.org/extending/building',
long_description = '''
This is really just a demo package.
''',
ext_modules = [module1])
In this example, :func:`setup` is called with additional meta-information, which
is recommended when distribution packages have to be built. For the extension
itself, it specifies preprocessor defines, include directories, library
directories, and libraries. Depending on the compiler, distutils passes this
information in different ways to the compiler. For example, on Unix, this may
result in the compilation commands ::
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -DMAJOR_VERSION=1 -DMINOR_VERSION=0 -I/usr/local/include -I/usr/local/include/python2.2 -c demo.c -o build/temp.linux-i686-2.2/demo.o
gcc -shared build/temp.linux-i686-2.2/demo.o -L/usr/local/lib -ltcl83 -o build/lib.linux-i686-2.2/demo.so
These lines are for demonstration purposes only; distutils users should trust
that distutils gets the invocations right.
.. _distributing:
Distributing your extension modules
===================================
When an extension has been successfully build, there are three ways to use it.
End-users will typically want to install the module, they do so by running ::
python setup.py install
Module maintainers should produce source packages; to do so, they run ::
python setup.py sdist
In some cases, additional files need to be included in a source distribution;
this is done through a :file:`MANIFEST.in` file; see the distutils documentation
for details.
If the source distribution has been build successfully, maintainers can also
create binary distributions. Depending on the platform, one of the following
commands can be used to do so. ::
python setup.py bdist_wininst
python setup.py bdist_rpm
python setup.py bdist_dumb

318
Doc/extending/embedding.rst Normal file
View File

@@ -0,0 +1,318 @@
.. highlightlang:: c
.. _embedding:
***************************************
Embedding Python in Another Application
***************************************
The previous chapters discussed how to extend Python, that is, how to extend the
functionality of Python by attaching a library of C functions to it. It is also
possible to do it the other way around: enrich your C/C++ application by
embedding Python in it. Embedding provides your application with the ability to
implement some of the functionality of your application in Python rather than C
or C++. This can be used for many purposes; one example would be to allow users
to tailor the application to their needs by writing some scripts in Python. You
can also use it yourself if some of the functionality can be written in Python
more easily.
Embedding Python is similar to extending it, but not quite. The difference is
that when you extend Python, the main program of the application is still the
Python interpreter, while if you embed Python, the main program may have nothing
to do with Python --- instead, some parts of the application occasionally call
the Python interpreter to run some Python code.
So if you are embedding Python, you are providing your own main program. One of
the things this main program has to do is initialize the Python interpreter. At
the very least, you have to call the function :c:func:`Py_Initialize`. There are
optional calls to pass command line arguments to Python. Then later you can
call the interpreter from any part of the application.
There are several different ways to call the interpreter: you can pass a string
containing Python statements to :c:func:`PyRun_SimpleString`, or you can pass a
stdio file pointer and a file name (for identification in error messages only)
to :c:func:`PyRun_SimpleFile`. You can also call the lower-level operations
described in the previous chapters to construct and use Python objects.
A simple demo of embedding Python can be found in the directory
:file:`Demo/embed/` of the source distribution.
.. seealso::
:ref:`c-api-index`
The details of Python's C interface are given in this manual. A great deal of
necessary information can be found here.
.. _high-level-embedding:
Very High Level Embedding
=========================
The simplest form of embedding Python is the use of the very high level
interface. This interface is intended to execute a Python script without needing
to interact with the application directly. This can for example be used to
perform some operation on a file. ::
#include <Python.h>
int
main(int argc, char *argv[])
{
Py_SetProgramName(argv[0]); /* optional but recommended */
Py_Initialize();
PyRun_SimpleString("from time import time,ctime\n"
"print 'Today is',ctime(time())\n");
Py_Finalize();
return 0;
}
The :c:func:`Py_SetProgramName` function should be called before
:c:func:`Py_Initialize` to inform the interpreter about paths to Python run-time
libraries. Next, the Python interpreter is initialized with
:c:func:`Py_Initialize`, followed by the execution of a hard-coded Python script
that prints the date and time. Afterwards, the :c:func:`Py_Finalize` call shuts
the interpreter down, followed by the end of the program. In a real program,
you may want to get the Python script from another source, perhaps a text-editor
routine, a file, or a database. Getting the Python code from a file can better
be done by using the :c:func:`PyRun_SimpleFile` function, which saves you the
trouble of allocating memory space and loading the file contents.
.. _lower-level-embedding:
Beyond Very High Level Embedding: An overview
=============================================
The high level interface gives you the ability to execute arbitrary pieces of
Python code from your application, but exchanging data values is quite
cumbersome to say the least. If you want that, you should use lower level calls.
At the cost of having to write more C code, you can achieve almost anything.
It should be noted that extending Python and embedding Python is quite the same
activity, despite the different intent. Most topics discussed in the previous
chapters are still valid. To show this, consider what the extension code from
Python to C really does:
#. Convert data values from Python to C,
#. Perform a function call to a C routine using the converted values, and
#. Convert the data values from the call from C to Python.
When embedding Python, the interface code does:
#. Convert data values from C to Python,
#. Perform a function call to a Python interface routine using the converted
values, and
#. Convert the data values from the call from Python to C.
As you can see, the data conversion steps are simply swapped to accommodate the
different direction of the cross-language transfer. The only difference is the
routine that you call between both data conversions. When extending, you call a
C routine, when embedding, you call a Python routine.
This chapter will not discuss how to convert data from Python to C and vice
versa. Also, proper use of references and dealing with errors is assumed to be
understood. Since these aspects do not differ from extending the interpreter,
you can refer to earlier chapters for the required information.
.. _pure-embedding:
Pure Embedding
==============
The first program aims to execute a function in a Python script. Like in the
section about the very high level interface, the Python interpreter does not
directly interact with the application (but that will change in the next
section).
The code to run a function defined in a Python script is:
.. literalinclude:: ../includes/run-func.c
This code loads a Python script using ``argv[1]``, and calls the function named
in ``argv[2]``. Its integer arguments are the other values of the ``argv``
array. If you compile and link this program (let's call the finished executable
:program:`call`), and use it to execute a Python script, such as:
.. code-block:: python
def multiply(a,b):
print "Will compute", a, "times", b
c = 0
for i in range(0, a):
c = c + b
return c
then the result should be:
.. code-block:: shell-session
$ call multiply multiply 3 2
Will compute 3 times 2
Result of call: 6
Although the program is quite large for its functionality, most of the code is
for data conversion between Python and C, and for error reporting. The
interesting part with respect to embedding Python starts with ::
Py_Initialize();
pName = PyString_FromString(argv[1]);
/* Error checking of pName left out */
pModule = PyImport_Import(pName);
After initializing the interpreter, the script is loaded using
:c:func:`PyImport_Import`. This routine needs a Python string as its argument,
which is constructed using the :c:func:`PyString_FromString` data conversion
routine. ::
pFunc = PyObject_GetAttrString(pModule, argv[2]);
/* pFunc is a new reference */
if (pFunc && PyCallable_Check(pFunc)) {
...
}
Py_XDECREF(pFunc);
Once the script is loaded, the name we're looking for is retrieved using
:c:func:`PyObject_GetAttrString`. If the name exists, and the object returned is
callable, you can safely assume that it is a function. The program then
proceeds by constructing a tuple of arguments as normal. The call to the Python
function is then made with::
pValue = PyObject_CallObject(pFunc, pArgs);
Upon return of the function, ``pValue`` is either *NULL* or it contains a
reference to the return value of the function. Be sure to release the reference
after examining the value.
.. _extending-with-embedding:
Extending Embedded Python
=========================
Until now, the embedded Python interpreter had no access to functionality from
the application itself. The Python API allows this by extending the embedded
interpreter. That is, the embedded interpreter gets extended with routines
provided by the application. While it sounds complex, it is not so bad. Simply
forget for a while that the application starts the Python interpreter. Instead,
consider the application to be a set of subroutines, and write some glue code
that gives Python access to those routines, just like you would write a normal
Python extension. For example::
static int numargs=0;
/* Return the number of arguments of the application command line */
static PyObject*
emb_numargs(PyObject *self, PyObject *args)
{
if(!PyArg_ParseTuple(args, ":numargs"))
return NULL;
return Py_BuildValue("i", numargs);
}
static PyMethodDef EmbMethods[] = {
{"numargs", emb_numargs, METH_VARARGS,
"Return the number of arguments received by the process."},
{NULL, NULL, 0, NULL}
};
Insert the above code just above the :c:func:`main` function. Also, insert the
following two statements directly after :c:func:`Py_Initialize`::
numargs = argc;
Py_InitModule("emb", EmbMethods);
These two lines initialize the ``numargs`` variable, and make the
:func:`emb.numargs` function accessible to the embedded Python interpreter.
With these extensions, the Python script can do things like
.. code-block:: python
import emb
print "Number of arguments", emb.numargs()
In a real application, the methods will expose an API of the application to
Python.
.. TODO: threads, code examples do not really behave well if errors happen
(what to watch out for)
.. _embeddingincplusplus:
Embedding Python in C++
=======================
It is also possible to embed Python in a C++ program; precisely how this is done
will depend on the details of the C++ system used; in general you will need to
write the main program in C++, and use the C++ compiler to compile and link your
program. There is no need to recompile Python itself using C++.
.. _link-reqs:
Compiling and Linking under Unix-like systems
=============================================
It is not necessarily trivial to find the right flags to pass to your
compiler (and linker) in order to embed the Python interpreter into your
application, particularly because Python needs to load library modules
implemented as C dynamic extensions (:file:`.so` files) linked against
it.
To find out the required compiler and linker flags, you can execute the
:file:`python{X.Y}-config` script which is generated as part of the
installation process (a :file:`python-config` script may also be
available). This script has several options, of which the following will
be directly useful to you:
* ``pythonX.Y-config --cflags`` will give you the recommended flags when
compiling:
.. code-block:: shell-session
$ /opt/bin/python2.7-config --cflags
-I/opt/include/python2.7 -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
* ``pythonX.Y-config --ldflags`` will give you the recommended flags when
linking:
.. code-block:: shell-session
$ /opt/bin/python2.7-config --ldflags
-L/opt/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic
.. note::
To avoid confusion between several Python installations (and especially
between the system Python and your own compiled Python), it is recommended
that you use the absolute path to :file:`python{X.Y}-config`, as in the above
example.
If this procedure doesn't work for you (it is not guaranteed to work for
all Unix-like platforms; however, we welcome :ref:`bug reports <reporting-bugs>`)
you will have to read your system's documentation about dynamic linking and/or
examine Python's :file:`Makefile` (use :func:`sysconfig.get_makefile_filename`
to find its location) and compilation
options. In this case, the :mod:`sysconfig` module is a useful tool to
programmatically extract the configuration values that you will want to
combine together. For example:
.. code-block:: python
>>> import sysconfig
>>> sysconfig.get_config_var('LIBS')
'-lpthread -ldl -lutil'
>>> sysconfig.get_config_var('LINKFORSHARED')
'-Xlinker -export-dynamic'
.. XXX similar documentation for Windows missing

1324
Doc/extending/extending.rst Normal file
View File

File diff suppressed because it is too large Load Diff

41
Doc/extending/index.rst Normal file
View File

@@ -0,0 +1,41 @@
.. _extending-index:
##################################################
Extending and Embedding the Python Interpreter
##################################################
This document describes how to write modules in C or C++ to extend the Python
interpreter with new modules. Those modules can not only define new functions
but also new object types and their methods. The document also describes how
to embed the Python interpreter in another application, for use as an extension
language. Finally, it shows how to compile and link extension modules so that
they can be loaded dynamically (at run time) into the interpreter, if the
underlying operating system supports this feature.
This document assumes basic knowledge about Python. For an informal
introduction to the language, see :ref:`tutorial-index`. :ref:`reference-index`
gives a more formal definition of the language. :ref:`library-index` documents
the existing object types, functions and modules (both built-in and written in
Python) that give the language its wide application range.
For a detailed description of the whole Python/C API, see the separate
:ref:`c-api-index`.
.. note::
This guide only covers the basic tools for creating extensions provided
as part of this version of CPython. Third party tools may offer simpler
alternatives. Refer to the `binary extensions section
<https://packaging.python.org/en/latest/extensions/>`__ in the Python
Packaging User Guide for more information.
.. toctree::
:maxdepth: 2
:numbered:
extending.rst
newtypes.rst
building.rst
windows.rst
embedding.rst

1590
Doc/extending/newtypes.rst Normal file
View File

File diff suppressed because it is too large Load Diff

137
Doc/extending/windows.rst Normal file
View File

@@ -0,0 +1,137 @@
.. highlightlang:: c
.. _building-on-windows:
****************************************
Building C and C++ Extensions on Windows
****************************************
This chapter briefly explains how to create a Windows extension module for
Python using Microsoft Visual C++, and follows with more detailed background
information on how it works. The explanatory material is useful for both the
Windows programmer learning to build Python extensions and the Unix programmer
interested in producing software which can be successfully built on both Unix
and Windows.
Module authors are encouraged to use the distutils approach for building
extension modules, instead of the one described in this section. You will still
need the C compiler that was used to build Python; typically Microsoft Visual
C++.
.. note::
This chapter mentions a number of filenames that include an encoded Python
version number. These filenames are represented with the version number shown
as ``XY``; in practice, ``'X'`` will be the major version number and ``'Y'``
will be the minor version number of the Python release you're working with. For
example, if you are using Python 2.2.1, ``XY`` will actually be ``22``.
.. _win-cookbook:
A Cookbook Approach
===================
There are two approaches to building extension modules on Windows, just as there
are on Unix: use the :mod:`distutils` package to control the build process, or
do things manually. The distutils approach works well for most extensions;
documentation on using :mod:`distutils` to build and package extension modules
is available in :ref:`distutils-index`. If you find you really need to do
things manually, it may be instructive to study the project file for the
:source:`winsound <PCbuild/winsound.vcxproj>` standard library module.
.. _dynamic-linking:
Differences Between Unix and Windows
====================================
.. sectionauthor:: Chris Phoenix <cphoenix@best.com>
Unix and Windows use completely different paradigms for run-time loading of
code. Before you try to build a module that can be dynamically loaded, be aware
of how your system works.
In Unix, a shared object (:file:`.so`) file contains code to be used by the
program, and also the names of functions and data that it expects to find in the
program. When the file is joined to the program, all references to those
functions and data in the file's code are changed to point to the actual
locations in the program where the functions and data are placed in memory.
This is basically a link operation.
In Windows, a dynamic-link library (:file:`.dll`) file has no dangling
references. Instead, an access to functions or data goes through a lookup
table. So the DLL code does not have to be fixed up at runtime to refer to the
program's memory; instead, the code already uses the DLL's lookup table, and the
lookup table is modified at runtime to point to the functions and data.
In Unix, there is only one type of library file (:file:`.a`) which contains code
from several object files (:file:`.o`). During the link step to create a shared
object file (:file:`.so`), the linker may find that it doesn't know where an
identifier is defined. The linker will look for it in the object files in the
libraries; if it finds it, it will include all the code from that object file.
In Windows, there are two types of library, a static library and an import
library (both called :file:`.lib`). A static library is like a Unix :file:`.a`
file; it contains code to be included as necessary. An import library is
basically used only to reassure the linker that a certain identifier is legal,
and will be present in the program when the DLL is loaded. So the linker uses
the information from the import library to build the lookup table for using
identifiers that are not included in the DLL. When an application or a DLL is
linked, an import library may be generated, which will need to be used for all
future DLLs that depend on the symbols in the application or DLL.
Suppose you are building two dynamic-load modules, B and C, which should share
another block of code A. On Unix, you would *not* pass :file:`A.a` to the
linker for :file:`B.so` and :file:`C.so`; that would cause it to be included
twice, so that B and C would each have their own copy. In Windows, building
:file:`A.dll` will also build :file:`A.lib`. You *do* pass :file:`A.lib` to the
linker for B and C. :file:`A.lib` does not contain code; it just contains
information which will be used at runtime to access A's code.
In Windows, using an import library is sort of like using ``import spam``; it
gives you access to spam's names, but does not create a separate copy. On Unix,
linking with a library is more like ``from spam import *``; it does create a
separate copy.
.. _win-dlls:
Using DLLs in Practice
======================
.. sectionauthor:: Chris Phoenix <cphoenix@best.com>
Windows Python is built in Microsoft Visual C++; using other compilers may or
may not work (though Borland seems to). The rest of this section is MSVC++
specific.
When creating DLLs in Windows, you must pass :file:`pythonXY.lib` to the linker.
To build two DLLs, spam and ni (which uses C functions found in spam), you could
use these commands::
cl /LD /I/python/include spam.c ../libs/pythonXY.lib
cl /LD /I/python/include ni.c spam.lib ../libs/pythonXY.lib
The first command created three files: :file:`spam.obj`, :file:`spam.dll` and
:file:`spam.lib`. :file:`Spam.dll` does not contain any Python functions (such
as :c:func:`PyArg_ParseTuple`), but it does know how to find the Python code
thanks to :file:`pythonXY.lib`.
The second command created :file:`ni.dll` (and :file:`.obj` and :file:`.lib`),
which knows how to find the necessary functions from spam, and also from the
Python executable.
Not every identifier is exported to the lookup table. If you want any other
modules (including Python) to be able to see your identifiers, you have to say
``_declspec(dllexport)``, as in ``void _declspec(dllexport) initspam(void)`` or
``PyObject _declspec(dllexport) *NiGetSpamData(void)``.
Developer Studio will throw in a lot of import libraries that you do not really
need, adding about 100K to your executable. To get rid of them, use the Project
Settings dialog, Link tab, to specify *ignore default libraries*. Add the
correct :file:`msvcrtxx.lib` to the list of libraries.