Skip to content

Update to v3.12.10 #190

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 122 commits into from
Apr 17, 2025
Merged

Update to v3.12.10 #190

merged 122 commits into from
Apr 17, 2025

Conversation

lazka
Copy link
Member

@lazka lazka commented Apr 16, 2025

summary:

  • only some refreshes due to context changes
  • squash one fixup

range-diff

1: 42595fa = 1: 70bab43 sysconfig: make _sysconfigdata.py relocatable
2: 95f152c ! 2: e2584af build: add --with-nt-threads and make it default on mingw

@@ Include/internal/pycore_condvar.h
  /* This means pthreads are not implemented in libc headers, hence the macro
     not present in unistd.h. But they still can be implemented as an external
 @@
- /* include windows if it hasn't been done before */
- #define WIN32_LEAN_AND_MEAN
- #include <windows.h>
+ #  define WIN32_LEAN_AND_MEAN
+ #endif
+ #include <windows.h>              // CRITICAL_SECTION
 +/* winpthreads are involved via windows header, so need undef _POSIX_THREADS after header include */
 +#if defined(_POSIX_THREADS)
 +#undef _POSIX_THREADS

3: 36f07a7 = 3: 651a0f8 Define MS_WINDOWS (and others) when compiling with MINGW
4: 689214c = 4: b5e144e configure: add MACHDEP and platform on MINGW
5: a283071 = 5: b0ffdfd Add default configuration for MINGW
6: b1629dc = 6: c870543 configure: enable largefile support by default for Mingw
7: d228beb = 7: 4ef28f8 Add PC/ to CPPFLAGS and to SRCDIRS on Mingw
8: aca28a0 = 8: 1056ca4 posixmodule.c: Define necessary headers for compiling on MINGW
9: 0ec01b9 = 9: 1dc9a69 Build winreg by default when compiling for MINGW
10: f99bc0b = 10: fb030fb pyport.h: Add support for MINGW
11: f396ab1 = 11: f89e260 configure: add options so that shared build is possible on MINGW
12: 12ee454 = 12: 2fa60d1 Add dynamic loading support for MINGW
13: 1b955b8 = 13: 622d44d Ignore main program for frozen scripts on MINGW
14: 2f52536 = 14: 15603f8 Detect winsock2 and setup _socket module on MINGW
15: 8b31f30 = 15: 4c4fa2f Enable some windows specific modules
16: 869a71f = 16: da21e49 sysconfig: MINGW build extensions with GCC
17: 5112900 = 17: b072673 sysconfig: treat MINGW builds as POSIX builds
18: 7e4271a = 18: 0316a86 Add support for stdcall without underscore
19: cf41939 = 19: 6291658 Customize site for MINGW
20: a32389d ! 20: 12201dc add python config sh

@@ Misc/python-config.sh.in
 +    # Since we don't know where the output from this script will end up
 +    # we keep all paths in Windows-land since MSYS2 can handle that
 +    # while native tools can't handle paths in MSYS2-land.
-+    if [ "$OSTYPE" = "msys" ]; then
++    if [ "$OSTYPE" = "cygwin" ] || [ "$OSTYPE" = "msys" ] ; then
 +        RESULT=$(cd "$RESULT" && pwd -W)
      fi
      echo $RESULT

21: 0ac7d8c = 21: c7e92a6 mingw: prefer unix sep if MSYSTEM environment variable
22: 8bf8750 = 22: 67aef22 msys cygwin semi native build sysconfig
23: 3eb79d0 = 23: 315d81f sysconfig: mingw sysconfig like posix
24: 3bb000b = 24: 072ff40 mingw use backslashes in compileall py
25: b3b1acd = 25: d1b24d4 mingw pdcurses_ISPAD
26: 44efc48 = 26: 64ea6e4 remove_path_max.default
27: 8100121 = 27: 303ceee dont link with gettext
28: 8653e12 = 28: 79a179c ctypes python dll
29: 46edd93 = 29: 1a19840 gdbm module includes
30: b7a2f35 = 30: 3d35ed8 use gnu_printf in format
31: 8a4b9f7 = 31: 98c7f82 mingw fix ssl dont use enum_certificates
32: 67627d2 ! 32: b7edd90 fix using dllhandle and winver mingw

@@ Commit message
 
  ## Python/sysmodule.c ##
 @@ Python/sysmodule.c: Data members:
- #include <windows.h>
+ #  include <windows.h>
  #endif /* MS_WINDOWS */
  
 -#ifdef MS_COREDLL

33: 2c79609 = 33: ee575ff Add AMD64 to sys config so msvccompiler get_build_version works
34: 8e83f3d = 34: 80a5479 MINGW link with additional library
35: 0ae17fc = 35: 419b585 install msilib
36: fcd75f3 = 36: 79b4fd3 fix signal module build
37: 49f9fe9 = 37: 0092433 build: build winconsoleio and _testconsole
38: 01ef4e0 = 38: ecec00a expose sem_unlink
39: 1da82fc ! 39: 3e5da55 Use cygpty while using isatty

@@ Python/bltinmodule.c
  #include "pycore_ast.h"           // _PyAST_Validate()
  #include "pycore_call.h"          // _PyObject_CallNoArgs()
 @@ Python/bltinmodule.c: builtin_input_impl(PyObject *module, PyObject *prompt)
-         Py_DECREF(tmp);
-         if (fd < 0 && PyErr_Occurred())
-             return NULL;
+         if (fd < 0 && PyErr_Occurred()) {
+             goto error;
+         }
 -        tty = fd == fileno(stdin) && isatty(fd);
 +        tty = fd == fileno(stdin) && (isatty(fd) || is_cygpty(fd));
      }
@@ Python/bltinmodule.c: builtin_input_impl(PyObject *module, PyObject *prompt)
 @@ Python/bltinmodule.c: builtin_input_impl(PyObject *module, PyObject *prompt)
              Py_DECREF(tmp);
              if (fd < 0 && PyErr_Occurred())
-                 return NULL;
+                 goto error;
 -            tty = fd == fileno(stdout) && isatty(fd);
 +            tty = fd == fileno(stdout) && (isatty(fd) || is_cygpty(fd));
          }

40: e8d69af = 40: 9417841 build: link win resource files and build pythonw
41: 1bc0601 = 41: 40d8928 fix isselectable
42: d2671a9 = 42: 4f7f990 configure.ac: fix inet_pton check
43: 2bec85a = 43: 4ba545d pass gen profile ldflags
44: 3fa80c7 = 44: 856611c pkg config windows must link ext with python lib
45: 4851c0c = 45: f1c012d importlib bootstrap path sep
46: 17a4e28 = 46: 90e63e3 warnings fixes
47: 025bf11 = 47: d4699e7 fix build testinternalcapi
48: a21087c = 48: f14104a clang arm64
49: 7a689b9 = 49: 9ca9a38 configure.ac: set MINGW stack reserve
50: 345d04d = 50: c7cfa8e Don't use os.pathsep to find EOF not all distributions in win32 have them as \ instead check using sys.platform
51: 84ceb66 = 51: 813b8b7 Fix extension suffix for c-extensions on mingw
52: b965482 = 52: 92f9d3c Change the get_platform() method in sysconfig
53: 3354131 = 53: 566627c build: Fix ncursesw include lookup
54: 8cb44d1 = 54: 9f78c20 tests: fix test_bytes
55: 68f2a11 = 55: 37d370c time: fix strftime not raising for invalid year values
56: 7358366 = 56: 06ce4a2 ctypes: find_library('c') should return None with ucrt
57: 1e81da6 = 57: c257585 build: Disable checks for dlopen/dlfcn
58: 929b197 = 58: 70aa50a Fix install location of the import library
59: 8a37e02 = 59: 1c3a6ce build: Integrate venvlauncher build/installation into the Makefile
60: 1cf8aee = 60: c20a829 configure.ac: set WIN32_WINNT version
61: ab89718 = 61: 4262bd5 configure.ac: don't check for clock
functions
62: a16be6e = 62: 75bd12c expanduser: normpath paths coming from env vars
63: b1c198d = 63: 6c4b8c8 CI: test the build and add some mingw specific tests
64: a00d7bd = 64: f61cd58 Define PY3_DLLNAME to fix build
65: 956e302 = 65: dace02e _testconsole.c: Fix casing & path sep
66: c4b4886 = 66: d7f45cc Return consistent architecture markers for python on mingw/armv7
67: a97b113 = 67: d162f1d handle ncursesw pkg-config when cross-compiling
68: da22ca4 = 68: 5520f77 mingw_smoketests: fix _UCRT condition
69: 687fec6 = 69: d77bb16 Modify sys.winver to match upstream
70: f918377 = 70: e808beb Change user site-packages path to include the environment info
71: e679a14 = 71: e2ca452 configure: Include a header in the check for _beginthread
72: edbdf21 = 72: dc09df5 configure.ac: Default to --without-c-locale-coercion on Windows
73: 93c0b0a = 73: 9d27166 Fix failing tests
74: 6ba7d4d = 74: 8f547fc Don't change os.sep with an empty MSYSTEM env var, not just a missing one
75: 7b49379 = 75: 86fe7ab def VPATH when compiling Python/sysmodule.c
76: 9d0cb2e = 76: 17921da Make _Py_CheckPython3 extern
77: 1c45126 = 77: 7f44e20 link with bcrypt
78: f1fa7c0 = 78: 7d377bd correctly find native python
79: 1a01952 = 79: 3cf81b7 Add extra flags for _bootstrap_python
80: 824e969 = 80: 435ac83 posixmodule: undefine HAVE_OPENDIR
81: e23dc68 = 81: 5d62a5d getpath.py: add support for mingw
82: 21748bb = 82: b8cde35 Don't build _posixsubprocess on Windows.
83: aaa5138 = 83: b54da9f _ssl: link with ws2_32
84: d85bf6b = 84: 1222380 Always normalize path in abspath
85: 7134817 = 85: 42f3b79 Include winsock.h when checking for netdb function
86: b4d654d = 86: c0c7ce1 include _multiprocessing/semaphore.c on win32
87: bf8ad27 = 87: beb7cea configure: build mmap module on win32
88: 708e46e = 88: 21f62ec venv creation fixes
89: 8d7075d = 89: f1073a3 move the shutdown function where winsock.h is included
90: 1d42f8f = 90: fc0dd6b configure.ac: set BUILDEXEEXT and EXEEXT
91: 9c2622c = 91: d35b2e5 configure.ac: fix building some test modules
92: 3a9be2a = 92: ee9142c Don't convert sysconfig.get_config_var('VPATH') to an absolute path
93: cf7d334 = 93: 9431777 Always convert / to \\ before passing though pathcch functions
94: 2f9b925 = 94: 316fbde Build venvlauncher.exe from PC/launcher.c
95: 470a8c8 = 95: 6e1d779 getpath.py: fix dirname
96: 0daa456 = 96: 638d1da getpath: use normpath on all generated paths
97: 516ba4d = 97: 50b88dd pathconfig: normpath sys.path[0]
98: b2feb7e = 98: 1b61d6b smoketests: add some tests for sys/site paths
99: 662ae51 = 99: 9825b41 Search DLLs only on paths added using add_dll_directory().
100: b194251 = 100: 4769814 Build and install libpython3.dll
101: 20b05ed = 101: 1cb727a Port GetPythonImport() to mingw
102: ba1318d = 102: e754b07 LoadLibraryExW: make sure to only use backslashes for paths
103: c65ef8c = 103: f95b319 mingw_smoketests: add a test to check if sysconfig returns correct values for LIBDIR
104: f7caf72 = 104: 442dc1b Add missing library for multiprocessing module
105: 7e39665 = 105: 05e22b5 selectmodule: add support for MINGW
106: c7203da = 106: 6968f18 readline: disable on MINGW
107: ad9e9e6 = 107: ab99d16 Add libraries to fix compile of ctypes on MINGW
108: 4101ea4 = 108: bf939ee Add -lpython<version> while linking stdlib
109: 6420a04 = 109: d6e7bc7 Enable _uuid on MINGW
110: ab95c18 = 110: f291913 Enable curses library on MINGW
111: 6edb9c3 = 111: 53f904e mingw_smoketests: disable distutils related tests
112: 7262c5d = 112: 904c421 configure.ac: use autoconf 2.69
113: f8db959 = 113: 780ad60 CI: clean up ignored tests
114: 25a4659 = 114: c202fe4 gcc: make incompatible-pointer-types as warning
115: 82468a4 = 115: f5a51b2 test_makefile: normalize path
116: bb09d84 = 116: bd5af5e sysconfig.get_platform(): use consistent naming
117: 3b1c83f = 117: 1e21203 Add ucrt to version string
118: 43f6304 = 118: 5a2828b socketmodule: fix captilization of headers
119: 437f206 = 119: 62b2ff3 mingw_smoketests: build extension in a venv
120: ce24efb = 120: 271f0da venvlauncher: try looking for the versioned .exe first and then fall back
121: 7f2569d = 121: 082ef86 CI: update actions
122: 1b241aa < -: ----------- fixup! add python config sh
123: 46151e9 = 122: a02e0a4 CI: test CLANGARM64

mingwandroid and others added 30 commits April 16, 2025 07:30
Co-authored-by: Алексей <[email protected]>
Co-authored-by: lovetox <[email protected]>
Co-authored-by: Naveen M K <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Christoph Reiter <[email protected]>
This is used throughout the Python code but is not defined
when compiling with Mingw-w64 compilers, so define it.
In MSVC these are defined in PC/pyconfig.h, we have to
define it manually because pyconfig.h is created by
autotools.

Co-authored-by: Алексей <[email protected]>
Co-authored-by: Christoph Reiter <[email protected]>
Co-authored-by: Naveen M K <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Christoph Reiter <[email protected]>
Largefile is supported on Windows.

Co-authored-by: Naveen M K <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Christoph Reiter <[email protected]>
Also, build `nt` module instead of `posix` when compiling
with MINGW.

Co-authored-by: Naveen M K <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Christoph Reiter <[email protected]>
Co-authored-by: Naveen M K <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Naveen M K <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Naveen M K <[email protected]>
This is the case used in MINGW

Co-authored-by: Алексей <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Christoph Reiter <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Christoph Reiter <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Christoph Reiter <[email protected]>
Co-authored-by: cat <[email protected]>
Co-authored-by: Naveen M K <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Christoph Reiter <[email protected]>
Co-authored-by: Алексей <[email protected]>
Co-authored-by: Алексей <[email protected]>
lazka and others added 22 commits April 16, 2025 08:35
This looks for DLL names in the import table but while with MSVC the DLL
is named python311.dll in our case it is named libpython3.11.dll.
Adjust the strings and lengths accordingly.
It seems like in case the path passed to it is absolute, but contains
forward slashes then LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR does not work
and DLLs in the same directory as the extension are not considered.

This occurs in our fork because in MSYS2-mode the extension loader will
normalize to forward slashes before.

Normalize everything to backslashes again before passing it to LoadLibraryExW.

Fixes #151
`ws2_32` is required when building that module
Generalize the `MINGW` case.
In theory, it should be correct for other system too
and would fix cross-compilation
llvm-mingw only has autoconf 2.69
Remove tests which are passing now, or no longer exist, and
move tests which only fail with msvcrt into a separate file.
gcc 14 changed `incompatible-pointer-types` from warning to error
for now, make it as warning to avoid build failure
so that it passes on mingw
Currently, the platform names are hardcoded in many places,
and are not named consistently. This commit fixes it by
standardizing the names to be returned by `sysconfig.get_platform`.
The naming is based on #167 (comment)

Similarly, `EXT_SUFFIX` is also standardized to be consistent with the platform names.

Signed-off-by: Naveen M K <[email protected]>
`Rpc` should be `rpc`. It error's out in
cross-compilation.
ensurepip no longer works since we implement PEP 668, so do
everything in a venv.
…back

By default venvlauncher only looks for python.exe in the python "home",
which fails in the MSYS2 case where there could be multiple python versions
in the same prefix i.e. python.exe could be 3.12 while the venv was created
with python3.13.exe.

Upstream CPython doesn't have this problem since they never have multiple
Python versions in the same prefix.

On the other hand the Python test suite (test_sysconfig specifically) creates
a venv for the uninstalled Python build dir, where only python.exe exists.

To cover both cases we first try to look for the python3.XY.exe and then
fall back to python.exe.
test cross-compiled arm builds too
@lazka lazka force-pushed the wip-mingw-v3.12.10 branch from 067a5c8 to a02e0a4 Compare April 16, 2025 06:36
@lazka lazka marked this pull request as ready for review April 16, 2025 06:40
Copy link
Member

@naveen521kk naveen521kk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@lazka lazka closed this Apr 17, 2025
@lazka lazka reopened this Apr 17, 2025
@lazka lazka merged commit a02e0a4 into mingw-v3.12.10 Apr 17, 2025
35 of 40 checks passed
@lazka lazka deleted the wip-mingw-v3.12.10 branch April 17, 2025 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants