• Python crash together with threads

    From Guenther Sohler@3:633/280.2 to All on Wed Oct 2 23:26:47 2024
    My Software project is working fine in most of the cases
    (www.pythonscad.org)
    however I am right now isolating a scenario, which makes it crash
    permanently.

    It does not happen with Python 3.11.6 (and possibly below), it happens with 3.12 and above
    It does not happen when not using Threads.

    However due to the architecture of the program I am forced to evaluate some parts in main thread and some parts in a dedicated Thread. The Thread is started with QThread(QT 5.0)
    whereas I am quite sure that program flows do not overlap.

    When I just execute my 1st very simple Python function inside the newly
    created thread, like:

    PyObject *a = PyFloat_FromDouble(3.3);

    my program crashes with this Stack trace

    0 0x00007f6837fe000f in _PyInterpreterState_GET () at ../Include/internal/pycore_pystate.h:179
    #1 get_float_state () at Objects/floatobject.c:38
    #2 PyFloat_FromDouble (fval=3.2999999999999998) at
    Objects/floatobject.c:136
    #3 0x00000000015a021f in python_testfunc() ()
    #4 0x0000000001433301 in CGALWorker::work() ()
    #5 0x0000000000457135 in CGALWorker::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
    #6 0x00007f68364d0f9f in void doActivate<false>(QObject*, int, void**) ()
    at /lib64/libQt5Core.so.5
    #7 0x00007f68362e66ee in QThread::started(QThread::QPrivateSignal) () at /lib64/libQt5Core.so.5
    #8 0x00007f68362e89c4 in QThreadPrivate::start(void*) () at /lib64/libQt5Core.so.5
    #9 0x00007f6835cae19d in start_thread () at /lib64/libc.so.6
    #10 0x00007f6835d2fc60 in clone3 () at /lib64/libc.so.6


    I suspect, that this is a Null pointer here
    See also _PyInterpreterState_Get()
    and _PyGILState_GetInterpreterStateUnsafe(). */
    static inline PyInterpreterState* _PyInterpreterState_GET(void) {
    PyThreadState *tstate = _PyThreadState_GET();
    #ifdef Py_DEBUG
    _Py_EnsureTstateNotNULL(tstate);
    #endif
    # <<----------- suspect state is nullpointer
    return tstate->interp;
    }

    any clues , whats going on here, and how I can mitigate that ?

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Louis Krupp@3:633/280.2 to All on Thu Oct 3 07:06:03 2024
    On 10/2/2024 7:26 AM, Guenther Sohler wrote:
    My Software project is working fine in most of the cases (www.pythonscad.org)
    however I am right now isolating a scenario, which makes it crash permanently.

    It does not happen with Python 3.11.6 (and possibly below), it happens with 3.12 and above
    It does not happen when not using Threads.

    However due to the architecture of the program I am forced to evaluate some parts in main thread and some parts in a dedicated Thread. The Thread is started with QThread(QT 5.0)
    whereas I am quite sure that program flows do not overlap.

    When I just execute my 1st very simple Python function inside the newly created thread, like:

    PyObject *a = PyFloat_FromDouble(3.3);

    my program crashes with this Stack trace

    0 0x00007f6837fe000f in _PyInterpreterState_GET () at ./Include/internal/pycore_pystate.h:179
    #1 get_float_state () at Objects/floatobject.c:38
    #2 PyFloat_FromDouble (fval=3.2999999999999998) at
    Objects/floatobject.c:136
    #3 0x00000000015a021f in python_testfunc() ()
    #4 0x0000000001433301 in CGALWorker::work() ()
    #5 0x0000000000457135 in CGALWorker::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
    #6 0x00007f68364d0f9f in void doActivate<false>(QObject*, int, void**) ()
    at /lib64/libQt5Core.so.5
    #7 0x00007f68362e66ee in QThread::started(QThread::QPrivateSignal) () at /lib64/libQt5Core.so.5
    #8 0x00007f68362e89c4 in QThreadPrivate::start(void*) () at /lib64/libQt5Core.so.5
    #9 0x00007f6835cae19d in start_thread () at /lib64/libc.so.6
    #10 0x00007f6835d2fc60 in clone3 () at /lib64/libc.so.6


    I suspect, that this is a Null pointer here
    See also _PyInterpreterState_Get()
    and _PyGILState_GetInterpreterStateUnsafe(). */
    static inline PyInterpreterState* _PyInterpreterState_GET(void) {
    PyThreadState *tstate = _PyThreadState_GET();
    #ifdef Py_DEBUG
    _Py_EnsureTstateNotNULL(tstate);
    #endif
    # <<----------- suspect state is nullpointer
    return tstate->interp;
    }

    any clues , whats going on here, and how I can mitigate that ?
    Can you post a small, self-contained program that demonstrates the problem?

    Louis

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: Newshosting.com - Highest quality at a great p (3:633/280.2@fidonet)
  • From Left Right@3:633/280.2 to All on Fri Oct 4 07:01:53 2024
    whereas I am quite sure that program flows do not overlap.

    You can never be sure of this in Python. Virtually all objects in
    Python are allocated on heap, so instantiating integers, doing simple arithmetic etc. -- all of this requires synchronization because it
    will allocate memory for a shared pool.

    The description of _PyThreadState_GET states that callers must hold
    GIL. Does your code do that? It's not possible to divine that from the
    stack trace, but you'd probably know that.

    On Wed, Oct 2, 2024 at 3:29=E2=80=AFPM Guenther Sohler via Python-list <python-list@python.org> wrote:

    My Software project is working fine in most of the cases (www.pythonscad.org)
    however I am right now isolating a scenario, which makes it crash permanently.

    It does not happen with Python 3.11.6 (and possibly below), it happens wi=
    th
    3.12 and above
    It does not happen when not using Threads.

    However due to the architecture of the program I am forced to evaluate so=
    me
    parts in main thread and some parts in a dedicated Thread. The Thread is started with QThread(QT 5.0)
    whereas I am quite sure that program flows do not overlap.

    When I just execute my 1st very simple Python function inside the newly created thread, like:

    PyObject *a =3D PyFloat_FromDouble(3.3);

    my program crashes with this Stack trace

    0 0x00007f6837fe000f in _PyInterpreterState_GET () at ./Include/internal/pycore_pystate.h:179
    #1 get_float_state () at Objects/floatobject.c:38
    #2 PyFloat_FromDouble (fval=3D3.2999999999999998) at Objects/floatobject.c:136
    #3 0x00000000015a021f in python_testfunc() ()
    #4 0x0000000001433301 in CGALWorker::work() ()
    #5 0x0000000000457135 in CGALWorker::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
    #6 0x00007f68364d0f9f in void doActivate<false>(QObject*, int, void**) (=
    )
    at /lib64/libQt5Core.so.5
    #7 0x00007f68362e66ee in QThread::started(QThread::QPrivateSignal) () at /lib64/libQt5Core.so.5
    #8 0x00007f68362e89c4 in QThreadPrivate::start(void*) () at /lib64/libQt5Core.so.5
    #9 0x00007f6835cae19d in start_thread () at /lib64/libc.so.6
    #10 0x00007f6835d2fc60 in clone3 () at /lib64/libc.so.6


    I suspect, that this is a Null pointer here
    See also _PyInterpreterState_Get()
    and _PyGILState_GetInterpreterStateUnsafe(). */
    static inline PyInterpreterState* _PyInterpreterState_GET(void) {
    PyThreadState *tstate =3D _PyThreadState_GET();
    #ifdef Py_DEBUG
    _Py_EnsureTstateNotNULL(tstate);
    #endif
    # <<----------- suspect state is nullpointer
    return tstate->interp;
    }

    any clues , whats going on here, and how I can mitigate that ?
    --
    https://mail.python.org/mailman/listinfo/python-list

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)