• Strategies for avoiding having to use --break-system-packages with pip

    From Chris Green@3:633/280.2 to All on Tue Jan 14 22:32:35 2025
    Subject: Strategies for avoiding having to use --break-system-packages with pip

    I have a (relatively) clean Debian 12 installation running on my two
    workhorse systems, a desktop server at home and my laptop that travels
    around with me.

    I moved from Xubuntu to Debian on both these systems a few months ago.

    I ran Xubuntu for many years and acquired a whole lot of python
    packages installed with pip, as root. For the last couple of years I
    had to use the --break-system-packages option to get things installed.

    As far as I'm aware I never hit any dependency problems doing this.
    It's probably because things I installed with pip were mostly quite
    small, specialised, packages that I used in just one or two utility
    programs that I had written myself. In quite a few cases these were
    realated to image processing and such things.


    So far I've managed to keep my Debian 12 installations 'pip free', I
    haven't even got pip installed. However I may have just come across
    something that would at least be very useful and it comes from PyPi.
    (It's tkintertable if that's of any interest or relevance)


    What are my options?

    Just install it using pip as root and --break-system-packages,
    what's likely to break?

    Use a virtual environment, what do I have to do then to make using
    my program (that uses tkintertable) 'transparent', i.e. I just
    want to be able to run the program from the command prompt like
    any other program.

    Download tkintertable from git into my development environment and
    use that. My PYTHONPATH will need to point to it but I can't see
    any further issues with doing this.

    Anything else? As far as I can see using pipx doesn't help me at
    all (see recent thread here).

    --
    Chris Green
    ú

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Stefan Ram@3:633/280.2 to All on Tue Jan 14 22:43:19 2025
    Subject: Re: Strategies for avoiding having to use --break-system-packages with pip

    Chris Green <cl@isbd.net> wrote or quoted:
    What are my options?

    You've got a few ways to skin this cat without turning your Debian
    setup into a dumpster fire.

    Virtual environment route:

    This is your best bet for keeping things kosher. Here's the lowdown:

    bash

    python3 -m venv ~/myenv
    source ~/myenv/bin/activate
    pip install tkintertable

    To make your program run smooth as butter, whip up a shell script
    that fires up the virtual environment and kicks off your program:

    bash

    #!/bin/bash
    source ~/myenv/bin/activate
    python /path/to/your/program.py

    Slap that bad boy in your PATH, make it executable, and you're
    golden.

    Source installation:

    Pulling tkintertable from git and tweaking your PYTHONPATH is
    solid. It's like growing your own organic produce - more work,
    but you know what you're getting.

    Pip with --break-system-packages:

    It's like jaywalking - you might get away with it, but one day
    you could end up in a world of hurt.

    DIY Debian package:

    For the overachievers out there. It's like building your
    own surfboard - cool if you can pull it off, but not for
    the faint of heart.

    Bottom line:

    The virtual environment play (option 1) is your ticket to ride.
    It keeps your system clean as a whistle while letting you run
    your program without breaking a sweat.



    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: Stefan Ram (3:633/280.2@fidonet)
  • From Mats Wichmann@3:633/280.2 to All on Wed Jan 15 03:06:06 2025
    Subject: Re: Strategies for avoiding having to use --break-system-packages
    with pip

    On 1/14/25 04:32, Chris Green via Python-list wrote:
    I have a (relatively) clean Debian 12 installation running on my two workhorse systems, a desktop server at home and my laptop that travels
    around with me.

    I moved from Xubuntu to Debian on both these systems a few months ago.

    I ran Xubuntu for many years and acquired a whole lot of python
    packages installed with pip, as root. For the last couple of years I
    had to use the --break-system-packages option to get things installed.

    As far as I'm aware I never hit any dependency problems doing this.
    It's probably because things I installed with pip were mostly quite
    small, specialised, packages that I used in just one or two utility
    programs that I had written myself. In quite a few cases these were
    realated to image processing and such things.


    So far I've managed to keep my Debian 12 installations 'pip free', I
    haven't even got pip installed. However I may have just come across something that would at least be very useful and it comes from PyPi.
    (It's tkintertable if that's of any interest or relevance)


    What are my options?

    Just install it using pip as root and --break-system-packages,
    what's likely to break?

    Use a virtual environment, what do I have to do then to make using
    my program (that uses tkintertable) 'transparent', i.e. I just
    want to be able to run the program from the command prompt like
    any other program.

    Download tkintertable from git into my development environment and
    use that. My PYTHONPATH will need to point to it but I can't see
    any further issues with doing this.

    Anything else? As far as I can see using pipx doesn't help me at
    all (see recent thread here).


    You might look at uv, which makes the managing of a virtualenv for your program pretty transparent. You declare your dependencies, and then just:

    uv run myscript.py

    And of course if you don't want to type three words to launch, you can
    put those in an executable shell script in your path, and invoke it with
    a single command.

    There was a nice article on this somewhere which I now can't find, but
    the project docs cover the topics pretty well:

    https://docs.astral.sh/uv/guides/scripts



    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From c.buhtz@posteo.jp@3:633/280.2 to All on Wed Jan 15 03:09:16 2025
    Subject: Re: Strategies for avoiding having to use --break-system-packages
    with pip

    Hello Chris,

    I do have similar "problems" and still try to get used to the "new way".

    Other might correct me. I am not sure yet.

    To my current understanding the way to go is to install Python
    applications via "pipx". That make the application available in your
    system but also isolate it in its own virtual environment. Of course you should prefer to install applications from your GNU/Linux distros
    official repository if available.

    If you install a Python package (library, not an application) you should create your own Python environment via venv for example. Pipx is not
    intended to install Python packages that are not applications.

    Regrads,
    Christian

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Thomas Passin@3:633/280.2 to All on Wed Jan 15 03:36:34 2025
    Subject: Re: Strategies for avoiding having to use --break-system-packages
    with pip

    On 1/14/2025 6:32 AM, Chris Green via Python-list wrote:
    I have a (relatively) clean Debian 12 installation running on my two workhorse systems, a desktop server at home and my laptop that travels
    around with me.

    I moved from Xubuntu to Debian on both these systems a few months ago.

    I ran Xubuntu for many years and acquired a whole lot of python
    packages installed with pip, as root. For the last couple of years I
    had to use the --break-system-packages option to get things installed.

    As far as I'm aware I never hit any dependency problems doing this.
    It's probably because things I installed with pip were mostly quite
    small, specialised, packages that I used in just one or two utility
    programs that I had written myself. In quite a few cases these were
    realated to image processing and such things.


    So far I've managed to keep my Debian 12 installations 'pip free', I
    haven't even got pip installed. However I may have just come across something that would at least be very useful and it comes from PyPi.
    (It's tkintertable if that's of any interest or relevance)


    What are my options?

    Just install it using pip as root and --break-system-packages,
    what's likely to break?

    You can also install with --user and --break-system-packages, but that
    doesn't really solve the problem. Also, as just happened to me, if an
    upgrade happens to change the system's python to a newer version (e.g.,
    3.12.x to 3.13.y), you would have to install all your packages again
    with the new Python install.

    Use a virtual environment, what do I have to do then to make using
    my program (that uses tkintertable) 'transparent', i.e. I just
    want to be able to run the program from the command prompt like
    any other program.

    You can write a shell script that activates the venv and then launches
    your program. This works pretty well.

    Download tkintertable from git into my development environment and
    use that. My PYTHONPATH will need to point to it but I can't see
    any further issues with doing this.

    This is an approach I use sometimes, mainly if I have cloned a project.
    I run the project's program(s) using a script that sets the PYTHONPATH.

    Anything else? As far as I can see using pipx doesn't help me at
    all (see recent thread here).

    To completely avoid problems when the system's Python install gets
    changed, you can install your own Python version outside of the package manager; it doesn't have to be the same version as the system's. I've
    done this when I wanted to run a later version of Python than the
    system's. You would have to take care of updating it yourself since the package manager won't know about it. On the system where I did this, I
    ran the system's python version using "python3" and my own using
    "python3.11" (I think the system was still on 3.8 or 3.9).

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Left Right@3:633/280.2 to All on Wed Jan 15 06:06:39 2025
    Subject: Re: Strategies for avoiding having to use --break-system-packages
    with pip

    I wouldn't trust pip to install anything into my system. It's not a
    reliable program that I'd recommend anyone to use for things that they
    might depend on.

    My typical course of action is to create a virtual environment for the
    package I need. Install the package into that virtual environment
    using pip or w/e the package wants to be installed with. Investigate
    and refine the dependencies I need (It's very common in the Python
    world to incorrectly specify dependencies, to require a lot of
    unnecessary dependencies, to depend on packages in the wrong way). And
    after I've figured out what exactly I need to get the package to work,
    copy the dependencies together with the package to the platlib
    directory of Python I'm using for this task.

    If platlib happens to be in the system directories or anywhere else: I
    wouldn't care about that. In the process of installing the program, I
    would've learned about the nature and the layout of dependencies, so
    that I could make informed decisions about what goes where and whether
    anything is needed or is in danger of being affected by system updates
    or endangers system updates.

    So far, this has worked every time. This worked for me personally,
    for small companies that do internal deployments and for big companies
    that distribute Python together with the entire Linux distribution in
    this way. The key ingredient is to know what you are doing and to be
    able to anticipate the bad things. Tools like pip take away from the
    users the need to know what they are doing in hopes of reducing
    complexity on the part of the users' knowledge necessary to accomplish
    program installation. However, there's no free lunch, and tools like
    pip bring an extra layer of complexity in trying to do what they
    claim. This layer of complexity is usually revealed when these tools
    fail to do what they claim, and users are forced to learn what they
    actually need to know and, on top of that, how to troubleshoot
    programs like pip.

    From working in infra / automation, I get the knowledge about program
    / package installation "free" on the job, so, I don't see a point in
    using a tool that automates that for me beyond what I already
    described. I'm probably also biased because of that, but I still think
    that learning to do the thing is more important than learning how to
    use the tool that does the thing.

    On Tue, Jan 14, 2025 at 4:42=E2=80=AFPM Chris Green via Python-list <python-list@python.org> wrote:

    I have a (relatively) clean Debian 12 installation running on my two workhorse systems, a desktop server at home and my laptop that travels
    around with me.

    I moved from Xubuntu to Debian on both these systems a few months ago.

    I ran Xubuntu for many years and acquired a whole lot of python
    packages installed with pip, as root. For the last couple of years I
    had to use the --break-system-packages option to get things installed.

    As far as I'm aware I never hit any dependency problems doing this.
    It's probably because things I installed with pip were mostly quite
    small, specialised, packages that I used in just one or two utility
    programs that I had written myself. In quite a few cases these were
    realated to image processing and such things.


    So far I've managed to keep my Debian 12 installations 'pip free', I
    haven't even got pip installed. However I may have just come across something that would at least be very useful and it comes from PyPi.
    (It's tkintertable if that's of any interest or relevance)


    What are my options?

    Just install it using pip as root and --break-system-packages,
    what's likely to break?

    Use a virtual environment, what do I have to do then to make using
    my program (that uses tkintertable) 'transparent', i.e. I just
    want to be able to run the program from the command prompt like
    any other program.

    Download tkintertable from git into my development environment and
    use that. My PYTHONPATH will need to point to it but I can't see
    any further issues with doing this.

    Anything else? As far as I can see using pipx doesn't help me at
    all (see recent thread here).

    --
    Chris Green
    =C2=B7
    --
    https://mail.python.org/mailman/listinfo/python-list

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Peter J. Holzer@3:633/280.2 to All on Sat Jan 18 10:22:08 2025
    Subject: Re: Strategies for avoiding having to use --break-system-packages
    with pip


    --rv6hssab3bxwuxfd
    Content-Type: text/plain; charset=us-ascii
    Content-Disposition: inline
    Content-Transfer-Encoding: quoted-printable

    On 2025-01-14 11:32:35 +0000, Chris Green via Python-list wrote:
    Use a virtual environment, what do I have to do then to make using
    my program (that uses tkintertable) 'transparent', i.e. I just
    want to be able to run the program from the command prompt like
    any other program.

    Just use the python interpreter in the venv in the hashbang line.

    For example, here's the first line of one my scripts:

    #!/usr/local/share/wds/venv/bin/python3

    As you can probably guess, the venv is in /usr/local/share/wds/venv.

    There is no need for wrapper scripts which activate the venv. Python
    does that all by itself.

    I have a small script, install-python[1], to assist with setting the
    hashbang, but if it's just a few scripts you can simply edit it manually.

    hp

    [1] https://git.hjp.at:3000/hjp/install-python/src/branch/master/install-py= thon


    --=20
    _ | Peter J. Holzer | Story must make more sense than reality.
    |_|_) | |
    | | | hjp@hjp.at | -- Charles Stross, "Creative writing
    __/ | http://www.hjp.at/ | challenge!"

    --rv6hssab3bxwuxfd
    Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEETtJbRjyPwVTYGJ5k8g5IURL+KF0FAmeK5hsACgkQ8g5IURL+ KF1avxAAsqo7v6IOH0VKMUnf+pLeoyzhlX6//Ky1WDE0MDPr5z+A1WOBRf888bOv kXkSFvG5TwxVFSsUjcVimuOyDs7h3tcK5N9ZSWWiDTR3GYnVAI7Fu4noRYQOPiPA MSeFAxDshv/86d9ZNtUL6U7LaDZotuOsNesg3UaT+LtNpmsMvyrhquZrSJp+5IJj zIFOyrm+y+hFhEHU3/N84LCn0KbsZQOVVwSSnFi8fC/7mqZlG4pmNnKGujIjUsO/ eMQ9M+zQZotk7R1xctIRAA+kPtxC0zj+LHp4yYwhWNgWt9geOwnigiuE1Bcqn/Gf pp65wbu9xx2rxS7FFlsGkcGv5awBe2HwXwkCBt1cAeSJ78fqMkH2QSvcPyXdJ9aY A+mlQ0erXl6tlG+hpbjOvhkc4Vyvfro66wCJGGI8VZ7xXEdSMtk3t87vc8PdCUMU ZdUx30bEwEOAToGSR4IgCu8QRBzHMra2ApzcE464ZEbz6Qgm/fzNOXR/49MuC1aT 1mR/5fvKq6ExVNE3WHyLMV1KUNVfaDWkLeyzfYolPFnzZZ1mPXKezTOTzJsMb+Cm s3QkTbgW90Dhc6v37hLa3sC6n3iIX4w0q49r+0kZ/Qg/bhML34+I76Q9QFgL051b w5DEwmt34rLfdb723Q7z5/3wwv7h6PVffNnkKI/zHbn5U2/VJrQ=
    =8GQl
    -----END PGP SIGNATURE-----

    --rv6hssab3bxwuxfd--

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Chris Green@3:633/280.2 to All on Sat Jan 18 22:47:57 2025
    Subject: Re: Strategies for avoiding having to use --break-system-packages with pip

    Peter J. Holzer <hjp-python@hjp.at> wrote:
    [-- text/plain, encoding quoted-printable, charset: us-ascii, 32 lines --]

    On 2025-01-14 11:32:35 +0000, Chris Green via Python-list wrote:
    Use a virtual environment, what do I have to do then to make using
    my program (that uses tkintertable) 'transparent', i.e. I just
    want to be able to run the program from the command prompt like
    any other program.

    Just use the python interpreter in the venv in the hashbang line.

    For example, here's the first line of one my scripts:

    #!/usr/local/share/wds/venv/bin/python3

    As you can probably guess, the venv is in /usr/local/share/wds/venv.

    There is no need for wrapper scripts which activate the venv. Python
    does that all by itself.

    I have a small script, install-python[1], to assist with setting the hashbang, but if it's just a few scripts you can simply edit it manually.

    OP here. Yes, thank you, I thought that the shebang line would do the
    trick but it's nioce to have it confirmed.

    --
    Chris Green
    ú

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)