• Best (simplest) way to share data between processes

    From Chris Green@3:633/280.2 to All on Sat Jul 6 17:28:41 2024
    I have a Raspberry Pi in my boat that uses I2C to read a number of
    voltages and currents (using ADS1115 A2D) so I can monitor the battery condition etc.

    At present various different scripts (i.e. processes) just read the
    values using the I2C bus whenever they need to but I'm pretty sure
    this (quite rarely) results in false readings because two processes
    try to read at the same time.

    Thus I'm looking for ways to prevent simultaneous access.

    One fairly obvious way is to have single process/script which reads
    the A2D values continuously and writes them to a file. All other
    scripts then read from the file as needed, a simple file lock can then
    be used to prevent simultaneous access (well, simultaneous access when
    the writing process is writing).

    Is this the simplest approach? Are there better ways using
    multiprocess? (They look more complicated though).

    The I2C bus itself has a mutex but I don't think this guarantees that
    (for example) an A2D reading is atomic because one reading takes more
    than one I2C bus access.

    Would a mutex of some sort around each I2C transaction (i.e. complete
    A2D reading) be a better way to go?

    --
    Chris Green
    ú

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Stefan Ram@3:633/280.2 to All on Sat Jul 6 21:32:53 2024
    Chris Green <cl@isbd.net> wrote or quoted:
    Would a mutex of some sort around each I2C transaction (i.e. complete
    A2D reading) be a better way to go?

    I not an expert here, but from what I heard, you could use
    a named POSIX semaphore as a kind of system-wide mutex.

    Or, the single process reading the A2D values could write
    them to a shared memory segment the access to which is
    controlled by a mutex.

    Or, you could use a message queue (like POSIX mq) to
    distribute readings.

    But why overengineer? If you feel comfortable with the file
    solution, go for it! The only drawback might be that it's a
    bit slower than other approaches.

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: Stefan Ram (3:633/280.2@fidonet)
  • From Piergiorgio Sartor@3:633/280.2 to All on Sun Jul 7 05:32:44 2024
    On 06/07/2024 09.28, Chris Green wrote:
    I have a Raspberry Pi in my boat that uses I2C to read a number of
    voltages and currents (using ADS1115 A2D) so I can monitor the battery condition etc.

    At present various different scripts (i.e. processes) just read the
    values using the I2C bus whenever they need to but I'm pretty sure
    this (quite rarely) results in false readings because two processes
    try to read at the same time.

    Thus I'm looking for ways to prevent simultaneous access.

    Why using "different scripts"?
    Is it there any particular reason?

    Maybe it would be better, if possible, to have
    a single script, which, sequentially, reads
    whatever needs to be read (or written).
    In a loop.

    This is even simpler than using a file.

    bye,

    pg

    One fairly obvious way is to have single process/script which reads
    the A2D values continuously and writes them to a file. All other
    scripts then read from the file as needed, a simple file lock can then
    be used to prevent simultaneous access (well, simultaneous access when
    the writing process is writing).

    Is this the simplest approach? Are there better ways using
    multiprocess? (They look more complicated though).

    The I2C bus itself has a mutex but I don't think this guarantees that
    (for example) an A2D reading is atomic because one reading takes more
    than one I2C bus access.

    Would a mutex of some sort around each I2C transaction (i.e. complete
    A2D reading) be a better way to go?


    --

    piergiorgio


    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: A noiseless patient Spider (3:633/280.2@fidonet)
  • From Barry@3:633/280.2 to All on Mon Jul 8 08:27:01 2024


    On 7 Jul 2024, at 22:13, Chris Green via Python-list <python-list@python.o=
    wrote:
    =20
    a simple file lock can then
    be used to prevent simultaneous access (well, simultaneous access when
    the writing process is writing).

    There is a simple pattern to make this robust.

    Write new values to a tmp file.
    Close the tmp file.
    Then use os.rename(tmpfile, productionfile).

    This is guaranteed that any process that reads the file will only see all th=
    e old file contents or all the new file contents, never a mix of both.

    Barry



    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From MRAB@3:633/280.2 to All on Mon Jul 8 08:47:01 2024
    On 2024-07-07 23:27, Barry via Python-list wrote:


    On 7 Jul 2024, at 22:13, Chris Green via Python-list <python-list@python.org> wrote:

    a simple file lock can then
    be used to prevent simultaneous access (well, simultaneous access when
    the writing process is writing).

    There is a simple pattern to make this robust.

    Write new values to a tmp file.
    Close the tmp file.
    Then use os.rename(tmpfile, productionfile).

    This is guaranteed that any process that reads the file will only see all the old file contents or all the new file contents, never a mix of both.

    For clarity I'd recommend os.replace instead. This is because on Windows os.rename it would complain if the target file already exists, but
    os.replace has the same behaviour on both Linux and Windows.

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Barry Scott@3:633/280.2 to All on Mon Jul 8 18:34:44 2024


    On 7 Jul 2024, at 23:47, MRAB via Python-list <python-list@python.org> =
    wrote:
    =20
    For clarity I'd recommend os.replace instead. This is because on =
    Windows os.rename it would complain if the target file already exists, =
    but os.replace has the same behaviour on both Linux and Windows.

    Agreed.

    In this case the OP is on an RPi.

    Barry


    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Left Right@3:633/280.2 to All on Mon Jul 8 07:55:09 2024
    If resource usage isn't an issue, then the _easy_ thing to do, that
    would also be easily correct is to have a server doing all the
    h/w-related reading and clients talking to that server. Use for the
    server the technology you feel most confident with. Eg. you may use
    Python's http package. I believe that the server from this package
    runs in a single thread, and thus processes all requests
    synchronously. So, you'll get synchronization for free.

    Then, the rest of the scripts that need to talk to h/w will instead be
    talking to this server.

    Again, this isn't an _efficient_ solution... but, sometimes you don't
    need one. And this one is easy to make, easy to debug, easy to expand.
    But, if instead you were looking for a more efficient solution, then,
    the general idea that allows the http server to work in this case
    would still apply: have a single synchronization program that takes
    requests asynchronously, and orders them. So, a basic TCP server would
    also work as well as a UNIX socket. Your idea with holding a lock on a
    file would also work (in fact, plenty of Linux utilities work that
    way, eg. apt-get or yum).

    ----

    If you don't want to change the existing script, then instead of
    running them directly, you could run them through batch: https://man7.org/linux/man-pages/man1/batch.1p.html this is a very
    simply queuing program that's available for Linux. It will take care
    of synchronization by putting the scripts you want to run in a queue
    and executing them one at a time.

    On Sun, Jul 7, 2024 at 11:12=E2=80=AFPM Chris Green via Python-list <python-list@python.org> wrote:

    I have a Raspberry Pi in my boat that uses I2C to read a number of
    voltages and currents (using ADS1115 A2D) so I can monitor the battery condition etc.

    At present various different scripts (i.e. processes) just read the
    values using the I2C bus whenever they need to but I'm pretty sure
    this (quite rarely) results in false readings because two processes
    try to read at the same time.

    Thus I'm looking for ways to prevent simultaneous access.

    One fairly obvious way is to have single process/script which reads
    the A2D values continuously and writes them to a file. All other
    scripts then read from the file as needed, a simple file lock can then
    be used to prevent simultaneous access (well, simultaneous access when
    the writing process is writing).

    Is this the simplest approach? Are there better ways using
    multiprocess? (They look more complicated though).

    The I2C bus itself has a mutex but I don't think this guarantees that
    (for example) an A2D reading is atomic because one reading takes more
    than one I2C bus access.

    Would a mutex of some sort around each I2C transaction (i.e. complete
    A2D reading) be a better way to go?

    --
    Chris Green
    =C2=B7
    --
    https://mail.python.org/mailman/listinfo/python-list

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)
  • From Chris Green@3:633/280.2 to All on Mon Jul 8 22:52:59 2024
    Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.removethis.de> wrote:
    On 06/07/2024 09.28, Chris Green wrote:
    I have a Raspberry Pi in my boat that uses I2C to read a number of
    voltages and currents (using ADS1115 A2D) so I can monitor the battery condition etc.

    At present various different scripts (i.e. processes) just read the
    values using the I2C bus whenever they need to but I'm pretty sure
    this (quite rarely) results in false readings because two processes
    try to read at the same time.

    Thus I'm looking for ways to prevent simultaneous access.

    Why using "different scripts"?
    Is it there any particular reason?

    Maybe it would be better, if possible, to have
    a single script, which, sequentially, reads
    whatever needs to be read (or written).
    In a loop.

    This is even simpler than using a file.

    Yes, but it's conceptually (and programming wise) much simpler to have
    separate scripts. Some of them are simple 'on demand' scripts that I
    run from the command line when I want to know something. Others are
    scripts that drive displays on control panels.

    --
    Chris Green
    ú

    --- MBSE BBS v1.0.8.4 (Linux-x86_64)
    * Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)