Hello,

When memory mapping a file with the MAP_PRIVATE flag, the modifications (writes) only exist in memory and are not written back to the file.

According to the man pages, calling msync (3) on a such a mapping does NOT writes the changes back:

"When the msync() function is called on MAP_PRIVATE mappings, any modified data shall not be written to the underlying object and shall not cause
such data to be made visible to other processes" https://linux.die.net/man/3/msync

So: is there a way to write the changes back to the file?

An obvious application is:
- mapping the file with MAP_PRIVATE
- make some modifications in memory only (fast) while keeping the original version on disk (safe)
- at some point (when the user decides, and once the consistency of the changes have been verified) writing the modifications to the disk

I'm pretty sure it exists some way or another, but I don't know how.

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: Nemoweb (3:633/280.2@fidonet)

Le 07/04/2024 … 15:34, pehache a ‚crit :

Hello,

When memory mapping a file with the MAP_PRIVATE flag, the modifications (writes)
only exist in memory and are not written back to the file.

According to the man pages, calling msync (3) on a such a mapping does NOT writes the changes back:

"When the msync() function is called on MAP_PRIVATE mappings, any modified data
shall not be written to the underlying object and shall not cause such data to be
made visible to other processes"
https://linux.die.net/man/3/msync

So: is there a way to write the changes back to the file?

An obvious application is:
- mapping the file with MAP_PRIVATE
- make some modifications in memory only (fast) while keeping the original version on disk (safe)
- at some point (when the user decides, and once the consistency of the changes
have been verified) writing the modifications to the disk

I'm pretty sure it exists some way or another, but I don't know how.

At the moment what I'm doing is something like:

========================================
fd = open("aaa",O_RDWR);
p = mmap ( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_PRIVATE | MAP_NORESERVE
, fd
, 0 );

// writing to p; the changes exist only in memory

void* p2 = mmap( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_SHARED | MAP_NORESERVE
, fd
, 0 );

memcpy(p2,p,n); // copying everything from p to p2
msync(p2,n);

// unmap/remap p so it's ready for new changes
munmap(p,n);
p = mmap ( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_PRIVATE | MAP_NORESERVE
, fd
, 0 );

========================================

This works, but:

- the whole content is copied, not only the changed content
- is this code legal? Is there any potential conflict between the 2
mapping, with an undefined behavior?

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: Nemoweb (3:633/280.2@fidonet)

On Sun, 07 Apr 2024 13:34:43 +0000, pehache wrote:

Hello,

Hi, pehache

When memory mapping a file with the MAP_PRIVATE flag, the modifications (writes) only exist in memory and are not written back to the file.

[snip]

So: is there a way to write the changes back to the file?

[snip]

The comp.unix.programmer newsgroup will be of better help for this sort
of question (it's not really on-topic for comp.lang.c).

But, of the top of my head; with my limited understanding of the unix
mmap() kernel call, the only ways to "write the changes" are to either
1) mmap(MAP_SHARED) and modify the mapped area, or
2) write() the mapped area back to the file.

But, as I said, comp.unix.programmer will be of better help.

--
Lew Pitcher
"In Skills We Trust"

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: A noiseless patient Spider (3:633/280.2@fidonet)

As advised, I'm copying this post here

Le 07/04/2024 … 17:18, pehache a ‚crit :

Le 07/04/2024 … 15:34, pehache a ‚crit :

Hello,

When memory mapping a file with the MAP_PRIVATE flag, the modifications (writes)
only exist in memory and are not written back to the file.

According to the man pages, calling msync (3) on a such a mapping does NOT >> writes the changes back:

"When the msync() function is called on MAP_PRIVATE mappings, any modified data
shall not be written to the underlying object and shall not cause such data to be
made visible to other processes"
https://linux.die.net/man/3/msync

So: is there a way to write the changes back to the file?

An obvious application is:
- mapping the file with MAP_PRIVATE
- make some modifications in memory only (fast) while keeping the original >> version on disk (safe)
- at some point (when the user decides, and once the consistency of the changes
have been verified) writing the modifications to the disk

I'm pretty sure it exists some way or another, but I don't know how.

At the moment what I'm doing is something like:

========================================
fd = open("aaa",O_RDWR);
p = mmap ( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_PRIVATE | MAP_NORESERVE
, fd
, 0 );

// writing to p; the changes exist only in memory

void* p2 = mmap( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_SHARED | MAP_NORESERVE
, fd
, 0 );

memcpy(p2,p,n); // copying everything from p to p2
msync(p2,n);

// unmap/remap p so it's ready for new changes
munmap(p,n);
p = mmap ( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_PRIVATE | MAP_NORESERVE
, fd
, 0 );

========================================

This works, but:

- the whole content is copied, not only the changed content
- is this code legal? Is there any potential conflict between the 2 mapping, with an undefined behavior?

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: Nemoweb (3:633/280.2@fidonet)

Le 07/04/2024 … 15:49, Lew Pitcher a ‚crit :

On Sun, 07 Apr 2024 13:34:43 +0000, pehache wrote:

Hello,

Hi, pehache

When memory mapping a file with the MAP_PRIVATE flag, the modifications
(writes) only exist in memory and are not written back to the file.

[snip]

So: is there a way to write the changes back to the file?

[snip]

The comp.unix.programmer newsgroup will be of better help for this sort
of question (it's not really on-topic for comp.lang.c).

But, of the top of my head; with my limited understanding of the unix
mmap() kernel call, the only ways to "write the changes" are to either
1) mmap(MAP_SHARED) and modify the mapped area, or
2) write() the mapped area back to the file.

I'm currenly using the first option in my code. Either way, the whole
mapped area is written back, even if there were only a few changes. I was hoping there could be another way to wrtite back only the modified pages.

But, as I said, comp.unix.programmer will be of better help.

fu2

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: Nemoweb (3:633/280.2@fidonet)

On Sun, 07 Apr 24 15:18:47 +0000, pehache wrote:

void* p2 = mmap( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_SHARED | MAP_NORESERVE
, fd
, 0 );

Not easy to remember what the arguments mean. Try this:

void * p2 = mmap
(
/*addr =*/ NULL,
/*length =*/ n,
/*prot =*/ PROT_READ | PROT_WRITE,
/*flags =*/ MAP_SHARED | MAP_NORESERVE,
/*fd =*/ fd,
/*offset =*/ 0
);

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: A noiseless patient Spider (3:633/280.2@fidonet)

On 2024-04-08, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

On Sun, 07 Apr 24 15:18:47 +0000, pehache wrote:

void* p2 = mmap( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_SHARED | MAP_NORESERVE
, fd
, 0 );

Not easy to remember what the arguments mean. Try this:

Nobody is going to follow your grotesque, poorly considered coding
conventions.

void * p2 = mmap
(
/*addr =*/ NULL,

^^^^ ^^^^ a null pointer is some kind of "addr"

/*length =*/ n,
/*prot =*/ PROT_READ | PROT_WRITE,

^^^^ ^^^^
| |
`---------|---- You know this
`---- from this.

/*flags =*/ MAP_SHARED | MAP_NORESERVE,

^^^^
Tells you nothing. You know that preprocessor constants
combined together with | are "flags".

/*fd =*/ fd,

^^ ^^
identical, tells you nothing.

/*offset =*/ 0
);

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: A noiseless patient Spider (3:633/280.2@fidonet)

On 08/04/2024 02:50, Lawrence D'Oliveiro wrote:

On Sun, 07 Apr 24 15:18:47 +0000, pehache wrote:

void* p2 = mmap( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_SHARED | MAP_NORESERVE
, fd
, 0 );

Not easy to remember what the arguments mean. Try this:

void * p2 = mmap
(
/*addr =*/ NULL,
/*length =*/ n,
/*prot =*/ PROT_READ | PROT_WRITE,
/*flags =*/ MAP_SHARED | MAP_NORESERVE,
/*fd =*/ fd,
/*offset =*/ 0
);

That's great but, how did you manage to figure out the meanings and
order of the parameters in order to be able to write those comments?

If you have 1000s of calls to such functions, will you have those
comments plastered everwhere?

In a language with proper named/keyword arguments, you don't need to
remember the order. You don't need to supply all the arguments (eg. addr
and offset in your example can have defaults, possibly those sets of
flags too).

If you get an argument name wrong, it will tell you. I suspect that if
you wrote this by mistake:

/*prot =*/ MAP_SHARED | MAP_NORESERVE,
/*flags =*/ PROT_READ | PROT_WRITE,

it would not be detected.

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: A noiseless patient Spider (3:633/280.2@fidonet)

Le 07/04/2024 … 23:49, Keith Thompson a ‚crit :

James Kuyper <jameskuyper@alumni.caltech.edu> writes:

On 4/7/24 15:09, pehache wrote:

Le 07/04/2024 … 15:49, Lew Pitcher a ‚crit :

...

But, as I said, comp.unix.programmer will be of better help.

fu2

Why does that advice annoy you? Why wouldn't you want to take your
question to the forum where people are best qualified to answer it?

I believe "fu2" refers to this header line in the article you're
replying to :
Followup-To: comp.unix.programmer

That's correct.

pehache: Apparently James thought "fu2" meant ... something else.

OK, sorry for the misunderstanding :( !

"fu2" is a routinely used abbreviation for "Followup-To" (and just that!)
on the fr.* hierarchy, and I thought it was kind of universal on all hierarchies. Apparently it's not...

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: Nemoweb (3:633/280.2@fidonet)

On Mon, 8 Apr 2024 12:14:30 +0100, bart wrote:

... how did you manage to figure out the meanings and
order of the parameters in order to be able to write those comments?

I read the docs.

If you have 1000s of calls to such functions, will you have those
comments plastered everwhere?

Yes.

In a language with proper named/keyword arguments, you don't need to
remember the order.

True. I take advantage of that--and defaults for omitted arguments--in
Python.

But there are people in this noisegroup who don’t like me mentioning that.

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: A noiseless patient Spider (3:633/280.2@fidonet)