Skip to content

P1883 std::file_handle and std::mapped_file_handle #633

@wg21bot

Description

@wg21bot
Collaborator

P1883R0 Walkthrough of P1031s file_handle for LEWG-I (Niall Douglas)

Activity

added this to the 2019-11 milestone on Oct 15, 2019
brycelelbach

brycelelbach commented on Nov 6, 2019

@brycelelbach

Belfast 2019-11 LEWGI Minutes, Day 1 (Tuesday)

P1883R0 std::file_handle: Direction Review (Tuesday)

Champion: Niall Douglas

Minute Taker: Ryan McDougall

Start Review: 11-05 13:45

Start Polling: 13:56

  1. Prioritize performance over ease of use.
  2. Designed around the POSIX race free filesystem path lookup extensions [POSIXext].
  3. Expose kernel i/o cache control and virtual memory control facilities.
  4. Support concurrent file i/o.
  5. Support mapped memory.

POLL: The above list should be our design goals for a modern C++ standard local I/O library.

Strongly For Weakly For Neutral Weakly Against Strongly Against
7 5 2 0 0

Attendance: 16

That has consensus.

POLL: We are comfortable with passing omnibus structs (like io_request) instead of individual parameters.

Strongly For Weakly For Neutral Weakly Against Strongly Against
1 4 5 1 1

Attendance: 14

Refer to ARG for review.

Break: 15:32

Resume: 15:56

path_handle p("/foo");
file_handle f1(p, "bar");

// Status quo
file_handle f0({}, "/foo/bar");

// With a new path_view_type constructor
file_handle f0("/foo/bar");

POLL: file_handle should be constructible from just a path_view_type (e.g. file_handle f0("/foo/bar")).

Strongly For Weakly For Neutral Weakly Against Strongly Against
0 3 5 5 3

Attendance: 18

There's consensus against that.

POLL: Uniquely named file generation APIs should support templating.

Strongly For Weakly For Neutral Weakly Against Strongly Against
1 6 6 3 2

Attendance: 21

That has no consensus.

  1. Priorities (for defaults): security > performance > ease of use.
  2. Designed around the POSIX race free filesystem path lookup extensions [POSIXext].
  3. Expose kernel i/o cache control and virtual memory control facilities.
  4. Support concurrent file i/o.
  5. Support mapped memory.
  6. No asynchronous i/o (for now).

POLL: The above list should be our design goals for a modern C++ standard local I/O library.

NO OBJECTION TO UNANIMOUS CONSENT

Attendance: 21

POLL: Based on what we've seen so far, we like the basic design of path_handle and file_handle from P1883, and we should promise more committee time to pursuing it, knowing that our time is scarce and this will leave less time for other work.

Strongly For Weakly For Neutral Weakly Against Strongly Against
7 9 1 0 0

Attendance: 21

That has strong consensus.

End: 17:31

Things this paper did well:

  • Limited our scope from what it originally was (P1031).

Things this paper hasn't done well:

  • Be short.
  • Be example-driven.

CONSENSUS: Bring a revision of P1883R0 (std::file_handle), with the guidance below, to LEWGI for further design review.

  • Remove constexpr from this proposal.
  • Dont allow clone to change the mode or caching, and add a reopen method that can change the mode or caching.
  • Don't depend on any novel error handling right now, like deterministic exception handling or expected.
  • Add handle method which returns the native handle.
  • Drop set_append_only; re-add it later if we need it.
  • Rename random_file to something like unique_file.
  • Make the static member functions free functions instead.

Refer to ARG for review of io_request as an ABI defense mechanism.

brycelelbach

brycelelbach commented on Nov 6, 2019

@brycelelbach

An idea we didn't poll on day 2: no default for deadlines, force people to always provide them.

ben-craig

ben-craig commented on Nov 6, 2019

@ben-craig

Belfast Day 2

Guest Chair: Ben Craig

Minutes: Guy Davidson

POLL: Do we want to provide the "extents()" function as presented

Strongly For Weakly For Neutral Weakly Against Strongly Against
1 6 5 2 0

Motion passes, current extents() function is acceptable

Attendance: 15

POLL: Do we want finite, non-zero duration deadlines by default for retrying operations (e.g. link, relink, maybe others)?

Strongly For Weakly For Neutral Weakly Against Strongly Against
1 4 3 3 3

No consensus for change in the paper

Attendance: 16

POLL: Do we want a default where the library makes a single attempt, without retries, for retryable operations (e.g. link, relink, maybe others)?

Strongly For Weakly For Neutral Weakly Against Strongly Against
2 3 4 3 1

No consensus for change in the paper

Attendance: 15

POLL: Do we want a non-infinite, implementation defined default for retryable operations (e.g. link, relink, maybe others)?

Strongly For Weakly For Neutral Weakly Against Strongly Against
1 2 5 4 1

No consensus for change in the paper

Attendance: 15

Suggestion: change retryable "deadline" parameters to "duration_to_retry"

Suggestion: in extent() function, return a vector of structs with good names, instead of vector of pair

Suggestion: [[nodiscard]] on the extent_guard constructor

Suggestion: unlock() on the extent_guard should not be noexcept by current LEWG policy

POLL: We don't want extent_guard locks that have the possibility of flag::byte_lock_insanity behavior

Strongly For Weakly For Neutral Weakly Against Strongly Against
4 0 3 4 1

No consensus for change in the paper

Attendance: 15

brycelelbach

brycelelbach commented on Nov 8, 2019

@brycelelbach

Belfast 2019-11 LEWGI Minutes, Day 2 (Wednesday)

P1883R0 std::file_handle: Design Review

Chair: Ben Craig

Champion: Niall Douglas

Minute Taker: Guy Davidson

Start Review: 11-06 13:35

End: 15:30

CONSENSUS: Bring a revision of P1883R0 (std::file_handle), with the guidance below, to LEWGI for further design review.

  • Change retryable deadline parameters to duration_to_retry.
  • In extent, return a vector of structs with good names, instead of a vector of pair.
  • Add [[nodiscard]] on the extent_guard constructor.
  • unlock on the extent_guard should not be noexcept by current LEWG policy.
removed this from the 2019-11 milestone on Dec 12, 2019
wg21bot

wg21bot commented on Jan 18, 2020

@wg21bot
CollaboratorAuthor

P1883R1 file_handle and mapped_file_handle (Niall Douglas)

added this to the 2020-02 milestone on Jan 18, 2020
ogiroux

ogiroux commented on Feb 10, 2020

@ogiroux
Collaborator

Is this paper replacing P1031 going forward? The SG1 feedback for P1031 isn't reflected in this P1883, so we won't be reviewing it. Just FYI.

ned14

ned14 commented on Feb 10, 2020

@ned14

@ogiroux Literally second page of P1883R1 there is a box full of changes entitled "Changes since R0 due to SG1 Concurrency feedback:"

I also have a proposed policy for SG1 to consider whether to recommend regarding naming and design of functions which have built-in timeouts due to denial of service attacks, which SG1 requested of me in Belfast. It incorporates my best understanding of the feedback from the room on that day.

P1883 does not replace P1031 going forward. It's just a breakout of two classes from P1031.

15 remaining items

added and removed
EWGEvolution
EWGIEvolution Incubator
on Jan 23, 2023
ned14

ned14 commented on Feb 7, 2023

@ned14

Errata on P1883R2 so I don't forget for R3:

  • A deadline snuck into the new clone_extents_to() which needs replacing with the usual chrono duration and time point overloads.

SG1 suggested that file_handle::buffer_type become span<volatile byte> to account for buffers returned by read() pointing into shared memory maps.

ben-craig

ben-craig commented on Feb 8, 2023

@ben-craig

2023-02-07 13:00 to 15:15 Issaquah Library Evolution Meeting

P1883R2: file_handle and mapped_file_handle

2023-02-07 13:00 to 15:15 UTC-8 Issaquah Library Evolution Minutes

Champion: Niall Douglas

Chair: Ben Craig

Minute Taker: Steve Downey

Start: 2023-02-07 14:05 UTC-8

Does this paper have:

  • Examples? Yes
  • Field experience? Yes
    • Implementation experience? Yes
    • Usage experience? Yes
    • Deployment experience? Yes
  • Performance considerations? Yes
  • Discussion of prior art? Yes
  • Changes Library Evolution previously requested? WIP
  • Wording? No
  • Breaking changes? No
  • Feature test macro? No
  • Freestanding? TBD
  • Format and/or iostream support? TBD

End: HH:MM

Summary

This was the first time this paper has been seen by LEWG (it has previously been seen by LEWGI). The author talked about the high level design and walked through some of the examples.

POLL: When opening a file, require the user to specify something like current_working_directory(), rather than use a default constructed handle {} as short-hand for current_working_directory() (the name is not part of the poll)

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
5 11 4 3 0

Attendance: 22 local, 7 remote

# of Authors: 1

Author Position: N

Outcome: consensus in favor

WA: Putting anything there besides default ctor is encouraging people to do the same thing elsewhere

Next Steps

Respond to the feedback, and await more LEWG time to further review the paper.

removed this from the 2023-02 milestone on Mar 31, 2023
deleted a comment from ned14 on May 22, 2023
deleted a comment from hatcat on May 22, 2023
ned14

ned14 commented on Jun 12, 2023

@ned14

@ogiroux In the lunch queue just passed you asked if SG1 was going to be seeing P1883 at the Varna meeting? I couldn't answer off the top of my head, so I said I'd go look up the paper history as P1883 hadn't been on my radar for the Varna meeting.

SG1 saw P1883R2 at Issaquah (https://wiki.edg.com/bin/view/Wg21issaquah2023/P1883R2Issaquah) and it requested a number of things from any future proposed normative wording (all of which I think were uncontroversial). At the same meeting, LEWG asked for an out-of-meeting telecon to review the proposed API of P1883, with @hatcat volunteering to organise that.

All my very limited spare bandwidth between Issaquah and Varna went into writing the proposed normative wording for P1028 status code, and thus I did no work on any of my other papers. @hatcat I believe was also unusually busy between these meetings, and did not have the spare bandwidth to organise the API review telecon.

I furthermore think there is probably no point in SG1 seeing P1883 until after LEWG has consumed and processed the results from the API review telecon, as only thereafter might the points raised in the SG1 review notes become applicable.

(Note that I think those notes from the wiki ought to be posted here for convenience, but I haven't done it as I'm unsure if I'm allowed to do so)

So tl;dr; there has been zero progress on P1883 since SG1 last saw the paper in Issaquah, and thus no need for SG1 to see P1883 at the Varna meeting.

Also be aware that due to conditions negotiated in my new work employment, I shall be pausing all WG21 activity for twelve months after this meeting, so I would not expect any further progress on ANY of my papers until this time next year Summer 2024. I would expect therefore no further progress on P1883 until Autumn 2024, and hopefully by 2024 @hatcat will have also freed up some spare cycles to do the API review telecon. So @ogiroux you might see this paper again early 2025 if all goes well, and that means P1883 most certainly will not be making the C++ 26 IS cutoff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    B3 - additionBucket 3 as described by P0592: material that is not mentioned in P0592C++29Targeted at C++29ISShip vehicle: ISLEWGLibrary EvolutionSG1Concurrencyneeds-revisionPaper needs changes before it can proceedready-for-library-evolution-meeting-reviewThis paper needs to be discussed at a Library Evolution meetingsize - hugepaper size estimate, biggest size

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @ned14@brycelelbach@ben-craig@cor3ntin@inbal2l

        Issue actions

          P1883 `std::file_handle` and `std::mapped_file_handle` · Issue #633 · cplusplus/papers