Hello, are there any update on this? Thanks! On Tuesday, January 25, 2022 at 11:17:48 PM UTC+1 Mike Kruskal wrote:
> We have no plans set to support arena-based strings yet. This would be > blocked on finishing the migration to string_view accessors, but *that* is > planned for 2022. > > On Tuesday, January 18, 2022 at 7:12:16 PM UTC-8 [email protected] > wrote: > >> Hi, I have a question. When arena-based strings are expected to be >> released ？ >> On Wednesday, September 9, 2020 at 6:17:26 AM UTC+8 [email protected] >> wrote: >> >>> Our StringPiece type has been made obsolete by the std::string_view type >>> introduced in C++17, so we will eventually get rid of StringPiece and >>> replace it with std::string_view (or absl::string_view, which has the same >>> API but is available in C++11). So if you are making a local modification >>> to support allocating strings on arenas, it would probably be best to go >>> straight to std::string_view and avoid our StringPiece type. To handle both >>> the arena and non-arena case, the simplest solution would be to store both >>> a std::string_view and a std::string. When arenas are used, you can have >>> the string_view point into arena-allocated memory, and when arenas are not >>> used, it can just point to the std::string's data. >>> >>> On Thu, Sep 3, 2020 at 3:19 AM X Ah <[email protected]> wrote: >>> >>>> Hi Feng, >>>> I have an API design problem, Since StringPiece doesn't own the data >>>> and it's impossible to get memory from it, So how should the behavior be >>>> if >>>> I do ParseFromString and the message is not in arena? I think the >>>> StringPiece field should be empty if current message doesn't own an arena, >>>> but it is strange for user. Could you introduce how Google internal use >>>> StringPiece in protobuf? >>>> Thanks! >>>> On Saturday, January 16, 2016 at 3:32:39 AM UTC+8 [email protected] >>>> wrote: >>>> >>>>> On Thu, Jan 14, 2016 at 6:06 PM, Austin Schuh <[email protected] >>>>> > wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I've got an application where I can't allocate memory while using >>>>>> protobufs. Arenas have been awesome for doing that. I'm able to >>>>>> allocate >>>>>> a big block of memory at startup time or stack allocate memory for the >>>>>> arena, and then use that for allocating protobufs. Thanks! >>>>>> >>>>>> I'd like to be able to allocate strings in the arena. I'm willing to >>>>>> do the implementation, and wouldn't mind up-streaming if my >>>>>> implementation >>>>>> is complete enough and there is interest. It looks like I should start >>>>>> by >>>>>> implementing ctype=STRING_PIECE and then allocate memory in the arena to >>>>>> back it. The class in //src/google/protobuf:arenastring.h looks like >>>>>> the >>>>>> place to do all the operations. It looks like I need to modify the >>>>>> interface to provide setters and getters to support STRING_PIECE there. >>>>>> >>>>>> Is that the right place to start? Is there any more guidance that >>>>>> you can give me? >>>>>> >>>>> Hi Austin, >>>>> >>>>> Thanks for contacting us and offering help! >>>>> >>>>> You are looking at the right direction. We actually already >>>>> opensourced the StringPiece implementation not very long ago: >>>>> >>>>> https://github.com/google/protobuf/blob/master/src/google/protobuf/stubs/stringpiece.h >>>>> >>>>> It's intended to be used to implement "ctype = STRING_PIECE" for >>>>> string fields and since it's merely a <const char*, size_t> pair, it can >>>>> be >>>>> directed at the buffer in the arena. Such features are implemented inside >>>>> Google but unfortunately it's not opensourced due to dependency issues. >>>>> We >>>>> plan to get them out eventually but hasn't have enough time to work on >>>>> it. >>>>> Since we already have an internal version of it, we probably won't be >>>>> able >>>>> to accept your contributions. I can't give a concrete timeline about when >>>>> we will get our implementation opensourced also. Sorry for that... >>>>> >>>>> If you need this soon, I suggest you try to implement it as simple as >>>>> possible. Better to only support lite runtime with arena enabled. Some >>>>> changes you want to make: >>>>> 1. Make ArenaStringPtr work with StringPiece, or introduce an >>>>> ArenaStringPiecePtr which might be easier to implement. >>>>> 2. Update protocol compiler to use ArenaStringPtr/ArenaStringPiecePtr >>>>> to store ctype=STRING_PIECE fields and expose a StringPiece API: >>>>> // proto >>>>> message Foo { >>>>> string bar = 1 [ctype = STRING_PIECE]; >>>>> } >>>>> // generated C++ code >>>>> message Foo { >>>>> public: >>>>> StringPiece bar() const; >>>>> void set_bar(StringPiece value); // Note that we need to do a deep >>>>> copy here because StringPiece doesn't own the underlying data. >>>>> void set_alias_bar(StringPiece value); // Make the field point to >>>>> the StringPiece data directly. Caller must make sure the underlying data >>>>> outlives the Foo message. >>>>> >>>>> private: >>>>> ArenaStringPiecePtr bar_; >>>>> }; >>>>> >>>>> Look at the string_field.cc implementation in the compiler directory >>>>> <https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/cpp/cpp_string_field.cc> >>>>> >>>>> and you can create a string_piece_field.cc implementation based on that. >>>>> Most of the work will be done here, including not only the generated API >>>>> but also all the parsing/serialization/copy/constructor/destructor >>>>> support. >>>>> >>>>> That's pretty all that needed to support StringPiece in lite-runtime + >>>>> arena. A lot more work will be needed to support other combinations >>>>> (lite-runtime + no arena, full-runtime + arena, full-runtime + >>>>> non-arena), >>>>> but since you have a specific targeted platform and we will opensource >>>>> the >>>>> StringPiece support eventually, it's probably not worthwhile to invest >>>>> time >>>>> to support anything you don't actually need right now. >>>>> >>>>> Hope this helps. >>>>> >>>>> Regards, >>>>> Feng >>>>> >>>>> >>>>>> Thanks, >>>>>> Austin >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Protocol Buffers" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at https://groups.google.com/group/protobuf. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Protocol Buffers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/protobuf/ccc70ecc-873a-4297-8102-8e2319ffd760n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/protobuf/ccc70ecc-873a-4297-8102-8e2319ffd760n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/e517c272-340a-4f84-8500-76133a239089n%40googlegroups.com.
