Custom Associative Container integration into entt meta

[bss-steen-mueller]

Hi,

I am trying to add my custom containers similar to std::(unordered_)map to the entt meta system. This mostly works quite straight forward. The custom containers do not use std (due to certain design decisions in there) and thus store the elements not as a pair but as a struct: struct Entry {KeyType key; ValueType value;}
The problem arises with the iterators. I can return an iterator from that but the part in https://github.com/skypjack/entt/blob/v3.13.0/src/entt/meta/meta.hpp#L1717 requires the returned type to support -> and have members named first and second. It works for std because std returns *std::pair<KeyType, ValueType>. For me, it gets more complicated because I can in theory just return *Entry but then the compiler does not find first on that. I could now rename my variable also in the struct but that is rather annoying because it does not match the rest of the naming scheme and I would find it rather odd that the meta system requires me to name my variables in certain way. I know that the required functions have also a requirement on the naming but these are additional functions which I can implement on top of the existing functions and group them together with comments. The first and second would leak into all implementations in the class. But it would be nice if I could somehow specify the used names for functions although I guess that will be kind of hard and a separate topic.
I could also return something like std::pair<KeyType, ValueType> from the iterator because the operator -> is unused and just implemented for entt::meta. This will fail to compile because the access in the meta.hpp is with -> and thus either requires a pointer or a type which supports that operator. If I try to return something like returning a pointer to std::pair, I need to store the pair on the iterator to make sure that it is still alive, making the iterator bigger than necessary.
I also tried a struct which operates kind of like a pointer by implementing -> but either *std::pair is returned which leads to the same lifetime issue or storing it on the iterator or having infinite recursion if the type itself is returned.
I was wondering how other people did solve this issue on custom containers.

[skypjack]

I see the point. Interesting problem actually. I did it for a couple of custom containers, namely the dense map and the dense set. However, they return a pair of references that fits the requirements. In fact, first and second are still a thing in this case, even if you don't have to store keys and values as pairs internally (and these containers don't do that).
In any case, I think that you can get the job done too with a meta-only layer that adapts your stuff to make things work as expected.
Note that basic_meta_associative_container_traits has begin and end functions that return a meta_associative_container::iterator. As an example:

static iterator begin(const meta_ctx &area, void *container, const void *as_const) {
    return container ? iterator{area, std::bool_constant<key_only>{}, static_cast<Type *>(container)->begin()}
                     : iterator{area, std::bool_constant<key_only>{}, static_cast<const Type *>(as_const)->begin()};
}

This is where you can hook into meta probably, by means of a specialization of basic_meta_associative_container_traits. For example, you can write a proxy iterator that returns a pair of references to key/value (or just a struct with two reference members named first and second, no need to use std::pair).

I can't write an example on the fly because, I mean, I'd need your container or a similar one and I don't have enough time today to pack something meaningful, sorry. However, let me know if it makes sense, then I can try to help a little further during the next days.

[bss-steen-mueller]

Thanks for the reply already, I did not expect to be that quick.
The containers are available here (picked one as an example):
https://github.com/ezEngine/ezEngine/blob/dev/Code/Engine/Foundation/Containers/HashTable.h
I guess I did wrong in the first place now because I tried to adapt the container to basic_meta_associative_container_traits instead of writing my own version of basic_meta_associative_container_traits.
I checked the dense_map implementation and if I understand it correctly, it seems to be something like this in the end:

    struct Pointer
    {
      std::pair<const KeyType&, const ValueType&> value;
      const std::pair<const KeyType&, const ValueType&>* operator->() const { return &value; }
    };

    Pointer operator->() const
    {
      return Pointer{.value = {Key(), Value()}};
    }

And I guess with a custom implementation of basic_meta_associative_container_traits, I could hide this in this conversion in there.

[skypjack]

You don't have to use std::pair though. Something like struct Proxy { KeyType &first; ValueType &second; }; is enough probably.
Let me know if you want to give it a try on your own or if you need more help before proceeding. 👍

[bss-steen-mueller]

I think I made it work, at least my unit tests are all successful now. I was slightly worried about the lifetime of the object which is returned by -> but I guess it is always used in the same expression, so it should be fine. Thanks for the help!

[skypjack]

Oh, ok, I'm glad it helped then. 🙂
Feel free to reach out if you have any other problems.

[skypjack]

I could now rename my variable also in the struct but that is rather annoying because it does not match the rest of the naming scheme and I would find it rather odd that the meta system requires me to name my variables in certain way.

Side note for the sake of completeness: the fact is that we're dealing with an associative container and meta needs a way to lookup the key and the value in the returned object.
Since first and second are the let's say most common names for this kind of container (or at least the standard ones as in those used by the standard), I decided to use first and second.
Specializing basic_meta_associative_container_traits is the way to get around my decision (because almost everything is customizable in EnTT), although it's a little more demanding because there is an albeit short specialization to be written.
However, a decision had to be made. I wouldn't have known how to implement it differently. 🤷‍♂️

[bss-steen-mueller]

I guess it totally makes sense looking at it from that perspective. Implementing things for std first is not a bad default to take.