Multi Agent model implementation proposal

[Tortar]

It is known that Agents.jl suffers from dynamic dispatch when working with a high number of agents types. I think that now that we have an abstract interface to rely on and to implement we can try to solve the issue. Currently, we just have some workarounds: representing every type in only one struct, but this is unnatural, and wastes some RAM (and it is probably not as performant as the possible implementation described below)

I tried to use Virtual.jl and ManualDispatch.jl and custom macros to solve the problem, but to no avail, even though the first package seems actually suited to improve the matter it didn't work well (performance was worse) when I tried to make use of it.

So I think we have only the hard way to move forward on the issue: implement all the interface for a model where the main data structure is a tuple where each type is inside its own Dict/Vector. This should be even better in terms of performance than compressing in only one struct different agent types without the need to change anything.

[Tortar]

I had an idea so that to make the access to the right container for a given id faster than other possible strategies, namely how could model[id] work for a model with a tuple of vectors/dict. The problem is identifying which is the container for a given id: the strategy I have thought is that we could try to set a new id with nextid such that if we have N types of agents we set it such that the remainder of the division by N gives the right value for the index of the container, this means that for example if we have 3 types and the id is 7 we can do mod1(7, 3) to identify the right container. The access to the agent then can be done with model.agents[mod1(7,3)][7]

However, this means that we can't use only nextid(model) to let the user set a new id since this is not enough to identify the right id to give to the new agent, we should extend it to nextid(model, agent_type).

[Tortar]

@Datseris I'd like to know what you think about all of this when you have the time :-)

[Datseris]

Problem is that you are still using the same type (the integers) to get agents of different type, so your idea would also be type unstable.

Limiting the id assinged to agents is totally fine for me. There is reallh no reason for users in the first place to care about the id, having it auto assigned would have been simpler from a design view point anyways, especially now with the improvements of @agent. I just don't see how this would help performance though.

[Datseris]

But then again, it may be much more performant to have a tuple of dicts, each with concrete type, rather than having a dict of Union type. But I truly do not know, we would need to test a minimal implementation (which really shouldn't be too hard).

[Datseris]

BTW, although this is a different issue, I have been thinking that in version 6 we should rework the model interface so that the stepping functions are included in the model. This is how the rest of hte packages that use step! operate: DynamicalSystems.jl / DiffEq: the evolution rules are given as functions when creating the central struct.

This would allow us to make a version where there is a different stepping function for each agent type (if that gives better performance), so that when iterating through agents we know what function to use with each and this way remove the necessity for dynamic dispatch.

[Tortar]

Limiting the id assinged to agents is totally fine for me. There is reallh no reason for users in the first place to care about the id, having it auto assigned would have been simpler from a design view point anyways, especially now with the improvements of @agent. I just don't see how this would help performance though.

It seems that doing so we could apply the speed-up described in https://discourse.julialang.org/t/poor-time-performance-on-dict/9656/14

[Datseris]

yes, that does seem promising.

[Tortar]

I found a blog post by Bogumił Kamiński which explains that a similar strategy can have huge benefits performance wise: https://juliasnippets.blogspot.com/2018/07/abc-of-abm-in-julia.html

[Tortar]

I had an idea so that to make the access to the right container for a given id faster than other possible strategies, namely how could model[id] work for a model with a tuple of vectors/dict. The problem is identifying which is the container for a given id: the strategy I have thought is that we could try to set a new id with nextid such that if we have N types of agents we set it such that the remainder of the division by N gives the right value for the index of the container, this means that for example if we have 3 types and the id is 7 we can do mod1(7, 3) to identify the right container. The access to the agent then can be done with model.agents[mod1(7,3)][7]

Another advantage of this is that we could (or the user could) group ids by type before interacting with the model with model[id] so that the operations can be type stable.

[Datseris]

Yes I have thought of that before already. Use the modulo operation. Module as many types. So .e.,g if 4 types, we cound Ids by modulo 4. So that all ids whose mod 4 == 1 is type 1, etc.

[Tortar]

ok, maybe this is a better solution:

https://discourse.julialang.org/t/what-materials-should-i-read-extending-the-compiler/106841/11

!!

somewhat related: #492

[Datseris]

right, i don't understand it but i hope you do!

[Tortar]

sorry wrong reply, I wanted to link this one: https://discourse.julialang.org/t/what-materials-should-i-read-extending-the-compiler/106841/9

[Tortar]

but probably you are referring to this one, yes it's still a bit unclear if suitable but I wanted to let you know :-)