Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: support pass unique key to Keeper #25

Open
Ehco1996 opened this issue May 25, 2023 · 1 comment
Open

RFC: support pass unique key to Keeper #25

Ehco1996 opened this issue May 25, 2023 · 1 comment

Comments

@Ehco1996
Copy link

Hi, first of all thank you for the library!

In the process of trying to use it, we found that because the worker's unique id is achieved through sonyflake, the worker requires an integer between 0 and 255 to achieve the unique id when it is started

func CheckWorkerKey(key string) (int, error) {

In practice, this restriction would lead to reliance on external storage to generate the unique integer in multi-process situations, so I propose that the global unique key can be passed directly from the caller and the keeper component no longer generates the unique key based on number

There are two advantages to this change

  • the logic of the keeper component is lighter, no need to generate its own unique keys, no need to introduce sonyflake
  • the limit of less than 255 workers is lifted

If you agree to this change, I will provide PR in my free time

thanks

@ShiningRush
Copy link
Owner

ShiningRush commented May 25, 2023

Hi echo, thanks for your proposal, basically I agree with it.
I want declare two points:

  1. Worker key limited 0~255 is incorrect, sonyflake is support to use uint16(0~65536) as machine_id, this should be a bug of fastflow
  2. using worker key suffix as unique number is very convenient in k8s, because you can deploy your application by stateful set, it can auto generate pod name like stateful-set-0, stateful-set-1 ...

however, support a new way to generate id is LGTM. I think we can decouple id generator:

  • if client give a id generator, we could do not check the format of worker key any more
  • or else, nothing change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants