Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: expose additional effect settings #294

Open
rbergen opened this issue May 31, 2023 · 15 comments
Open

Feature: expose additional effect settings #294

rbergen opened this issue May 31, 2023 · 15 comments

Comments

@rbergen
Copy link
Collaborator

rbergen commented May 31, 2023

Recently, the ability was added for effects to expose settings that can be changed via the API. A few basic settings have been exposed by LEDStripEffect; the PatternSubscribers effect adds two more for itself.

Obviously, there are more effects for which it makes sense to expose properties as settings that can be changed via the device API and (later) the device web UI.

When working on this:

@Louis-Riel
Copy link
Contributor

Louis-Riel commented Jun 4, 2023

I'm POCing something to save UI effect settings in the browser storage facilities. In this POC, I'm setting an effect image URL option. This is what the effect image feature looks like:
image
The little gear opens a config dialig:
image
The favicon.ico is the default, and the one in the screenshot is the overridden value:
image

So all and all, this is more for the UIness of this parameter configuration, rather than using browser storage to save effect params. This is not the best, as this parameter will only be effective for the effect on the browser you set the option in. I'm looking to use browser storage to save site options like color schemes and things of the sort. So maybe this effect image url option could be in the chips effect config, and if that is so, the dialog approach can still be used and the prefs saved to the chip.

@rbergen
Copy link
Collaborator Author

rbergen commented Jun 7, 2023

@Louis-Riel Would you envisage using this UX for other chip-stored effect settings as well? Because that's what this issue is primarily about.

Concerning the image per effect: I can definitely see that being a chip-stored effect setting, provided we indeed make it a URL (not an actual image upload that is stored to SPIFFS, for instance.)

@robertlipe
Copy link
Contributor

If a setting is changed, how is the effect notified? Is it torn down and recreated (ouch!) where it should pick the config settings out of the received JSON or is there intended to be some kind of notification that emits a std::vector to some kind of receiver that you've registered? An event would allow interactive tuning where the user beats on the up and down keys (or tweaks a slider or whatever) while nuking and restarting the animation would be much more jarring.

To the notion of global settings "effect speed" is a common pattern in the ones I worked on. One of our competitors (and I still can't find who because there's so much copying ...) defined Brightness, Speed, and Scale for every effect, but the result was a UX nightmare as effects would rob bits and overload things to weird meanings. "Scale is 0-100, but odds mean that PacMan is self-driving and the lower digit controls the number of ghosts and the top digit controls the maze/map level". I think it was meant to be drive by an IR remote but I just can't imagine it was usable.

How do our settings interact with the IR remote?

Speed from 0-100 is something we can all feed into a map and use. It's more than just frame rate. Brightness (#284) is probably another global setting that we should broadcast down to the events.

Both of those would be nicely represented by actual keys on remotes like https://www.amazon.com/dp/B0C1Z44XVW - They're also good pleas to NOT teardown and reconstruct the animation.

@rbergen
Copy link
Collaborator Author

rbergen commented Jul 25, 2023

There are two types of settings:

  • Device settings. These are kept in the DeviceSettings class (declared and implemented in devicesettings.h and devicesettings.cpp), which is made available via g_ptrSystem->DeviceSettings(). Notifications of updates to these settings are not pushed to effects that use them. This means that effects that rely on device settings for their own configuration currently need to pull and check changes to device setting values at intervals that make sense to them. An example of an effect doing this is PatternWeather, which gets a number of settings from DeviceSettings. An effect that just pulls a setting it needs from DeviceSettings whenever it needs it is PatternPongClock.
  • Effect settings. Changes to these are pushed to an individual effect directly; LEDStripEffect includes an overloadable SetSetting() member function for this. Examples of an effect that adds its own effect settings to the standard set (that being friendlyName, maximumEffectTime, the read-only hasMaximumEffectTime, and the write-only clearMaximumEffectTime) is PatternSubscriber.

Both DeviceSettings and individual effects should publish the specifications for all settings they keep in a list of SettingSpec instances, and return a list of std::reference_wrappers to them, when asked. Both DeviceSettings and LEDStripEffect include a GetSettingSpecs() member function that can be used as the starting point for exploring exactly how this is currently set up.
The SettingSpecs are published via a number of endpoints in the on-board web server's API, for consumption by JSON-savvy humans and/or a semi-dynamic (web) interface. The endpoints are documented in REST_API.md.

I didn't look into it in much detail, but I'd say that things like Speed, Brightness, Scale, etc. are candidates for effect settings. This is also why these properties are persisted to JSON on a per-effect basis.

The IR remote does most of its interaction with EffectManager, via the RemoteControl class - and that class' handle() function as implemented in remotecontrol.cpp specifically. I don't think there's currently any direct IR remote interaction with individual effects. In either case YMMV, but I remember finding the code in RemoteControl::handle() very digestible.

@robertlipe
Copy link
Contributor

robertlipe commented Jul 25, 2023 via email

@davepl
Copy link
Contributor

davepl commented Jul 25, 2023 via email

@robertlipe
Copy link
Contributor

robertlipe commented Jul 25, 2023 via email

@davepl
Copy link
Contributor

davepl commented Jul 25, 2023 via email

@robertlipe
Copy link
Contributor

robertlipe commented Jul 25, 2023 via email

@davepl
Copy link
Contributor

davepl commented Jul 25, 2023 via email

@rbergen
Copy link
Collaborator Author

rbergen commented Jul 25, 2023

Thank you. This is helpful. Since I have a BUNCH of effects in L1 cerebral cache right now, I was thinking about the interactions.

Circling back to this comment, I think I may have caused some confusion with my earlier comment. In an attempt to clear that up:

  • SettingSpec objects are "just" metadata specifications for the actual settings that can be be retrieved and set at the device or effect level. They are meant to tell a future web interface what settings exist for an effect, what their description is, what's their type and if there are lower and upper boundaries to their values. As we seem to be focusing on effect settings in this conversation, I will not mention device settings any further. In any case, the key thing here is that the SettingSpec blobs do not play any role in the actual retrieval or changing of settings.
  • The actual setting( value)s for an effect can be retrieved with LEDStipEffect's SerializeSettingsToJSON() member function - which does yield something that's very much intended to be turned into a JSON blob. They can be set with the same class' SetSetting() member function. The latter can be used to set the value for one effect setting at a time. As this function is called on the effect class' instance when an effect setting's value is changed, the effect can decide on the spot what needs to be interpreted/reinitialized/redrawn/reset/etc.
  • The aforementioned member functions are exposed in the on-board web server's API using the Retrieve effect settings and Change effect settings endpoints.

How I see things is that any remote control's interactions with individual effects would not be routed through the web server's API, but through the RemoteControl::handle() function - and then somehow the member functions of LEDStripEffect I just mentioned. A first practical challenge I see there is finding remote control buttons that are available for use. On the remote controls I have at my disposal (those admittedly being the very simple ones that were "randomly thrown in" with some of the LED devices I acquired), the vast majority of buttons has already been assigned a function.

@robertlipe
Copy link
Contributor

robertlipe commented Jul 28, 2023 via email

@robertlipe
Copy link
Contributor

@davepl , going back to #294 (comment)

We have some dependency issues that make this not quite mechanical. For example, patternBounce is using the _boids that are in g(). (I didn't even know that g() had Boids[]. I just created and managed my own, such as https://github.com/robertlipe/NightDriverStrip/blob/469e24808063a42bc40c2bd10adad9690e0de142/include/effects/matrix/PatternSMBoidExplosion.h#L31 - maybe those should be psram allocated, but that's not for here. It's a weird thing to stash in gfxbase.)

So we can't just blindly move the bodies of Starts into Inits() that are called by all the ctors. Maybe you know the precise construction order, but for whatever reason, g() is invalid at constructor time.

Likewise, we have a bunch of g()->Clear calls in Starts. Maybe those shouldn't be there at all to allow cross fades, but it's not UNreasonable for an effect to want to start with a clear screen. (Some of the ones I worked on really do require it because they immediately start smearing or blobbing the contents of the pixel buffer and droppings from a previous effect weird them out.) Other existing effects are accessing g()->GetNoise() inside Start - that won't work in a constructor and it would be unsightly to do it on an effect change. PongClock takes a similar nullptr deref if we move that up to a ctor, but it's doing work that needs rethought in an effects system anyway. I didn't analyze the one in Spectrumeffects, but it probably will crash too - I just didn't get through all the other crashes, I think.

Of the existing eleven Starts, eight of them would cause visual weirdities if called at runtime. This is why we were trying to push them into the ctors.

Therefore, it seems reasonable to have an entry point that's after the ctors, after we have a global g() state, and before the first Draw(). I propose we call that entry point Start() :-).

That puts us needing a new entry point in the effects. It's going to need an argument, so we'd be changing the footprint of Start even if we got through the above. I propose an OnSettingsChanged(SettingSpec[] specs)

I suppose an alternative, if you really want Start() to mean both "effect is ready to run and, well Start" and to mean "there has been an effects change" that most Starts will need to maintain an internal flags for first invocation and treat itself specially on that invocation before the first Draw (so it can access g()->GetNoise or whatever) and then act differently and look at its new argument on subsequent calls. I'm not a fan of that "on the first call to read() call open()" kind of design.

Again, I'm willing to do or help with the work. I'm just saying that (my understanding of) the proposal in #294 (comment) seems to hit an unexpected iceberg. The Rest interface (and Remote) "just" need to emit/broadcast these for speed and brightness and probably just the Rest interface (which I hope is used by the web and mobile apps) need to emit it for specific changes in the app.

Is this a reasonable path forward on the top-level goal in this issue?

P.S. I've gotta get a debugger on this thing. Debugging constructor crashes from the assembly - on a processor I don't really speak - is no fun. Is a JTAG pod the way to go? That'd help upload speeds, but would really hurt my portability.

@robertlipe
Copy link
Contributor

Oof, but we might have USE_REMOTE but not USE_WEBSERVER and include/webserver.h is where all the SettingSpec manipulation is done.

I don't know enough about how remote and REST/web interact to propose an API that's ideal. Rutger, is that your area?

@rbergen
Copy link
Collaborator Author

rbergen commented Jul 29, 2023

To be frank, I'm not sure why we are talking about a triangle that has the internal API (as provided by LEDStripEffect), the web server and the IR remote as its corners. In my view:

  • The LEDStripEffect::SerializeSettingsToJSON and LEDStripEffect::SetSetting member functions are the actual C++ API (or ABI, if you will) for getting and pushing settings values from and to an individual effect. The name SerializeSettingsToJSON is actually mildly misleading. The settings end up in a JSONObject, which is not (yet) a JSON blob. It's more of a "dynamic object structure", with the clever conversions to and from a range of (primitive) types one would want. Those can also be extended with conversions to and from more complex types; this has already been done for CRGB and CRGBPalette16 in jsonserializer.h.
  • What the web server does is "convert" the internal API to and from the language the Internet speaks, i.e. a semi-RESTful API. This is where JSONObjects get converted to and from the actual JSON blobs. Also, the SettingSpecs are externally published by the web server, to aid in the "dynamic implementation" of code that allows human users to interpret and change the settings - this is what the SettingSpecs were created for.
  • If we want the IR remote to modify effect settings, then I think RemoteServer should interact directly with the functions mentioned in the first bullet. It should most definitely remain totally unaware of the fact that the web server even exists. In this situation, the SettingSpecs objects could be used to see if the current effect publishes a particular setting that RemoteServer is aware of, and manipulate its value when a particular button is pressed.

So: the core C++ API for changing effect settings is provided by LEDStripEffect, the webserver takes care of translating that to and from "web lingo", and the IR remote server would do the translation from IR button codes. SettingSpecs can be used to discover if a particular effect publishes a particular setting.
Concerning the latter point, as I'm typing this I'm inclined to say that RemoteServer could probably blindly invoke SetSetting when a particular effect setting button is pushed on the remote. SetSetting's declaration is such that it accepts a const String& setting name and a const String& setting value, and should just ignore any name/value combinations it doesn't understand. This is what the current implementations in LEDStripEffect and PatternSubscribers do as well.

If pulling the value for a particular setting of an effect from JSONObject is too cumbersome, then we could extend the LEDStripEffect API/ABI with a "GetSetting" function that accepts a const String& name and a String& value as parameters. It could then return a bool to indicate if it actually recognized the setting name, and thus put the setting value in the second parameter.
That would be a pretty simple thing to add. It does mean that an effect that exposes settings has to return them in two ways - through the JSONObject and individually. That we could then address by only implementing GetSetting in the effect, and moving the composition of the JSONObject to the web server. The reason I'm reluctant to do that is that it would increase the number of function calls between the webserver and the effect - but maybe I'm worrying too much about the impact of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants