pf4j plugin needs to refresh and detect changes automatically with spring

[sme124]

Hello,

I am building a system where it requires someone to be able to add/remove, just adding the plugins in a specific folder. All plugin systems should detect that and work without refreshing or rebuilding the jars.
Currently, we have to rebuild the plugins, need to add updated plugins into our plugin container, and re-run that container to detect the changes.

Could you please suggest some ways to achieve this?

[decebals]

No need to rebuild the main project jars, in any circumstances.
It's a possibility to add, remove, enable and disable the plugins at runtime via pf4j's API or pf4j-update [1]. In any case you don't need to touch/rebuild the code/project.
In certain situations it may be necessary to restart the application after some plugin changes (the same approach is present in Eclipse - OSGI based, IDEA IntelliJ, ...).

Maybe I don't understand well this part with "rebuild the plugins", can you add more details (why is necessary to rebuild the plugins, ...)?

[1] https://github.com/pf4j/pf4j-update

[sme124]

Hello @decebals ,

Thank you for the guidance. We have tried to achieve auto refreshment for plugins without rebuilding/restarting the server but facing an issue while refreshing those plugins with registering their endpoints to access their functionalities.
We have tried to use pf4j-spring along with pf4j-update, and below is the code that refreshes (add/updating) the plugins from the given plugin repo.

here is the code for that along with what we getting after adding new plugin with this code -

public Mono<APIResponse> refreshPlugin() throws MalformedURLException {
      DefaultUpdateRepository defaultUpdateRepository = new DefaultUpdateRepository("folder", new URL("file:///F:/var/downloads/"));
      // We can provide repo by adding list in update manager
      // create update manager
      UpdateManager updateManager = new UpdateManager(pluginManager, List.of(defaultUpdateRepository));

      // >> keep system up-to-date <<
      boolean systemUpToDate = true;

      // check for updates
      if (updateManager.hasUpdates()) {
          List<PluginInfo> updates = updateManager.getUpdates();
          log.debug("Found {} updates", updates.size());
          for (PluginInfo plugin : updates) {
              log.debug("Found update for plugin '{}'", plugin.id);
              PluginInfo.PluginRelease lastRelease = updateManager.getLastPluginRelease(plugin.id);
              String lastVersion = lastRelease.version;
              String installedVersion = pluginManager.getPlugin(plugin.id).getDescriptor().getVersion();
              log.debug("Update plugin '{}' from version {} to version {}", plugin.id, installedVersion, lastVersion);
              boolean updated = updateManager.updatePlugin(plugin.id, lastVersion);
              if (updated) {
                  log.debug("Updated plugin '{}'", plugin.id);
              } else {
                  log.error("Cannot update plugin '{}'", plugin.id);
                  systemUpToDate = false;
              }
          }
      } else {
          log.debug("No updates found");
      }

      // check for available (new) plugins
      if (updateManager.hasAvailablePlugins()) {
          List<PluginInfo> availablePlugins = updateManager.getAvailablePlugins();
          log.debug("Found {} available plugins", availablePlugins.size());
          for (PluginInfo plugin : availablePlugins) {
              log.debug("Found available plugin '{}'", plugin.id);
              PluginInfo.PluginRelease lastRelease = updateManager.getLastPluginRelease(plugin.id);
              String lastVersion = lastRelease.version;
              log.debug("Install plugin '{}' with version {}", plugin.id, lastVersion);
              boolean installed = updateManager.installPlugin(plugin.id, lastVersion);
              if (installed) {
                  log.debug("Installed plugin '{}'", plugin.id);
                  pluginManager.startPlugin(plugin.id);
                  injectExtensions(pluginManager.getPlugin(plugin.id));

              } else {
                  log.error("Cannot install plugin '{}'", plugin.id);
                  systemUpToDate = false;
              }
          }

          pluginConfig.registerMvcEndpoints(pluginManager);
          RouterFunction<ServerResponse> routes = pluginManager.getExtensions(PluginInterface.class).stream()
                  .flatMap(g -> g.reactiveRoutes().stream())
                  .map(r -> (RouterFunction<ServerResponse>) r)
                  .reduce((o, r) -> (RouterFunction<ServerResponse>) o.andOther(r))
                  .orElse(null);

          if (routes != null)
              pluginEndpoints.andOther(routes);



      } else {
          log.debug("No available plugins found");
      }

      if (systemUpToDate) {
          log.debug("System up-to-date");
      }

      return userHelper.mappedToResponse(null);
  }

  public void injectExtensions(PluginWrapper plugin) {
      Set<String> extensionClassNames;
      log.debug("Registering extensions of the plugin '{}' as beans", plugin.getPluginId());
      extensionClassNames = pluginManager.getExtensionClassNames(plugin.getPluginId());
      Iterator var5 = extensionClassNames.iterator();

      while (var5.hasNext()) {
          String extensionClassName = (String) var5.next();

          try {
              log.debug("Register extension '{}' as bean", extensionClassName);
              Class<?> extensionClass = plugin.getPluginClassLoader().loadClass(extensionClassName);
              this.registerExtension(extensionClass);
          } catch (ClassNotFoundException var8) {
              log.error(var8.getMessage(), var8);
          }
      }
  }

  public void registerExtension(Class<?> extensionClass) {
      Map<String, ?> extensionBeanMap = pluginManager.getApplicationContext().getBeansOfType(extensionClass);
      if (extensionBeanMap.isEmpty()) {
          Object extension = pluginManager.getExtensionFactory().create(extensionClass);
          AbstractAutowireCapableBeanFactory beanFactory = (AbstractAutowireCapableBeanFactory) pluginManager.getApplicationContext().getAutowireCapableBeanFactory();
          beanFactory.registerSingleton(extensionClass.getName(), extension);
      } else {
          log.debug("Bean registeration aborted! Extension '{}' already existed as bean!", extensionClass.getName());
      }

  }

Please guide us on, whether it is possible to get plugin features like that.

[decebals]

You add some business/logical code (RouterFunction, pluginConfig, ..) which makes it difficult for me to read and understand.
Try with a simple (quickstart [1]) project. Try to debug a little bit the application for more context. See troubleshooting [2] section if you encounter problems. It's not clear for me if your extensions are discovered correctly, if the extensions.idx contains information about extensions. As I said, try to investigate a little bit your application and come with concrete issues (using this small code in isolation, I observed this issue ..).

[1] - https://pf4j.org/dev/quickstart.html
[2] - https://pf4j.org/doc/troubleshooting.html

[sme124]

@decebals,
As per your suggestion, we have investigated our application. In our application, it is able to get plugins along with their features too at the time of initializing.

• When I refresh it with the "/plugins/refresh" endpoint, it refreshes it by downloading or updating plugins. But when I try to access the endpoint defined in plugins, it gives a 404-not found error.
• So, as per my understanding it does not register endpoint came from the plugin, as the plugin is installed by refreshing. But in the same state, if I restart the application plugin endpoint is accessible.

In the above zip project spring-plugin-container is the application in which I have added code to refresh the plugin. It has the endpoint "/plugins" through which I can check registered plugins.

References :

• https://github.com/ralemy/pf4j-spring-tutorial
• https://ralemy.github.io/pf4j-spring-tutorial/spring-plugin-container/

pf4j-demo.zip

[gurucube]

@decebals,
We are now able to add and remove plugins, use them inside our application using pf4j-spring (upgraded) we can enable and disable them managing the JSON file so we can use all functionalities, also we can update plugins from a remote repo using pf4j-update so all is working BUT we still not able to use it at runtime. We need that all this changes happen without to have to restart the entire app.
SpringBoot mapping must be updated with the new endpoint (RouterFunction) defined in the plugin so basically we need to add reactive routing at run time.
Have you got some examples or point us to the solution?
Thank you in advance

[decebals]

BUT we still not able to use it at runtime. We need that all this changes happen without to have to restart the entire app.

I understand your intention but I don't understand what's stopping you. It 's probably the following statement:

SpringBoot mapping must be updated with the new endpoint (RouterFunction) defined in the plugin so basically we need to add reactive routing at run time.

I remember seeing a project on Github that uses PF4J to add new routes (a plugin adds routes in application).
Maybe #8 (comment) is useful for you. I don't work with Spring Web so I cannot help you here.