New plugin type: Thin jar with POM

[janhoy]

This is an early proposal for a third plugin type. Currently pf4j supports two types: Fat/shade/one-jar and ZIP with lib folder.

The new plugin type would be a thin jar with only plugin code, and a list of dependencies in the form of Maven coordinates (e.g. a pom.xml file or a gradle file or simply a list of coordinates). Pf4j would, once plugin type is identified, resolve dependencies from Maven repos and download the jar dependencies to a myplugin-1.0.0.lib folder next to the plugin itself. A corresponding PluginLoader will then add this lib folder to the class loader.

Identification of this plugin type could be through a special name suffix (*.pf4j), or some hint in plugin descriptor/manifest. The dependency resolution and download could either happen during installation (pf4j-update) or in run-time at plugin load phase; this should be configurable. For dependency resolution we can use the maven-resolver library.

Alternative: If we choose to do this in pf4j-update only, another option could be to download into lib folder and then re-package everything as a .zip plugin which would then be installed into plugins root. This would not need any changes in core at all, but a disadvantage is that core project will not understand this plugin type natively.

[decebals]

The idea is good. I think that I will come with proposes.

[janhoy]

The simplest to start with would be to add a new key to MANIFEST: mvn-dependencies=package:component:version, package:component:version... where you could list the coordinates. But it would also be cool to support, say mvn-dependencies-file=META-INF/{pom.xml,ivy.xml,build.gradle} as well as mvn-dependencies-transitive=false|true

[decebals]

@janhoy If you don't started already to write some code, I am ready to try to create a POC.

[janhoy]

Not started, feel free to give it a shot!

[sashank]

Is this ready ?

[decebals]

@sashank
I started work but nothing functional yet. I didn't know that it's a real interest for this feature, for me it's like a research work.

[decebals]

After I played a little bit with this concept, my opinion is that it doesn't put enough value on table.
I prefer actual plugin packages (fat jar and zip with classes and lib). I wish to have all information/resources of plugin in a single place. The concept presented in this issue only move the dependencies (external libraries) outside of plugin, that dependencies are resolved lazy, at runtime. I think that is much better to resolve these dependencies at build time (of plugin) and to encapsulate all the plugin's resources in the the plugin's pack (zip or jar). Using Maven repository at runtime to resolve the plugin's (external) dependencies comes with new points of fail (define Maven repo, have connection to internet, set http proxy, ...) and I don't see a real benefit.

I found few projects that try to use the same concept (more or less):

• https://github.com/nanosai/modrun
• https://github.com/microsofia/microsofia-boot

but these projects didn't convince me about utility of this concept.

It's relative easy to implement this concept in PF4J but I want to see the advantages before I add this feature.
My two cents. If you have other opinion please let me know.

[HJacco]

It's so easy to get a thin jar that only contains th plugin code, you just only put these two maven plugin into you pom file and then run maven command

<plugin>
    <artifactId>maven-jar-plugin</artifactId>
    <version>2.4</version>
    <configuration>
        <archive>
            <manifest>
                <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
                <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
            </manifest>
            <manifestEntries>
                <Plugin-Id>${plugin.id}</Plugin-Id>
                <Plugin-Version>${plugin.version}</Plugin-Version>
                <Plugin-Provider>${plugin.provider}</Plugin-Provider>
                <Plugin-Class>${plugin.class}</Plugin-Class>
                <Plugin-Dependencies>${plugin.dependencies}</Plugin-Dependencies>
            </manifestEntries>
        </archive>
    </configuration>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>2.5.1</version>
    <configuration>
        <annotationProcessors>
            <annotationProcessor>org.pf4j.processor.ExtensionAnnotationProcessor</annotationProcessor>
        </annotationProcessors>
    </configuration>
</plugin>

[janhoy]

Using Maven repository at runtime to resolve the plugin's (external) dependencies comes with new points of fail (define Maven repo, have connection to internet, set http proxy, ...) and I don't see a real benefit.

I'm aware that delayed download has its problems, and we should not assume that all users of pf4j are able or willing to have their application connect to the internet for downloading dependencies. They may have security policies that prevent that etc. However, pf4j-core could very well support the new plugin format, and simply require that the myplugin-1.0.0.lib folder already exists, if not don't load the plugin.

That leaves the resolving and downloading part to either pf4j-update or some command-line tool folks can run after copying the jar into plugins folder, e.g. java -jar installDeps.jar /my/plugins/path, or include in core but disabled as default resolve.thinjar.deps=false.

The benefits I see with such a new plugin type are

• Much smaller plugin size
• Explicit dependency declarations, which could be used by host application to preemptively detect colliding or blacklisted deps
• Principle of DRY - no need to store 1000 copies of commons-lang.jar when we know that it's in maven

Of course you make valid points that it must be unambiguous what maven repo to use, and downloads must be checksum validated, but I think that's part of the maven-resolver library already.

[JiaRG]

I think some implementations of this project can help you.😀
https://github.com/apiman/apiman/tree/master/common/plugin/src/main/java/io/apiman/common/plugin

[decebals]

@JiaRG
I fixed the link's url in your comment. I don't understand how apiman can help us, specially on this issue/feature.
The new plugin type presented in this page is interesting, but because I don't have a clear use case for it in the near feature, I postponed the implementation.

[Purus]

Having plugins based on Maven artificats is that in a large organizations, it makes easy to manage the plugins in a centralized location. Individual members does not need to download and keep plugins in multiple versions. But when from Maven, just changing the version number fetches the new version.

[decebals]

I will try to keep thinking about this feature and maybe in the end we will have POC.
I think it is better to implement this concept in a new `pf4j-maven 'project (or as a subproject in the actual project) in order not to affect the core module (size, dependencies, ...).