Which storage option should I choose and how would that affect performance?

[sagikazarmark]

I'd like to setup a HA Harbor instance mostly following this guide and I'm thinking about choosing S3 over PVs for storage.

Does anyone have any metrics/measurements about the two? Is there a measurable difference in performance? I'd assume that PVs would be faster (obviously), but I don't know if it would be noticeable given there is Harbor in between.

For Helm charts, I've used Chartmuseum in the past and it works perfectly with S3 as a storage, but those are relatively small files and are not accessed in an environment where performance matters that much (user computer, CI, CD, etc).

Container images on the other hand are pulled from various places where performance could matter (eg. production Kubernetes clusters).

Appreciate any insights. Thanks!