Clarification on Implementation of Row-Level Security for Column-Based Tables in ParadeDB

[razakpm]

I've been exploring the capabilities of ParadeDB and am particularly impressed with its support for both row and column-based tables. This dual support offers a versatile approach to handling various data storage and retrieval scenarios, which is greatly appreciated.

However, I have a question regarding the implementation of Row-Level Security (RLS) within ParadeDB, specifically in the context of column-based tables. Understanding that RLS is crucial for ensuring data access controls and security, I'm curious about the following aspects:

• Implementation Details: Could you provide some insights into how ParadeDB implements RLS for column-based tables? It would be beneficial to understand the underlying mechanism or architecture that supports this feature.
• Performance Impact: Additionally, I'm interested in learning about any potential performance impacts when RLS is applied to column-based tables. Given the structural differences between row and column-based storage, how does ParadeDB maintain performance while ensuring security?
• Configuration and Usage: Lastly, could you offer some guidance on how to configure and use RLS with column-based tables in ParadeDB? Any examples or documentation on this would be highly valuable.

I believe understanding these aspects will not only clarify the capabilities of ParadeDB regarding security and data access control but also assist others who might have similar questions.

Thank you for your time and effort in developing ParadeDB. I look forward to your response and any insights you can share on this matter.

[rebasedming]

Hey @razakpm great questions. I've converted this to a Discussion as it's more appropriate.

We haven't yet implemented RLS support for Parquet tables but it's something we're actively working on. We have a plan for doing so that I'm happy to share:

1. Implementation Details: In Postgres, the way that RLS works is Postgres will rewrite the query plan to incorporate RLS. For instance, if you have an RLS policy on SELECT statements, Postgres will apply a filter to all SELECT statements. We would support RLS by translating and sending the Postgres query plan to DataFusion (our query engine). In theory, this means that all RLS policies would also get sent over.
2. Performance Impact RLS policies on SELECT statements typically manifest themselves as filters, which DataFusion filter pushdown handles very well.
3. Configuration and Usage Once implemented, configuring/using security policies over Parquet tables would be identical to that of heap tables.