Handling Row filters and column masks accross multiple engines
Reddit r/dataengineering5d4 min read
Hey guys, I'm building a custom Iceberg REST catalog that returns table metadata and short-lived storage credentials to our query engines (eg Spark and Trino). Currently we're trying to add row filters and column masking so users could only see the data they’re allowed to access based on their role. If you’ve set this up in your place, I’d love to know how you’re actually enforcing it in the engines. Passing the rules: Are you embedding filter/mask SQL expressions into table properties in the table metadata response, or using any other method? Engine enforcement : Did you have to write custom
