Data Science Wire

Handling Row filters and column masks accross multiple engines

Reddit r/dataengineering5d4 min read

Hey guys, I'm building a custom Iceberg REST catalog that returns table metadata and short-lived storage credentials to our query engines (eg Spark and Trino). Currently we're trying to add row filters and column masking so users could only see the data they’re allowed to access based on their role. If you’ve set this up in your place, I’d love to know how you’re actually enforcing it in the engines. Passing the rules: Are you embedding filter/mask SQL expressions into table properties in the table metadata response, or using any other method? Engine enforcement : Did you have to write custom

Read the full story at Reddit r/dataengineering

More in Data Engineering