Snowflake lineage configuration

Include and exclude lists

You can include or exclude assets up to the schema level. Provide databases and schemas in the format database/schema. Each part is evaluated as a regular expression. Assets which are added later in the data source will also be included or excluded if they match the conditions specified in the lists. Example values:

• myDB/: all schemas in myDB database.
• myDB2/.*: all schemas in myDB2 database.
• myDB3/mySchema1: mySchema1 schema from myDB3 database.
• myDB4/mySchema[1-5]: any schema in my myDB4 database with a name that starts with mySchema and ends with a digit between 1 and 5.

External inputs

If you use external Snowflake SQL scripts, you can add them in a .zip file as an external input. You can organize the structure of a .zip file as subfolders that represent databases and schemas. After the scripts are scanned, they are added under respective databases and schemas in the selected catalog or project. The .zip file can have the following structure:

    <database_name>
        <schema_name>
           <script_name.sql>
    <database_name>
        <script_name.sql>
    <script_name.sql>
    replace.csv

The replace.csv file contains placeholder replacements for the scripts that are added in the .zip file. For more information about the format, see Placeholder replacements.

Advanced import options

Table stages extraction

You can add a regular expression to list table stages from which you want staged files to be extracted. Use a fully qualified name and enclose each segment with double quotation marks. Leave the field empty if you do not want to extract staged files from any table stages. Example value:

\\\"mydb\\\"\\.\\\"schema1\\\"\\.\\\".*\\\"|\\\"mydb\\\"\\.\\\"myschema\\\"\\.\\\"abc.*\\\

Transformation logic extraction

You can enable building transformation logic descriptions from SQL code in SQL scripts.