How to import data from Snowflake

Use Snowflake’s COPY INTO command to send data directly to a folder in Lokad. Snowflake connects to Lokad’s S3-compatible endpoint and writes the exported files into the shared folder. No intermediate download or manual upload is required.

This guide exports a table as Parquet files with Snappy compression.

Obtain folder credentials

Follow How to transfer files with an S3 client to share the destination folder and obtain its S3 access key ID and secret key.

Allow writes on the share: Read only must be unchecked. These examples assume an empty Prefix Path. Use the S3 secret key shown after creating the share, rather than the generated FTP/SFTP password.

The bucket name is the numeric Lokad account ID. The credentials determine which folder is accessible within that account.

For example, with account ID 85055 and a share pointing to /incoming/snowflake, the object exports/lineitem/data_0_0_0.snappy.parquet appears in Files at:

/incoming/snowflake/exports/lineitem/data_0_0_0.snappy.parquet

Do not include /incoming/snowflake in the object path: the shared folder is already the root exposed by the credentials.

Create an external stage

An external stage saves the destination and credentials used by Snowflake to transfer files.

In a Snowflake SQL worksheet, select a warehouse and a regular database and schema where you can create stages.

Run the following command. Replace in the URL with your account ID, and replace the credential placeholders with the values from your folder share.

CREATE STAGE LOKAD_EXPORT_STAGE
    URL = 's3compat://<AccountId>/exports/lineitem/'
    ENDPOINT = 's3.lokad.com'
    CREDENTIALS = (
        AWS_KEY_ID = '<Lokad S3 access key ID>'
        AWS_SECRET_KEY = '<Lokad S3 secret key>'
    );

The AWS_KEY_ID and AWS_SECRET_KEY parameters take your Lokad credentials. An AWS account is not required.

Keep the endpoint as s3.lokad.com, without https://, a bucket name, or a port number. Snowflake uses HTTPS and virtual-hosted addressing, such as https://85055.s3.lokad.com.

The stage is created in the selected database and schema. Use that same context for the following commands, or qualify the stage name as DATABASE.SCHEMA.LOKAD_EXPORT_STAGE.

Export the data

This example exports Snowflake’s sample LINEITEM table. Replace the table name with your own source table for a customer integration. The sample database must be available in your Snowflake account to run the example unchanged.

COPY INTO @LOKAD_EXPORT_STAGE/
FROM SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.LINEITEM
FILE_FORMAT = (
    TYPE = PARQUET
    COMPRESSION = SNAPPY
)
HEADER = TRUE
MAX_FILE_SIZE = 134217728;

COPY INTO @LOKAD_EXPORT_STAGE/ writes into exports/lineitem/ within the shared folder. COPY INTO @LOKAD_EXPORT_STAGE/subFolder/ writes into exports/lineitem/subFolder/ within that folder.

HEADER = TRUE preserves the source column names in the Parquet schema. It does not add a row to the data. MAX_FILE_SIZE targets output files of up to 128 MiB; Snowflake may produce several smaller files.

See Snowflake’s COPY INTO reference for more export options.