Data Product Hub provides several options for delivering data products to the consumer. Each item in a data product can be delivered by using different methods. The connector type determines the available delivery methods.
Types of delivery methods for Data Product Hub
| Name | Description | Notes |
|---|---|---|
| Download | Consumers receive a URL to download a data product from a connection. | Requires a data source connection that supports download. |
| Data extract | Consumers can download a data product as a file extract from a target connection. | Requires two connections: A source connection and a target connection. |
| Open URL | Consumers receive a URL to directly access a data product. | Does not require a data source connection. |
| Live access with Flight service | Consumers receive a code snippet to connect with a data source and receive a data product. | The provided code snippet can be added to notebooks and other applications. |
Download
The Download delivery method generates a URL that allows consumers to download directly from a connected data source.
Open URL
The Open URL delivery method provides a URL that points to a website containing the items in a data product.
Data extract
You can extract data from a database and deliver the extract as a file to consumers. The data extract delivery method requires two connections, one for connecting to the database you are extracting from (the source) and another connection for delivering the file to the consumer (the target). To view which connectors support source and target connections for data extracts, see Delivery methods for connectors.
The target connection is a file storage location that is used exclusively for storing data extracts. This connection delivers the extract to consumers through a download URL. Since data extracts can be large, a file storage location with adequate capacity is recommended. The target connections require read/write credentials.
Create two connections for the data extract delivery method:
- Source connection: The source connection is the location where you extract data.
- Target connection: The target connection is the location where the extracted file is stored and where consumers download a data extract from a URL.
Live access with Flight service
The Flight service provides real-time, read/write access to many data sources through a common open source API. The Flight service provides a single interface for accessing many different data sources. With the Flight service, Data Product Hub no longer needs a connection asset to the data source with personal credentials. The Flight service retrieves the credentials transparently. To determine if your data source supports the Flight service, see Delivery methods for connectors.
When you subscribe to a data product and select delivery by the Flight service, a Flight URL is provided. You download a code snippet in your preferred language (Python or R). The code snippet contains an asset ID and a catalog ID to connect to a data source to receive delivery of the items in a data product. The code snippet can be inserted into a notebook and other applications.
You can also access the data programmatically by using an Arrow client. Arrow libraries are available for C, C++, C#, Go, Java, JavaScript, Julia, MATLAB, Python, R and Ruby. See Apache Arrow for instructions on installing the libraries for each language.
See Flight client example for an example of how to access your data product using Python.
For more information about Arrow Flight RPC, see Arrow Flight RPC.
Learn more
Delivery methods for connectors
Parent topic: Publishing a data product