Fetch project information

You can use the following functions to fetch project-related information programmatically.

• get_name()

  This function returns the name of the project.

• get_description()

  This function returns the description of the project.

• get_metadata()

  This function returns the project metadata.

• get_storage_metadata()

  This function returns the metadata of the object storage associated with the project.

• get_project_bucket_name()

  This function returns the project bucket name in the associated object storage. All project files are stored in this bucket.

• get_files()

  This function returns a list of the files in your project. Each element in the returned list contains the ID and the name of the file. The list of returned files is not sorted by any criterion and can change when you call the function again.

• get_assets()

  This function returns a list of all project assets. You can pass the optional parameter assetType to the function get_assets which allows you to filter assets by type. The accepted values for this parameter are data_asset, connection and asset. The value asset returns all of the assets in your project. For example, to get only the data assets, use the function get_assets("data_asset").

• get_connections()

  This function returns a list of the connections you have in your project. Each element in the returned list contains the ID and the name of the connection.

Fetch files

You can use the following functions to fetch files stored in the object storage associated with your project.

You can fetch files in two ways:

• get_file_url(filename) where filename is the name of the file you want to fetch.

  This function returns the URL to fetch a file from the object storage using Spark. The URL is constructed based on the type of object storage associated with the project. Hadoop configurations are set up automatically when you interact with the object storage of your project.

  The following example shows you how to use this function to fetch data from the object storage using Spark:

  # Import the lib
  library(projectLib)
  project <- projectLib::Project$new(sc, "<ProjectId>", "<ProjectToken>")
  
  # Get the url
  url <- project$get_file_url("myFile.csv")
  
  # Fetch the CSV file from the object storage using Spark
  invisible(sparkR.session(appName = "SparkSession R"))
  
  df.data <- read.df(
      url,
      source = "org.apache.spark.sql.execution.datasources.csv.CSVFileFormat",
      header = "true")
  head(df.data)
  

• get_file(filename) where filename is the name of the file you want to fetch.

  This function fetches a file into the memory of the running kernel. The function returns a byte buffer which can be used to bind into kernel-specific data structures, for example, an R data frame. This method of fetching files is not recommended for very large files.

  The following example shows you how to fetch a file and read the data into an R data frame:

  # Import project lib
  library(projectLib)
  project <- projectLib::Project$new(sc , "<ProjectId>", "<ProjectToken>")
  
  # Fetch data
  my.file <- project$get_file("my_file.csv")
  
  # Read the CSV data file into a data frame
  df.data <-  read.csv(text = rawToChar(my.file))
  head(df.data)
  

Save data

You can use the following function to save data to the object storage associated with your project. The data will be added as a file to the project bucket in the associated Cloud Object Storage. This function does multiple things. Firstly, it puts the data into the object storage and then it adds this data as a data asset to your project so you can see the data that you saved as a file in the data assets list in your project.

save_data(filename, data, setProjectAsset=TRUE, overwrite=FALSE)

The function takes the following parameters:

• filename: the name of the created file.
• data: the data to upload. The accepted types for this parameter is R raw objects or string buffers.
• setProjectAsset[optional]: adds the file to the project as a data asset after the data was successfully uploaded to the object storage. It takes a boolean value and the value true is set by default.
• overwrite[optional]: overwrites the file if the file already exists in the object storage or the project. By default it is set to false.

Here is an example, which shows you how you can save data to a file in the object storage:

library("projectLib")
project <- access_project()

# Capture CSV data from console output
csv_lines <- capture.output(write.csv(df.data, row.names=FALSE), type="output")
csv_raw <- charToRaw(paste0(csv_lines, collapse='\n'))

project$save_data("file.csv", csv_raw)

Read data from a connection

You can use the following function to get the metadata (credentials) of a given connection.

get_connection: the function takes as input the ID of the connection or the name of the connection. You can get these values by using the get_assets() function which returns the id, name and type of all the assets listed in project.

The function get_connection returns the connection credentials which you can use to fetch data from the connection data source.

Here is an example, which shows you how you can fetch the credentials of a connection by using the get_connection function:

# Import project lib
library(projectLib)
project <- projectLib::Project$new(sc , "<ProjectId>", "<ProjectToken>")

# Fetch connection
conn.cred <- project$get_connection(name="<ConnectionName>")

If your connection is a connection to dashDB for example, you can fetch your data by running the following code:

library(ibmdbR)

props <- paste("DASHDB;DATABASE=BLUDB;HOSTNAME=", conn.cred$host, ";PORT=50000;PROTOCOL=TCPIP;", sep="")
conn <- idaConnect(props, uid = conn.cred$username, pwd = conn.cred$password, conType = "odbc")
idaInit(conn)

idf.1 <- ida.data.frame('<TableName>')
head(idf.1)

Fetch connected data

You can use the following function to fetch the credentials of connected data. The function returns a dictionary that contains the connection credentials in addition to a datapath attribute that points to specific data in that connection, for example, a table in a dashDB instance or a database in a Cloudant instance.

get_connected_data: this function takes as input the ID of the connected data or the name of the connected data. You can get these values by using the get_assets() function which returns the id, name and type of all the assets listed in project.

Here is an example, which shows you how to fetch the credentials of connected data in a dashDB instance by using the get_connected_data function:

# Import project lib
library(projectLib)
project <- projectLib::Project$new(sc , "<ProjectId>", "<ProjectToken>")

# Fetch credentials of the connected data
conn.data <- project$get_connected_data(id="<ConnectedDataId>")

Parent topic: Loading and accessing data in a notebook