Data Source Details (2.14)
This page provides details for each type of data source for InterSystems Data Studio™. For guidance on how to create, edit, delete and test a data source, see Defining Data Sources.
If the connection test or data source creation fails, review the relevant log files in System Management > Admin Tools > Log Files.
Please note that some data sources require credentials. Create credentials before creating these data sources.
Structured Data Sources
Structured data sources provide access to data in a defined format, such as delimited files, database tables, APIs, and feeds.
AzureDelimited
An AzureDelimited data source provides access to an Azure Blob Storage container that contains delimited files.
For an AzureDelimited data source, specify the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog.
Optional. Displayed on when Authentication Method is Microsoft Entra ID. Select the credential that contains the Service Principal Client ID as the username and the Client Secret as the password.
Required. The URL of the Azure Storage account.
Required. The name of the Azure Blob Storage container to access.
Required. Choose one of the following:
-
Access Key
-
Microsoft Entra ID
-
Azure Managed Identity — Azure Managed Identity is available only when Data Studio is running on Azure. With this authentication method, Data Studio uses the managed identity and permissions assigned to the Azure resource hosting Data Studio. No additional credentials are required.
-
Shared Access Signature
Optional. Displayed only when Authentication Method is Microsoft Entra ID. Enter the Microsoft Entra ID tenant (directory) ID.
Optional. Displayed only when Authentication Method is Access Key or Shared Access Signature. Enter the storage account access key or the SAS token.
Optional. Location of the folder from which to load files. You can optionally include %RUNDATE in the path for the ISO date to be injected when the files are loaded. The root folder is used if one is not provided.
Optional. Location of the folder from which to import file schemas. You can optionally include %RUNDATE in the path for the ISO date to be injected at the time the files are listed. The root folder is used if one is not provided.
Optional. Location of folder to archive previously loaded files. You can optionally include %RUNDATE in the path for the ISO date to be injected at the time of the archiving. The root folder is used if one is not provided.
Optional. Location of the folder to write files to when a recipe promotes data to the Azure Blob Storage container. The root folder is used if one is not provided.
ExcelSingleFileDir
An ExcelSingleFileDir data source provides access to data stored in Microsoft Excel files that are periodically written to a directory accessible by Data Studio. Apart from the file format, this data source is the same as FileDir.
FileDir
A FileDir source is a source that provides data via files, periodically written to a specific directory within the file system. The file should contain data separated by delimiters; a common example is a comma-separated value (CSV) file. Another common delimiter is the tab.
For a FileDir source, specify the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog .
Required. The directory that the system creates for this data source, relative to the system base path. This directory appears in the File Manager and is used to store files associated with the source. Because this is a relative path, it should not start with / or \ (but any leading path separator is automatically removed).
To avoid confusion, this directory should not be used by any other data source.
When you create a FileDir source, the system automatically creates the directory named by Interface Path Location , as well as the subdirectories Samples, Source, Work, and Archive. Using the File Manager explains the purpose of these directories.
Guardian
A Guardian data source provides access to The Guardian OpenPlatformOpens in a new tab news feed, provided by The Guardian News & Media LimitedOpens in a new tab; see https://open-platform.theguardian.com/documentation/Opens in a new tab.
For such a data source, provide the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog.
Required. The API key needed to access this data source.
JDBC
A JDBC data source provides access to a database via a JDBC connection.
For a JDBC data source, specify the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog.
Required. The credential that defines the username and password to access the database. Select the applicable credential from the dropdown list.
For JDBC connections that use key-based or certificate-based authentication, a credential is still required. If the JDBC driver does not require a password, leave the password field blank.
Required. Enter the JDBC connection string needed to access the database.
For secure JDBC connections, include any driver-specific authentication parameters, such as key file or certificate information, in the JDBC connection string as required by the JDBC driver documentation.
If your JDBC connection uses certificate, keystore, or truststore files managed by Data Studio, reference each uploaded file in the JDBC connection string using %key_name% and, if applicable, %key_name_password% tokens.
Required. Select the database vendor and version.
If you select this check box, it will be possible to project tables from the selected database as foreign tables within Data Studio. This is useful when it is not feasible or reasonable to load data directly into Data Studio. For example, a table may be extremely large and might not be queried frequently. A foreign table is read-only but can otherwise be accessed in the same way as local tables.
If you select this check box, also specify JDBC Foreign Table Local Schema. This schema name appears in Data Studio when browsing or querying foreign tables projected from this data source.
Optional. Upload certificate, key store, trust store, or other certificate-related file required by the JDBC connection. Each entry is identified by a Key Name, which is referenced in the JDBC URL using a %key_name% token. If the uploaded file requires a password, reference it in the JDBC URL using the corresponding %key_name_password% token.
For each entry, specify the following:
-
Key Name — A unique, case-insensitive name used to reference the uploaded file in the JDBC URL.
-
Certificate File — Upload the certificate, keystore, truststore, or other authentication file by selecting Select Files... or by dragging and dropping the file.
-
Password — Enter the password used to access the uploaded file, if required.
Select Add Certificate Entry to add additional entries.
NYT
An NYT data source provides access to a news feed APIOpens in a new tab provided by The New York Times CompanyOpens in a new tab ; see https://developer.nytimes.com/get-startedOpens in a new tab .
For such a data source, provide the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog .
Required. The API key needed to access this data source.
REST
A REST data source provides access to a RESTful endpoint. Use this type of data source to connect to APIs that expose data over HTTP. For this type of data source, supply the following details:
Required. The unique name of the data source. This is the name users see when browsing the Data Catalog.
Optional. Select the saved credential that authorizes access to this data source.
Optional. The API key or token required to access this data source.
Optional. The base URL for the REST API endpoint (for example, https://test.example.com).
Optional. A REST endpoint used to verify the connection. This value is appended to the API Server path (for example, /oauth2/token).
Optional. The local FileDir data source folder where files are staged.
To use the File Manager interface for staged files, create a FileDir data source before creating this data source, and select it here. If you do not specify a FileDir source, the system stages files in a temporary folder, and the File Manager interface is not available.
Optional. REST endpoints that require custom processing.
After creating the REST data source, navigate to Data Catalog > REST Registry to add additional metadata.
RSS
An RSS data source provides access to an RSS feed. For such a data source, provide the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog .
Required. The URL of the RSS feed.
Required. The length of time to wait, in seconds, for a response from the RSS feed URL before throwing an error.
S3Delimited
An S3Delimited data source provides access to an S3 bucket that contains delimited files.
For an S3Delimited data source, specify the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog .
Required. The credential that defines the username and password to access the given S3 bucket. Select the applicable credential from the dropdown list.
If you are not using an IAM credential, create a credential using the AWS access key ID as the username and the AWS secret access key as the password.
Required. The name of the S3 bucket to access.
Optional. The session token to use when accessing the S3 bucket.
Optional. Location of the folder from which to load files. You can optionally include %RUNDATE in the path for the ISO date to be injected at the time the files are loaded. The root folder is used if one is not provided.
Optional. Location of the folder from which to import file schemas. You can optionally include %RUNDATE in the path for the ISO date to be injected at the time the files are listed. The root folder is used if one is not provided.
Optional. Location of the folder to archive previously loaded files. You can optionally include %RUNDATE in the path for the ISO date to be injected at the time of the archiving. The root folder is used if one is not provided.
Optional. Location of the folder to write files to, in the case when a recipe promotes data to a file on the given S3 bucket.
SAP4HanaAPI
A SAP4HanaAPI data source provides access to SAP S/4HANA data through OData v2 APIs.
For a SAP data source, specify the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog .
Required. The credential that defines the username and password to access the given SAP server.
Select the appropriate credential from the dropdown list.
Required. The host name of the SAP server (for example, from https://{host}:{port}/sap/opu/odata/sap/... ).
Required. The port to use on the SAP server.
Required. The number of times to retry if the SAP server returns a 500 error.
Required. Specifies which schemas should appear as options.
Salesforce
A Salesforce data source provides access to a Salesforce instance via the Salesforce API. Depending on the selected authorization flow, some connection details may differ.
For a Salesforce data source, specify the following details:
Required. Select the appropriate authorization flow for your Salesforce instance. Choose either password or client_credentials. The choice you make will alter what details are necessary to provide when creating this data source.
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog .
Required for the password authorization flow. The credential that defines the username and password to authenticate the Salesforce API. Select the applicable credential from the dropdown list.
Required. Select the appropriate SDS Datasource (DS) Credentials record for the ClientId and Client Secret.
Required. Specify the server used for authentication (which does not have to be the same server on which Salesforce is running).
Required. Specify the server on which the Salesforce API is running.
Optional. Specify the web server port to use, if that is not the standard port number.
SftpDelimited
An SftpDelimited data source provides access to an SFTP server that contains delimited files.
For an SftpDelimited data source, specify the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog .
Required. The credential that defines the username and password to access the given SFTP server. Select the applicable credential from the dropdown list.
Optional. Passphrase for key file.
Required. The host name of the SFTP server.
Required. The port to use on the SFTP server.
Optional. Additional command options to use when connecting to the SFTP server.
Optional. Location of folder to load files from in SFTP server. You can optionally include %RUNDATE in the path for the ISO date to be injected at the time the files are loaded. The root folder is used if one is not provided.
Optional. Location of folder to place files promoted to SFTP server. The root folder is used if one is not provided.
Optional. The local FileDir data source folder where files are staged.
The selected FileDir source determines where staged files appear in the File Manager. If you do not specify a FileDir source, the system stages files in a temporary folder and the File Manager interface is not available
Optional. Upload your certificate file. Either click Select files... to open your computer’s file explorer, or drag and drop files into the section of the screen that says Drop files here to upload.
Unstructured Data Sources
Unstructured data sources provide access to document files stored in external systems such as S3 buckets and SFTP servers.
AzureDocument
An AzureDocument data source provides access to an Azure Blob Storage container that contains unstructured documents.
For an AzureDocument data source, specify the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog.
Optional. Displayed on when Authentication Method is Microsoft Entra ID. Select the credential that contains the Service Principal Client ID as the username and the Client Secret as the password.
Required. The URL of the Azure Storage account.
Required. The name of the Azure Blob Storage container to access.
Required. Choose one of the following:
-
Access Key
-
Microsoft Entra ID
-
Azure Managed Identity — Azure Managed Identity is available only when Data Studio is running on Azure. With this authentication method, Data Studio uses the managed identity and permissions assigned to the Azure resource hosting Data Studio. No additional credentials are required.
-
Shared Access Signature
Optional. Displayed only when Authentication Method is Access Key or Shared Access Signature. Enter the storage account access key or the SAS token.
Optional. Displayed only when Authentication Method is Microsoft Entra ID. Enter the Microsoft Entra ID tenant (directory) ID.
S3Document
An S3Document data source provides access to an S3 bucket that contains document files.
For an S3Document data source, specify the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog .
Optional. The credential that defines the username and password to access the given S3 bucket. Select the applicable credential from the dropdown list.
If you are not using an IAM credential, create a credential using the AWS access key ID as the username and the AWS secret access key as the password.
Required. The name of the S3 bucket to access.
Optional. The session token to use when accessing the S3 bucket.
SftpDocument
An SftpDocument data source provides access to an SFTP server that contains document files.
For an SftpDocument data source, specify the following details:
Required. The unique name of this data source. This is the name that users see when they browse the Data Catalog .
Required. The credential that defines the username and password to access the given SFTP server. Select the applicable credential from the dropdown list.
Optional. Passphrase for key file.
Required. The host name of the SFTP server.
Required. The port to use on the SFTP server.
Optional. Additional command options to use when connecting to the SFTP server.
Optional. Upload your certificate file. Either click Select files... to open your computer’s file explorer, or drag and drop files into the section of the screen that says Drop files here to upload.
