Upload CSV files to Conduit via API (Guide)

Create a new integration

Before sending data, complete the necessary API configuration.

Find the “Push API” integration https://app.getconduit.app/credentials/?q=push+api

Click the Connect button

Enter a name for the integration

Set up columns for your virtual table

  1. Add a new column

  2. Name your column.

  3. Select the appropriate data type for dimensions.

  4. Name another column.

  5. Select “Metric” for numeric columns.

  6. Specify a particular data type for numeric columns.

  7. Copy the URL and incorporate it into your scripts.

Click the “Save” button.

The integration is ready. You can create your script.

Basic Usage

Uploading Files

To upload a file, make a POST request to the provided URL. You can specify a date by appending `?date=YYYY-MM-DD` to the URL; otherwise, today's date is used based on your profile's time zone settings.

Example using curl:

curl -X POST -F "file=@stats_2024_01_01.csv" "https://api.getconduit.app/push/{uniqtoken}/receive/?date=2024-01-01"

Replace {uniqtoken} with a real token.

You'll receive a response with a file ID, for instance:

{"event_id": 1}

(Optional)Checking File Status

File processing is asynchronous. To check a file's status, use its file ID in a GET request.

Example using curl:

curl "https://api.getconduit.app/push/{uniqtoken}/status/1/"

Responses indicate processing success or failure and provide details accordingly.

Examples of API Responses:

- File not yet processed: `{"processed": true, "failed": false, "details": null}`

- Successful processing: `{"processed": true, "failed": false, "details": null}`

- Unsuccessful processing: `{"processed": true, "failed": true, "details": "[Campaign] column not found"}`

CSV File Requirements:

- The first row must include header names corresponding to your specific API configuration.

- Columns should be separated by commas.

- Use double quotes to escape characters.

- Do not include thousand separators in numbers; use a period for decimals.

- Format dates as YYYY-MM-DD and datetime values as YYYY-MM-DDTHH:MM:SS.

- Include time zone in numeric format (+0300) for datetime; UTC is default if unspecified.

Limitations

- Maximum file size: 10 MB.

- Rate limit: No more than one API call per second.

After Processing

Once processed, data becomes accessible within Conduit's various modules, including Copilot, Imports, Workflows, and Dashboard.

Upon uploading data, it is integrated into the virtual table. All separate CSV file uploads will be consolidated into a single continuous table.

You can update previously uploaded data chunks by specifying a specific DAY. See the `?date=YYYY-MM-DD` parameter.

Uploading a CSV file with multiple days

By default, the server updates rows for a single day, which is either today or the day specified in the date parameter.

If your CSV file contains data for multiple days, you can specify which column contains the date to be used as the key for selecting the appropriate day.

To do this, go to the integration settings, check the "Use the specified column as a primary dimension for date/time" option, and specify the column name. This column must contain date values.

Updating rows

When you send a CSV file, the rows from the new file will completely overwrite the rows for the specified day, which can be inconvenient for many use cases.

To replace an existing row, add the `mode=update` parameter to the URL. It is recommended to use this parameter along with one indicating a specific day; otherwise, you risk updating the row on an unintended day.

With the `mode=update` parameter, our server operates as follows:

- The server searches for a unique identifier for each row:

- You can provide a unique ID in the table.

- Alternatively, the server will create a unique identifier by calculating the hash for all dimensions (but not metrics) of the row.

If your data already has a natural unique identifier, it is best to use it directly. To do this, specify the name of the column in the "Column with a unique row ID" field within the integration settings.

The server will search for a row with this ID. If it finds the row, it will update it with the new data. If it doesn't find the row, it will add the new line to the table.

More Options

Mode

Optional parameter mode can be set to [append | update | overwrite].

  • overwrite: default,overwrites the day.

  • append: Appends rows at the end of the day.

All data types support all modes

Default value:

CSV - mode=overwrite

FORM - mode=append

HTTP FORM Example

To send a row with 3 columns:

  • col1 = abc

  • col2 = 123

  • col3 = 2024-01-01

curl -X POST "https://api.getconduit.app/push/{uniqtoken}/receive/?date=2024-01-01" \
 -d "col1=abc" \
 -d "col2=123" \
 -d "col3=2024-01-01"

JSON Submission Example

To send the same data using JSON:

curl -X POST "https://api.getconduit.app/push/{uniqtoken}/receive/?date=2024-01-01" \
 -H "Content-Type: application/json" \
 -d '{"col1": "abc", "col2": "123", "col3": "2024-01-01"}'

Content-Type

Supported HTTP Header Content-Type values:

  • multipart/form-data

  • application/x-www-form-urlencoded

  • application/json

To send CSV, you must specify the header Content-Type = multipart/form-data

For FORM, all three types of headers are supported. However, if JSON is being sent, the Content-Type header must be set to application/json.

Payload Type Specification

Optionally, the user can specify the parameter ?payload_type=[csv | form] to indicate the type of content being sent.

Limits

  • File size limit: 10 MB

  • HTTP BODY size limit for FORM: 10 KB

Rate Limits

  • File requests: 1 request per 5 seconds.

  • FORM requests: 5 requests per 5 seconds.

PreviousManage your AI Copilot usersNextInstagram Integration

Last updated