Upload CSV files to Conduit via API (Guide)
Last updated
Last updated
Before sending data, complete the necessary API configuration.
Find the “Push API” integration https://app.getconduit.app/credentials/?q=push+api
Click the Connect button
Enter a name for the integration
Set up columns for your virtual table
Add a new column
Name your column.
Select the appropriate data type for dimensions.
Name another column.
Select “Metric” for numeric columns.
Specify a particular data type for numeric columns.
Copy the URL and incorporate it into your scripts.
Click the “Save” button.
The integration is ready. You can create your script.
Uploading Files
To upload a file, make a POST request to the provided URL. You can specify a date by appending `?date=YYYY-MM-DD` to the URL; otherwise, today's date is used based on your profile's time zone settings.
Example using curl:
curl -X POST -F "file=@stats_2024_01_01.csv" "https://api.getconduit.app/push/{uniqtoken}/receive/?date=2024-01-01"
Replace {uniqtoken} with a real token.
You'll receive a response with a file ID, for instance:
{"event_id": 1}
(Optional)Checking File Status
File processing is asynchronous. To check a file's status, use its file ID in a GET request.
Example using curl:
curl "https://api.getconduit.app/push/{uniqtoken}/status/1/"
Responses indicate processing success or failure and provide details accordingly.
Examples of API Responses:
- File not yet processed: `{"processed": true, "failed": false, "details": null}`
- Successful processing: `{"processed": true, "failed": false, "details": null}`
- Unsuccessful processing: `{"processed": true, "failed": true, "details": "[Campaign] column not found"}`
CSV File Requirements:
- The first row must include header names corresponding to your specific API configuration.
- Columns should be separated by commas.
- Use double quotes to escape characters.
- Do not include thousand separators in numbers; use a period for decimals.
- Format dates as YYYY-MM-DD and datetime values as YYYY-MM-DDTHH:MM:SS.
- Include time zone in numeric format (+0300) for datetime; UTC is default if unspecified.
Limitations
- Maximum file size: 10 MB.
- Rate limit: No more than one API call per second.
After Processing
Once processed, data becomes accessible within Conduit's various modules, including Copilot, Imports, Workflows, and Dashboard.
Upon uploading data, it is integrated into the virtual table. All separate CSV file uploads will be consolidated into a single continuous table.
You can update previously uploaded data chunks by specifying a specific DAY. See the `?date=YYYY-MM-DD` parameter.
By default, the server updates rows for a single day, which is either today or the day specified in the date parameter.
If your CSV file contains data for multiple days, you can specify which column contains the date to be used as the key for selecting the appropriate day.
To do this, go to the integration settings, check the "Use the specified column as a primary dimension for date/time" option, and specify the column name. This column must contain date values.
When you send a CSV file, the rows from the new file will completely overwrite the rows for the specified day, which can be inconvenient for many use cases.
To replace an existing row, add the `mode=update` parameter to the URL. It is recommended to use this parameter along with one indicating a specific day; otherwise, you risk updating the row on an unintended day.
With the `mode=update` parameter, our server operates as follows:
- The server searches for a unique identifier for each row:
- You can provide a unique ID in the table.
- Alternatively, the server will create a unique identifier by calculating the hash for all dimensions (but not metrics) of the row.
If your data already has a natural unique identifier, it is best to use it directly. To do this, specify the name of the column in the "Column with a unique row ID" field within the integration settings.
The server will search for a row with this ID. If it finds the row, it will update it with the new data. If it doesn't find the row, it will add the new line to the table.
Optional parameter mode can be set to [append | update | overwrite].
overwrite: default,overwrites the day.
append: Appends rows at the end of the day.
All data types support all modes
Default value:
CSV - mode=overwrite
FORM - mode=append
To send a row with 3 columns:
col1 = abc
col2 = 123
col3 = 2024-01-01
To send the same data using JSON:
Supported HTTP Header Content-Type values:
multipart/form-data
application/x-www-form-urlencoded
application/json
To send CSV, you must specify the header Content-Type = multipart/form-data
For FORM, all three types of headers are supported. However, if JSON is being sent, the Content-Type header must be set to application/json.
Optionally, the user can specify the parameter ?payload_type=[csv | form] to indicate the type of content being sent.
Limits
File size limit: 10 MB
HTTP BODY size limit for FORM: 10 KB
File requests: 1 request per 5 seconds.
FORM requests: 5 requests per 5 seconds.
