Upload large files

A multipart upload sends a file in parts straight to storage, so no single request has to carry the whole file. A single PUT works up to 512 MB, but use multipart for files over about 100 MB, on slow or unreliable connections, or whenever a proxy in your network rejects large requests. The maximum file size is 512 MB (536,870,912 bytes) either way.

The SDK uses multipart automatically for uploads between 1 MiB and its 50 MiB limit. For anything larger, use the script below or the raw steps after it.

Prerequisites

An API key in STROVIO_API_KEY.
Node.js 18 or later for the script, on macOS, Windows or Linux.

Upload with a script (any platform)

Save this as upload-large.mjs:

import { open, stat } from "node:fs/promises";

const [file, key, contentType = "application/octet-stream", access = "private"] = process.argv.slice(2);
if (!process.env.STROVIO_API_KEY) throw new Error("Set STROVIO_API_KEY first: https://strovio.xyz/docs/get-an-api-key");
if (!file || !key) throw new Error("Usage: node upload-large.mjs <file> <key> [contentType] [public|private]");
const api = "https://strovio.xyz/api/v1";
const auth = { Authorization: `Bearer ${process.env.STROVIO_API_KEY}` };

async function post(path, body) {
  const res = await fetch(`${api}${path}`, {
    method: "POST",
    headers: { ...auth, "Content-Type": "application/json" },
    body: JSON.stringify(body),
  });
  const text = await res.text();
  if (!res.ok) throw new Error(`${path} failed with status ${res.status}: ${text}`);
  return JSON.parse(text);
}

const { size } = await stat(file);
const start = await post("/files/multipart/start", { key, size, contentType, access });
const handle = await open(file);
const parts = [];
let offset = 0;
for (const part of start.parts) {
  const chunk = Buffer.alloc(part.contentLength);
  await handle.read(chunk, 0, part.contentLength, offset);
  offset += part.contentLength;
  const res = await fetch(part.url, { method: "PUT", body: chunk });
  if (!res.ok) throw new Error(`Part ${part.partNumber} failed with status ${res.status}`);
  parts.push({ PartNumber: part.partNumber, ETag: res.headers.get("etag") });
  console.error(`Uploaded part ${part.partNumber} of ${start.parts.length}`);
}
await handle.close();
console.log(await post("/files/multipart/complete", { s3Key: start.s3Key, uploadId: start.uploadId, key, contentType, access, parts }));

Run it:

node upload-large.mjs video.mp4 videos/demo.mp4 video/mp4 private

Expected result: a progress line per part, then the file object with "access": "private" and the full size.

The raw steps

1. Start

curl -X POST "https://strovio.xyz/api/v1/files/multipart/start" \
  -H "Authorization: Bearer $STROVIO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key":"videos/demo.mp4","contentType":"video/mp4","size":134217728,"access":"private"}'

Expected result: uploadId, s3Key, bucket and a parts list. Each part has a partNumber, a signed url and a contentLength. Use each part's contentLength rather than assuming a size: today every part is 5 MiB (5242880 bytes) except the last, which holds the remainder.

2. Upload each part

PUT bytes [offset, offset + contentLength) of the file to each part's url, and keep the ETag response header of each, quotes included. On macOS or Linux, for part n (counting from 1) with 5 MiB parts:

dd if=video.mp4 of=part.bin bs=5242880 skip=$((n - 1)) count=1 2>/dev/null
curl -sf -D - -o /dev/null -X PUT -H "Content-Type:" --data-binary @part.bin "<parts[n-1].url>" \
  | grep -i '^etag:' | cut -d' ' -f2 | tr -d '\r'

The last command prints the ETag, for example "9b2cf535f27731c974343645a3985328". -H "Content-Type:" stops curl adding a form content type. On Windows, use the script above. Parts can be sent one at a time or in parallel. Do not send an Authorization header to the part URLs; they are already signed. Part URLs stay valid for two hours.

3. Complete

curl -X POST "https://strovio.xyz/api/v1/files/multipart/complete" \
  -H "Authorization: Bearer $STROVIO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"s3Key":"<s3Key>","uploadId":"<uploadId>","key":"videos/demo.mp4","contentType":"video/mp4","access":"private","parts":[{"PartNumber":1,"ETag":"\"<etag 1>\""},{"PartNumber":2,"ETag":"\"<etag 2>\""}]}'

Note the field names: start returns partNumber, while complete takes PartNumber and ETag. Send the same key and access as in step 1.

Expected result: 201 with the same fields as a normal upload.

There is no abort call. An upload that is never completed does not create a file.

Confirm success

curl -I https://strovio.xyz/api/v1/files/videos/demo.mp4 -H "Authorization: Bearer $STROVIO_API_KEY" returns 200 and the full Content-Length.

Common errors

400 invalid_request with "s3Key does not match key and access": send the same key and access you sent to start.
403 forbidden: the s3Key came from a different account's upload. Use the s3Key from your own start call.
413 file_too_large or storage_quota_exceeded: the file is over 512 MB or your storage is full.
A part PUT returns 403: the part URL expired. Start again.
500 internal_error on complete, often an ETag that does not match: start again and copy the ETags exactly.

Small files without multipart

For a single signed upload, call POST /api/v1/files/presign-put with key, contentType, size and access, PUT the bytes to the returned putUrl with the same Content-Type within one hour, then call POST /api/v1/files/complete with key, contentType and access. See the API reference.