Overview

This guide outlines the process of uploading a MP3 file from a file path, especially suitable for large files. The upload is performed in chunks to ensure efficiency and reliability, particularly over unstable networks. This process is intended for a Node.js environment.

1. Start Upload

Begin by initiating the upload process.

start-upload.ts
import axios from 'axios';

async function startUpload(fileName: string, fileSize: number, fileType: string) {
  const response = await axios.post('/api/startUpload', {
    fileName,
    fileSize,
    fileType,
  });
  return response.data; // Returns { uploadId, s3UploadId }
}

2. Reading the File and Chunking

Read the MP3 file from the file path and divide it into chunks.

chunk-file.ts
import fs from 'fs';

function chunkFile(filePath: string, chunkSize: number): Buffer[] {
  const fileBuffer = fs.readFileSync(filePath);
  const chunks = [];
  let currentChunkStart = 0;

  while (currentChunkStart < fileBuffer.length) {
    chunks.push(fileBuffer.slice(currentChunkStart, currentChunkStart + chunkSize));
    currentChunkStart += chunkSize;
  }

  return chunks;
}

3. Request Upload Permit and Upload Chunks

For each chunk, request an upload permit and upload it.

upload-chunk.ts
async function requestUploadPermit(s3UploadId: string, uploadId: string, partNumber: number) {
  const response = await axios.post('/api/requestUploadPermit', {
    s3UploadId,
    uploadId,
    partNumber,
  });
  return response.data; // Returns { uploadUrl }
}

async function uploadChunk(chunkUploadUrl: string, chunk: Buffer, partNumber: number) {
  const chunkDataResp = await axios.put(chunkUploadUrl, chunk);
  return {
    ETag: chunkDataResp.headers.etag.replace(/"/g, ""),
    PartNumber: partNumber,
  };
}

4. Finish Upload

Complete the upload process by merging all the uploaded chunks.

finish-upload.ts
async function finishUpload(uploadId: string, s3UploadId: string, completedParts: Array<{ ETag: string, PartNumber: number }>) {
  await axios.post('/api/finishUpload', {
    uploadId,
    s3UploadId,
    completedParts,
  });
}

Combining all steps

The complete process includes reading the audio file from a path, chunking, and uploading.

upload-full-file.ts
import path from 'path';

async function uploadMp3File(filePath: string, chunkSize: number = 10_485_760) { // chunk size of 10MB
  const fileName = path.basename(filePath);
  const fileSize = fs.statSync(filePath).size;
  const { uploadId, s3UploadId } = await startUpload(fileName, fileSize, 'audio/mpeg');

  const chunks = chunkFile(filePath, chunkSize);
  const completedParts = [];

  for (let i = 0; i < chunks.length; i++) {
    const partNumber = i + 1;
    const { uploadUrl } = await requestUploadPermit(s3UploadId, uploadId, partNumber);
    const completedPart = await uploadChunk(uploadUrl, chunks[i], partNumber);
    completedParts.push(completedPart);
  }

  await finishUpload(uploadId, s3UploadId, completedParts);
}

// Example usage with a file path
const filePath = '/path/to/your/file.mp3';
uploadMp3File(filePath);

Notes

  • This process is intended for a Node.js environment.
  • Use asynchronous file reading methods for large files in a production environment.
  • Adjust the chunk size based on the file size and network conditions.

Support

If you have any questions or need further assistance, please reach out. This guide is designed to facilitate the efficient upload of large MP3 files from a file path, ensuring robustness even in challenging network scenarios.

Was this page helpful?