​
Overview

This guide outlines the process of uploading a MP3 file from a file path, especially suitable for large files. The upload is performed in chunks to ensure efficiency and reliability, particularly over unstable networks. This process is intended for a Node.js environment.

​
1. Start Upload

Begin by initiating the upload process.

start-upload.ts
import axios from 'axios';

async function startUpload(fileName: string, fileSize: number, fileType: string) {
  const response = await axios.post('/api/startUpload', {
    fileName,
    fileSize,
    fileType,
  });
  return response.data; // Returns { uploadId, s3UploadId }
}

​
2. Reading the File and Chunking

Read the MP3 file from the file path and divide it into chunks.

chunk-file.ts
import fs from 'fs';

function chunkFile(filePath: string, chunkSize: number): Buffer[] {
  const fileBuffer = fs.readFileSync(filePath);
  const chunks = [];
  let currentChunkStart = 0;

  while (currentChunkStart < fileBuffer.length) {
    chunks.push(fileBuffer.slice(currentChunkStart, currentChunkStart + chunkSize));
    currentChunkStart += chunkSize;
  }

  return chunks;
}

​
3. Request Upload Permit and Upload Chunks

For each chunk, request an upload permit and upload it.

upload-chunk.ts
async function requestUploadPermit(s3UploadId: string, uploadId: string, partNumber: number) {
  const response = await axios.post('/api/requestUploadPermit', {
    s3UploadId,
    uploadId,
    partNumber,
  });
  return response.data; // Returns { uploadUrl }
}

async function uploadChunk(chunkUploadUrl: string, chunk: Buffer, partNumber: number) {
  const chunkDataResp = await axios.put(chunkUploadUrl, chunk);
  return {
    ETag: chunkDataResp.headers.etag.replace(/"/g, ""),
    PartNumber: partNumber,
  };
}

​
4. Finish Upload

Complete the upload process by merging all the uploaded chunks.

finish-upload.ts
async function finishUpload(uploadId: string, s3UploadId: string, completedParts: Array<{ ETag: string, PartNumber: number }>) {
  await axios.post('/api/finishUpload', {
    uploadId,
    s3UploadId,
    completedParts,
  });
}

​
Combining all steps

The complete process includes reading the audio file from a path, chunking, and uploading.

upload-full-file.ts
import path from 'path';

async function uploadMp3File(filePath: string, chunkSize: number = 10_485_760) { // chunk size of 10MB
  const fileName = path.basename(filePath);
  const fileSize = fs.statSync(filePath).size;
  const { uploadId, s3UploadId } = await startUpload(fileName, fileSize, 'audio/mpeg');

  const chunks = chunkFile(filePath, chunkSize);
  const completedParts = [];

  for (let i = 0; i < chunks.length; i++) {
    const partNumber = i + 1;
    const { uploadUrl } = await requestUploadPermit(s3UploadId, uploadId, partNumber);
    const completedPart = await uploadChunk(uploadUrl, chunks[i], partNumber);
    completedParts.push(completedPart);
  }

  await finishUpload(uploadId, s3UploadId, completedParts);
}

// Example usage with a file path
const filePath = '/path/to/your/file.mp3';
uploadMp3File(filePath);

​
Notes

  • This process is intended for a Node.js environment.
  • Use asynchronous file reading methods for large files in a production environment.
  • Adjust the chunk size based on the file size and network conditions.

​
Support

If you have any questions or need further assistance, please reach out. This guide is designed to facilitate the efficient upload of large MP3 files from a file path, ensuring robustness even in challenging network scenarios.

​
Overview

This guide outlines the process of uploading a MP3 file from a file path, especially suitable for large files. The upload is performed in chunks to ensure efficiency and reliability, particularly over unstable networks. This process is intended for a Node.js environment.

​
1. Start Upload

Begin by initiating the upload process.

start-upload.ts
import axios from 'axios';

async function startUpload(fileName: string, fileSize: number, fileType: string) {
  const response = await axios.post('/api/startUpload', {
    fileName,
    fileSize,
    fileType,
  });
  return response.data; // Returns { uploadId, s3UploadId }
}

​
2. Reading the File and Chunking

Read the MP3 file from the file path and divide it into chunks.

chunk-file.ts
import fs from 'fs';

function chunkFile(filePath: string, chunkSize: number): Buffer[] {
  const fileBuffer = fs.readFileSync(filePath);
  const chunks = [];
  let currentChunkStart = 0;

  while (currentChunkStart < fileBuffer.length) {
    chunks.push(fileBuffer.slice(currentChunkStart, currentChunkStart + chunkSize));
    currentChunkStart += chunkSize;
  }

  return chunks;
}

​
3. Request Upload Permit and Upload Chunks

For each chunk, request an upload permit and upload it.

upload-chunk.ts
async function requestUploadPermit(s3UploadId: string, uploadId: string, partNumber: number) {
  const response = await axios.post('/api/requestUploadPermit', {
    s3UploadId,
    uploadId,
    partNumber,
  });
  return response.data; // Returns { uploadUrl }
}

async function uploadChunk(chunkUploadUrl: string, chunk: Buffer, partNumber: number) {
  const chunkDataResp = await axios.put(chunkUploadUrl, chunk);
  return {
    ETag: chunkDataResp.headers.etag.replace(/"/g, ""),
    PartNumber: partNumber,
  };
}

​
4. Finish Upload

Complete the upload process by merging all the uploaded chunks.

finish-upload.ts
async function finishUpload(uploadId: string, s3UploadId: string, completedParts: Array<{ ETag: string, PartNumber: number }>) {
  await axios.post('/api/finishUpload', {
    uploadId,
    s3UploadId,
    completedParts,
  });
}

​
Combining all steps

The complete process includes reading the audio file from a path, chunking, and uploading.

upload-full-file.ts
import path from 'path';

async function uploadMp3File(filePath: string, chunkSize: number = 10_485_760) { // chunk size of 10MB
  const fileName = path.basename(filePath);
  const fileSize = fs.statSync(filePath).size;
  const { uploadId, s3UploadId } = await startUpload(fileName, fileSize, 'audio/mpeg');

  const chunks = chunkFile(filePath, chunkSize);
  const completedParts = [];

  for (let i = 0; i < chunks.length; i++) {
    const partNumber = i + 1;
    const { uploadUrl } = await requestUploadPermit(s3UploadId, uploadId, partNumber);
    const completedPart = await uploadChunk(uploadUrl, chunks[i], partNumber);
    completedParts.push(completedPart);
  }

  await finishUpload(uploadId, s3UploadId, completedParts);
}

// Example usage with a file path
const filePath = '/path/to/your/file.mp3';
uploadMp3File(filePath);

​
Notes

  • This process is intended for a Node.js environment.
  • Use asynchronous file reading methods for large files in a production environment.
  • Adjust the chunk size based on the file size and network conditions.

​
Support

If you have any questions or need further assistance, please reach out. This guide is designed to facilitate the efficient upload of large MP3 files from a file path, ensuring robustness even in challenging network scenarios.