Multiple multipart large files (1GB And Beyond) uploads to the AWS S3 server using pre-signed URLs and react.js
Table of contents
No headings in the article.
Hi there, Good day. Hope everyone is well. Today I will share with you how to upload multi-part files to AWS S3 with React. File uploading is a common scenario in web applications. But today we will see a little complex file uploading.
I am sharing the used-case scenario first. Suppose you need to develop a web application where:
Unlimited photos and videos to upload.
Each file should show a separate upload progress bar.
The average photo size is 15/20 MB and the video size is above 1 GB.
One Thumbnail and one Preview file should be created from each photo and video.
The user can cancel the upload if he wants to during the upload.
The user will select multiple files to upload but you will not send all requests to the server at once. You will upload files one by one. When one upload is finished, upload another one.
Files must be uploaded to AWS S3.
Here the entire work has to be done on the client side using React.js
There are two main challenges here, one is to create a thumbnail from the video and the other is to upload a file above 1 GB
Please have a look at this link: Generate thumbnail from video using react custom hook
If the user selects 1000 images and some 1/2 GB sized video files and uploads them, then there is a possibility of crashing the browser and also the server may shut down if the server's request capacity is low. Another issue is that requests may time out for large file uploads.
So what is the solution? Today we will solve this issue. I have personally tested this solution by uploading 5000+ files and it works 99%. So, without further delay, let's get started.
I will share the main working code here. There are some common tasks that I will not show you. Like creating the React project and installing various packages etc.
I first created a React project and used TypeScript in it and designed it using Tailwind CSS to make it user-friendly. Hope everyone can understand them after seeing the code. Then I will share the Git repository of the entire project code with you.
I have used the following npm packages.
aws-sdk
react-dropzone
axios
rc-progress
react-toastify
Step-1: We will create a service file called S3FileUploadService.ts which will contain all the file upload logic.
// Initiate a multipart upload request
async initUpload() {
try {
const Key = `test/${cuid()}.${this.item?.fileExtension}`;
const Bucket = this.bucketName;
const initResp: any = await this.s3Client.createMultipartUpload({ Bucket, Key }).promise();
this.uploadId = initResp.UploadId;
this.uploadKey = initResp.Key;
this.numberOfParts = Math.ceil(this.item?.fileSize / this.chunkSize);
this.getSignedUrls();
} catch (error) {
this.complete(error);
}
}
// Create pre-signed URLs for each part
async getSignedUrls() {
try {
const Bucket = this.bucketName;
const Key = this.uploadKey;
const UploadId = this.uploadId;
const numberOfParts = this.numberOfParts;
const promises = [];
for (let i = 0; i < numberOfParts; i++) {
promises.push(this.s3Client.getSignedUrlPromise('uploadPart', { Bucket, Key, UploadId, PartNumber: i + 1 }));
}
const resp: any = await Promise.all(promises);
const urls: any[] = await resp?.map((el: any, index: number) => ({
partNumber: index + 1,
url: el,
}));
this.preSignedUrls = urls;
this.onStartChunkUpload();
} catch (error) {
this.complete(error);
}
}
// Uploading start
async onStartChunkUpload() {
try {
const axiosInstant = axios.create();
delete axiosInstant.defaults.headers.put['Content-Type'];
const urls: any = this.preSignedUrls;
const file = this.item?.originalFile;
const fileName = this.item?.fileName;
const numberOfParts = this.numberOfParts;
const chunkSize = this.chunkSize;
let _progressList: IProgressList[] = await urls.map((item: any) => ({
fileName,
numberOfParts,
partNumber: item?.partNumber,
percentage: 0,
}));
const cancelRequest = axios.CancelToken.source();
const chunkUploadedPromises = await urls?.map(async (part: any, index: number) => {
const chunkStartFrom = index * chunkSize;
const chunkEndTo = (index + 1) * chunkSize;
const blobSlice = index < urls.length ? file.slice(chunkStartFrom, chunkEndTo) : file.slice(chunkStartFrom);
const resp = await axiosInstant.put(part?.url, blobSlice, {
onUploadProgress: async (progressEvent: any) => {
if (this.aborted) {
cancelRequest.cancel(`${fileName} uploading canceled.`);
}
const { loaded, total } = progressEvent;
let percentage = Math.floor((loaded * 100) / total);
_progressList[
_progressList.findIndex(
(el: IProgressList) => el?.fileName === this.item?.fileName && el?.partNumber === part?.partNumber
)
] = {
fileName,
numberOfParts,
partNumber: part?.partNumber,
percentage,
};
this.onProgressFn(_progressList);
},
cancelToken: cancelRequest.token,
});
return resp;
});
const respParts = await Promise.all(chunkUploadedPromises);
const uploadedParts: IPart[] = await respParts.map((part, index) => ({
ETag: (part as any).headers.etag,
PartNumber: index + 1,
}));
this.uploadedParts = uploadedParts;
this.finalizeMultipartUpload();
} catch (error) {
this.complete(error);
}
}
// Finalize multipart upload
async finalizeMultipartUpload() {
try {
const Bucket = this.bucketName;
const Key = this.uploadKey;
const UploadId = this.uploadId;
const parts: IPart[] = this.uploadedParts;
const params = { Bucket, Key, UploadId, MultipartUpload: { Parts: parts } };
const resp = await this.s3Client.completeMultipartUpload(params).promise();
this.complete();
} catch (error) {
this.complete(error);
}
}
// Complete the multipart upload request on success or failure.
async complete(error?: any) {
try {
if (error) {
this.onErrorFn(error);
const Bucket = this.bucketName;
const Key = this.uploadKey;
const UploadId = this.uploadId;
const params = { Bucket, Key, UploadId };
await this.s3Client.abortMultipartUpload(params).promise();
return;
}
this.onSuccessFn(`${this.item?.fileName} successfully uploaded.`);
} catch (error) {
this.onErrorFn(error);
}
}
onProgress(onProgress: any) {
this.onProgressFn = onProgress;
return this;
}
onError(onError: any) {
this.onErrorFn = onError;
return this;
}
onSuccess(resp: any) {
this.onSuccessFn = resp;
return this;
}
abort() {
this.aborted = true;
}
}
interface IPart {
ETag: string;
PartNumber: number;
}
interface IProgressList {
fileName: string;
numberOfParts: number;
partNumber: number;
percentage: number;
}
export interface S3UploadProps {
item: IQueueFileProcessing;
}
Here is the full code of the S3FileUploadService.ts service
import S3 from 'aws-sdk/clients/s3';
import axios from 'axios';
import cuid from 'cuid';
import { IQueueFileProcessing } from 'hooks/useQueueFileProcessing';
import { CHUNK_SIZE } from 'utils/AppConstant';
export class S3FileUploadService {
s3Client: S3;
bucketName: string;
item: IQueueFileProcessing;
uploadId: string;
uploadKey: string;
chunkSize: number;
numberOfParts: number;
preSignedUrls: any[];
uploadedParts: IPart[];
onProgressFn: (progress: any) => void;
onErrorFn: (err: any) => void;
onSuccessFn: (resp: any) => void;
aborted: boolean;
constructor(options: S3UploadProps) {
this.s3Client = new S3({
apiVersion: '2006-03-01',
accessKeyId: process.env.REACT_APP_S3_ACCESS_KEY_ID,
secretAccessKey: process.env.REACT_APP_S3_SECRET_ACCESS_KEY,
region: process.env.REACT_APP_S3_REGION,
signatureVersion: 'v4',
});
this.bucketName = process.env.REACT_APP_S3_BUCKET_NAME!;
this.item = options?.item;
this.uploadId = '';
this.uploadKey = '';
this.chunkSize = CHUNK_SIZE;
this.numberOfParts = 0;
this.preSignedUrls = [];
this.uploadedParts = [];
this.onProgressFn = () => {};
this.onErrorFn = () => {};
this.onSuccessFn = () => {};
this.aborted = false;
}
// Starting the multipart (chunk) upload
start() {
this.initUpload();
}
// Initiate a multipart upload request
async initUpload() {
try {
const Key = `test/${cuid()}.${this.item?.fileExtension}`;
const Bucket = this.bucketName;
const initResp: any = await this.s3Client.createMultipartUpload({ Bucket, Key }).promise();
this.uploadId = initResp.UploadId;
this.uploadKey = initResp.Key;
this.numberOfParts = Math.ceil(this.item?.fileSize / this.chunkSize);
this.getSignedUrls();
} catch (error) {
this.complete(error);
}
}
// Create pre-signed URLs for each part
async getSignedUrls() {
try {
const Bucket = this.bucketName;
const Key = this.uploadKey;
const UploadId = this.uploadId;
const numberOfParts = this.numberOfParts;
const promises = [];
for (let i = 0; i < numberOfParts; i++) {
promises.push(this.s3Client.getSignedUrlPromise('uploadPart', { Bucket, Key, UploadId, PartNumber: i + 1 }));
}
const resp: any = await Promise.all(promises);
const urls: any[] = await resp?.map((el: any, index: number) => ({
partNumber: index + 1,
url: el,
}));
this.preSignedUrls = urls;
this.onStartChunkUpload();
} catch (error) {
this.complete(error);
}
}
// Uploading start
async onStartChunkUpload() {
try {
const axiosInstant = axios.create();
delete axiosInstant.defaults.headers.put['Content-Type'];
const urls: any = this.preSignedUrls;
const file = this.item?.originalFile;
const fileName = this.item?.fileName;
const numberOfParts = this.numberOfParts;
const chunkSize = this.chunkSize;
let _progressList: IProgressList[] = await urls.map((item: any) => ({
fileName,
numberOfParts,
partNumber: item?.partNumber,
percentage: 0,
}));
const cancelRequest = axios.CancelToken.source();
const chunkUploadedPromises = await urls?.map(async (part: any, index: number) => {
const chunkStartFrom = index * chunkSize;
const chunkEndTo = (index + 1) * chunkSize;
const blobSlice = index < urls.length ? file.slice(chunkStartFrom, chunkEndTo) : file.slice(chunkStartFrom);
const resp = await axiosInstant.put(part?.url, blobSlice, {
onUploadProgress: async (progressEvent: any) => {
if (this.aborted) {
cancelRequest.cancel(`${fileName} uploading canceled.`);
}
const { loaded, total } = progressEvent;
let percentage = Math.floor((loaded * 100) / total);
_progressList[
_progressList.findIndex(
(el: IProgressList) => el?.fileName === this.item?.fileName && el?.partNumber === part?.partNumber
)
] = {
fileName,
numberOfParts,
partNumber: part?.partNumber,
percentage,
};
this.onProgressFn(_progressList);
},
cancelToken: cancelRequest.token,
});
return resp;
});
const respParts = await Promise.all(chunkUploadedPromises);
const uploadedParts: IPart[] = await respParts.map((part, index) => ({
ETag: (part as any).headers.etag,
PartNumber: index + 1,
}));
this.uploadedParts = uploadedParts;
this.finalizeMultipartUpload();
} catch (error) {
this.complete(error);
}
}
// Finalize multipart upload
async finalizeMultipartUpload() {
try {
const Bucket = this.bucketName;
const Key = this.uploadKey;
const UploadId = this.uploadId;
const parts: IPart[] = this.uploadedParts;
const params = { Bucket, Key, UploadId, MultipartUpload: { Parts: parts } };
const resp = await this.s3Client.completeMultipartUpload(params).promise();
this.complete();
} catch (error) {
this.complete(error);
}
}
// Complete the multipart upload request on success or failure.
async complete(error?: any) {
try {
if (error) {
this.onErrorFn(error);
const Bucket = this.bucketName;
const Key = this.uploadKey;
const UploadId = this.uploadId;
const params = { Bucket, Key, UploadId };
await this.s3Client.abortMultipartUpload(params).promise();
return;
}
this.onSuccessFn(`${this.item?.fileName} successfully uploaded.`);
} catch (error) {
this.onErrorFn(error);
}
}
onProgress(onProgress: any) {
this.onProgressFn = onProgress;
return this;
}
onError(onError: any) {
this.onErrorFn = onError;
return this;
}
onSuccess(resp: any) {
this.onSuccessFn = resp;
return this;
}
abort() {
this.aborted = true;
}
}
interface IPart {
ETag: string;
PartNumber: number;
}
interface IProgressList {
fileName: string;
numberOfParts: number;
partNumber: number;
percentage: number;
}
export interface S3UploadProps {
item: IQueueFileProcessing;
}
Step-2: Then we will create a React Custom Hook called useQueueFileProcessing.ts. The work of this hook is to take our given file list, process the file from there, create Thumbnail and Preview files and return a processed array.
Here is the full code of the useQueueFileProcessing.ts hook
import { useState } from 'react';
import { toast } from 'react-toastify';
import { getFileType, randomNumber } from 'utils/helpers';
export interface IQueueFileProcessing {
fileId: number;
fileName: string;
fileExtension: string;
fileSize: number;
fileType: string;
thumbFile: any;
previewFile: any;
originalFile: any;
progress: number;
isProcessing: boolean;
isCompleted: boolean;
isCanceled: boolean;
}
const useQueueFileProcessing = () => {
const [processedFiles, setProcessedFiles] = useState<IQueueFileProcessing[]>([]);
const [isProcessing, setIsProcessing] = useState<boolean>(false);
const [processingMessage, setProcessingMessage] = useState<string>('');
const startFileProcessing = async (rawFiles: any, oldProcessedFiles: IQueueFileProcessing[] = []) => {
const startProcessingTime = performance.now();
const queueList: IQueueFileProcessing[] = [];
setIsProcessing(true);
await queueProcessing(0);
async function queueProcessing(nextProcessIndex: number) {
const file = rawFiles[nextProcessIndex];
setProcessingMessage(`Processing left ${rawFiles?.length - nextProcessIndex} out of ${rawFiles?.length}`);
if (!!file?.name) {
const fileType = getFileType(file);
let queueItem: IQueueFileProcessing = {
fileId: randomNumber(),
fileName: file?.name,
fileExtension: file?.name?.split('.').pop(),
fileSize: file?.size,
fileType: fileType ?? '',
thumbFile: '',
previewFile: '',
originalFile: file,
progress: 0,
isCompleted: false,
isProcessing: true,
isCanceled: false,
};
await generateNewFiles(file, fileType)
.then((newFile: any) => {
const newQueueItem = {
...queueItem,
thumbFile: newFile?.thumbFile,
previewFile: newFile?.previewFile,
};
queueItem = newQueueItem;
})
.catch((error) => console.log(`Thumbnail and preview generation failed. ${error}`));
queueList.push(queueItem);
await queueProcessing(nextProcessIndex + 1);
}
}
await Promise.resolve([...oldProcessedFiles, ...queueList])
.then((result) => {
setProcessedFiles(result.filter((el) => !!el?.thumbFile && !!el?.previewFile));
setIsProcessing(false);
const endProcessingTime = performance.now();
console.info(
`Total processing time: ${((endProcessingTime - startProcessingTime) / 1000).toFixed(3)} seconds.`
);
})
.catch(() => setProcessedFiles([...oldProcessedFiles]));
};
return [
(rawFiles: any[], oldProcessedFiles: IQueueFileProcessing[] = []) =>
startFileProcessing(rawFiles, oldProcessedFiles),
processedFiles,
isProcessing,
processingMessage,
] as const;
};
// Generate low resolution thumbnail and preview file
const generateNewFiles = async (file: File, fileType: string = 'image') => {
let thumbFile: any = '';
let previewFile: any = '';
try {
thumbFile = previewFile = await generateCanvas(file, fileType, 2, 0.75);
if (fileType === 'image') {
thumbFile = await generateCanvas(previewFile, fileType, 2, 0.5);
}
return { thumbFile, previewFile };
} catch (error) {
toast.error(`${error}`);
return { thumbFile, previewFile };
}
};
const generateCanvas = (file: any, fileType: string, divisionBy = 5, quality = 0.75) => {
return new Promise((resolve, reject) => {
let canvas: any = document.createElement('canvas');
canvas.imageSmoothingQuality = 'medium'; // [low, medium, high] Reference:https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/imageSmoothingQuality
let context: any = canvas.getContext('2d');
if (fileType === 'image') {
let img = document.createElement('img');
img.src = URL.createObjectURL(file);
img.onload = () => {
canvas.width = img.width / divisionBy;
canvas.height = img.height / divisionBy;
context.drawImage(img, 0, 0, canvas.width, canvas.height);
return context.canvas.toBlob((blob: any) => resolve(blob), 'image/jpeg', quality);
};
img.onerror = () => reject(`${file.name} is invalid image format.`);
} else if (fileType === 'video') {
const video = document.createElement('video');
video.autoplay = true;
video.muted = true;
video.src = URL.createObjectURL(file);
video.onloadeddata = () => {
canvas.width = video.videoWidth / divisionBy;
canvas.height = video.videoHeight / divisionBy;
context.drawImage(video, 0, 0, canvas.width, canvas.height);
video.pause();
return context.canvas.toBlob((blob: any) => resolve(blob), 'image/jpeg', quality);
};
video.onerror = () => reject(`${file.name} is invalid video format.`);
}
});
};
export default useQueueFileProcessing;
Step-3: Then we will create a helper file called helpers.ts. where we will have some helper functions we need.
Here is the full code of helpers.ts
export const getFileType = (file: File) => {
if (file && !!file?.type) {
if (file.type.match('image.*')) {
return 'image';
} else if (file.type.match('video.*')) {
return 'video';
} else if (file.type.match('audio.*')) {
return 'audio';
} else if (file.type.match('application/pdf')) {
return 'pdf';
} else if (file.type.match('application.*')) {
return 'doc';
}
}
return 'other';
};
export const randomNumber = (start = 100000, end = 999999) => {
return Math.floor(Math.random() * end) + start;
};
export const bytesToMB = (bytes: number) => (bytes / (1024 * 1024)).toFixed(2);
Step-4: Then we will create a file called AppConstant.ts. where we will have some constant value.
Here is the full code of AppConstant.ts
import pkg from '../../package.json';
export const APP_VERSION = pkg.version;
export const APP_NAME =
'Multiple multipart large files (1GB And Beyond) uploads to the AWS S3 server using pre-signed URLs and react.js';
export const CHUNK_SIZE = 1024 * 1024 * 100; // 100MB, This must be bigger than or equal to 5MB, otherwise AWS will respond with: "Your proposed upload is smaller than the minimum allowed size"
Step-5: Then we will create an index.tsx, a FilePreview.tsx, and a FileUploadProgressBar.tsx file for our visible design and upload script respectively.
Here is the full code of index.tsx
import { ChangeEvent, useCallback, useEffect, useMemo, useState } from 'react';
import { FileRejection, useDropzone } from 'react-dropzone';
import { toast } from 'react-toastify';
import Icon from 'components/Icon';
import useQueueFileProcessing, { IQueueFileProcessing } from 'hooks/useQueueFileProcessing';
import FilePreview from './FilePreview';
import FileUploadProgressBar from './FileUploadProgressBar';
const UploadFeature = () => {
const [queueFiles, setQueueFiles] = useState<IQueueFileProcessing[]>([]);
const [isUploading, setIsUploading] = useState<boolean>(false);
// File processing
const [fileProcessingFn, processedFiles, isProcessing, processingMessage] = useQueueFileProcessing();
useEffect(() => {
if (!!processedFiles?.length && !isProcessing) {
setQueueFiles(processedFiles);
}
}, [processedFiles, isProcessing]);
// React-dropzone
const onDrop = useCallback(
async (acceptedFiles: any) => await fileProcessingFn(acceptedFiles, queueFiles),
[queueFiles]
);
const { getRootProps, getInputProps, open, isDragActive } = useDropzone({
onDrop,
accept: { 'image/jpeg': [], 'image/jpg': [], 'image/png': [], 'image/gif': [], 'video/*': [] },
noClick: true,
noKeyboard: true,
onDropRejected: (fileRejections: FileRejection[]) => {
fileRejections?.map((item: any) => {
toast.error(`${item?.errors[0]?.message}. ${item?.file?.name}.`);
});
},
});
const fileRemoveHandler = useMemo(
() => (itemIndex: number) => {
queueFiles.splice(itemIndex, 1);
setQueueFiles([...queueFiles]);
},
[queueFiles]
);
const onSubmitHandler = async (e: ChangeEvent<HTMLFormElement>) => {
e.preventDefault();
try {
setIsUploading(true);
} catch (error: any) {
toast.error(error);
}
};
return (
<div className="relative px-5 py-2">
<div className="flex justify-center">
<form method="POST" onSubmit={onSubmitHandler} encType="multipart/form-data" className="w-[50vw] h-full">
<h2 className="flex justify-between items-center font-semibold text-yellow-300 mb-2">
Photo & Video Upload
<button
disabled={!queueFiles?.length || isUploading}
className="flex justify-center items-center bg-yellow-300 text-yellow-900 text-sm py-1 w-32 space-x-2 rounded-lg font-semibold cursor-pointer hover:bg-yellow-200 disabled:cursor-not-allowed disabled:bg-yellow-200 disabled:opacity-50"
>
<Icon name="cloud-upload" className="w-6 h-6" />
<span>{isUploading ? 'Uploading...' : 'Upload'}</span>
</button>
</h2>
<div className="flex w-full h-[75vh] justify-center items-center border border-dashed border-yellow-300 text-white p-4">
<div className="w-full">
<FilePreview
isProcessing={isProcessing}
processingMessage={processingMessage}
queueFiles={queueFiles}
fileRemoveHandler={fileRemoveHandler}
/>
<div {...getRootProps()} className="w-full text-center pt-4">
<input {...getInputProps()} />
<div className="w-full mb-5 border border-dashed py-5">
<div className="flex items-center justify-center">
<button type="button" onClick={open} className="relative">
<Icon name="photo" className="text-gray-600 w-20 h-20" />
<Icon
name="plus-circle"
className="text-gray-500 w-8 h-8 bg-gray-700 absolute bottom-2 right-0 rounded-full"
/>
</button>
</div>
<p className="text-xs text-gray-400">
Upload your image by <span className="font-bold">add button</span>
</p>
{isDragActive ? (
'Drop the files here ...'
) : (
<>
Drop photos or videos to upload Or{' '}
<button type="button" onClick={open} className="underline">
Browse
</button>
</>
)}
<p className="text-yellow-300">{!!queueFiles?.length && `${queueFiles?.length} files selected`}</p>
</div>
</div>
</div>
</div>
</form>
</div>
{!!queueFiles?.length && isUploading && (
<FileUploadProgressBar queueFiles={queueFiles} setQueueFiles={setQueueFiles} setIsUploading={setIsUploading} />
)}
</div>
);
};
export default UploadFeature;
Here is the full code of FileUploadProgressBar.tsx
import clsx from 'clsx';
import { Circle } from 'rc-progress';
import { memo, useEffect, useRef, useState } from 'react';
import { toast } from 'react-toastify';
import Icon from 'components/Icon';
import Spinner from 'components/Spinner';
import { IQueueFileProcessing } from 'hooks/useQueueFileProcessing';
import { S3FileUploadService, S3UploadProps } from 'services/S3FileUploadService';
const s3UploaderInstList: any[] = [];
interface Props {
queueFiles: IQueueFileProcessing[];
setQueueFiles: Function;
setIsUploading: Function;
}
const FileUploadProgressBar = (props: Props) => {
const { queueFiles, setQueueFiles, setIsUploading } = props;
const isUploadingStartRef = useRef(false);
const [progressList, setProgressList] = useState<any>({ list: [] });
const [isMinimized, setIsMinimized] = useState<boolean>(false);
const [completedProcessCount, setCompletedProcessCount] = useState<number>(0);
useEffect(() => {
if (!!isUploadingStartRef?.current) {
setCompletedProcessCount(0);
isMinimized ? setIsMinimized(true) : setIsMinimized(false);
let _progressList: IQueueFileProcessing[] = queueFiles;
setProgressList({ list: _progressList });
startUploadingQueue(0, _progressList);
}
return () => {
isUploadingStartRef.current = true;
};
}, [queueFiles]);
const startUploadingQueue = async (currentProcessIndex: number, _progressList: IQueueFileProcessing[]) => {
try {
const item: any = queueFiles?.find((el, index) => currentProcessIndex === index && !el?.isCompleted);
if (!!item?.fileId) {
const s3UploaderPayload: S3UploadProps = { item };
const s3Uploader = new S3FileUploadService(s3UploaderPayload);
s3UploaderInstList.push(s3Uploader);
let _updatedProgressList = [..._progressList];
await s3Uploader
.onProgress((progress: any) => {
const sumTotal = progress.reduce((acc: number, cv: { percentage: number }) => acc + cv.percentage, 0);
const percentage = Math.floor(sumTotal / progress[0].numberOfParts);
if (percentage < 100 && percentage % 2 === 0) {
_updatedProgressList[currentProcessIndex] = { ...item, progress: percentage };
setProgressList({ list: _updatedProgressList });
}
})
.onError((error: any) => {
_updatedProgressList[currentProcessIndex] = { ...item, isCanceled: true, isProcessing: false };
setProgressList({ list: _updatedProgressList });
setCompletedProcessCount((prevCount) => prevCount + 1);
startUploadingQueue(currentProcessIndex + 1, _updatedProgressList);
toast.error(error?.message);
})
.onSuccess((resp: any) => {
_updatedProgressList[currentProcessIndex] = { ...item, isCompleted: true, progress: 100 };
setProgressList({ list: _updatedProgressList });
setCompletedProcessCount((prevCount) => prevCount + 1);
startUploadingQueue(currentProcessIndex + 1, _updatedProgressList);
toast.success(`${item?.fileName} successfully uploaded.`);
});
s3Uploader.start();
}
} catch (error: any) {
toast.error(error);
}
};
const onCancelHandler = (cancelIndex: number) => {
s3UploaderInstList.filter((el: IQueueFileProcessing, index) => cancelIndex === index)[0].abort();
};
const onCloseHandler = () => {
setIsUploading(false);
setQueueFiles([]);
};
return (
<div className="absolute right-0 bottom-0 w-[20vw] h-auto bg-gray-400 shadow mr-[1px]">
<div className="">
<h2 className="flex justify-between items-center font-semibold text-yellow-300 p-2 bg-gray-700">
{`${completedProcessCount} out of ${progressList?.list?.length} uploads complete`}
<button onClick={() => setIsMinimized(!isMinimized)} className="">
<Icon name={isMinimized ? 'go-back' : 'minus'} className="w-4 h-4 text-yellow-400" />
</button>
<button onClick={() => onCloseHandler()} className="">
<Icon name="x" className="w-4 h-4 text-yellow-400" />
</button>
</h2>
<div
className={clsx('w-full max-h-[55vh] overflow-hidden overflow-y-auto bg-gray-600', isMinimized && 'hidden')}
>
{progressList?.list.map((item: IQueueFileProcessing, index: number) => (
<div key={item?.fileId}>
<div className="flex justify-between items-center text-white p-1">
<div className="w-full flex justify-between items-center space-x-2">
<div className="flex justify-between items-center space-x-1">
<span>
<Icon name={item?.fileType === 'image' ? 'photo' : 'video'} className="w-4 h-4" />
</span>
<span className="text-[10px]">
{item?.fileName} - (<span className="text-yellow-300">{item?.progress}%)</span>
</span>
</div>
<div className="pr-2">
{item?.isProcessing && item?.progress > 0 && item?.progress < 100 && (
<button onClick={() => onCancelHandler(index)} className="text-red-600">
<Icon name="x" className="w-3 h-3" />
</button>
)}
</div>
</div>
<div className="flex justify-center w-auto">
{item?.isProcessing && item?.progress <= 0 && <Spinner className="w-4 h-4" />}
{item?.isProcessing && item?.progress > 0 && item?.progress < 100 && (
<div className="w-3 flex justify-center items-center">
<Circle
percent={item?.progress}
trailWidth={3}
strokeWidth={3}
trailColor="white"
strokeColor="green"
/>
</div>
)}
{item?.progress >= 100 && <Icon name="check" className="w-4 h-4 text-green-700" />}
{item?.isCanceled && <Icon name="error" className="w-4 h-w-4 text-red-600" />}
</div>
</div>
</div>
))}
</div>
</div>
</div>
);
};
export default memo(FileUploadProgressBar);
Here is the full code of FilePreview.tsx
import { memo } from 'react';
import Icon from 'components/Icon';
import Spinner from 'components/Spinner';
import { IQueueFileProcessing } from 'hooks/useQueueFileProcessing';
interface FilePreviewProps {
isProcessing: boolean;
processingMessage: string;
queueFiles: IQueueFileProcessing[];
fileRemoveHandler: Function;
}
const FilePreview = (props: FilePreviewProps) => {
const { isProcessing, processingMessage, queueFiles, fileRemoveHandler } = props;
if (isProcessing)
return (
<div className="max-h-[200px] overflow-hidden shadow-2xl bg-gray-400 rounded-sm">
<div className="flex items-center justify-center h-full py-5 text-yellow-300">
<Spinner className="w-8 h-8" /> <span>{processingMessage}</span>
</div>
</div>
);
return (
<>
{!!queueFiles.length && (
<div className="max-h-[200px] overflow-hidden overflow-y-auto mt-2 border-1 shadow-2xl p-2 bg-gray-400 rounded-sm">
<div className="grid gap-2 grid-cols-6">
{queueFiles.map((item: IQueueFileProcessing, index: number) => {
return (
<div key={index} className="relative">
<div className="relative w-full h-14 rounded-sm">
<img
src={URL.createObjectURL(item?.thumbFile)}
alt={item?.fileName}
className="object-cover h-full w-full rounded-sm"
/>
</div>
<button
className="absolute text-red-600 top-0 right-0 bottom-0 left-0 z-10 opacity-0 hover:opacity-100"
type="button"
onClick={() => fileRemoveHandler(index)}
>
<p className="flex w-full h-full justify-center items-center">
<Icon name="trash" className="w-6 h-6 rounded-full bg-gray-600 p-1" />
</p>
</button>
</div>
);
})}
</div>
</div>
)}
</>
);
};
export default memo(FilePreview);
Our project directory will look like this
Now we will start our project by running the following yarn or npm command and uploading files.
yarn start
or
npm run start
Download full source code from GitHub
Note: I have discussed only the client side here. How to account for and configure AWS S3 is not shown here. I will write about it in detail in another post later.
Summary
Here we have seen how to easily upload large files in queue format by multi-parting them. Hope this will be useful for you.