Scribd
Upload
22 views3 pages

Multipart Upload

The document describes the steps to upload a large file to S3 using multipart upload. It explains how to split the file, initiate a multipart upload, upload individual parts, check parts, assemble parts metadata and complete the upload.

Uploaded by

Vishnu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
22 views3 pages

Multipart Upload

The document describes the steps to upload a large file to S3 using multipart upload. It explains how to split the file, initiate a multipart upload, upload individual parts, check parts, assemble parts metadata and complete the upload.

Uploaded by

Vishnu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 3

1)split command:-

split the file into smaller chunks of our desired size.

cmd:-

split -b <chunck size> <file name>.

ex: split -b 200MB 512MB.zip

use ls command to check the chunks file.

response: 512MB.zip xaa xab xac

xaa xab xac - chuncks created from split command

2)initate the multipart upload

aws s3api create-multipart-upload --bucket <bucket name> --key <file>

ex:-
aws s3api create-multipart-upload --bucket vishnu-demo-multipart-s3 --key
512MB.zip

this command returns a json response.copy the upload id in the json for future
use.

upload id:

tiHVWa4hxvkr66CLU_3c131ktrmqJjFVfbLB.R0LyN9k7NOpV4XYRS7PeIqbVFizR5GrIQIGV.9rWUL5Im_
m257h.jtsWGpMz1PiDnya4YBe5296MrzgjQDN.iyr6LVb9wJbqNwh8fcGNrJntZfrUgwAuW7sRKkctnsONa
Eyavg-

3)upload indivial chuncks

aws s3api upload-part --bucket vishnu-demo-multipart-s3 --key 512MB.zip --


part-number 1 --body xaa --upload-id
tiHVWa4hxvkr66CLU_3c131ktrmqJjFVfbLB.R0LyN9k7NOpV4XYRS7PeIqbVFizR5GrIQIGV.9rWUL5Im_
m257h.jtsWGpMz1PiDnya4YBe5296MrzgjQDN.iyr6LVb9wJbqNwh8fcGNrJntZfrUgwAuW7sRKkctnsONa
Eyavg-

aws s3api upload-part --bucket vishnu-demo-multipart-s3 --key 512MB.zip --


part-number 2 --body xab --upload-id
tiHVWa4hxvkr66CLU_3c131ktrmqJjFVfbLB.R0LyN9k7NOpV4XYRS7PeIqbVFizR5GrIQIGV.9rWUL5Im_
m257h.jtsWGpMz1PiDnya4YBe5296MrzgjQDN.iyr6LVb9wJbqNwh8fcGNrJntZfrUgwAuW7sRKkctnsONa
Eyavg-

aws s3api upload-part --bucket vishnu-demo-multipart-s3 --key 512MB.zip --


part-number 3 --body xac --upload-id
tiHVWa4hxvkr66CLU_3c131ktrmqJjFVfbLB.R0LyN9k7NOpV4XYRS7PeIqbVFizR5GrIQIGV.9rWUL5Im_
m257h.jtsWGpMz1PiDnya4YBe5296MrzgjQDN.iyr6LVb9wJbqNwh8fcGNrJntZfrUgwAuW7sRKkctnsONa
Eyavg-

each above command will give a json response.copy th etag in the json for
future use

"ETag": "\"fda5b58447db7d781df6ca14218ad95b\""
"ETag": "\"6ba3177bdfcab6aac08b10b5162734e6\""

"ETag": "\"ce656311eed1560a072a246985394c5b\""

4)check all indivial parts are uploaded in s3.

aws s3api list-parts --bucket vishnu-demo-multipart-s3 --key 512MB.zip --


upload-id
tiHVWa4hxvkr66CLU_3c131ktrmqJjFVfbLB.R0LyN9k7NOpV4XYRS7PeIqbVFizR5GrIQIGV.9rWUL5Im_
m257h.jtsWGpMz1PiDnya4YBe5296MrzgjQDN.iyr6LVb9wJbqNwh8fcGNrJntZfrUgwAuW7sRKkctnsONa
Eyavg-

Above command send json response like below and check all parts are uploaded or
not.

{
"Parts": [
{
"PartNumber": 1,
"LastModified": "2023-12-17T06:35:31+00:00",
"ETag": "\"fda5b58447db7d781df6ca14218ad95b\"",
"Size": 200000000
},
{
"PartNumber": 2,
"LastModified": "2023-12-17T06:36:33+00:00",
"ETag": "\"6ba3177bdfcab6aac08b10b5162734e6\"",
"Size": 200000000
},
{
"PartNumber": 3,
"LastModified": "2023-12-17T06:36:54+00:00",
"ETag": "\"ce656311eed1560a072a246985394c5b\"",
"Size": 136870912
}
],
"ChecksumAlgorithm": null,
"Initiator": {
"ID": "arn:aws:sts::971973167117:assumed-role/s3_full_access/i-
050291194911719f9",
"DisplayName": "s3_full_access/i-050291194911719f9"
},
"Owner": {
"ID": "e6cb1fc53c49974213a0a27bed7021da6c93d88a62b672aa379d3b159fc25a43"
},
"StorageClass": "STANDARD"
}

5) create a json file with etag.

{
"Parts": [
{
"ETag": "fda5b58447db7d781df6ca14218ad95b",
"PartNumber":1
},
{
"ETag": "6ba3177bdfcab6aac08b10b5162734e6",
"PartNumber":2
},

{
"ETag": "ce656311eed1560a072a246985394c5b",
"PartNumber":3
}]
}

6) complete the multipart download.

aws s3api complete-multipart-upload --multipart-upload file://multipart.json --


bucket vishnu-demo-multipart-s3 --key 512MB.zip --upload-id
tiHVWa4hxvkr66CLU_3c131ktrmqJjFVfbLB.R0LyN9k7NOpV4XYRS7PeIqbVFizR5GrIQIGV.9rWUL5Im_
m257h.jtsWGpMz1PiDnya4YBe5296MrzgjQDN.iyr6LVb9wJbqNwh8fcGNrJntZfrUgwAuW7sRKkctnsONa
Eyavg-

The above command send a json response like below that means upload success.

{
"ServerSideEncryption": "AES256",
"Location":
"https://vishnu-demo-multipart-s3.s3.ap-south-1.amazonaws.com/512MB.zip",
"Bucket": "vishnu-demo-multipart-s3",
"Key": "512MB.zip",
"ETag": "\"2b7102f7b6cfe3317ba4f6c55f70bf07-3\""
}

You might also like