I am using S3 static website. There is a build process that recreates the website locally in the build. Then, I use aws s3 sync to update the changes to the existing objects.
However, aws s3 sync will reupload the unchanged files each and every time unnecessarily.
I know s3 sync has the following logic:
A local file will require uploading if one of the following conditions is true:
• The local file does not exist under the specified bucket and prefix.
• The size of the local file is different than the size of the s3 object.
• The last modified time of the local file is newer than the last modified time of the s3 object.
That’s weird. Pretty explicitly not what the docs say. Recursively copies new and updated files from the source directory to the destination.
Perhaps those files are being “updated” without being changed?
Tell your build process to not rewrite files that don’t change?
I reuploads based on the third condition “The last modified time of the local file is newer than the last modified time of the s3 object.”
Hmm, I generating a static site using mkdocs in the build. I will have to check out the options in mkdocs.
It is annoying that s3 sync doesn’t have the option to ignore timestamps. It has for the size.
It only replaces with newer files if the size is the same so you could touch the files before uploading and just make them really old
That way it will only upload the files that are different sizes or new
As long as you set the same time stamp on all the files and force the sync one time The files will always have the same time stamp
Not sure if this is a hacky work around or the actual solution but at least it’s a hack you work around worst case
