Am I doing something wrong with POST uploads or is this a bug?

[zanzlender]

[EDIT] Using the superagent library as described in the docs fixes this... Will post the answer once I find what it is but the catch is in the .attach() method superagent.attach("file", formData.get("file"), "sdas.png");

After looking into the superagent library, it would appear that .attach() is just a replacement for formData.append() with the same syntax, but my quick reproduction with the .append() method doesn't work... Will post info after I explore it deeper.

POST uploads fail

I am trying to solve my problem with PresignedPutUrls (which work perfectly) but cannot enforce any restrictions as PresignedPostUrls can - e.g. file size.

However, no matter what I do I get the following error: The body of your POST request is not well-formed multipart/form-data. (The name of the uploaded key is missing). I cannot find any reference to this in the documentation nor do I see the "name" property mentioned anywhere...

For reference, my bucket's name is tasky-app inside of which I have a profile-images folder.

Creating policy

const postPolicy = minioClient.newPostPolicy();
postPolicy.setBucket("tasky-app");
// tried with both .setKey() and .setKeyStartsWith(), but get same error
//postPolicy.setKey("somenameidk.png");
postPolicy.setKeyStartsWith("profile-images/somenameidk.png");
postPolicy.setExpires(new Date(Date.now() + 60 * 60 * 24));
postPolicy.setContentType("text/plain");
postPolicy.setContentDisposition("attachment; filename=example.png");
// 1KB - 10MB
postPolicy.setContentLengthRange(1024, 1024 * 1024 * 100);
postPolicy.setUserMetaData({
  key: "value",
  smoke: "data",
});
const presignedPost = await minioClient.presignedPostPolicy(postPolicy);
console.log(presignedPost);

After which I get:

{
  postURL: 'http://localhost:8001/tasky-app',
  formData: {
    bucket: 'tasky-app',
    key: 'profile-images/somenameidk.png',
    'Content-Type': 'image/png',
    'Content-Disposition': 'attachment; filename=example.png',
    'x-amz-meta-key': 'value',
    'x-amz-meta-smoke': 'data',
    'x-amz-date': '20240513T103640Z',
    'x-amz-algorithm': 'AWS4-HMAC-SHA256',
    'x-amz-credential': 'JJtetbdFP9nyGrG4x7iQ/20240513/us-east-1/s3/aws4_request',
    policy: 'eyJjb25kaXRpb25zIjpbWyJlcSIsIiRidWNrZXQiLCJ0YXNreS1hcHAiXSxbInN0YXJ0cy13aXRoIiwiJGtleSIsInByb2ZpbGUtaW1hZ2VzL3NvbWVuYW1laWRrLnBuZyJdLFsiZXEiLCIkQ29udGVudC1UeXBlIiwidGV4dC9wbGFpbiJdLFsiZXEiLCIkQ29udGVudC1EaXNwb3NpdGlvbiIsImF0dGFjaG1lbnQ7IGZpbGVuYW1lPWV4YW1wbGUucG5nIl0sWyJjb250ZW50LWxlbmd0aC1yYW5nZSIsMTAyNCwxMDQ4NTc2MDBdLFsiZXEiLCIkeC1hbXotbWV0YS1rZXkiLCJ2YWx1ZSJdLFsiZXEiLCIkeC1hbXotbWV0YS1zbW9rZSIsImRhdGEiXSxbImVxIiwiJHgtYW16LWRhdGUiLCIyMDI0MDUxM1QxMDM2NDBaIl0sWyJlcSIsIiR4LWFtei1hbGdvcml0aG0iLCJBV1M0LUhNQUMtU0hBMjU2Il0sWyJlcSIsIiR4LWFtei1jcmVkZW50aWFsIiwiSkp0ZXRiZEZQOW55R3JHNHg3aVEvMjAyNDA1MTMvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJdXSwiZXhwaXJhdGlvbiI6IjIwMjQtMDUtMTNUMTA6Mzg6MDYuNjczWiJ9',
    'x-amz-signature': '90c4bdaf683fb13e7dc7701b5a45d506e44e2fda693145a2c112fd48ca98d359'
  }
}

Uploading image

I have a simple form which only accespts one file and handles the upload. (Policies is copied here for simplicity)

      <form
        name="myform"
        encType="multipart/form-data"
        method="POST"
        onSubmit={async (e) => {
          e.preventDefault();

          const formData = new FormData(e.currentTarget);

          const postPolicy = {
              postURL: 'http://localhost:8001/tasky-app',
              formData: {
                bucket: 'tasky-app',
                key: 'profile-images/somenameidk.png',
                'Content-Type': 'text/plain',
                'Content-Disposition': 'attachment; filename=example.png',
                'x-amz-meta-key': 'value',
                'x-amz-meta-smoke': 'data',
                'x-amz-date': '20240513T103640Z',
                'x-amz-algorithm': 'AWS4-HMAC-SHA256',
                'x-amz-credential': 'JJtetbdFP9nyGrG4x7iQ/20240513/us-east-1/s3/aws4_request',
                policy: 'eyJjb25kaXRpb25zIjpbWyJlcSIsIiRidWNrZXQiLCJ0YXNreS1hcHAiXSxbInN0YXJ0cy13aXRoIiwiJGtleSIsInByb2ZpbGUtaW1hZ2VzL3NvbWVuYW1laWRrLnBuZyJdLFsiZXEiLCIkQ29udGVudC1UeXBlIiwidGV4dC9wbGFpbiJdLFsiZXEiLCIkQ29udGVudC1EaXNwb3NpdGlvbiIsImF0dGFjaG1lbnQ7IGZpbGVuYW1lPWV4YW1wbGUucG5nIl0sWyJjb250ZW50LWxlbmd0aC1yYW5nZSIsMTAyNCwxMDQ4NTc2MDBdLFsiZXEiLCIkeC1hbXotbWV0YS1rZXkiLCJ2YWx1ZSJdLFsiZXEiLCIkeC1hbXotbWV0YS1zbW9rZSIsImRhdGEiXSxbImVxIiwiJHgtYW16LWRhdGUiLCIyMDI0MDUxM1QxMDM2NDBaIl0sWyJlcSIsIiR4LWFtei1hbGdvcml0aG0iLCJBV1M0LUhNQUMtU0hBMjU2Il0sWyJlcSIsIiR4LWFtei1jcmVkZW50aWFsIiwiSkp0ZXRiZEZQOW55R3JHNHg3aVEvMjAyNDA1MTMvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJdXSwiZXhwaXJhdGlvbiI6IjIwMjQtMDUtMTNUMTA6Mzg6MDYuNjczWiJ9',
                'x-amz-signature': '90c4bdaf683fb13e7dc7701b5a45d506e44e2fda693145a2c112fd48ca98d359'
              }
            }
          Object.keys(postPolicy.formData).forEach((key) => {
            formData.append(key, postPolicy.formData[key]);
          });

          try {
            const response = await fetch(postPolicy.postURL, {
              method: "POST",
              mode: "cors",
              body: formData,
              headers: {
                "Access-Control-Allow-Origin": "*",
              },
            });

            console.log("response", response);
          } catch (err) {
            console.log("UERRRR", err);
          }
        }}
      >
        <input type="file" name="file" />

        <button className="border-4 bg-red-500 text-white" type="submit">
          Submit
        </button>
      </form>

Errors

MinIO Trace

"type": "S3",
 "host": "localhost:8001",
 "api": "s3.PostPolicyBucket",
 "time": "2024-05-13T10:30:33.30987239Z",
 "duration": 366077,
 "path": "/tasky-app",
 "request": {
  "time": "2024-05-13T10:30:33.30987239Z",
  "proto": "HTTP/1.1",
  "method": "POST",
  "path": "/tasky-app",
  "headers": {
   "Accept": "*/*",
   "Accept-Encoding": "gzip, deflate, br, zstd",
   "Accept-Language": "en-US,en;q=0.9,bs;q=0.8,hr;q=0.7,de;q=0.6",
   "Access-Control-Allow-Origin": "*",
   "Cache-Control": "no-cache",
   "Connection": "keep-alive",
   "Content-Length": "117025",
   "Content-Type": "multipart/form-data; boundary=----WebKitFormBoundarymRCrHWgTcAFQD28d",
   "Host": "localhost:8001",
   "Origin": "http://localhost:3000",
   "Pragma": "no-cache",
   "Referer": "http://localhost:3000/",
   "Sec-Ch-Ua": "\"Chromium\";v=\"124\", \"Google Chrome\";v=\"124\", \"Not-A.Brand\";v=\"99\"",
   "Sec-Ch-Ua-Mobile": "?0",
   "Sec-Ch-Ua-Platform": "\"Windows\"",
   "Sec-Fetch-Dest": "empty",
   "Sec-Fetch-Mode": "cors",
   "Sec-Fetch-Site": "same-site",
   "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
  },
  "body": "<BLOB>"
 },
 "response": {
  "time": "2024-05-13T10:30:33.310233267Z",
  "headers": {
   "Accept-Ranges": "bytes",
   "Access-Control-Allow-Credentials": "true",
   "Access-Control-Allow-Origin": "http://localhost:3000",
   "Access-Control-Expose-Headers": "Date, Etag, Server, Connection, Accept-Ranges, Content-Range, Content-Encoding, Content-Length, Content-Type, Content-Disposition, Last-Modified, Content-Language, Cache-Control, Retry-After, X-Amz-Bucket-Region, Expires, X-Amz*, X-Amz*, *",
   "Content-Length": "402",
   "Content-Type": "application/xml",
   "Server": "MinIO",
   "Strict-Transport-Security": "max-age=31536000; includeSubDomains",
   "Vary": "Origin Accept-Encoding",
   "X-Amz-Id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8",
   "X-Amz-Request-Id": "17CF05AF406E9C47",
   "X-Content-Type-Options": "nosniff",
   "X-Xss-Protection": "1; mode=block"
  },
  "body": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>MalformedPOSTRequest</Code><Message>The body of your POST request is not well-formed multipart/form-data. (The name of the uploaded key is missing)</Message><BucketName>tasky-app</BucketName><Resource>/tasky-app</Resource><RequestId>17CF05AF406E9C47</RequestId><HostId>dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8</HostId></Error>",
  "statusCode": 400
 },
 "callStats": {
  "rx": 3581,
  "tx": 402,
  "duration": 366077,
  "timeToFirstByte": 311981
 }
}

[klauspost]

I am not a JS expert at any level, but key: 'profile-images/somenameidk.png', is no coming through.

My guess is that Object.keys(postPolicy.formData).forEach((key) never runs.

[zanzlender]

The key: 'profile-images/somenameidk.png' is coming through, because I logged it out and saw it... And the same goes for the .forEach part... in the request that is sent every property inside the policy.formData is applied...

And I ran the same exact code but only replaced formData.append() with superagent.post.append() and it worked...

[zanzlender]

I found the solution to this problem, although I am still not sure why it works like that...

I compared the request the superagent library made vs my own XHR and/or fetch request. The only difference I noticed was the order of the properties in the formData part of the request. Specifically, in my own requests the file was the first property - since it had been automatically inserted when the form was submitted - while the superagent had it as the last property, as is generated with the code from the docs.

However, when I put the file as the last property everything worked.

In order, it's the superagent library XHR request, then my Fetch request followed by my own XHR.

I don't know if this is intended behaviour with XHR or something but it really took me some time to realize 😅

[klauspost]

https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html

Important
When constructing your request, make sure that the file field is the last field in the form.

[zanzlender]

Ok I didn't know that... But thanks for confirming my issue 😂