OSS client PutObject not support file larger than 5Gi

[AlbeeSo]

Pre-requisites

• I have double-checked my configuration
• I can confirm the issue exists when I tested with :latest
• I have searched existing issues and could not find a match for this bug
• I'd like to contribute the fix myself (see contributing guide)

What happened/what did you expect to happen?

When I tried to put a huge output file larger than 5Gi to aliyun OSS, I received such error messages in wait container:

time="2024-04-03T10:10:37.725Z" level=info msg="OSS Save path: /tmp/argo/outputs/artifacts/out.tgz, key: argo-workflows/test/big"
time="2024-04-03T10:10:37.804Z" level=warning msg="failed to put file: oss: service returned error: StatusCode=400, ErrorCode=InvalidArgument, ErrorMessage=\"Post body size must be less than 5G.\", RequestId=660D2B1DAFD89A3732CCE935, Ec=0017-00000102"
time="2024-04-03T10:10:37.804Z" level=warning msg="Non-transient error: oss: service returned error: StatusCode=400, ErrorCode=InvalidArgument, ErrorMessage=\"Post body size must be less than 5G.\", RequestId=660D2B1DAFD89A3732CCE935, Ec=0017-00000102"
time="2024-04-03T10:10:37.804Z" level=info msg="Save artifact" artifactName=out duration=79.453168ms error="oss: service returned error: StatusCode=400, ErrorCode=InvalidArgument, ErrorMessage=\"Post body size must be less than 5G.\", RequestId=660D2B1DAFD89A3732CCE935, Ec=0017-00000102" key=argo-workflows/test/big
time="2024-04-03T10:10:37.804Z" level=error msg="executor error: oss: service returned error: StatusCode=400, ErrorCode=InvalidArgument, ErrorMessage=\"Post body size must be less than 5G.\", RequestId=660D2B1DAFD89A3732CCE935, Ec=0017-00000102"

Version

latest

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: artifact-
spec:
  entrypoint: main
  templates:
    - name: main
      container:
        image: alpine:latest
        command:
          - sh
          - -c
        args:
          - |
            mkdir -p /out
            dd if=/dev/random of=/out/testfile.txt bs=20M count=1024
            echo "created files!"
      outputs:
        artifacts:
          - name: out
            path: /out/testfile.txt
            oss:
              endpoint: http://oss-cn-zhangjiakou-internal.aliyuncs.com
              bucket: bucket
              key: argo-workflows/test/bigfile
              accessKeySecret:
                name: my-argo-workflow-credentials
                key: accessKey
              secretKeySecret:
                name: my-argo-workflow-credentials
                key: secretKey

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

[shuangkun]

It does seem to be a problem, could you please submit a PR to fix it?

[AlbeeSo]

It does seem to be a problem, could you please submit a PR to fix it?

Sure.

[agilgur5]

time="2024-04-03T10:10:37.804Z" level=error msg="executor error: oss: service returned error: StatusCode=400, ErrorCode=InvalidArgument, ErrorMessage=\"Post body size must be less than 5G.\", RequestId=660D2B1DAFD89A3732CCE935, Ec=0017-00000102"

That error message is coming from the artifact server -- it's a 400 error.

OSS client PutObject not support file larger than 5Gi

Why do you think this is a client error or bug?

[AlbeeSo]

time="2024-04-03T10:10:37.804Z" level=error msg="executor error: oss: service returned error: StatusCode=400, ErrorCode=InvalidArgument, ErrorMessage=\"Post body size must be less than 5G.\", RequestId=660D2B1DAFD89A3732CCE935, Ec=0017-00000102"

That error message is coming from the artifact server -- it's a 400 error.

OSS client PutObject not support file larger than 5Gi

Why do you think this is a client error or bug?

argo-workflows/workflow/artifacts/oss/oss.go

Line 342 in 0b625a6

return bucket.PutObjectFromFile(objectName, path)

Hi, OSS SDK client offers several functions help upload local files to server. And PutObjectFromFile uses Simple Upload which usually causes (or always from my experience) InvalidArgument or EntityTooLarge error when uploading files larger than 5Gi. https://www.alibabacloud.com/help/en/oss/you-cannot-upload-large-objects-by-using-simple-upload?spm=a3c0i.23458820.2359477120.55.7ef56e9boHbYeC
Multipart Upload is the recommended way.