JSCPD Doesn't Run Unless TEST_CASE_RUN is set #5483

ccolella-mdc · 2024-04-05T19:53:34Z

Is there an existing issue for this?

I have searched the existing issues

Current Behavior

JSCPD only runs if you specify TEST_CASE_RUN as true or false

Expected Behavior

JSCPD should still run even if you don't specify a value for TEST_CASE_RUN

Super-Linter version

v6.3.1

Relevant log output

  2024-04-05 01:20:31 [DEBUG]   Categorizing the following files: /github/workspace/.github/workflows/linting.yml /github/workspace
  2024-04-05 01:20:31 [DEBUG]   FILTER_REGEX_INCLUDE: , FILTER_REGEX_EXCLUDE: 
  2024-04-05 01:20:31 [DEBUG]   IGNORE_GENERATED_FILES has a valid boolean string value: false
  2024-04-05 01:20:31 [DEBUG]   IGNORE_GITIGNORED_FILES has a valid boolean string value: false
  2024-04-05 01:20:31 [DEBUG]   FILE: /github/workspace/.github/workflows/linting.yml, FILE_TYPE: yml, BASE_FILE: linting.yml, FILE_DIR_NAME: /github/workspace/.github/workflows
  2024-04-05 01:20:31 [DEBUG]   File:[/github/workspace/.github/workflows/linting.yml], File extension:[yml], File type: [/github/workspace/.github/workflows/linting.yml: ASCII text]
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/workflows/linting.yml is NOT a supported shell script. Skipping
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is a GitHub Actions file...
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/workflows/linting.yml is GitHub Actions file.
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is a Cloud Formation file...
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is an OpenAPI file...
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/workflows/linting.yml is NOT an OpenAPI descriptor
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is a Tekton file...
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is a Kubernetes descriptor...
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/workflows/linting.yml is NOT a Kubernetes descriptor
  2024-04-05 01:20:31 [DEBUG]   FILE: /github/workspace, FILE_TYPE: /github/workspace, BASE_FILE: workspace, FILE_DIR_NAME: /github
  2024-04-05 01:20:31 [DEBUG]   /github/workspace matches with /github/workspace. Adding it to the list of directories to lint for linters that are expected to lint the whole codebase
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/linters/.checkov.yml contains a 'directory:' configuration option
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/linters/.checkov.yml doesn't contain a 'directory:' statement
  2024-04-05 01:20:31 [DEBUG]   Adding /github/workspace to the list of directories to analyze with Checkov.
  ",
      "Stderr": ""
    }
  ]
  2024-04-05 01:20:31 [INFO]   Command output when building the file list:
  ------
  2024-04-05 01:20:31 [DEBUG]   Categorizing the following files: /github/workspace/.github/workflows/linting.yml /github/workspace
  2024-04-05 01:20:31 [DEBUG]   FILTER_REGEX_INCLUDE: , FILTER_REGEX_EXCLUDE: .*\.dll|MDClarityWeb\/.*\.min\.(js|css)|MDClarityWeb\/(js|css)\/modules\/.*
  2024-04-05 01:20:31 [DEBUG]   IGNORE_GENERATED_FILES has a valid boolean string value: false
  2024-04-05 01:20:31 [DEBUG]   IGNORE_GITIGNORED_FILES has a valid boolean string value: false
  2024-04-05 01:20:31 [DEBUG]   FILE: /github/workspace/.github/workflows/linting.yml, FILE_TYPE: yml, BASE_FILE: linting.yml, FILE_DIR_NAME: /github/workspace/.github/workflows
  2024-04-05 01:20:31 [DEBUG]   File:[/github/workspace/.github/workflows/linting.yml], File extension:[yml], File type: [/github/workspace/.github/workflows/linting.yml: ASCII text]
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/workflows/linting.yml is NOT a supported shell script. Skipping
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is a GitHub Actions file...
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/workflows/linting.yml is GitHub Actions file.
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is a Cloud Formation file...
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is an OpenAPI file...
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/workflows/linting.yml is NOT an OpenAPI descriptor
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is a Tekton file...
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/workflows/linting.yml is a Kubernetes descriptor...
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/workflows/linting.yml is NOT a Kubernetes descriptor
  2024-04-05 01:20:31 [DEBUG]   FILE: /github/workspace, FILE_TYPE: /github/workspace, BASE_FILE: workspace, FILE_DIR_NAME: /github
  2024-04-05 01:20:31 [DEBUG]   /github/workspace matches with /github/workspace. Adding it to the list of directories to lint for linters that are expected to lint the whole codebase
  2024-04-05 01:20:31 [DEBUG]   Checking if /github/workspace/.github/linters/.checkov.yml contains a 'directory:' configuration option
  2024-04-05 01:20:31 [DEBUG]   /github/workspace/.github/linters/.checkov.yml doesn't contain a 'directory:' statement
  2024-04-05 01:20:31 [DEBUG]   Adding /github/workspace to the list of directories to analyze with Checkov.
  ------
  2024-04-05 01:20:31 [DEBUG]   Stderr when building the file list is empty
  2024-04-05 01:20:31 [INFO]   Successfully gathered list of files...
  2024-04-05 01:20:31 [DEBUG]   Ending GitHub Actions log group: Super-Linter initialization

Steps To Reproduce

Run super-linter in github-actions with no value set for TEST_CASE_RUN

ex:

      - name: Run Super Linter
        uses: super-linter/super-linter@v6
        env:
          VALIDATE_ALL_CODEBASE: ${{ github.event_name == 'push' }}
          DEFAULT_BRANCH: master
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          CREATE_LOG_FILE: true
          LOG_LEVEL: 'DEBUG'

Anything else?

Relevant piece of code causing the issue:

super-linter/lib/functions/buildFileList.sh

Lines 239 to 243 in 25cb7af

    
           # JSCPD test cases are handled below because we first need to exclude non-relevant test cases 
        
           if [[ "${TEST_CASE_RUN}" == "false" ]]; then 
        
             debug "Add ${FILE} to the list of items to lint with JSCPD" 
        
             echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JSCPD" 
        
           fi

It should either default to false somewhere or this check should be changed to [[ "${TEST_CASE_RUN}" != "true" ]]"

The text was updated successfully, but these errors were encountered:

ferrarimarco · 2024-04-08T07:29:56Z

Hi! Thanks for reporting this.

Since super-linter v6, JSCPD lints the entire code base and not individual files.

That snippet that you reference is to handle testing that particular linter and it's not needed otherwise.

PS: TEST_CASE_RUN default to false already:

super-linter/lib/linter.sh

Line 95 in 25cb7af

TEST_CASE_RUN="${TEST_CASE_RUN:-"false"}"

ccolella-mdc · 2024-04-08T15:59:23Z

Sure but when I run it without a value for TEST_CASE_RUN JSCPD doesn’t run at all. When I run it with TEST_CASE_RUN: false it does run.

I should not have to include TEST_CASE_RUN to run JSCPD.

ferrarimarco · 2024-04-08T16:03:15Z

Sure but when I run it without a value for TEST_CASE_RUN JSCPD doesn’t run at all. When I run it with TEST_CASE_RUN: false it does run.

I should not have to include TEST_CASE_RUN to run JSCPD.

This is correct. Also, when you TEST_CASE_RUN=true, JSCPD runs in a way that checks duplicates within each file, which is not likely what you want.

Can you include the full debug log, not just a fragment, so we can see what's going on?

ccolella-mdc · 2024-04-08T16:11:06Z

TEST_CASE_RUN is not documented as a required input. If it is required the docs should say so.

Here's the full log file

super-linter.log

ferrarimarco · 2024-04-08T16:31:46Z

Let me reopen this while we investigate.

On line 1814, the process to build the file list terminates successfully, but it doesn't seem to print the expected log line about JSCPD:

super-linter/lib/functions/buildFileList.sh

Lines 239 to 243 in 25cb7af

    
           # JSCPD test cases are handled below because we first need to exclude non-relevant test cases 
        
           if [[ "${TEST_CASE_RUN}" == "false" ]]; then 
        
             debug "Add ${FILE} to the list of items to lint with JSCPD" 
        
             echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JSCPD" 
        
           fi

,

despite TEST_CASE_RUN being false (line 1782 in your log)

Export the TEST_CASE_RUN variable because subprocesses reference it when building the file list. Fix #5483

ferrarimarco · 2024-04-08T16:47:26Z

I was able to reproduce this. #5499 should fix the bug :)

Thanks for reporting!

ccolella-mdc · 2024-04-08T16:49:22Z

Awesome thank you! And of course

- Export the TEST_CASE_RUN variable because subprocesses reference it when building the file list. - Remove a duplicate file that we didn't catch because Jscpd was not running due to the TEST_CASE_RUN variable not being exported. Fix #5483

ccolella-mdc added the bug Something isn't working label Apr 5, 2024

ferrarimarco closed this as not planned Won't fix, can't repro, duplicate, stale Apr 8, 2024

ferrarimarco reopened this Apr 8, 2024

ferrarimarco added a commit that referenced this issue Apr 8, 2024

fix: export test_case_run

d6c55bf

Export the TEST_CASE_RUN variable because subprocesses reference it when building the file list. Fix #5483

ferrarimarco mentioned this issue Apr 8, 2024

Export TEST_CASE_RUN #5499

Merged

8 tasks

ferrarimarco closed this as completed in #5499 Apr 8, 2024

github-actions bot mentioned this issue Apr 8, 2024

chore(main): release 6.4.0 #5482

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JSCPD Doesn't Run Unless TEST_CASE_RUN is set #5483

JSCPD Doesn't Run Unless TEST_CASE_RUN is set #5483

ccolella-mdc commented Apr 5, 2024 •

edited

ferrarimarco commented Apr 8, 2024

ccolella-mdc commented Apr 8, 2024

ferrarimarco commented Apr 8, 2024

ccolella-mdc commented Apr 8, 2024

ferrarimarco commented Apr 8, 2024

ferrarimarco commented Apr 8, 2024

ccolella-mdc commented Apr 8, 2024

JSCPD Doesn't Run Unless TEST_CASE_RUN is set #5483

JSCPD Doesn't Run Unless TEST_CASE_RUN is set #5483

Comments

ccolella-mdc commented Apr 5, 2024 • edited

Is there an existing issue for this?

Current Behavior

Expected Behavior

Super-Linter version

Relevant log output

Steps To Reproduce

Anything else?

ferrarimarco commented Apr 8, 2024

ccolella-mdc commented Apr 8, 2024

ferrarimarco commented Apr 8, 2024

ccolella-mdc commented Apr 8, 2024

ferrarimarco commented Apr 8, 2024

ferrarimarco commented Apr 8, 2024

ccolella-mdc commented Apr 8, 2024

ccolella-mdc commented Apr 5, 2024 •

edited