Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix 3871 workflow concurrency #3903

Merged
merged 2 commits into from
Jul 6, 2023
Merged

Conversation

Mzack9999
Copy link
Member

@Mzack9999 Mzack9999 commented Jul 4, 2023

Proposed changes

This PR attempts to mitigate nested template execution within a workflow context by increasing the concurrency of the sizedwaitgroup structure by one unit.

Notes:

  • This operation should be done at each nested level where the template being executed is a workflow, but this should cover most cases.
  • A better approach is switching back to a declarative execution (as in the old PR improved workflow logic #142)

Example

$ cat test-workflow.yaml
id: test-workflow

info:
  name: test-workflow
  author: aaa
  severity: info
  description: test


workflows:
  - template: baidu.yaml
    matchers:
      - name: baidu-detect
        subtemplates:
          - template: ali.yaml

$ cat ali.yaml
id: ali

info:
  name: ali
  author: aaa
  severity: info
  description: test

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    host-redirects: true
    max-redirects: 2
    matchers:
      - type: word
        name: ali-detect
        words:
          - "ali.com"
        part: body

$ cat baidu.yaml
id: baidu

info:
  name: baidu
  author: aaa
  severity: info
  description: test

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    host-redirects: true
    max-redirects: 2
    matchers:
      - type: word
        name: baidu-detect
        words:
          - "baidu"
        part: body

$ go run . -w test-workflow.yaml -u  http://baidu.com -c 1          

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.8

                projectdiscovery.io

[WRN] Found 24 template[s] loaded with deprecated paths, update before v3 for continued support.
[INF] Current nuclei version: v2.9.8 (latest)
[INF] Current nuclei-templates version: v9.5.4 (latest)
[INF] Workflows loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[INF] No results found. Better luck next time!

Checklist

  • Pull request is created against the dev branch
  • All checks passed (lint, unit/integration/regression tests etc.) with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

@Mzack9999 Mzack9999 added the Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. label Jul 4, 2023
@Mzack9999 Mzack9999 self-assigned this Jul 4, 2023
@Mzack9999 Mzack9999 marked this pull request as ready for review July 4, 2023 09:15
Copy link
Contributor

@RamanaReddy0M RamanaReddy0M left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

From ^example:

 go run . -w workflow-test.yaml -u  http://baidu.com -c 1 -dreq

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.8

                projectdiscovery.io

[INF] Current nuclei version: v2.9.8 (outdated)
[INF] Current nuclei-templates version: v9.5.4 (latest)
[INF] Workflows loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[INF] [baidu] Dumped HTTP request for http://baidu.com

GET / HTTP/1.1
Host: baidu.com
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2224.3 Safari/537.36
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip

[INF] No results found. Better luck next time!

@ehsandeep ehsandeep merged commit 305ac6a into dev Jul 6, 2023
11 checks passed
@ehsandeep ehsandeep deleted the bugfix-3871-workflow-concurrency branch July 6, 2023 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

nuclei workflow get stuck when concurrency set to 1 at certain workflows
3 participants