Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

txscript: backport tokenizer from dcrd #1769

Merged

Commits on Nov 17, 2021

  1. Configuration menu
    Copy the full SHA
    843d760 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    47806df View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    bcb9643 View commit details
    Browse the repository at this point in the history
  4. txscript: Introduce zero-alloc script tokenizer.

    This implements an efficient and zero-allocation script tokenizer that
    is exported to both provide a new capability to tokenize scripts to
    external consumers of the API as well as to serve as a base for
    refactoring the existing highly inefficient internal code.
    
    It is important to note that this tokenizer is intended to be used in
    consensus critical code in the future, so it must exactly follow the
    existing semantics.
    
    The current script parsing mechanism used throughout the txscript module
    is to fully tokenize the scripts into an array of internal parsed
    opcodes which are then examined and passed around in order to implement
    virtually everything related to scripts.
    
    While that approach does simplify the analysis of certain scripts and
    thus provide some nice properties in that regard, it is both extremely
    inefficient in many cases, and makes it impossible for external
    consumers of the API to implement any form of custom script analysis
    without manually implementing a bunch of error prone tokenizing code or,
    alternatively, the script engine exposing internal structures.
    
    For example, as shown by profiling the total memory allocations of an
    initial sync, the existing script parsing code allocates a total of
    around 295.12GB, which equates to around 50% of all allocations
    performed.  The zero-alloc tokenizer this introduces will allow that to
    be reduced to virtually zero.
    
    The following is a before and after comparison of tokenizing a large
    script with a high opcode count using the existing code versus the
    tokenizer this introduces for both speed and memory allocations:
    
    benchmark                    old ns/op     new ns/op     delta
    BenchmarkScriptParsing-8     63464         677           -98.93%
    
    benchmark                    old allocs     new allocs     delta
    BenchmarkScriptParsing-8     1              0              -100.00%
    
    benchmark                    old bytes     new bytes     delta
    BenchmarkScriptParsing-8     311299        0             -100.00%
    
    The following is an overview of the changes:
    
    - Introduce new error code ErrUnsupportedScriptVersion
    - Implement zero-allocation script tokenizer
    - Add a full suite of tests to ensure the tokenizer works as intended
      and follows the required consensus semantics
    - Add an example of using the new tokenizer to count the number of
      opcodes in a script
    - Update README.md to include the new example
    - Update script parsing benchmark to use the new tokenizer
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    c997417 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    0997842 View commit details
    Browse the repository at this point in the history
  6. txscript: Optimize script disasm.

    This converts the DisasmString function to make use of the new
    zero-allocation script tokenizer instead of the far less efficient
    parseScript thereby significantly optimizing the function.
    
    In order to facilitate this, the opcode disassembly functionality is
    split into a separate function called disasmOpcode that accepts the
    opcode struct and data independently as opposed to requiring a parsed
    opcode.  The new function also accepts a pointer to a string builder so
    the disassembly can be more efficiently be built.
    
    While here, the comment is modified to explicitly call out the script
    version semantics.
    
    The following is a before and after comparison of a large script:
    
    benchmark                   old ns/op     new ns/op     delta
    BenchmarkDisasmString-8     102902        40124         -61.01%
    
    benchmark                   old allocs     new allocs     delta
    BenchmarkDisasmString-8     46             51             +10.87%
    
    benchmark                   old bytes     new bytes     delta
    BenchmarkDisasmString-8     389324        130552        -66.47%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    f980c9a View commit details
    Browse the repository at this point in the history
  7. txscript: Introduce raw script sighash calc func.

    This introduces a new function named calcSignatureHashRaw which accepts
    the raw script bytes to calculate the script hash versus requiring the
    parsed opcode only to unparse them later in order to make it more
    flexible for working with raw scripts.
    
    Since there are several places in the rest of the code that currently
    only have access to the parsed opcodes, this modifies the existing
    calcSignatureHash to first unparse the script before calling the new
    function.
    
    Backport of decred/dcrd:f306a72a16eaabfb7054a26f9d9f850b87b00279
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    af757d3 View commit details
    Browse the repository at this point in the history
  8. txscript: Optimize CalcSignatureHash.

    This modifies the CalcSignatureHash function to make use of the new
    signature hash calculation function that accepts raw scripts without
    needing to first parse them.  Consequently, it also doubles as a slight
    optimization to the execution time and a significant reduction in the
    number of allocations.
    
    In order to convert the CalcScriptHash function and keep the same
    semantics, a new function named checkScriptParses is introduced which
    will quickly determine if a script can be fully parsed without failure
    and return the parse failure in the case it can't.
    
    The following is a before and after comparison of analyzing a large
    multiple input transaction:
    
    benchmark                  old ns/op     new ns/op     delta
    BenchmarkCalcSigHash-8     3627895       3619477       -0.23%
    
    benchmark                  old allocs     new allocs     delta
    BenchmarkCalcSigHash-8     1335           801            -40.00%
    
    benchmark                  old bytes     new bytes     delta
    BenchmarkCalcSigHash-8     1373812       1293354       -5.86%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    c19535b View commit details
    Browse the repository at this point in the history
  9. txscript/reference_test: Convert sighash calc test

    This converts the tests for calculating signature hashes to use the
    exported function which handles the raw script versus the now deprecated
    variant requiring parsed opcodes.
    
    Backport of 06f769e
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    c6f4caf View commit details
    Browse the repository at this point in the history
  10. txscript: Make isSmallInt accept raw opcode.

    This converts the isSmallInt function to accept an opcode as a byte
    instead of the internal opcode data struct in order to make it more
    flexible for raw script analysis.
    
    The comment is modified to explicitly call out the script version
    semantics.
    
    Finally, it updates all callers accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    583b740 View commit details
    Browse the repository at this point in the history
  11. txscript: Make asSmallInt accept raw opcode.

    This converts the asSmallInt function to accept an opcode as a byte
    instead of the internal opcode data struct in order to make it more
    flexible for raw script analysis.
    
    It also updates all callers accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    dfb1a67 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    05aa488 View commit details
    Browse the repository at this point in the history
  13. txscript: Optimize IsPayToPubKey

    This converts the IsPayToScriptHash function to analyze the raw script
    instead of using the far less efficient parseScript, thereby
    significantly optimizing the function.
    
    In order to accomplish this, it introduces four new functions:
    extractCompressedPubKey, extractUncompressedPubKey, extractPubKey, and
    isPubKeyScript.  The extractPubKey function makes use of
    extractCompressedPubKey and extractUncompressedPubKey to combine their
    functionality as a convenience and isPubKeyScript is defined in terms of
    extractPubKey.
    
    The extractCompressedPubKey works with the raw script bytes to
    simultaneously determine if the script is a pay-to-compressed-pubkey
    script, and in the case it is, extract and return the raw compressed
    pubkey bytes.
    
    Similarly, the extractUncompressedPubKey works in the same way except it
    determines if the script is a pay-to-uncompressed-pubkey script and
    returns the raw uncompressed pubkey bytes in the case it is.
    
    The extract function approach was chosen because it is common for
    callers to want to only extract relevant details from a script if the
    script is of the specific type.  Extracting those details requires
    performing the exact same checks to ensure the script is of the correct
    type, so it is more efficient to combine the two into one and define the
    type determination in terms of the result so long as the extraction does
    not require allocations.
    
    The following is a before and after comparison of analyzing a large
    script:
    
    benchmark                     old ns/op     new ns/op     delta
    BenchmarkIsPubKeyScript-8     62323         2.97          -100.00%
    
    benchmark                     old allocs     new allocs     delta
    BenchmarkIsPubKeyScript-8     1              0              -100.00%
    
    benchmark                     old bytes     new bytes     delta
    BenchmarkIsPubKeyScript-8     311299        0             -100.00%
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    99cb679 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    2d2608c View commit details
    Browse the repository at this point in the history
  15. txscript: Optimize IsPayToPubKeyHash

    This converts the IsPayToPubKeyHash function to analyze the raw script
    instead of using the far less efficient parseScript, thereby
    significantly optimization the function.
    
    In order to accomplish this, it introduces two new functions.  The first
    one is named extractPubKeyHash and works with the raw script bytes
    to simultaneously determine if the script is a pay-to-pubkey-hash script,
    and in the case it is, extract and return the hash.  The second new
    function is named isPubKeyHashScript and is defined in terms of the
    former.
    
    The extract function approach was chosen because it is common for
    callers to want to only extract relevant details from a script if the
    script is of the specific type.  Extracting those details requires
    performing the exact same checks to ensure the script is of the correct
    type, so it is more efficient to combine the two into one and define the
    type determination in terms of the result so long as the extraction does
    not require allocations.
    
    The following is a before and after comparison of analyzing a large
    script:
    
    benchmark                         old ns/op     new ns/op     delta
    BenchmarkIsPubKeyHashScript-8     62228         0.45          -100.00%
    
    benchmark                         old allocs     new allocs     delta
    BenchmarkIsPubKeyHashScript-8     1              0              -100.00%
    
    benchmark                         old bytes     new bytes     delta
    BenchmarkIsPubKeyHashScript-8     311299        0             -100.00%
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    c771f4f View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    665c298 View commit details
    Browse the repository at this point in the history
  17. txscript: Optimize IsPayToScriptHash.

    This converts the IsPayToScriptHash function to analyze the raw script
    instead of using the far less efficient parseScript thereby
    significantly optimizing the function.
    
    In order to accomplish this, it introduces two new functions.  The first
    one is named extractScriptHash and works with the raw script bytes to
    simultaneously determine if the script is a p2sh script, and in the case
    it is, extract and return the hash.  The second new function is named
    isScriptHashScript and is defined in terms of the former.
    
    The extract function approach was chosen because it is common for
    callers to want to only extract relevant details from a script if the
    script is of the specific type.  Extracting those details requires
    performing the exact same checks to ensure the script is of the correct
    type, so it is more efficient to combine the two into one and define the
    type determination in terms of the result so long as the extraction does
    not require allocations.
    
    Finally, this also deprecates the isScriptHash function that requires
    opcodes in favor of the new functions and modifies the comment on
    IsPayToScriptHash to explicitly call out the script version semantics.
    
    The following is a before and after comparison of analyzing a large
    script that is not a p2sh script:
    
    benchmark                        old ns/op     new ns/op     delta
    BenchmarkIsPayToScriptHash-8     62393         0.60          -100.00%
    
    benchmark                        old allocs     new allocs     delta
    BenchmarkIsPayToScriptHash-8     1              0              -100.00%
    
    benchmark                        old bytes     new bytes     delta
    BenchmarkIsPayToScriptHash-8     311299        0             -100.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    215af7f View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    4d31d15 View commit details
    Browse the repository at this point in the history
  19. txscript: Optimize IsMultisigScript.

    This converts the IsMultisigScript function to make use of the new
    tokenizer instead of the far less efficient parseScript thereby
    significantly optimizing the function.
    
    In order to accomplish this, it introduces two new functions.  The first
    one is named extractMultisigScriptDetails and works with the raw script
    bytes to simultaneously determine if the script is a multisignature
    script, and in the case it is, extract and return the relevant details.
    The second new function is named isMultisigScript and is defined in
    terms of the former.
    
    The extract function accepts the script version, raw script bytes, and a
    flag to determine whether or not the public keys should also be
    extracted.  The flag is provided because extracting pubkeys results in
    an allocation that the caller might wish to avoid.
    
    The extract function approach was chosen because it is common for
    callers to want to only extract relevant details from a script if the
    script is of the specific type.  Extracting those details requires
    performing the exact same checks to ensure the script is of the correct
    type, so it is more efficient to combine the two into one and define the
    type determination in terms of the result so long as the extraction does
    not require allocations.
    
    It is important to note that this new implementation intentionally has a
    semantic difference from the existing implementation in that it will now
    correctly identify a multisig script with zero pubkeys whereas
    previously it incorrectly required at least one pubkey.  This change is
    acceptable because the function only deals with standardness rather than
    consensus rules.
    
    Finally, this also deprecates the isMultiSig function that requires
    opcodes in favor of the new functions and deprecates the error return on
    the export IsMultisigScript function since it really does not make sense
    given the purpose of the function.
    
    The following is a before and after comparison of analyzing both a large
    script that is not a multisig script and a 1-of-2 multisig public key
    script:
    
    benchmark                            old ns/op     new ns/op     delta
    BenchmarkIsMultisigScriptLarge-8     64166         5.52          -99.99%
    BenchmarkIsMultisigScript-8          630           59.4          -90.57%
    
    benchmark                            old allocs     new allocs     delta
    BenchmarkIsMultisigScriptLarge-8     1              0              -100.00%
    BenchmarkIsMultisigScript-8          1              0              -100.00%
    
    benchmark                            old bytes     new bytes     delta
    BenchmarkIsMultisigScriptLarge-8     311299        0             -100.00%
    BenchmarkIsMultisigScript-8          2304          0             -100.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    0eaae26 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    02dab16 View commit details
    Browse the repository at this point in the history
  21. txscript: Optimize IsMultisigSigScript.

    This converts the IsMultisigSigScript function to analyze the raw script
    and make use of the new tokenizer instead of the far less efficient
    parseScript thereby significantly optimizing the function.
    
    In order to accomplish this, it first rejects scripts that can't
    possibly fit the bill due to the final byte of what would be the redeem
    script not being the appropriate opcode or the overall script not having
    enough bytes.  Then, it uses a new function that is introduced named
    finalOpcodeData that uses the tokenizer to return any data associated
    with the final opcode in the signature script (which will be nil for
    non-push opcodes or if the script fails to parse) and analyzes it as if
    it were a redeem script when it is non nil.
    
    It is also worth noting that this new implementation intentionally has
    the same semantic difference from the existing implementation as the
    updated IsMultisigScript function in regards to allowing zero pubkeys
    whereas previously it incorrectly required at least one pubkey.
    
    Finally, the comment is modified to explicitly call out the script
    version semantics.
    
    The following is a before and after comparison of analyzing a large
    script that is not a multisig script and both a 1-of-2 multisig public
    key script (which should be false) and a signature script comprised of a
    pay-to-script-hash 1-of-2 multisig redeem script (which should be true):
    
    benchmark                               old ns/op     new ns/op     delta
    BenchmarkIsMultisigSigScriptLarge-8     69328         2.93          -100.00%
    BenchmarkIsMultisigSigScript-8          2375          146           -93.85%
    
    benchmark                               old allocs     new allocs     delta
    BenchmarkIsMultisigSigScriptLarge-8     5              0              -100.00%
    BenchmarkIsMultisigSigScript-8          3              0              -100.00%
    
    benchmark                               old bytes     new bytes     delta
    BenchmarkIsMultisigSigScriptLarge-8     330035        0             -100.00%
    BenchmarkIsMultisigSigScript-8          9472          0             -100.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    34ebf0f View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    ce1513d View commit details
    Browse the repository at this point in the history
  23. txscript: Optimize IsPushOnlyScript.

    This converts the IsPushOnlyScript function to make use of the new
    tokenizer instead of the far less efficient parseScript thereby
    significantly optimizing the function.
    
    It also deprecates the isPushOnly function that requires opcodes in
    favor of the new function and modifies the comment on IsPushOnlyScript
    to explicitly call out the script version semantics.
    
    The following is a before and after comparison of analyzing a large
    script:
    
    benchmark                       old ns/op     new ns/op     delta
    BenchmarkIsPushOnlyScript-8     62412         622           -99.00%
    
    benchmark                       old allocs     new allocs     delta
    BenchmarkIsPushOnlyScript-8     1              0              -100.00%
    
    benchmark                       old bytes     new bytes     delta
    BenchmarkIsPushOnlyScript-8     311299        0             -100.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    2d5f7cf View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    e422d42 View commit details
    Browse the repository at this point in the history
  25. txscript: Optimize IsPayToWitnessPubKeyHash

    This converts the IsPayToWitnessPubKeyHash function to analyze the raw
    script instead of the far less efficient parseScript, thereby
    significantly optimizing the function.
    
    In order to accomplish this, it introduces two new functions. The first
    one is named extractWitnessPubKeyHash and works with the raw script
    bytes to simultaneously deteremine if the script is a p2wkh, and in case
    it is, extract and return the hash. The second new function is name
    isWitnessPubKeyHashScript which is defined in terms of the former.
    
    The extract function is approach was chosen because it is common for
    callers to want to only extract relevant details from the script if the
    script is of the specific type. Extracting those details requires the
    exact same checks to ensure the script is of the correct type, so it is
    more efficient to combine the two and define the type determination in
    terms of the result so long as the extraction does not require
    allocations.
    
    Finally, this deprecates the isWitnessPubKeyHash function that requires
    opcodes in favor of the new functions and modifies the comment on
    IsPayToWitnessPubKeyHash to explicitly call out the script version
    semantics.
    
    The following is a before and after comparison of executing
    IsPayToWitnessPubKeyHash on a large script:
    
    benchmark                          old ns/op     new ns/op     delta
    BenchmarkIsWitnessPubKeyHash-8     68927         0.53          -100.00%
    
    benchmark                          old allocs     new allocs     delta
    BenchmarkIsWitnessPubKeyHash-8     1              0              -100.00%
    
    benchmark                          old bytes     new bytes     delta
    BenchmarkIsWitnessPubKeyHash-8     311299        0             -100.00%
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    54d08eb View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    728ce10 View commit details
    Browse the repository at this point in the history
  27. txscript: Optimize IsPayToWitnessScriptHash

    This converts the IsPayToWitnessScriptHash function to analyze the raw
    script instead of using the far less efficient parseScript, thereby
    significantly optimizing the function.
    
    In order to accomplish this, it introduces two new functions. The first
    one is named extractWitnessScriptHash and works with the raw script byte
    to simultaneously deteremine if the script is a p2wsh script, and in the
    case that is is, extract and return the hash. The second new function is
    named isWitnessScriptHashScript and is defined in terms of the former.
    
    The extract function approach was chosed because it is common for
    callers to want to only extract relevant details from a script if the
    script is of the specific type. Extracting those details requires
    performing the exact same checks to ensure the script is of the correct
    type, so it is more efficient to combine the two into one and define the
    type determination in terms of the result, so long as the extraction
    does not require allocations.
    
    Finally, this also deprecates the isWitnessScriptHash function that
    requires opcodes in favor of the new functions and modifies the comment
    on IsPayToWitnessScriptHash to call out the script version semantics.
    
    The following is a before and after comparison of executing
    IsPayToWitnessScriptHash on a large script:
    
    benchmark                          old ns/op     new ns/op     delta
    BenchmarkIsWitnessScriptHash-8     62774         0.63          -100.00%
    
    benchmark                          old allocs     new allocs     delta
    BenchmarkIsWitnessScriptHash-8     1              0              -100.00%
    
    benchmark                          old bytes     new bytes     delta
    BenchmarkIsWitnessScriptHash-8     311299        0             -100.00%
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    55a6bb5 View commit details
    Browse the repository at this point in the history
  28. txscript: Add benchmark for IsNullData

    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    e4118c9 View commit details
    Browse the repository at this point in the history
  29. txscript: Optimize IsNullData

    This converts the IsNullData function to analyze the raw script instead
    of using the far less efficient parseScript, thereby significantly
    optimizing the function.
    
    The following is a before and after comparison of analyzing a large
    script:
    
    benchmark                       old ns/op     new ns/op     delta
    BenchmarkIsNullDataScript-8     62495         2.65          -100.00%
    
    benchmark                       old allocs     new allocs     delta
    BenchmarkIsNullDataScript-8     1              0              -100.00%
    
    benchmark                       old bytes     new bytes     delta
    BenchmarkIsNullDataScript-8     311299        0             -100.00%
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    5f771c1 View commit details
    Browse the repository at this point in the history
  30. Configuration menu
    Copy the full SHA
    77660b7 View commit details
    Browse the repository at this point in the history
  31. txscript: Optimize IsUnspendable.

    This converts the IsUnspendable function to make use of a combination of
    raw script analysis and the new tokenizer instead of the far less
    efficient parseScript thereby significantly optimizing the function.
    
    It is important to note that this new implementation intentionally has a
    semantic difference from the existing implementation in that it will now
    report scripts that are larger than the max allowed script size are
    unspendable as well.
    
    Finally, the comment is modified to explicitly call out the script
    version semantics.
    
    Note: this function was recently optimized in master, so the gains here
    are less noticable than other optimizations.
    
    The following is a before and after comparison of analyzing a large
    script:
    
    benchmark                    old ns/op     new ns/op     delta
    BenchmarkIsUnspendable-8     656           584           -10.98%
    
    benchmark                    old allocs     new allocs     delta
    BenchmarkIsUnspendable-8     1              0              -100.00%
    
    benchmark                    old bytes     new bytes     delta
    BenchmarkIsUnspendable-8     1             0             -100.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    e98b7c1 View commit details
    Browse the repository at this point in the history
  32. txscript/engine: Optimize new engine push only script

    This modifies the check for whether or not a pay-to-script-hash
    signature script is a push only script to make use of the new and more
    efficient raw script function.
    
    Also, since the script will have already been checked further above when
    the ScriptVerifySigPushOnly flags is set, avoid checking it again in
    that case.
    
    Backport of af67951
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    549a1f2 View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    a59e01c View commit details
    Browse the repository at this point in the history
  34. Configuration menu
    Copy the full SHA
    1be3450 View commit details
    Browse the repository at this point in the history
  35. txscript/engine: Check ps2h push before parsing script

    This moves the check for non push-only pay-to-script-hash signature
    scripts before the script parsing logic when creating a new engine
    instance to avoid the extra overhead in the error case.
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    98dd6a9 View commit details
    Browse the repository at this point in the history
  36. Configuration menu
    Copy the full SHA
    8316a06 View commit details
    Browse the repository at this point in the history
  37. txscript: Optimize GetSigOpCount.

    This converts the GetSigOpCount function to make use of the new
    tokenizer instead of the far less efficient parseScript thereby
    significantly optimizing the function.
    
    A new function named countSigOpsV0 which accepts the raw script is
    introduced to perform the bulk of the work so it can be reused for
    precise signature operation counting as well in a later commit.  It
    retains the same semantics in terms of counting the number of signature
    operations either up to the first parse error or the end of the script
    in the case it parses successfully as required by consensus.
    
    Finally, this also deprecates the getSigOpCount function that requires
    opcodes in favor of the new function and modifies the comment on
    GetSigOpCount to explicitly call out the script version semantics.
    
    The following is a before and after comparison of analyzing a large
    script:
    
    benchmark                    old ns/op     new ns/op     delta
    BenchmarkGetSigOpCount-8     61051         677           -98.89%
    
    benchmark                    old allocs     new allocs     delta
    BenchmarkGetSigOpCount-8     1              0              -100.00%
    
    benchmark                    old bytes     new bytes     delta
    BenchmarkGetSigOpCount-8     311299        0             -100.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    4d5c0b2 View commit details
    Browse the repository at this point in the history
  38. Configuration menu
    Copy the full SHA
    cc802d1 View commit details
    Browse the repository at this point in the history
  39. txscript: Optimize GetPreciseSigOpCount.

    This converts the GetPreciseSigOpCount function to use a combination of
    raw script analysis and the new tokenizer instead of the far less
    efficient parseScript thereby significantly optimizing the function.
    
    In particular it uses the recently converted isScriptHashScript,
    IsPushOnlyScript, and countSigOpsV0 functions along with the recently
    added finalOpcodeData functions.
    
    It also modifies the comment to explicitly call out the script version
    semantics.
    
    The following is a before and after comparison of analyzing a large
    script:
    
    benchmark                           old ns/op     new ns/op     delta
    BenchmarkGetPreciseSigOpCount-8     130223        742           -99.43%
    
    benchmark                           old allocs     new allocs     delta
    BenchmarkGetPreciseSigOpCount-8     3              0              -100.00%
    
    benchmark                           old bytes     new bytes     delta
    BenchmarkGetPreciseSigOpCount-8     623367        0             -100.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    a4a21c0 View commit details
    Browse the repository at this point in the history
  40. Configuration menu
    Copy the full SHA
    26d63c6 View commit details
    Browse the repository at this point in the history
  41. txscript: Optimize GetWitnessSigOpCount

    This converts the GetWitnessSigOpCount function to use a combination of
    raw script analysis and the new tokenizer instead of the far less
    efficeint parseScript, thereby significantly optimizing the funciton.
    
    In particular, it use the recently added countSigOpsv0 in precise mode
    to avoid calling paseScript.
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    77863f5 View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    69d560c View commit details
    Browse the repository at this point in the history
  43. txscript: Make typeOfScript accept raw script.

    This converts the typeOfScript function to accept a script version and
    raw script instead of an array of internal parsed opcodes in order to
    make it more flexible for raw script analysis.
    
    Also, this adds a comment to CalcScriptInfo to call out the specific
    version semantics and deprecates the function since nothing currently
    uses it, and the relevant information can now be obtained by callers
    more directly through the use of the new script tokenizer.
    
    All other callers are updated accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    9b06388 View commit details
    Browse the repository at this point in the history
  44. txscript: Optimize typeOfScript pay-to-script-hash.

    This begins the process of converting the typeOfScript function to use a
    combination of raw script analysis and the new tokenizer instead of the
    far less efficient parsed opcodes with the intent of significantly
    optimizing the function.
    
    In order to ease the review process, each script type will be converted
    in a separate commit and the typeOfScript function will be updated such
    that the script is only parsed as a fallback for the cases that are not
    already converted to more efficient raw script variants.
    
    In particular, for this commit, since the ability to detect
    pay-to-script-hash via raw script analysis is now available, the
    function is simply updated to make use of it.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    3b86e0a View commit details
    Browse the repository at this point in the history
  45. Configuration menu
    Copy the full SHA
    671b5fe View commit details
    Browse the repository at this point in the history
  46. txscript: Optimize typeOfScript multisig.

    This continues the process of converting the typeOfScript function to
    use a combination of raw script analysis and the new tokenizer instead
    of the far less efficient parsed opcodes.
    
    In particular, for this commit, since the ability to detect multisig
    scripts via the new tokenizer is now available, the function is simply
    updated to make use of it.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    71bf51e View commit details
    Browse the repository at this point in the history
  47. Configuration menu
    Copy the full SHA
    6e86f0d View commit details
    Browse the repository at this point in the history
  48. txscript: Optimze typeOfScript pay-to-pubkey

    This continues the process of converting the typeOfScript function to
    use a combination of raw script analysis and the new tokenizer instead
    of the face less efficient parsed opcodes, with the intent of
    significantly optimizing the function.
    
    In particular, it converts the detection of pay-to-pubkey scripts to use
    raw script analysis.
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    8b64adc View commit details
    Browse the repository at this point in the history
  49. Configuration menu
    Copy the full SHA
    1133ea0 View commit details
    Browse the repository at this point in the history
  50. txscript: Optimize typeOfScript pay-to-pubkey-hash.

    This continues the process of converting the typeOfScript function to
    use a combination of raw script analysis and the new tokenizer instead
    of the far less efficient parsed opcodes.
    
    In particular, it converts the detection of pay-to-pubkey-hash scripts
    to use raw script analysis.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    13f6462 View commit details
    Browse the repository at this point in the history
  51. Configuration menu
    Copy the full SHA
    d02f97e View commit details
    Browse the repository at this point in the history
  52. txscript: Optimize typeOfScript for null data scripts

    This continues the process of converting the typeOfScript function to
    use a combination of raw script analysize and the tokenizer instead of
    parsed opcode, with the intent of significanty optimizing the function.
    
    In particular, it converts the detection of null data scripts to use raw
    script analysis.
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    d80863d View commit details
    Browse the repository at this point in the history
  53. Configuration menu
    Copy the full SHA
    78046b3 View commit details
    Browse the repository at this point in the history
  54. txscript: Optimize typeOfScript witness-pubkey-hash

    This continues the process of converting the typeOfScript function to
    use a combination of raw script analysis and the new tokenizer instead
    of the far less efficient parsed opcodes.
    
    In particular, it converts the detection of witness pubkey hash scripts
    to use raw script analysis and the new tokenizer.
    
    The following is a before and after comparison of analyzing a large
    script:
    
    benchmark                          old ns/op     new ns/op     delta
    BenchmarkIsWitnessPubKeyHash-8     61688         62839         +1.87%
    
    benchmark                          old allocs     new allocs     delta
    BenchmarkIsWitnessPubKeyHash-8     1              1              +0.00%
    
    benchmark                          old bytes     new bytes     delta
    BenchmarkIsWitnessPubKeyHash-8     311299        311299        +0.00%
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    847a262 View commit details
    Browse the repository at this point in the history
  55. txscript: Optimize typeOfScript for witness-script-hash

    This concludes the process of converting the typeOfScript function to
    use a combination of raw script analysis and the new tokenizer instead
    of the far less efficient parsed opcodes.
    
    In particular, it converts the detection of witness script hash scripts
    to use raw script analysis and the new tokenizer.
    
    With all of the limbs now useing optimized variants, the following is a
    before an after comparison of calling GetScriptClass on a large script:
    
    benchmark                     old ns/op     new ns/op     delta
    BenchmarkGetScriptClass-8     61515         15.3          -99.98%
    
    benchmark                     old allocs     new allocs     delta
    BenchmarkGetScriptClass-8     1              0              -100.00%
    
    benchmark                     old bytes     new bytes     delta
    BenchmarkGetScriptClass-8     311299        0             -100.00%
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    1a60e11 View commit details
    Browse the repository at this point in the history
  56. Configuration menu
    Copy the full SHA
    43846b1 View commit details
    Browse the repository at this point in the history
  57. txscript: Convert CalcScriptInfo.

    This converts CalcScriptInfo and dependent expectedInputs to make use of
    the new script tokenizer as well as several of the other recently added
    raw script analysis functions in order to remove the reliance on parsed
    opcodes as a step towards utlimately removing them altogether.
    
    It is worth noting that this has the side effect of significantly
    optimizing the function as well, however, since it is deprecated, no
    benchmarks are provided.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    705d24c View commit details
    Browse the repository at this point in the history
  58. Configuration menu
    Copy the full SHA
    6c212fd View commit details
    Browse the repository at this point in the history
  59. Configuration menu
    Copy the full SHA
    8c54905 View commit details
    Browse the repository at this point in the history
  60. txscript: Optimize CalcMultiSigStats.

    This converts the CalcMultiSigStats function to make use of the new
    extractMultisigScriptDetails function instead of the far less efficient
    parseScript thereby significantly optimizing the function.
    
    The tests are also updated accordingly.
    
    The following is a before and after comparison of analyzing a standard
    multisig script:
    
    benchmark                    old ns/op    new ns/op    delta
    ---------------------------------------------------------------
    BenchmarkCalcMultiSigStats   972          79.5         -91.82%
    
    benchmark                    old allocs   new allocs   delta
    ---------------------------------------------------------------
    BenchmarkCalcMultiSigStats   1            0            -100.00%
    
    benchmark                    old bytes    new bytes    delta
    ---------------------------------------------------------------
    BenchmarkCalcMultiSigStats   2304         0            -100.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    7791f92 View commit details
    Browse the repository at this point in the history
  61. txscript: Add benchmark for PushedData.

    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    c4f6302 View commit details
    Browse the repository at this point in the history
  62. txscript: Optimize PushedData.

    This converts the PushedData function to make use of the new tokenizer
    instead of the far less efficient parseScript thereby significantly
    optimizing the function.
    
    Also, the comment is modified to explicitly call out the script version
    semantics.
    
    The following is a before and after comparison of extracting the data
    from a very large script:
    
    benchmark                 old ns/op     new ns/op     delta
    BenchmarkPushedData-8     64837         1790          -97.24%
    
    benchmark                 old allocs     new allocs     delta
    BenchmarkPushedData-8     7              6              -14.29%
    
    benchmark                 old bytes     new bytes     delta
    BenchmarkPushedData-8     312816        1520          -99.51%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    0a4f228 View commit details
    Browse the repository at this point in the history
  63. txscript: Make canonicalPush accept raw opcode.

    This renames the canonicalPush function to isCanonicalPush and converts
    it to accept an opcode as a byte and the associate data as a byte slice
    instead of the internal parse opcode data struct in order to make it
    more flexible for raw script analysis.
    
    It also updates all callers and tests accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    da9fdab View commit details
    Browse the repository at this point in the history
  64. Configuration menu
    Copy the full SHA
    81b8032 View commit details
    Browse the repository at this point in the history
  65. Configuration menu
    Copy the full SHA
    6ec9b73 View commit details
    Browse the repository at this point in the history
  66. txscript: Optimize ExtractAtomicSwapDataPushes.

    This converts the ExtractAtomicSwapDataPushes function to make use of
    the new tokenizer instead of the far less efficient parseScript thereby
    significantly optimizing the function.
    
    The new implementation is designed such that it should be fairly easy to
    move the function into the atomic swap tools where it more naturally
    belongs now that the tokenizer makes it possible to analyze scripts
    outside of the txscript module.  Consequently, this also deprecates the
    function.
    
    The following is a before and after comparison of attempting to extract
    from both a typical atomic swap script and a very large non-atomic swap
    script:
    
    benchmark                                       old ns/op     new ns/op     delta
    BenchmarkExtractAtomicSwapDataPushesLarge-8     61332         44.4          -99.93%
    BenchmarkExtractAtomicSwapDataPushes-8          990           260           -73.74%
    
    benchmark                                       old allocs     new allocs     delta
    BenchmarkExtractAtomicSwapDataPushesLarge-8     1              0              -100.00%
    BenchmarkExtractAtomicSwapDataPushes-8          2              1              -50.00%
    
    benchmark                                       old bytes     new bytes     delta
    BenchmarkExtractAtomicSwapDataPushesLarge-8     311299        0             -100.00%
    BenchmarkExtractAtomicSwapDataPushes-8          3168          96            -96.97%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    367a75a View commit details
    Browse the repository at this point in the history
  67. Configuration menu
    Copy the full SHA
    33ee3e2 View commit details
    Browse the repository at this point in the history
  68. txscript: Optimize ExtractPkScriptAddrs scripthash.

    This begins the process of converting the ExtractPkScriptAddrs function
    to use the optimized extraction functions recently introduced as part of
    the typeOfScript conversion.
    
    In order to ease the review process, the detection of each script type
    will be converted in a separate commit such that the script is only
    parsed as a fallback for the cases that are not already converted to
    more efficient variants.
    
    In particular, this converts the detection for pay-to-script-hash
    scripts.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    055be98 View commit details
    Browse the repository at this point in the history
  69. txscript: Optimize ExtractPkScriptAddrs pubkeyhash.

    This continues the process of converting the ExtractPkScriptAddrs
    function to use the optimized extraction functions recently introduced
    as part of the typeOfScript conversion.
    
    In particular, this converts the detection for pay-to-pubkey-hash
    scripts.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    16bd663 View commit details
    Browse the repository at this point in the history
  70. txscript: Optimize ExtractPkScriptAddrs pubkey.

    This continues the process of converting the ExtractPkScriptAddrs
    function to use the optimized extraction functions recently introduced
    as part of the typeOfScript conversion.
    
    In particular, this converts the detection for pay-to-pubkey scripts.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    0e810b4 View commit details
    Browse the repository at this point in the history
  71. txscript: Optimize ExtractPkScriptAddrs multisig.

    This continues the process of converting the ExtractPkScriptAddrs
    function to use the optimized extraction functions recently introduced
    as part of the typeOfScript conversion.
    
    In particular, this converts the detection for multisig scripts.
    
    Also, since the remaining slow path cases are all recursive calls,
    the parsed opcodes are no longer used, so parsing is removed.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    0bc1825 View commit details
    Browse the repository at this point in the history
  72. txscript: Optimize ExtractPkScriptAddrs nulldata.

    This continues the process of converting the ExtractPkScriptAddrs
    function to use the optimized extraction functions recently introduced
    as part of the typeOfScript conversion.
    
    In particular, this converts the detection for nulldata scripts, removes
    the slow path fallback code since it is the final case, and modifies the
    comment to call out the script version semantics.
    
    The following is a before and after comparison of analyzing both a
    typical standard script and a very large non-standard script:
    
    benchmark                            old ns/op    new ns/op    delta
    -----------------------------------------------------------------------
    BenchmarkExtractPkScriptAddrsLarge   132400       44.4         -99.97%
    BenchmarkExtractPkScriptAddrs        1265         231          -81.74%
    
    benchmark                            old allocs   new allocs   delta
    -----------------------------------------------------------------------
    BenchmarkExtractPkScriptAddrsLarge   1            0            -100.00%
    BenchmarkExtractPkScriptAddrs        5            2            -60.00%
    
    benchmark                            old bytes    new bytes    delta
    -----------------------------------------------------------------------
    BenchmarkExtractPkScriptAddrsLarge   466944       0            -100.00%
    BenchmarkExtractPkScriptAddrs        1600         48           -97.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    507a4dc View commit details
    Browse the repository at this point in the history
  73. txscript: Optimize ExtractPkScriptAddrs witness pubkey hash

    This continues the process of converting the ExtractPkScriptAddrs
    function to use the optimized extraction functions recently introduced
    as part of the typeOfScript conversion.
    
    In particular, this converts the extraction for witness-pubkey-hash
    scripts.
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    ae7fffb View commit details
    Browse the repository at this point in the history
  74. txscript: Optimize ExtractPkScriptAddrs witness script hash

    This continues the process of converting the ExtractPkScriptAddrs
    function to use the optimized extraction functions recently introduced
    as part of the typeOfScript conversion.
    
    In particular, this converts the extract of witness-pay-to-script-hash
    scripts.
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    a831522 View commit details
    Browse the repository at this point in the history
  75. txscript: Optimize ExtractPkScriptAddr assume non-standard if no success

    This completes the process of converting the ExtractPkScriptAddr
    function to use the optimized extraction functions recently introduced
    as part of the typeOfScript conversion.
    
    In particular, this cleans up the final remaining case for non-standard
    transactions. The method now returns NonStandardTy direclty if no other
    branch was taken.
    
    The following is a before and after comparison of attempting to extract
    pkscript addrs from a very large, non-standard script.
    
    benchmark                                old ns/op     new ns/op     delta
    BenchmarkExtractPkScriptAddrsLarge-8     60713         17.0          -99.97%
    BenchmarkExtractPkScriptAddrs-8          289           17.0          -94.12%
    
    benchmark                                old allocs     new allocs     delta
    BenchmarkExtractPkScriptAddrsLarge-8     1              0              -100.00%
    BenchmarkExtractPkScriptAddrs-8          1              0              -100.00%
    
    benchmark                                old bytes     new bytes     delta
    BenchmarkExtractPkScriptAddrsLarge-8     311299        0             -100.00%
    BenchmarkExtractPkScriptAddrs-8          768           0             -100.00%
    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    1034a66 View commit details
    Browse the repository at this point in the history
  76. txscript: Optimize IsWitnessProgram

    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    6fb1c82 View commit details
    Browse the repository at this point in the history
  77. Configuration menu
    Copy the full SHA
    8b70634 View commit details
    Browse the repository at this point in the history
  78. Configuration menu
    Copy the full SHA
    4b03b59 View commit details
    Browse the repository at this point in the history
  79. Configuration menu
    Copy the full SHA
    d410d7d View commit details
    Browse the repository at this point in the history
  80. txscript: mergeMultiSig function def order cleanup.

    This moves the function definition for mergeMultiSig so it is more
    consistent with the preferred order used through the codebase.  In
    particular, the functions are defined before they're first used and
    generally as close as possible to the first use when they're defined in
    the same file.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    7ad3a10 View commit details
    Browse the repository at this point in the history
  81. Configuration menu
    Copy the full SHA
    dd609d6 View commit details
    Browse the repository at this point in the history
  82. Configuration menu
    Copy the full SHA
    ed9e17a View commit details
    Browse the repository at this point in the history
  83. Configuration menu
    Copy the full SHA
    e00fec1 View commit details
    Browse the repository at this point in the history
  84. txscript: Use raw scripts in SignTxOutput.

    This converts SignTxOutput and supporting funcs, namely sign,
    mergeScripts and mergeMultiSig, to make use of the new tokenizer as well
    as some recently added funcs that deal with raw scripts in order to
    remove the reliance on parsed opcodes as a step towards utlimately
    removing them altogether and updates the comments to explicitly call out
    the script version semantics.
    
    It is worth noting that this has the side effect of optimizing the
    function as well, however, since this change is not focused on the
    optimization aspects, no benchmarks are provided.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    f3354be View commit details
    Browse the repository at this point in the history
  85. txscript: Implement efficient opcode data removal.

    This introduces a new function named removeOpcodeByDataRaw which accepts
    the raw scripts and data to remove versus requiring the parsed opcodes
    to both significantly optimize it as well as make it more flexible for
    working with raw scripts.
    
    There are several places in the rest of the code that currently only
    have access to the parsed opcodes, so this only introduces the function
    for use in the future and deprecates the existing one.
    
    Note that, in practice, the script will never actually contain the data
    that is intended to be removed since the function is only used during
    signature verification to remove the signature itself which would
    require some incredibly non-standard code to create.
    
    Thus, as an optimization, it avoids allocating a new script unless there
    is actually a match that needs to be removed.
    
    Finally, it updates the tests to use the new function.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    30874ff View commit details
    Browse the repository at this point in the history
  86. txscript: Optimize removeOpcodeRaw

    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    a4720f3 View commit details
    Browse the repository at this point in the history
  87. txscript: Remove unused removeOpcode

    cfromknecht authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    2ddcdb9 View commit details
    Browse the repository at this point in the history
  88. Configuration menu
    Copy the full SHA
    a2ab5b6 View commit details
    Browse the repository at this point in the history
  89. txscript: Make isDisabled accept raw opcode.

    This converts the isDisabled function defined on a parsed opcode to a
    standalone function which accepts an opcode as a byte instead in order
    to make it more flexible for raw script analysis.
    
    It also updates all callers accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    484f7b1 View commit details
    Browse the repository at this point in the history
  90. txscript: Make alwaysIllegal accept raw opcode.

    This converts the alwaysIllegal function defined on a parsed opcode to a
    standalone function named isOpcodeAlwaysIllegal which accepts an opcode
    as a byte instead in order to make it more flexible for raw script
    analysis.
    
    It also updates all callers accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    c641025 View commit details
    Browse the repository at this point in the history
  91. txscript: Make isConditional accept raw opcode.

    This converts the isConditional function defined on a parsed opcode to a
    standalone function named isOpcodeConditional which accepts an opcode as
    a byte instead in order to make it more flexible for raw script
    analysis.
    
    It also updates all callers accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    62c608f View commit details
    Browse the repository at this point in the history
  92. txscript: Make min push accept raw opcode and data.

    This converts the checkMinimalDataPush function defined on a parsed
    opcode to a standalone function which accepts an opcode and data slice
    instead in order to make it more flexible for raw script analysis.
    
    It also updates all callers accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    710bd56 View commit details
    Browse the repository at this point in the history
  93. txscript: Convert to use non-parsed opcode disasm.

    This converts the engine's current program counter disasembly to make
    use of the standalone disassembly function to remove the dependency on
    the parsed opcode struct.
    
    It also updates the tests accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    54036e8 View commit details
    Browse the repository at this point in the history
  94. txscript: Refactor engine to use raw scripts.

    This refactors the script engine to store and step through raw scripts
    by making using of the new zero-allocation script tokenizer as opposed
    to the less efficient method of storing and stepping through parsed
    opcodes.  It also improves several aspects while refactoring such as
    optimizing the disassembly trace, showing all scripts in the trace in
    the case of execution failure, and providing additional comments
    describing the purpose of each field in the engine.
    
    It should be noted that this is a step towards removing the parsed
    opcode struct and associated supporting code altogether, however, in
    order to ease the review process, this retains the struct and all
    function signatures for opcode execution which make use of an individual
    parsed opcode.  Those will be updated in future commits.
    
    The following is an overview of the changes:
    
    - Modify internal engine scripts slice to use raw scripts instead of
      parsed opcodes
    - Introduce a tokenizer to the engine to track the current script
    - Remove no longer needed script offset parameter from the engine since
      that is tracked by the tokenizer
    - Add an opcode index counter for disassembly purposes to the engine
    - Update check for valid program counter to only consider the script
      index
      - Update tests for bad program counter accordingly
    - Rework the NewEngine function
      - Store the raw scripts
      - Setup the initial tokenizer
      - Explicitly check against version 0 instead of DefaultScriptVersion
        which would break consensus if changed
      - Check the scripts parse according to version 0 semantics to retain
        current consensus rules
      - Improve comments throughout
    - Rework the Step function
      - Use the tokenizer and raw scripts
      - Create a parsed opcode on the fly for now to retain existing
        opcode execution function signatures
      - Improve comments throughout
    - Update the Execute function
      - Explicitly check against version 0 instead of DefaultScriptVersion
        which would break consensus if changed
      - Improve the disassembly tracing in the case of error
    - Update the CheckErrorCondition function
      - Modify clean stack error message to make sense in all cases
      - Improve the comments
    - Update the DisasmPC and DisasmScript functions on the engine
      - Use the tokenizer
      - Optimize construction via the use of strings.Builder
    - Modify the subScript function to return the raw script bytes since the
      parsed opcodes are no longer stored
    - Update the various signature checking opcodes to use the raw opcode
      data removal and signature hash calculation functions since the
      subscript is now a raw script
      - opcodeCheckSig
      - opcodeCheckMultiSig
      - opcodeCheckSigAlt
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    d6b968c View commit details
    Browse the repository at this point in the history
  95. Configuration menu
    Copy the full SHA
    06c8bea View commit details
    Browse the repository at this point in the history
  96. Configuration menu
    Copy the full SHA
    03d1fb0 View commit details
    Browse the repository at this point in the history
  97. Configuration menu
    Copy the full SHA
    07ab66b View commit details
    Browse the repository at this point in the history
  98. txscript: Rename removeOpcodeByDataRaw func.

    This renames the removeOpcodeByDataRaw to removeOpcodeByData now that
    the old version has been removed.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    911db90 View commit details
    Browse the repository at this point in the history
  99. Configuration menu
    Copy the full SHA
    94e99cf View commit details
    Browse the repository at this point in the history
  100. Configuration menu
    Copy the full SHA
    69f3a39 View commit details
    Browse the repository at this point in the history
  101. Configuration menu
    Copy the full SHA
    7533672 View commit details
    Browse the repository at this point in the history
  102. Configuration menu
    Copy the full SHA
    6e5fbf8 View commit details
    Browse the repository at this point in the history
  103. Configuration menu
    Copy the full SHA
    e06b11a View commit details
    Browse the repository at this point in the history
  104. txscript: Remove unused unparseScript func.

    Also remove tests associated with unparsing opcodes accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    491b7b5 View commit details
    Browse the repository at this point in the history
  105. Configuration menu
    Copy the full SHA
    ca044fe View commit details
    Browse the repository at this point in the history
  106. txscript: Remove unused parseScriptTemplate func.

    Also remove tests associated with the func accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    595d379 View commit details
    Browse the repository at this point in the history
  107. txscript: Make executeOpcode take opcode and data.

    This converts the executeOpcode function defined on the engine to accept
    an opcode and data slice instead of a parsed opcode as a step towards
    removing the parsed opcode struct and associated supporting code altogether.
    
    It also updates all callers accordingly.
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    ef3d06e View commit details
    Browse the repository at this point in the history
  108. txscript: Make op callbacks take opcode and data.

    This converts the callback function defined on the internal opcode
    struct to accept the opcode and data slice instead of a parsed opcode as
    the final step towards removing the parsed opcode struct and associated
    supporting code altogether.
    
    It also updates all of the callbacks and tests accordingly and finally
    removes the now unused parsedOpcode struct.
    
    The final results for the raw script analysis and tokenizer
    optimizations are as follows:
    
    benchmark                                       old ns/op     new ns/op     delta
    BenchmarkIsPayToScriptHash-8                    62393         0.51          -100.00%
    BenchmarkIsPubKeyHashScript-8                   62228         0.56          -100.00%
    BenchmarkGetSigOpCount-8                        61051         658           -98.92%
    BenchmarkExtractPkScriptAddrsLarge-8            60713         17.2          -99.97%
    BenchmarkExtractPkScriptAddrs-8                 289           17.9          -93.81%
    BenchmarkIsWitnessPubKeyHash-8                  61688         0.42          -100.00%
    BenchmarkIsUnspendable-8                        656           520           -20.73%
    BenchmarkExtractAtomicSwapDataPushesLarge-8     61332         44.0          -99.93%
    BenchmarkExtractAtomicSwapDataPushes-8          990           260           -73.74%
    BenchmarkDisasmString-8                         102902        39754         -61.37%
    BenchmarkGetPreciseSigOpCount-8                 130223        715           -99.45%
    BenchmarkScriptParsing-8                        63464         681           -98.93%
    BenchmarkIsMultisigScriptLarge-8                64166         5.83          -99.99%
    BenchmarkIsMultisigScript-8                     630           58.5          -90.71%
    BenchmarkPushedData-8                           64837         1779          -97.26%
    BenchmarkCalcSigHash-8                          3627895       3605459       -0.62%
    BenchmarkIsPubKeyScript-8                       62323         2.83          -100.00%
    BenchmarkIsPushOnlyScript-8                     62412         569           -99.09%
    BenchmarkIsWitnessScriptHash-8                  61243         0.56          -100.00%
    BenchmarkGetScriptClass-8                       61515         16.4          -99.97%
    BenchmarkIsNullDataScript-8                     62495         2.53          -100.00%
    BenchmarkIsMultisigSigScriptLarge-8             69328         2.52          -100.00%
    BenchmarkIsMultisigSigScript-8                  2375          141           -94.06%
    BenchmarkGetWitnessSigOpCountP2WKH-8            504           72.0          -85.71%
    BenchmarkGetWitnessSigOpCountNested-8           1158          136           -88.26%
    BenchmarkIsWitnessPubKeyHash-8                  68927         0.53          -100.00%
    BenchmarkIsWitnessScriptHash-8                  62774         0.63          -100.00%
    
    benchmark                                       old allocs     new allocs     delta
    BenchmarkIsPayToScriptHash-8                    1              0              -100.00%
    BenchmarkIsPubKeyHashScript-8                   1              0              -100.00%
    BenchmarkGetSigOpCount-8                        1              0              -100.00%
    BenchmarkExtractPkScriptAddrsLarge-8            1              0              -100.00%
    BenchmarkExtractPkScriptAddrs-8                 1              0              -100.00%
    BenchmarkIsWitnessPubKeyHash-8                  1              0              -100.00%
    BenchmarkIsUnspendable-8                        1              0              -100.00%
    BenchmarkExtractAtomicSwapDataPushesLarge-8     1              0              -100.00%
    BenchmarkExtractAtomicSwapDataPushes-8          2              1              -50.00%
    BenchmarkDisasmString-8                         46             51             +10.87%
    BenchmarkGetPreciseSigOpCount-8                 3              0              -100.00%
    BenchmarkScriptParsing-8                        1              0              -100.00%
    BenchmarkIsMultisigScriptLarge-8                1              0              -100.00%
    BenchmarkIsMultisigScript-8                     1              0              -100.00%
    BenchmarkPushedData-8                           7              6              -14.29%
    BenchmarkCalcSigHash-8                          1335           712            -46.67%
    BenchmarkIsPubKeyScript-8                       1              0              -100.00%
    BenchmarkIsPushOnlyScript-8                     1              0              -100.00%
    BenchmarkIsWitnessScriptHash-8                  1              0              -100.00%
    BenchmarkGetScriptClass-8                       1              0              -100.00%
    BenchmarkIsNullDataScript-8                     1              0              -100.00%
    BenchmarkIsMultisigSigScriptLarge-8             5              0              -100.00%
    BenchmarkIsMultisigSigScript-8                  3              0              -100.00%
    BenchmarkGetWitnessSigOpCountP2WKH-8            2              0              -100.00%
    BenchmarkGetWitnessSigOpCountNested-8           4              0              -100.00%
    BenchmarkIsWitnessPubKeyHash-8                  1              0              -100.00%
    BenchmarkIsWitnessScriptHash-8                  1              0              -100.00%
    
    benchmark                                       old bytes     new bytes     delta
    BenchmarkIsPayToScriptHash-8                    311299        0             -100.00%
    BenchmarkIsPubKeyHashScript-8                   311299        0             -100.00%
    BenchmarkGetSigOpCount-8                        311299        0             -100.00%
    BenchmarkExtractPkScriptAddrsLarge-8            311299        0             -100.00%
    BenchmarkExtractPkScriptAddrs-8                 768           0             -100.00%
    BenchmarkIsWitnessPubKeyHash-8                  311299        0             -100.00%
    BenchmarkIsUnspendable-8                        1             0             -100.00%
    BenchmarkExtractAtomicSwapDataPushesLarge-8     311299        0             -100.00%
    BenchmarkExtractAtomicSwapDataPushes-8          3168          96            -96.97%
    BenchmarkDisasmString-8                         389324        130552        -66.47%
    BenchmarkGetPreciseSigOpCount-8                 623367        0             -100.00%
    BenchmarkScriptParsing-8                        311299        0             -100.00%
    BenchmarkIsMultisigScriptLarge-8                311299        0             -100.00%
    BenchmarkIsMultisigScript-8                     2304          0             -100.00%
    BenchmarkPushedData-8                           312816        1520          -99.51%
    BenchmarkCalcSigHash-8                          1373812       1290507       -6.06%
    BenchmarkIsPubKeyScript-8                       311299        0             -100.00%
    BenchmarkIsPushOnlyScript-8                     311299        0             -100.00%
    BenchmarkIsWitnessScriptHash-8                  311299        0             -100.00%
    BenchmarkGetScriptClass-8                       311299        0             -100.00%
    BenchmarkIsNullDataScript-8                     311299        0             -100.00%
    BenchmarkIsMultisigSigScriptLarge-8             330035        0             -100.00%
    BenchmarkIsMultisigSigScript-8                  9472          0             -100.00%
    BenchmarkGetWitnessSigOpCountP2WKH-8            1408          0             -100.00%
    BenchmarkGetWitnessSigOpCountNested-8           3200          0             -100.00%
    BenchmarkIsWitnessPubKeyHash-8                  311299        0             -100.00%
    BenchmarkIsWitnessScriptHash-8                  311299        0             -100.00%
    davecgh authored and Roasbeef committed Nov 17, 2021
    Configuration menu
    Copy the full SHA
    b95ba0a View commit details
    Browse the repository at this point in the history