Parsing of field fails after manual domain change

[skleber]

My intention is to refine a vocabulary model after it has been inferred by clusterByAlignment. Using ASCII datatype as test case, I stumbled across an error in the MessageParser that I could not trace to the source of the problem. Could you please have a look and help me in debugging?

Here the test script that produces the error:

from netzob.Model.Vocabulary.Domain.Parser.MessageParser import InvalidParsingPathException
from netzob.all import *
import pprint

# do pretty printing of variable contents later
pp = pprint.PrettyPrinter(indent=2)

# import two NTP message payloads that get clustered/aligned by clusterByAlignment
# into one symbol with 13 fields at minEquivalence=60
messages = [
    RawMessage(b'\x1a\x02\n\xec\x00\x00\x0ep\x00\x00\x0f\xea\xc6\x1e\\\x02\xc5\x01\xfe@\xd7j\xd7\x0e\xc5\x02\x04\xec\xee\xd3<R\xc5\x02\x04\xeb\xd8\xfd\x8f\x0c\xc5\x02\x04\xeb\xd8\xff2z'),
    RawMessage(b'\x1a\x02\n\xec\x00\x00\x07\xc3\x00\x00/\x80\xc6\x1e\\\x02\xc5\x01\xf9\x95BP\x82\xcf\xc5\x02\x04\xec\xecB\xee\x92\xc5\x02\x04\xeb\xd25.\xb5\xc5\x02\x04\xeb\xd25\xd6{')
    ]
symbols = Format.clusterByAlignment(messages, minEquivalence=60, internalSlick=True)
# this is the only symbol.
sym = symbols[0]
print(sym)

# print field before any manual change
print(sym.fields[1])
dt = sym.fields[1].domain.dataType

mp = MessageParser()

for msg in sym.messages:
    # test parsing of the message before any manual change
    before = mp.parseMessage(msg, sym)
    pp.pprint(before)
    # this works in all cases. Interestingly one of the parsed field values
    # results in a byte, the other in a str:
    # Field
    # -----------
    # b'\x07\xc3'
    # '\x0ep'
    # -----------

# change the domain type of field 1 into ASCII
sym.fields[1].domain = Data(ASCII(nbChars=(dt.size[0]//8, dt.size[1]//8)))

for msg in sym.messages:
    try:
        after = mp.parseMessage(msg, sym)
        # the parsing fails for one message

        for (a,b) in zip(before,after):
            if not a == b:
                print("field parse mismatch: {} | {}".format(a,b))
    except InvalidParsingPathException:
        print("message parsing failed for message {}".format(msg.data))

The stack trace I get if I remove the try around the parseMessage() is:

Traceback (most recent call last):
  File "#11_change-field-domain.py", line 36, in <module>
    print(sym.fields[1])
  File "/home/stephan/REUP-common/Tools/netzob-official/netzob/src/netzob/Model/Vocabulary/AbstractField.py", line 702, in __str__
    result = self.getCells(encoded=True)
  File "/home/stephan/REUP-common/Tools/netzob-official/netzob/src/netzob/Common/Utils/Decorators.py", line 188, in wrapped_f
    return func(*args, **kwargs)
  File "/home/stephan/REUP-common/Tools/netzob-official/netzob/src/netzob/Model/Vocabulary/AbstractField.py", line 281, in getCells
    return DataAlignment.align(data, self, encoded=encoded)
  File "/home/stephan/REUP-common/Tools/netzob-official/netzob/src/netzob/Common/Utils/Decorators.py", line 188, in wrapped_f
    return func(*args, **kwargs)
  File "/home/stephan/REUP-common/Tools/netzob-official/netzob/src/netzob/Common/Utils/DataAlignment/DataAlignment.py", line 260, in align
    return dAlignment.execute()
  File "/home/stephan/REUP-common/Tools/netzob-official/netzob/src/netzob/Common/Utils/DataAlignment/DataAlignment.py", line 163, in execute
    alignedMsg = next(mp.parseRaw(d, targetedFieldLeafFields))
  File "/home/stephan/REUP-common/Tools/netzob-official/netzob/src/netzob/Model/Vocabulary/Domain/Parser/MessageParser.py", line 241, in parseBitarray
    TypeConverter.convert(bitArrayToParse, BitArray, Raw)))