rlp.inspect on recursive lists does not validate invalid list size?

[kdeme]

Payloads with containing many rlp lists inside lists takes very long.
e.g. this payload ((it takes much less if the whole output part of the inspect is removed, but even without that this payload is >10s):

c5c2c1c3c5c2c1c5c2c5c2c1c5c3c5c2c1c3c5c5c2c5c2c1c5c3c1c3c5c2
c1c5c2c5c2c1c5c2c530c1c5c2c5c2c1c5c3c1c3c5c2c1c5c2c5c2c1c5c3
c5c2c1c3c2c1c3c5c2c1c5c2c5c2c1c5c3c5c2c1c3c5c2c530c1c5c1c5c2
c352c545c230c5c2c187c5c2c1423f3a3a3dc5

However, this payload is also incorrect, as several sizes of lists are not valid.
So rlp.inspect does not seem to validate the list sizes.

rlp.inspect is not used except for testing, so perhaps it is not really an issue.

[jlokier]

I suspect this issue affects RLP parsing in general, not just rlp.inspect, and that it's a few different vulnerabilities when parsing untrusted RLP data from the network:

• Iterating over the items in an RLP list doesn't check that the last item ends at the end of the containing list, so the last item in the list can overlap the item following the list.
• Iterating over the fields of an object doesn't check that the object ends at the end of the containing list.
• Iterating over a list sometimes does one scan to count the number of items, then another to process the items.
  With enough nesting could result in quadratic time behaviour.
• There's a lot of repeated analysis of the first bytes of an item.
• There's a lot of copying. Compined with quadratic time, this might explain the slow rlp.inspect on your payload.
• Sometimes .blobLen > 0 is used to decide whether to parse an optional value or skip the value as "none". But this unintentionally treats a list as "none" instead of an error.

A number of places need to treat blobs like list containers as well, i.e. blobs whose content is itself RLP encoded. So I think a bit of reworking how nested RLP structures are parsed is in order, to validate boundaries more carefully (end reached exactly without overrun).