Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TransformBlock() behaves incorrectly when input data length is not a multiple of 16 #10

Open
itai opened this issue Sep 12, 2022 · 1 comment

Comments

@itai
Copy link

itai commented Sep 12, 2022

When the length of the input data is not a multiple of 16, TransformBlock() does not behave correctly. In the following example, the same hash is generated for different byte sequences:

using System.Security.Cryptography;
using System.Text;
using Murmur;

{
    using HashAlgorithm hashAlgorithm = MurmurHash.Create128(preference: AlgorithmPreference.X64);
    TransformSingleByte(hashAlgorithm, 1);
    TransformSingleByte(hashAlgorithm, 1);
    TransformSingleByte(hashAlgorithm, 1);
    TransformSingleByte(hashAlgorithm, 1);
    hashAlgorithm.TransformFinalBlock(Array.Empty<byte>(), 0, 0);
    PrintBytes(hashAlgorithm.Hash!);
}

{
    using HashAlgorithm hashAlgorithm = MurmurHash.Create128(preference: AlgorithmPreference.X64);
    TransformSingleByte(hashAlgorithm, 0);
    TransformSingleByte(hashAlgorithm, 0);
    TransformSingleByte(hashAlgorithm, 0);
    TransformSingleByte(hashAlgorithm, 0);
    hashAlgorithm.TransformFinalBlock(Array.Empty<byte>(), 0, 0);
    PrintBytes(hashAlgorithm.Hash!);
}

{
    using HashAlgorithm hashAlgorithm = MurmurHash.Create128(preference: AlgorithmPreference.X64);
    TransformSingleByte(hashAlgorithm, 0);
    TransformSingleByte(hashAlgorithm, 0);
    TransformSingleByte(hashAlgorithm, 1);
    TransformSingleByte(hashAlgorithm, 1);
    hashAlgorithm.TransformFinalBlock(Array.Empty<byte>(), 0, 0);
    PrintBytes(hashAlgorithm.Hash!);
}

{
    using HashAlgorithm hashAlgorithm = MurmurHash.Create128(preference: AlgorithmPreference.X64);
    TransformSingleByte(hashAlgorithm, 0);
    TransformSingleByte(hashAlgorithm, 1);
    TransformSingleByte(hashAlgorithm, 0);
    TransformSingleByte(hashAlgorithm, 1);
    hashAlgorithm.TransformFinalBlock(Array.Empty<byte>(), 0, 0);
    PrintBytes(hashAlgorithm.Hash!);
}

static void TransformSingleByte(
    HashAlgorithm hashAlgorithm,
    byte @byte) =>
    hashAlgorithm.TransformBlock(new[] {@byte}, 0, 1, null, 0);

static void PrintBytes(IEnumerable<byte> bytes)
{
    var stringBuilder = new StringBuilder();
    foreach (var @byte in bytes)
    {
        stringBuilder.AppendFormat("{0:X2}", @byte);
    }
    Console.WriteLine(stringBuilder.ToString());
}

Output:

BC764CD8DDF7A0CFF126F51C16239658
BC764CD8DDF7A0CFF126F51C16239658
BC764CD8DDF7A0CFF126F51C16239658
BC764CD8DDF7A0CFF126F51C16239658
@itai
Copy link
Author

itai commented Sep 12, 2022

In the above example, all inputs to TransformSingleByte() are of size 1, which is less than the MurmurHash block size (16).

What happens is that HashCore() calls Body(), which calls Tail() with the input. Tail() does a calculation on the input and then XORs the result with H1 and H2. This means that when we call TransformBlock() twice with the same input, the effect cancels out (because X ^ Y ^ Y == X).

What HashCore() should have done is aggregate complete blocks and run the algorithm on them. The "tail" logic should only be used when HashFinal() is called.

@itai itai changed the title TransformBlock() behaves incorrectly on unaligned input TransformBlock() behaves incorrectly when input data length is not a multiple of 16 Sep 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant