medium int unsigned and any negative int signed is handled incorrectly

[GoogleCodeExporter]

print_data has this for unsigned ints, and the case for mediumint is
incorrect. This code clears the most signficant bit. That should not be done.

        switch (field->fixed_length) {
                case 1: return mach_read_from_1(value);
                case 2: return mach_read_from_2(value);
                case 3: return mach_read_from_3(value) & 0x3FFFFFUL;

The code for signed bits does not support negative values. It clears the
most signficant bit, which is correct to do for positive values. For
negative values the bit should be flipped from 0 to 1. I think the code has
other problems for negative values from sign extension.

Something like this handled negative ints:

                        ulint ur = mach_read_from_3(value);
                        ulint b0 = ur & 0xff;
                        ulint b1 = (ur >> 8) & 0xff;
                        ulint b2 = ((ur >> 16) & 0xff) ^ 0x80;
                        ulint r = b0 + (b1 << 8) + (b2 << 16);
                        if (r > 0x7fffff) {
                                return -((0xffffff - r) + 1);
                        } else {
                                return r;
                        }

Original issue reported on code.google.com by [email protected] on 11 Oct 2009 at 10:56