Represent percentage values with maximum available resolution

[jhbruhn]

Is your feature request related to a problem? Please describe.
When encoding a percentage value in a BTHome packet, the value is usually scaled by a factor of 10^n. The data type used is an unsigned integer value with a fixed size. This results in data transmissions which are never utilizing the maximum possible resolution for the used data type.

Describe the solution you'd like
To utilize the maximum possible resolution, percentage values should be encoded with the full resolution possible for the data type. E.g. if using an 8 bit unsigned integer, 100%=255. Thus the scale factor is 1/2^8. If using a 16 bit unsigned integer, the scale factor is 1/2^16

Additional context
This idea arose while implementing the BTHome protocol for b-parasites.

This is obviously not fully required as the current solution functions sufficiently as well. It is just something that is bothering me with the protocol definition as an embedded engineer.

[Ernst79]

I understand what you are saying. The only issue I saw was that you will get results like 99.6078 % (254/255). From a user point of view, I don't care about the .6078...., I just want to see if it is 100% or 99%. But if you really need it, we can add it, by adding a object_id with factor 1/255.

[pvvx]

But if you really need it, we can add it, by adding a object_id with factor 1/255.

Thus, all object_id will end tomorrow. And the old version will not be able to accept a packet from a new device with an added object_id.

For household devices, there is not enough id for the mains voltage (110..380+ V):
Id: 0x0C,"voltage", uint16 (2 bytes), step 0.001 -> max 65V.

... Etc.

The v2 format is very complex - it is impossible to implement reception on the MCU if there is an unknown object_id in the transmission. Can't recommend for use.

---

I have a logged meter that requires transmission:
Voltages: +-32V, 24 bits, step 0.0000038V (3.8uV)
Current: +-500mA, 24 bits, step 0.0000000596A (59.6nA)
Similarly for a simple BLE logger:
The INA228/229 is an ultra-precise digital power monitor with a 20-bit delta-sigma ADC specifically designed for current-sensing applications.
And many other DIY devices...

object_id is definitely not enough due to the size limit and units in the data.

[pvvx]

Why couldn't the size of the object be set to 2 bits of object_id?
00b - 64 types with 1 data byte
01b - 64 types with 2 data bytes
10b - 64 types with 3 data bytes
11b - 64 types with 4 data bytes
Total 256 different types.

Then it will be possible to parse a packet with an unknown identifier.

The current id table contains a total of 52 id types, 6 with different sizes.
Of these, the maximum number of id with one data byte is 35 types.

[Ernst79]

We have thought of this and solved this in a slightly different way in V2 format.
A sensor should send the data with object_id's from low to high (numerical order), see this part in the documentation

In that way, the receiver is able to parse all the object_id's with the lower numbers up till it sees an unknown object_id (with a high number). New object_id's will always be added with a higher number than the latest supported number.

There is a warning if the object_id's are not in numerical order.

                if prev_obj_meas_type > obj_meas_type:
                    _LOGGER.warning(
                        "BTHome device is not sending object ids in numerical order (from low to "
                        "high object id). This can cause issues with your BTHome receiver, "
                        "payload: %s",
                        payload.hex(),
                    )

[Ernst79]

About the measurement types you need/want, it is possible to add these, no problem. Just let me know how many bytes and which factor you want for each measurement

e.g. Voltage, 2 bytes, factor 0.01, unsigned integer (0-655,35 V)

We have also thought of running out of object_ids. We can use the object_ids above 127 (0x7F`) (or something else) to use an extra byte, such that we have 128*256 = 32768 "secondary" object_ids with 2 bytes and 128 "primary" object_ids.

0x7E
0x7F
0x8000
0x8001
etc.

[pvvx]

We have thought of this and solved this in a slightly different way in V2 format. A sensor should send the data with object_id's from low to high (numerical order), see this part in the documentation

This does not allow a third-party device to get a selection of only the data it needs, without having the entire decryption table.

[Ernst79]

Yes, but one can make a smaller decryption table with only the object_id and length of the objects you don’t want. Shouldn’t be that hard. I understand it isn’t ideal, but having to split up the object_ids already in 4 byte lengths isn’t ideal either, as it might limits the number of objects ids of a certain length.

But let’s keep this in mind when we start to develop a BTHome V3 in the future.

[afarago]

Here is a potential solution for a one-byte per object_id decoder table.

static const uint8_t PROGMEM MEAS_TYPES_FLAGS[] = { /* 8th bit Unused | 6-7th bits Factor | 4-5th bits DataType | 1-2-3rd bits DataLen */ 
  0b00000001, /* 0x00 | packet_id | packet_id | uint8 (1 byte) | 0 */
  0b00000001, /* 0x01 | battery | battery | uint8 (1 byte) | 0 */
  0b01001010, /* 0x02 | temperature | temperature | sint16 (2 bytes) | 2 */
  0b01000010, /* 0x03 | humidity | humidity | uint16 (2 bytes) | 2 */
  0b01000011, /* 0x04 | pressure | pressure | uint24 (3 bytes) | 2 */
  0b01000011, /* 0x05 | illuminance | illuminance | uint24 (3 bytes) | 2 */
  0b01000010, /* 0x06 | mass_kg | mass_kg | uint16 (2 byte) | 2 */
...

https://github.com/afarago/esphome_component_bthome/blob/master/components/bthome_base/bthome_common_generated.h