Real-time Environmental Monitoring

This topic has 7 replies, 2 voices, and was last updated 7 years, 3 months ago by Robert.

Viewing 7 posts - 1 through 7 (of 7 total)

Author

Posts

January 9, 2018 at 9:36 am #5467

Participant

The erroneous values seem to be inversions of the actual values. When mirroring these erroneous values around the X-axis, the result seems to make sense. This sign-flipping seems to occur at temperatures just around +1°C.

Attachments:

Temp-uRad-2.png

January 9, 2018 at 2:36 pm #5473

Radu

Keymaster

Thanks to your findings, I was able to identify a bug in the temperature encoding scheme:

uint16_t double2int(double val) {
	int8_t sintp = (int8_t)val;				//signed integer part
	int8_t sdecp = (val - sintp ) * 100;	//signed decimal part
	uint8_t udecp = sdecp>0?sdecp:-1*sdecp; //removed decimal sign
	uint8_t uintp = sintp + 127;			//convert to unsigned
	uint16_t res = (udecp << 8) | (uint16_t)uintp;		//pack it together
	return res;
}

The sign is lost when the integer part is 0, exactly all values between 0 and -1. The fix is added to the firmware as we speak, unless it can be solved on the server side.

For CO2, let’s continue herE: https://www.uradmonitor.com/topic/co2-sensor/

January 9, 2018 at 3:22 pm #5475

Robert

Participant

Thank you for sharing. I’ve put the code in a little test program:


uint16_t double2int(double val)
{
	int8_t   sintp = (int8_t)val;                     /* signed integer part */
	int8_t   sdecp = (val - sintp) * 100;             /* signed decimal part */
	uint8_t  udecp = (sdecp > 0) ? sdecp : -sdecp;    /* removed decimal sign */
	uint8_t  uintp = sintp + 127;                     /* convert to unsigned */
	uint16_t res   = (udecp << 8) | (uint16_t)uintp;  /* pack it together */

	return res;
}

int main(int argc, char *argv[])
{
	double  value;

	for (value = -2.0; value < 2.01; value += 0.1) {
		uint16_t  converted = double2int(value);
		printf("value = %f, converted = %u\n", value, converted);
	}
}

And I’m getting the following result:


value = -2.000000, converted = 125
value = -1.900000, converted = 22910
value = -1.800000, converted = 20350
value = -1.700000, converted = 17790
value = -1.600000, converted = 15230
value = -1.500000, converted = 12670
value = -1.400000, converted = 10110
value = -1.300000, converted = 7550
value = -1.200000, converted = 4990
value = -1.100000, converted = 2430
value = -1.000000, converted = 25471
value = -0.900000, converted = 22911
value = -0.800000, converted = 20351
value = -0.700000, converted = 17791
value = -0.600000, converted = 15231
value = -0.500000, converted = 12671
value = -0.400000, converted = 10111
value = -0.300000, converted = 7551
value = -0.200000, converted = 4991
value = -0.100000, converted = 2431
value = 0.000000, converted = 127
value = 0.100000, converted = 2687
value = 0.200000, converted = 5247
value = 0.300000, converted = 7807
value = 0.400000, converted = 10367
value = 0.500000, converted = 12927
value = 0.600000, converted = 15487
value = 0.700000, converted = 18047
value = 0.800000, converted = 20607
value = 0.900000, converted = 23167
value = 1.000000, converted = 128
value = 1.100000, converted = 2688
value = 1.200000, converted = 5248
value = 1.300000, converted = 7808
value = 1.400000, converted = 10368
value = 1.500000, converted = 12928
value = 1.600000, converted = 15488
value = 1.700000, converted = 18048
value = 1.800000, converted = 20608
value = 1.900000, converted = 23168
value = 2.000000, converted = 129

Though I’m not really sure what I’m looking at. Can you tell me what goes in and what’s supposed to come out? Is the output supposed to be a fixed-point value?

January 9, 2018 at 5:59 pm #5476

Radu

Keymaster

I’ve provided more details on email, please check it. But for the continuity of the topic, the bug becomes apparent when you decode the “converted” values. Please add the decoding function:

double int2double(uint16_t val) {
	int8_t d = (val >> 8) & 0xFF;
	int8_t i = (val & 0xFF) - 127;
	if (i < 0) d *= -1;
	return i + d / 100.0;
}

The if (i < 0) d *= -1; won't trigger when i is 0. The problem with this algorithm is that we have no way of knowing the sign, when the integer byte is 0. Exactly the bug you have spotted. Try this code to the "converted" values, and check the more details I sent on your email. I tried to see if this is fixable on the server side, but apparently information is lost on the conversion. A new encoding scheme has been added to the firmware. We will need to install this new firmware on all affected units.

January 14, 2018 at 9:39 pm #5484

Robert

Participant

As suggested per email, but to keep this top compleet: Why not using standard 16-bit signed integers?

Conversions back and forth with a little test program would look like this:


#include <stdio.h>
#include <stdint.h>

#define FIXED_POINT_FACTOR	100.0;

int16_t double2fixed(double val)
{
	val *= FIXED_POINT_FACTOR;

	if (val > INT16_MAX)
		return INT16_MAX;
	if (val < INT16_MIN)
		return INT16_MIN;

	return val;
}

double fixed2double(int16_t val)
{
	return val / FIXED_POINT_FACTOR;
}

int main(int argc, char *argv[])
{
	double  value;

//	for (value = -328.70; value < -326.01; value += 0.1)  /* Test bottom edge case */
//	for (value = 326.00; value < 329.01; value += 0.1)    /* Test top edge case */
	for (value = -2.0; value < 2.01; value += 0.1)        /* Test around zero */
	{
		int16_t  converted = double2fixed(value);
		double   back = fixed2double(converted);

		printf("value = %f, converted = %d, back = %f\n", value, converted, back);
	}
}

January 15, 2018 at 10:11 am #5485

Radu

Keymaster

Yes, that would work too. Something similar was added in firmware v130 as a fix:

// an uint16_t stores 16 bits, we use first bit for sign, and following 15 bits for number (0..32767)
// result is divided by 100 for a real with 2 decimals, max is 327.00
uint16_t di(double val) {
	uint16_t res = 0;
	uint16_t mask = 1<<15;
	if (val > 327 || val < -327) return res;
	if (val < 0) {
		val *= -1000;
		res = val / 10; // fix double approximation issue
		res |= mask;
	} else {
		val *= 1000;
		res = val / 10; // fix double approximation issue
	}
	return res;
}

and


// an uint16_t stores 16 bits, we use first bit for sign, and following 15 bits for number (0..32767)
// result is divided by 100 for a real with 2 decimals, max is 327.00
double id(uint16_t val) {
	double res = 0;
	uint16_t mask = 1<<15;
	// check negative number
	if (val & mask) {
		val &= ~mask; // remove sign bit
		res = val;
		res = -res;
	} else {
		res = val;
	}
	res /= 100.0; // restore the 2 decimals
	return res;
}

It’s pretty much the same thing.

January 15, 2018 at 10:56 am #5486

Robert

Participant

Yes, that would work too.

And it’s standard. And it’s a lot easier to read.

It’s pretty much the same thing.

Yes and no, but mostly no.
The intermediate 16-bit value function di() produces uses a proprietary bit layout. Besides being non-standard and harder to read code, another consequence is that standard arithmetics – like adding, subtracting, multiplying and dividing – do not work on negative numbers. Perhaps there’s still a good reason to use this proprietary format, but it must be a really good one to justify the disadvantages.

Standard negative numbers use two’s complement for negative counter parts of positive integers. This is necessary to keep standard arithmetics intact and hence 0 – 1 = -1. With your proprietary format, you have two zeroes: +0 and -0. And standard arithmetics are broken: +0 – 1 = -32767 and -0 – 1 = 32767


Double:      | Standard conversion (binary)            | Proprietary conversion (binary)| Standard reverse    | Proprietary reverse
-------------+-----------------------------------------+--------------------------------+---------------------+---------------------
value = -2.00, double2fixed = -200 (0b1111111100111000), di = 32968 (0b1000000011001000), fixed2double = -2.00, id = -2.00
value = -1.90, double2fixed = -190 (0b1111111101000010), di = 32958 (0b1000000010111110), fixed2double = -1.90, id = -1.90
value = -1.80, double2fixed = -179 (0b1111111101001101), di = 32947 (0b1000000010110011), fixed2double = -1.79, id = -1.79
value = -1.70, double2fixed = -169 (0b1111111101010111), di = 32937 (0b1000000010101001), fixed2double = -1.69, id = -1.69
value = -1.60, double2fixed = -159 (0b1111111101100001), di = 32927 (0b1000000010011111), fixed2double = -1.59, id = -1.59
value = -1.50, double2fixed = -149 (0b1111111101101011), di = 32917 (0b1000000010010101), fixed2double = -1.49, id = -1.49
value = -1.40, double2fixed = -139 (0b1111111101110101), di = 32907 (0b1000000010001011), fixed2double = -1.39, id = -1.39
value = -1.30, double2fixed = -129 (0b1111111101111111), di = 32897 (0b1000000010000001), fixed2double = -1.29, id = -1.29
value = -1.20, double2fixed = -119 (0b1111111110001001), di = 32887 (0b1000000001110111), fixed2double = -1.19, id = -1.19
value = -1.10, double2fixed = -109 (0b1111111110010011), di = 32877 (0b1000000001101101), fixed2double = -1.09, id = -1.09
value = -1.00, double2fixed =  -99 (0b1111111110011101), di = 32867 (0b1000000001100011), fixed2double = -0.99, id = -0.99
value = -0.90, double2fixed =  -89 (0b1111111110100111), di = 32857 (0b1000000001011001), fixed2double = -0.89, id = -0.89
value = -0.80, double2fixed =  -79 (0b1111111110110001), di = 32847 (0b1000000001001111), fixed2double = -0.79, id = -0.79
value = -0.70, double2fixed =  -69 (0b1111111110111011), di = 32837 (0b1000000001000101), fixed2double = -0.69, id = -0.69
value = -0.60, double2fixed =  -59 (0b1111111111000101), di = 32827 (0b1000000000111011), fixed2double = -0.59, id = -0.59
value = -0.50, double2fixed =  -49 (0b1111111111001111), di = 32817 (0b1000000000110001), fixed2double = -0.49, id = -0.49
value = -0.40, double2fixed =  -39 (0b1111111111011001), di = 32807 (0b1000000000100111), fixed2double = -0.39, id = -0.39
value = -0.30, double2fixed =  -29 (0b1111111111100011), di = 32797 (0b1000000000011101), fixed2double = -0.29, id = -0.29
value = -0.20, double2fixed =  -19 (0b1111111111101101), di = 32787 (0b1000000000010011), fixed2double = -0.19, id = -0.19
value = -0.10, double2fixed =   -9 (0b1111111111110111), di = 32777 (0b1000000000001001), fixed2double = -0.09, id = -0.09
value =  0.00, double2fixed =    0 (0b0000000000000000), di =     0 (0b0000000000000000), fixed2double =  0.00, id =  0.00
value =  0.10, double2fixed =   10 (0b0000000000001010), di =    10 (0b0000000000001010), fixed2double =  0.10, id =  0.10
value =  0.20, double2fixed =   20 (0b0000000000010100), di =    20 (0b0000000000010100), fixed2double =  0.20, id =  0.20
value =  0.30, double2fixed =   30 (0b0000000000011110), di =    30 (0b0000000000011110), fixed2double =  0.30, id =  0.30
value =  0.40, double2fixed =   40 (0b0000000000101000), di =    40 (0b0000000000101000), fixed2double =  0.40, id =  0.40
value =  0.50, double2fixed =   50 (0b0000000000110010), di =    50 (0b0000000000110010), fixed2double =  0.50, id =  0.50
value =  0.60, double2fixed =   60 (0b0000000000111100), di =    60 (0b0000000000111100), fixed2double =  0.60, id =  0.60
value =  0.70, double2fixed =   70 (0b0000000001000110), di =    70 (0b0000000001000110), fixed2double =  0.70, id =  0.70
value =  0.80, double2fixed =   80 (0b0000000001010000), di =    80 (0b0000000001010000), fixed2double =  0.80, id =  0.80
value =  0.90, double2fixed =   90 (0b0000000001011010), di =    90 (0b0000000001011010), fixed2double =  0.90, id =  0.90
value =  1.00, double2fixed =  100 (0b0000000001100100), di =   100 (0b0000000001100100), fixed2double =  1.00, id =  1.00
value =  1.10, double2fixed =  110 (0b0000000001101110), di =   110 (0b0000000001101110), fixed2double =  1.10, id =  1.10
value =  1.20, double2fixed =  120 (0b0000000001111000), di =   120 (0b0000000001111000), fixed2double =  1.20, id =  1.20
value =  1.30, double2fixed =  130 (0b0000000010000010), di =   130 (0b0000000010000010), fixed2double =  1.30, id =  1.30
value =  1.40, double2fixed =  140 (0b0000000010001100), di =   140 (0b0000000010001100), fixed2double =  1.40, id =  1.40
value =  1.50, double2fixed =  150 (0b0000000010010110), di =   150 (0b0000000010010110), fixed2double =  1.50, id =  1.50
value =  1.60, double2fixed =  160 (0b0000000010100000), di =   160 (0b0000000010100000), fixed2double =  1.60, id =  1.60
value =  1.70, double2fixed =  170 (0b0000000010101010), di =   170 (0b0000000010101010), fixed2double =  1.70, id =  1.70
value =  1.80, double2fixed =  180 (0b0000000010110100), di =   180 (0b0000000010110100), fixed2double =  1.80, id =  1.80
value =  1.90, double2fixed =  190 (0b0000000010111110), di =   190 (0b0000000010111110), fixed2double =  1.90, id =  1.90
value =  2.00, double2fixed =  200 (0b0000000011001000), di =   200 (0b0000000011001000), fixed2double =  2.00, id =  2.00

Author

Posts

Viewing 7 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic.

Jumps in measured data

Attachments: