Converting CPM to uSv/h

[Slyke]

Hello,

I know that this project is just a throw-me-up and abandoned, but I've been scouring Google for days for an answer.

Just wondering where you got the data from for converting CPM to uSv/hr in https://github.com/johndoe31415/gammascoututil/blob/master/DosisConversion.py#L26 ?

I'm currently trying to analyze some data from my GammaScout, and when converting the pulse counts to microseiverts I always get results that are slightly off.

For example, If I use the LND712 conversion factor to calculate uSv/h as such (663 pulses taken over an hour):
(663 / 60) / (0.005940 * (663 / 60) + 122.5) = 0.090155775064425
Or if I use the table from your file:
(663 / 60) / (-0.01931769722814499 * (663 / 60) + 122.5) = 0.090361539790837

But when using the GammaScout Toolbox software and exporting the data manually from there, it gives me a different result of: 0.076111 uSv/h

The first 2 results are not that different from each other, as they are only off by about ~0.0002uSv/h, but the result from the GammaScout Toolbox software is off by almost ~0.025uSv/h!

I was wondering if you have any insight into which I should use, and if it's going to be an issue (say I taken readings over the course of days with pulse counts in the 1000s)?

[johndoe31415]

Hey there,

thanks for the question -- it's kind of semi-abandoned to be honest. When I picked the project up and pushed it on GitHub, I had made unpublished modifications to the code and have not used my GS in maybe two years or so. Still have it, just don't record data anymore as I used to. Therefore, I couldn't even judge if it'd work and just pushed it all up with that fat disclaimer.

Regarding the dosis conversion code, that was one of the new features that I added before I stopped working on the code. It is very hazy in my memory, but I do believe that I reverse engineered that from the GS toolbox software. It could well be that the Toolbox also uses calibration values from the GS that are tube-specific, in which case the conversion I reverse engineered would only apply to my tube and not be generally valid.

I would be super interested in finding this out and maybe properly reverse engineer the Toolbox software to have the same functionality. However, I'll be on travel a lot the coming weeks so my availability to work on this will be limited. Could you maybe create a demo text file that showcases 1, 2, 4, 8, 16, etc counts per minute and that can be read in my the toolbox software? Then I can compare with my conversion results over here and we can see if we can track down the issue.

Out of curiosity, you're comparing the DosisConversion code from GSUtil to some other conversion -- where is that other conversion from? Are these official calibration curves for the LND712? Would you mind linking to them?

All the best, Joe

[Slyke]

Hello! Thank you for the quick reply.

What did you use to reverse engineer it? I don't have much experience reversing Windows executables. Looks like it was made in VB.NET or something. I was thinking about just creating some serial port application and seeing what the software produced in ways of calculated values. I could create some fake readings as you suggested and have the app read them. If you have your scout still available, could you send me the calibration data from the C command, the memory dump from the B command, and the calculated data from the gammascout software?

I'm not sure if these are from official sources or not, but I did find this calibration table:

		case GEIGER_TUBE_SBM20: 	return 0.006315;
		case GEIGER_TUBE_SI29BG: 	return 0.010000;
		case GEIGER_TUBE_SBM19: 	return 0.001500;
		case GEIGER_TUBE_STS5: 		return 0.006666;
		case GEIGER_TUBE_SI22G: 	return 0.001714;
		case GEIGER_TUBE_SI3BG: 	return 0.631578;
		case GEIGER_TUBE_SBM21: 	return 0.048000;
		case GEIGER_TUBE_LND712: 	return 0.005940;
		case GEIGER_TUBE_SBT9: 		return 0.010900;
		case GEIGER_TUBE_SI1G:		return 0.006000;

Source: https://www.uradmonitor.com/topic/hardware-conversion-factor/

The one for the LND712 produces results similar to your table.

I also tried emailing the actual manufacturers, but they never replied.

[Slyke]

I tried putting fake data into a file and loading it up as RAW data, but the application seems to know somehow. I think the last char on the end of each line is some sort of checksum. Anyway, I've created a spreadsheet with some data points, and my calibration data. I'm going to continue to investigate trying to load some fake data into the GammaScout app and see if that's of any fruition.

Spreadsheet: https://docs.google.com/spreadsheets/d/1tJwyVtDJXl3pweDp-A-F4SXmrqg8IxPvMP8J9RjMQng/edit#gid=0

Edit
Figured out the checksum calculation. It's using CheckSum8 Modulo 256
I fabricated a long list of logs, starting from 0 pulses and up to 266 during a 10 minute pulse count cycle.
Results here: https://docs.google.com/spreadsheets/d/1BXXK9yTNEAgkrPIEmpB8Hm7Cdr3gC237y_yix0gO1Jo/edit#gid=0

Here's the GammaScout dump file I generated and could feed into the GammaScout toolbox (place your GS serial where the Xs are):

v
Version 7.02Lb07 1020 XXXXXX 0220 15.03.19 21:38:54
c
GAMMA-SCOUT SoftCal gueltig
b0 00 07 7e 00
cd0900001a05f6da2d00
9a210000c1e1acb12d00
9a7d00002f0995c63c00
00a0020002009af24b00
9a46080004001cc23c00
c47d130085010e9e2d00
15dd4400990210b32d00
b
GAMMA-SCOUT Protokoll
f5ed452821150319f5080000000100020003000400050006000700080009000ad5
000b000c000d000e000f0010001100120013001400150016001700180019001a28
001b001c001d001e001f0020002100220023002400250026002700280029002a28
002b002c002d002e002f0030003100320033003400350036003700380039003a28
003b003c003d003e003f0040004100420043004400450046004700480049004a28
004b004c004d004e004f0050005100520053005400550056005700580059005a28
005b005c005d005e005f0060006100620063006400650066006700680069006a28
006b006c006d006e006f0070007100720073007400750076007700780079007a28
007b007c007d007e007f0080008100820083008400850086008700880089008a28
008b008c008d008e008f0090009100920093009400950096009700980099009a28
009b009c009d009e009f00a000a100a200a300a400a500a600a700a800a900aa28
00ab00ac00ad00ae00af00b000b100b200b300b400b500b600b700b800b900ba28
00bb00bc00bd00be00bf00c000c100c200c300c400c500c600c700c800c900ca28
00cb00cc00cd00ce00cf00d000d100d200d300d400d500d600d700d800d900da28
00db00dc00dd00de00df00e000e100e200e300e400e500e600e700e800e900ea28
00eb00ec00ed00ee00ef00f000f100f200f300f400f500f600f700f800f900fa28
00fb00fc00fd00fe00ff0100010101020103010401050106010701080109010a33

Generates this:

[johndoe31415]

Hey there! Ah, very cool and interesting, thank you so much for this. The linear progression is not what I saw when reversing the Toolbox, but that has been ages ago. I'm still on business travel today so I can't do anything, but will try to have a look at the weekend. I'm 99% sure that it was a piecewise linear function, as in the code. We shall see. Thanks a bunch for your efforts, @Slyke!

[Slyke]

Hey, that's no worries!
Yeah, it's cool, when ever you have time, no rush.

I have figured out some more:
The data in the calibration blob does change how it's calculated. How exactly it works I'm unsure, but it could be the data that feeds the piecewise linear function somehow. Changing the spaced numbers at the top also seems to change the calculated value's scale. I will need to crunch the numbers a little more and try to reverse engineer the hex data.

In the meantime, I generated a huge dataset (See attached big_range.log, just rename to a .dat file). The generate.txt can be run from your Javascript console by adjusting the 2 top variables, or you can run it from node (by renaming it to a js file) and feed in the startAt and generateLines as the 1st and 2nd params.

Spreadsheet with the large dataset: https://docs.google.com/spreadsheets/d/1k_cOgu6SYNzsBhoehYx-P0uL_6FM_-MyF7GlicalNqk/edit#gid=0

You can set the memory pointer (YYYY) in the dat file to either 5500 to get some sane amounts, or up to f500 to get most of the data. The uSv/h seems to go up into the millions towards the end since they use some of the first bits for the exponent.

Here's a graph using 5500:

big_range.log

big_range.txt

generate.txt

I think if I get a few calibrations of real GammaScouts I'll be able to work it out.

[Slyke]

Hello,

just wondering if you had a chance to get your GammaScout? Need another calibration reading so that I can compare it with my own.

GAMMA-SCOUT SoftCal gueltig

[johndoe31415]

Heya Slyke,

sorry again for taking so long to get back to you -- first I was on quite a long business trip and after my return I had to code for something else. You know the saying: Happy wife, happy life :-)

Anyways, I've had a chance to take a look now. Very interesting, a LOT has changed since I've last looked at the toolbox. It seems like a complete rewrite of the code. To be honest, all of this was such a long time ago that I don't really recall how I reverse engineered the table. Now there's something the new toolbox does that the old toolbox doesn't: It can write the raw ".DAT" data. I'm assuming that this includes the calibration data. Previously, this was 100% not the case and I think -- again, unsure of it -- that the values that can be found in gammascoututil therefore reflect only my GS, not all of them.

I think the best thing is to export a short "raw" data export using GSU, then peek for the calibration values inside. Then, switch calibration data of yours and mine and see how the values change. From that, reverse engineer how the curves are stored inside the calibration data blob and then write the code. I'm currently reading out mine, but since it's been in storage for VERY long, this might take a while:

Log buffer fill      : 65280 bytes

Anyways, I'll erase the data, record some short test data, then proceed from there. In any case, very interesting findings of the piecewise curves you generated. After reading out is complete, I'll also post my calibration values.

Out of curiousity, how did you acquire your calibration values? Could you also use GSU and do a "readcfg" and post your file here? That'd be super helpful, I'll also post mine so we can compare. Actually think mine might already be in git somewhere? Not sure. Again, this has been ages.

Thanks for your patience, let's see if we can crack this nut.

[johndoe31415]

Ah, very nice. The Gammascout Toolbox DAT format is pretty much just a RS232 capture. I think it should be possible to easily support reading that file format as well then. Cool. Need to fiddle a bit with the values and see what I get. In my DAT file, the calibration blob is this:

c
08 04 ab310b00
f3020000fb1de1f68601
cb2c00000600e6ff7602
ffffffff5b16acfc6702
00000000000000000000
00000000000000000000
00000000000000000000
00000000000000000000
00000000000000000000

and the readcfg is this:

00000000  08 04 ab 31 0b 00 f3 02  00 00 fb 1d e1 f6 86 01  |...1............|
00000010  cb 2c 00 00 06 00 e6 ff  76 02 ff ff ff ff 5b 16  |.,......v.....[.|
00000020  ac fc 67 02 00 00 00 00  00 00 00 00 00 00 00 00  |..g.............|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

So at least we're now sure they're the same thing. Neat.

[johndoe31415]

One last thought before I go to bed: Yours seems to be individually calibrated, could this be? While mine (GS Alert) seems to only have a stock factory calibration. So the calibration blobs might even vary between protocol versions, there've been plenty of changes to the protocol so far. Hmmmm. Absolutely need to have a closer look comparing your values to mine, see if they're even the same format.

[johndoe31415]

Looks promising. Here's my preliminary findings:

cd09000 01a0 5f6da2d00

The first little-endian uint32_t seems to indicate a X value for the piecewise interpolation. The second little-endial uint16_t is some sort of coefficient and the last uint32_t seems to be a y offset. It would make sense that all values are given in something linear to counts/sec and indeed the given value 0x9cd = 2509 seems to correspond pretty much with 5.0 cts/sec. About the two last I'm still playing around a fair bit. It seems like it's not a linear, but rather maybe quadratic piecewise function. Not entirely sure. But I'll try to figure this out for sure, fix this once and for all.

Edit: Here's the preliminary work: 06d80b2

[johndoe31415]

Hey, I think I figured it out!

So my guess from yesterday was wrong. It's not piecewise linear. Then I thought piecewise parabolic, but that wasn't it either. It's piecewise hyperbolic interpolation. And there's not two coefficients, but three, i.e., every line contains x, y1, y2, y3. The piecewise function is, for a given counts/second x:

f(x) = ax / (b - cx)

Where a = y3 (the linear coefficeint), b = y2 and c = y1 / 256 or y1 / 65536. There's a byte value that's in the header line (for your calibration it's 0x7e, for mine it's 0x04) that indicates for every bit (first line is LSB) it it's divided by 256 or 65536. That's pretty much it. Note that probably due to some rounding errors in how the internal calculation of GammaScout Toolbox works, the values that gammascoututil generates deviate a small bit. I've created a large testbench that's also included (testdata_*.json):

ERROR: testdata_v605.json.gz 1 counts over 60 seconds, 0.0 cts/sec expected 0.00617 µSv/hr but calculcated 0.00617 µSv/hr. Error 0.1%
ERROR: testdata_v605.json.gz 2 counts over 60 seconds, 0.0 cts/sec expected 0.01234 µSv/hr but calculcated 0.01235 µSv/hr. Error 0.1%
ERROR: testdata_v605.json.gz 3 counts over 60 seconds, 0.1 cts/sec expected 0.01852 µSv/hr but calculcated 0.01854 µSv/hr. Error 0.1%
ERROR: testdata_v605.json.gz 4 counts over 60 seconds, 0.1 cts/sec expected 0.02472 µSv/hr but calculcated 0.02473 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 5 counts over 60 seconds, 0.1 cts/sec expected 0.03091 µSv/hr but calculcated 0.03093 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 6 counts over 60 seconds, 0.1 cts/sec expected 0.03712 µSv/hr but calculcated 0.03713 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 7 counts over 60 seconds, 0.1 cts/sec expected 0.04333 µSv/hr but calculcated 0.04334 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 10 counts over 60 seconds, 0.2 cts/sec expected 0.06822 µSv/hr but calculcated 0.06823 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 11 counts over 60 seconds, 0.2 cts/sec expected 0.07446 µSv/hr but calculcated 0.07447 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 12 counts over 60 seconds, 0.2 cts/sec expected 0.08070 µSv/hr but calculcated 0.08072 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 15 counts over 60 seconds, 0.2 cts/sec expected 0.09947 µSv/hr but calculcated 0.09949 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 16 counts over 60 seconds, 0.3 cts/sec expected 0.10574 µSv/hr but calculcated 0.10576 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 17 counts over 60 seconds, 0.3 cts/sec expected 0.11202 µSv/hr but calculcated 0.11203 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 20 counts over 60 seconds, 0.3 cts/sec expected 0.13087 µSv/hr but calculcated 0.13089 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 22 counts over 60 seconds, 0.4 cts/sec expected 0.14348 µSv/hr but calculcated 0.14349 µSv/hr. Error 0.0%
ERROR: testdata_v605.json.gz 708 counts over 60 seconds, 11.8 cts/sec expected 7.25853 µSv/hr but calculcated 7.26269 µSv/hr. Error 0.1%
[...]

I.e., the error really is VERY small.

I have yet to work this into gammascoututil, might rework the whole thing as well since the protocol has changed significantly since I've last worked with it and some major refactoring/cleanups seem to be necessary. But I'll call it a day for today. All the code is currently in the caltest/ subdirectory. Commit is bead22b.

[Slyke]

Hey! Yah! That's all good! I'm actually about to go on holidays myself! Just jumped on to check emails.

Wow, you've done so much work! I think you have actually figured it out too. I have a GammaScout Online, not sure why it would need extra calibrations, but it was recently purchased so they have probably smoothed the formula a bit or or added in more sections for the piecewise functions something.

I think the GammaScoutToolbox can't do floating point calculations well and is losing some accuracy by truncating some lower decimals which is why different results are emerging from your data, but as you said they are so minor it probably doesn't matter. I noticed this too when generating the large datasets. There would be a bunch of pulses they all ended up being the same uSv/hr after it had calculated them.

In about a week when I get back and I'll check out everything and integrate it into the project I'm working on. If I find anything I will let you know! Thanks heaps!