× RadLog

Edit Long Term Trend?

More
1 week 3 days ago #7042 by Simomax
Replied by Simomax on topic Edit Long Term Trend?

Do you suppose the E-05 values are my offending data points?  I did not see out of range data in the CPM section for the same XValue. 

No, they are actually very small numbers. They are powers of ten. E-05 means times ten to the power of -5, so 4.74670369340925E-05 is actually 0.0000474670369340925. A quick way of calculating this is to multiply by 0.00001. If the number had E+05, then this is a large number, and is ten to the power of 5, and can be quickly calculated by multiplying by 100000. E-03 would be multiply by 0.001, and E+03 would be multiply by 1000.

I used ChatGPT to verify that for me (for sanity's sake) but then also asked ChatGPT if it could decode the timestamp, and I think it did. I would have to verify this though. ChatGPT said the following about the timestamp 44678.682294444443:

The number 44678.682294444443 is likely a date in Excel or .NET serial date format, where the value represents the number of days since a specific start date (also known as the epoch). For Excel and similar systems, the epoch is usually January 1, 1900.To convert the number 44678.682294444443 into a date, we'll follow these steps:

  1. 44678 represents the number of days since January 1, 1900.
  2. The decimal part (0.682294444443) represents the fraction of a day (which we'll convert into hours, minutes, and seconds).
Let me calculate this for you.The number 44678.682294444443 converts to the date and time April 29, 2022, at 16:22:30 (4:22:30 PM).

And this is apparently how it got the result:
from datetime import datetime, timedelta

# The given serial date number
serial_date = 44678.682294444443

# Excel's base date (epoch): January 1, 1900
base_date = datetime(1900, 1, 1)

# Calculate the date by adding the number of days to the base date
result_date = base_date + timedelta(days=serial_date)

# Display the full date and time
result_date.isoformat()

Not exactly the answer you were after, but hopefully offers another step forward in understanding the xml file.  I'll check out the date thing later today, after work. There is a good chance the spike you are looking for will be most prevalent in the 'CPM max' section (4th down in a file I'm looking at now), so I would try looking in there first, then once you find the spike, copy the corresponding date code and do a search for that in a text editor to find the other values of that same time stamp. Maybe the entire section could be copied into excel or something similar, then order by max number and get the record and date code that way. Or maybe even just feed the whole thing into ChatGPT and ask it to find the spike. That would be interesting!

Please Log in or Create an account to join the conversation.

More
1 week 3 days ago #7043 by Simomax
Replied by Simomax on topic Edit Long Term Trend?
Doh! When I was trying to work out the date code previously I thought epoch was 01/01/1970. I didn't realise it is 1900! I tried counting the days from 1970, which threw me way off. I'll have another stab at this later tonight.

Please Log in or Create an account to join the conversation.

More
1 week 3 days ago #7044 by GetSirius
Replied by GetSirius on topic Edit Long Term Trend?
Yes, I should have known that was a small number (feeling rather small here now).  I gave editing my .xml a try, the large CPM showed right up. We shall see what happens when the system updates. The second image on my first post is a live graph. Hopefully it will show good info soon. 

Thanks again.

Mineral Point radiation/lightning monitor: getsirius.com/radiation

Please Log in or Create an account to join the conversation.

More
1 week 3 days ago #7045 by Simomax
Replied by Simomax on topic Edit Long Term Trend?
Most welcome.

I think I have my head around the date/time in the xml. Going from what ChatGPT said about the two parts of the date code I have been doing some calcs and they all seem to pan out right. I have the working equations now for converting the time stamp to a human readable date time format, and vice versa. I'm working on a php page to do this right now. Using the calculator (when I finish it) should make it much easier to locate spikes. Simply look at the graph, get a round-about date (maybe time too?) feed it into the calculator and that will spit out the Excel formatted date (and time if used) in the format of days.percentage of the day, which is what the time stamps are in the xml. I'll post back here when it is done.

Please Log in or Create an account to join the conversation.

More
1 week 3 days ago #7046 by Simomax
Replied by Simomax on topic Edit Long Term Trend?
Alright, I got there and made a php page for converting the crazy time stamp (45578.87503472222222) in the xml to a human readable date and time, and also the reverse so you can generate the crazy time stamp from a given date and time. It's quite simple to use:

To generate a crazy time stamp simply choose the date and time at the top, and click the 'Date Time to Days and Percentage' button.

To decode a crazy time stamp simply copy and paste it into the box at the bottom and click the 'Days and Percentage To Date Time' button.

 

Usage cases; 'I want to find something in the xml file at a certain time date': Use the top section and punch in a date and time so you can see the readings around that time. It should get you in the ball park.
'I see something in my xml file and I want to know the date and time of the thing.': Use the bottom section and copy the crazy time stamp from the xml file into the box in the bottom section. It will show you the date and time in a human readable format.

Whilst this isn't a panacea for finding spikes, it should allow for getting in the ball park, getting somewhere close. It will still require some manual work browsing the xml file to find the spike, but it should get you reasonably close. I have a couple of other spikes in my long term log so I'll try finding them with this tool. But tomorrow. My brain is tired from all of those calculations.

There was one anomaly I came across. When I took a crazy time stamp from the xml, decoded it and then looked in the long term text file, the date was a couple of days out. I did this with the very last log in both the text file and the xml, so they should have been the same. Maybe not though as I'm not sure what the text file really does. It might be for shared data.

The tool is here:  https://www.schmoozie.co.uk/radmon/time_tools.php

And source here for hosting yourself (.php):
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<?php
$excelDate = 0;
$dayPercentage = 0;
$daysDate = '0000-00-00';
$days = 0;
$date = '';
$time = '';
$dayPerc = '';
$dated = date('Y-m-d');
$datet = date('H:i:s');
$daysPercentage = '';

extract($_REQUEST);
if(isset($submitDateTimeToDaysAndPercentage)) {
    $days = dateToExcelDate($dated);
    $dayPercentage = timeToDayPercentage($datet);
    $tempArr = explode(".", $dayPercentage); // use temp array to strip (1st index) '0.' from percentage to show only past the decimal
    $dayPercentage = $tempArr[1];
    $daysPercentage = $days . '.' . $dayPercentage;
}
else if(isset($submitDaysAndPercentageToDateTime)) {
    //echo $dayPerc;
    $tempArr = explode(".", $dayPerc);
    $date = daysToDate($tempArr[0]);
    $time = percentageToTime('0.' . $tempArr[1]);
}
// Function - date to Excel date format
function dateToExcelDate($dateStamp) {
    $date1 = new DateTime('1900-01-01'); // Epoch date
    $date2 = new DateTime($dateStamp); // Our date
    $interval = $date1->diff($date2); // Get the date difference
    return $interval->days; // Return number of days of the date difference
   }

// Function - date to Excel date format
function timeToDayPercentage($timeStamp) {
    $timeArray = explode(":",$timeStamp); // Insert hours, minutes and seconds into an array from our time
    $hours = $timeArray[0];
    $minutes = $timeArray[1];
    $seconds = $timeArray[2];
    $hoursP = $hours/24; // Do some maths to get the percentage of the day for the hours, minutes and seconds
    $minutesP = $minutes/1440;
    $secondsP = $seconds/86400;
    return ($hoursP + $minutesP + $secondsP); // Return the sum of the three percentages
   }

// Function - date after x days after 1900:01:01
function daysToDate($days) {
    $startDate = new DateTime('1900-01-01'); // Epoch date
    $startDate->modify("+$days days"); // get date x days since epoch date
    return $startDate->format('Y-m-d'); // Return date in readable form
    }
     
// Function - convert percentage of a day to a time
function percentageToTime($percentage) { // This is messy to comment. It is easier to just comment with the actual math. The percentage for this is 0.682294444443
    $totalHours = 24 * $percentage; // Total hours = 24 * 0.682294444443 = 16.374706666632 hours
    $pHours = floor($totalHours); // this is the whole hours number (16). We loose the decimal and everything after
    $totalMinutes = 60 * ($totalHours - $pHours); // Then calculate the total minutes from the decimal of total hours: Total Minutes = 60 * (total hours - whole hours)
    $pMinutes = floor($totalMinutes); // this is the whole minutes number (22). We loose the decimal and everything after
    $totalSeconds = 60 * ($totalMinutes - $pMinutes); // Then calculate the total seconds from the decimal of total minutes: Total Seconds = 60 * (total minutes - whole whole)
    $pSeconds = floor($totalSeconds); // this is the whole seconds number (28). We loose the decimal and everything after
    return  $pHours . ':' . $pMinutes . ':' . $pSeconds; // Return the time formatted as ##:##:##
    // Phew......
}
// echo '<br /><br />';
// $dateStamp = '2022-04-25';
// $excelDate = dateToExcelDate($dateStamp);
// echo "Date to days since 1900:01:01: " . $excelDate;
// echo '<br /><br />';
// $timeStamp = '16:22:29';
// $dayPercentage = timeToDayPercentage($timeStamp);
// echo "Time to percentage of a day: " . $dayPercentage;
// echo '<br /><br />';
// $days = 44678;
// $daysDate = daysToDate($days);
// echo "The date " . $days . " days after 1900:01:01 is " . $daysDate;
// echo '<br /><br />';
// $percentage = 0.68228009259259;
// $timeStamp = percentageToTime($percentage);
// echo "The time is: " . $timeStamp;

?>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Radlog.exe XML time tools</title>
<link rel="icon" type="image/x-icon" href="favicon.ico">
</head>
<body>
<form method="post" enctype="multipart/form-data">
Convert readable date time into Radlog.exe XML date code:<br /><br />
<label>Date Time</label>
<input required type="date" value="<?php echo $dated; ?>" name="dated" >
<input required type="time" step="1" value="<?php echo $datet; ?>" name="datet" > = <?php echo $daysPercentage; ?><br /><br />
<input type="submit" name="submitDateTimeToDaysAndPercentage" value="Date Time to Days and Percentage">
</form>
<br /><br /><br /><br /><br /><br />
Convert Radlog.exe XML date code to readable date time:<br /><br />
<form method="post" enctype="multipart/form-data">
<label for="dayPerc">Days and day percentage:</label>
<input type="text" id="dayPerc" value="<?php echo $dayPerc; ?>" name="dayPerc"> = <?php echo $date . ' ' . $time; ?><br /><br />
<input type="submit" name="submitDaysAndPercentageToDateTime" value="Days and Percentage To Date Time">
</form>
   
</body>
</html>
Attachments:

Please Log in or Create an account to join the conversation.

More
1 week 3 days ago #7047 by GetSirius
Replied by GetSirius on topic Edit Long Term Trend?
Thanks for the code. I will probably put that on my server for use should I need it some day. 
Success! I now have a useable graph. Thanks for your help!
I removed the live graph from my post, really don't need Google finding the folder. Here is a copy:


Mineral Point radiation/lightning monitor: getsirius.com/radiation

Please Log in or Create an account to join the conversation.

Moderators: Gamma-Man
Time to create page: 0.212 seconds
Powered by Kunena Forum
Everything's free. Please support us by considering a donation. Log in first!
Solar powered Raspberry Pi 4 server stats: CPU 25% Memory 14% Swap 42% CPU temp=56.9'C Uptime 44 Days