DCF77 Meteo Data 1 full day decode

[brettoliver]

DCF77MeteoDataDecode01..txt

Edit -data is missing from the code I will resend it complete as sson as the new data is ready!

Enclosed 1 full days Meteo Data from 00:00 to 23:59. Maybe you will find this useful.

I have added time to the data at the end of each section using basic serial.print with no leading zeroes added eg 0:3 is 00:03

I am transcribing the data onto a spreadsheet so sections of data can be broken down to see what and when data is sent.

It will take some time to do but can post it if it's of any use.

[tobozo]

Thanks !

My ESP32 can't get a full day from my indoor antenna either, its logs show some holes in the weather data too, and sometimes redundancy between two decodings, which makes me wonder what sort of bug I've created there 🤔

I haven't checked if the DCF77 signal decoding task was still properly timed after the last code changes, it could be a possible regression of a time-shift bug, eventually losing a whole minute of data before being back in sync, or it could just be QRN from the TFT.

Hopefully this will get sorted out after the refactoring.

Anyway, the weathermemory part needs a rewrite now, so I've started storing this data in NVS.

With one full weather data block (=3 bytes) every 3mn, the whole 24h data set can't contain more than 480 entries, some of which may address more than one type of forecast.

I'll go with 480 of those:

struct WeatherForecast {
  int hour;
  int minute;
  uint8_t data[3];
};
WeatherForecast** AllCitiesForecast; // cache all forecasts

and 8 of those for the watched city:

struct WeatherData {

  int hour;
  int minute;

  int dcw;  // Day critical weather
  int ncw;  // Night critical weather
  int xwh;  // Extremeweather
  int anm1; // Weather anomality 1
  int anm2; // Weather anomality 2
  int rpb;  // Rainprobability
  int anm;  // Anomaly
  int tem;  // Temperature
  int twodayforecast;
  int fourdayforecast;
};

I'll probably use 20 slots of another struct for wind directions and strengths, one for each city.

Meanwhile I've been looking for interfaces examples to show DCF77 weather data, and a 160x1218 display obviously can't fit all that.

This made me realize using three icons to describe one weather data instance was wrong, so I'll probably, put the wind direction icon in the middle (replacing the atomic icon), remove the rightmost weather icon and reuse the space for windspeed + humidity + temperature.

It's a good thing the ButtonC short-push event is currently unassigned, it could end up as a browsing control for the different stored forecasts.

Since the 2-3-4 days forecasts ('DO', 'Fr' and 'SA' on the screen capture) don't have humidity, wind strength and windspeed, the display pattern only requires one icon and one text block, this could be scroll material :-)

[brettoliver]

I am hoping to get a full days decode by 23:00hrs tonight. I use an Udo Klein Super Filter between my aerial and my ESP32. The filter fills in the gaps if signal is lost. It does not decode the Meteodata though.
I have used a couple of days data to build up a spreadsheet showing what is sent minute by minute. I'll paste a copy here prob Tuesday morning.

I 'll also load your latest code once I have finished recording the serial out. Looks like you have made a number of changes in the last few days.
I like the serial output and have got it working in part on another TFT display using a Arduino as a serial monitor.
Is there a way of making the ESP32 transmit serial at 9600 as this would work better on my serial monitor?

I like your idea of the wind and other data on the display

[tobozo]

Is there a way of making the ESP32 transmit serial at 9600 as this would work better on my serial monitor?

you just need to change this:

M5.begin();

into this:

M5.begin(true, true, false); // LCDEnable, SDEnable, SerialEnable
Serial.begin(9600);

[brettoliver]

Just loaded your latest version. Thanks for all those updates.

I think I have a full days decode now and have included it.
DCF77 Meteo Data Decode13JulyBrett.txt

Have also gone through a days decode minute by minute and put it onto a spreadsheet so the data can be analized.
DCF77 Meteo Data Decode04.xlsx

[brettoliver]

Thanks for the info on changing the baud rate. I now have a good output to my 2nd TFT serial monitor.
I have tweeked the serial.print outputs to make it fit my display.

On the TFT display where it says "Rain Showers" I think there is a bit of serial output missing. it should say "Night = 1100 03 Rain Showers"

If you look at my spreadsheet this output is missing on all the Night colums bit there for the day column.
After 21:00 hrs both day and night outputs are complete for the 2 day forecasts.

[tobozo]

Thanks for this data 👍

So if there's only one label missing in the text version, is it safe to assume the weather decoding logic tree is correct ?

I'm rewriting the whole weather display routine where I separate the TFT output from the Serial output (one is more verbose than the other), but there is colateral damage : the binary representation of data chunks is lost.

I'll try to come up later with something to restore that low level debug, but I'll stick with the 32 columns model for the Serial output rendering.

[brettoliver]

Hi the weather decoding logic looks fine.

I am just waiting for my LOLIN D32 Pro V2.0 to arrive then I can have a play with a 240x320 TFT.

I will just mix and match my serial display output to match your changes as they happen.

[brettoliver]

Have received my LOLIN D32 Pro V2.0. Compiled and loaded OK to my board.
Do I need to set the pins for my board in your code?

[tobozo]

If you select the corresponding LoLinD32Pro device from the boards menu you don't have any pins to configure provided you're using the 1.4'' TFT model (a ST7735) fitting the special SPI connector, however I'm quite sure it'll also work with an ILI9341 as they share many instructions, and all necessary pins are labelled on the silkscreen.

[edit] here's a successful implementation of lolin d-32-pro with an ili9341 with touch spi sharing the tft spi.

[brettoliver]

I have checked and I have the correct board selected.
The code shows the DCF77 pin as 21 and silkscreen on the back of board has this as the SDA pin. I have changed the the DCF77 pin to 13.
I am using a 2.2" ILI9341 TFT. The backlight comes on but nothing is displayed.
The serial out shows
H⸮F�ȭ⸮⸮�� �%�⸮XN�
11Q8⸮⸮)�!⸮Q⸮⸮[E][UI.h:1126] InitUI(): Successfully created spritePtr
[E][UI.h:1133] InitUI(): Successfully created logoSpritePtr

Then nothing else. Do I need to config a 320x240 board in the code?
Any ideas?

[brettoliver]

Maybe I have the TFT connection wrong?
I had CS on Pin5 SS- moved it to Pin 14 T-CS and now get this

[E][UI.h:1126] InitUI(): Successfully created spritePtr
[E][UI.h:1133] InitUI(): Successfully created logoSpritePtr

Scanning I2C Bus:

 0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
         -- -- -- -- -- -- -- -- -- -- -- -- --

01: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
02: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
03: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
04: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
05: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
06: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
07: -- -- -- -- -- -- -- --
Millistart 12752
Millioffset 0
E (34822) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (34822) task_wdt: - IDLE0 (CPU 0)
E (34822) task_wdt: Tasks currently running:
E (34822) task_wdt: CPU 0: buttonsTask
E (34822) task_wdt: CPU 1: IDLE1
E (34822) task_wdt: Aborting.
abort() was called at PC 0x400e4ccf on core 0

Backtrace: 0x4008c9b4:0x3ffbe170 0x4008cbe5:0x3ffbe190 0x400e4ccf:0x3ffbe1b0 0x40085159:0x3ffbe1d0 0x400dbe12:0x3ffaf630 0x400da404:0x3ffaf650 0x400d7285:0x3ffaf670 0x400d7383:0x3ffaf690 0x4008900d:0x3ffaf6b0

Rebooting...

[tobozo]

Indeed CS should be on 14 as indicated by the "T-CS" label on the back of your LoLin D-32-Pro board.
Does the display flicker during the boot loop or is it always off ?

I can't decode the backtrace for you (it requires the ELF from your compilation) but there's a tool in the Arduino menu where you can paste the whole message and get a hint.

Decoding the exception must be done right after compiling or it won't work.
Sometimes the decoded content il also useless or has bad precision but still better than nothing :-)

The exception decoder is available in you "tools" menu / ESP Exception decoder, or you can install it from here.

Another thing you can do is check the pins for the DCF77, also I2C SDA and SCL, and eventually comment out those lines in esp32-DCF77_Analyzer_Clock/Config.h
:

#define USE_RTC
#define USE_BUTTONS

... to see if you still get the same error

[edit] managed to reproduce the same error message after flashing my TTGO-T1 with the wrong board, the timeout was related to the buttons pins not responding, so keeping USE_BUTTONS commented out to see if the display responds is indeed a valid test

[tobozo]

Not exactly a UI enhancement but I'm testing a lighter weather cache system (maintaining one NVS key per city was a bad idea) this time using an array fed with weather data for all cities, and backed up to NVS (max 5Kb) every hour.

Note how the last three minutes of every hour never succeed to decode any weather data, same a in your logs.
This seems to affect at least cities 19,39 and 59 and indicates there's a bug somewhere between the signal decoding logic and the tasks management.

The weather display logic is still a work in progress and I need to achieve a visible result before I push the changes, so I'll consider this bug as a tech debt unless you tell me you need the weather data for those specific cities.

[brettoliver]

Indeed CS should be on 14 as indicated by the "T-CS" label on the back of your LoLin D-32-Pro board.
Does the display flicker during the boot loop or is it always off ?

I can't decode the backtrace for you (it requires the ELF from your compilation) but there's a tool in the Arduino menu where you can paste the whole message and get a hint.

Decoding the exception must be done right after compiling or it won't work.
Sometimes the decoded content il also useless or has bad precision but still better than nothing :-)

The exception decoder is available in you "tools" menu / ESP Exception decoder, or you can install it from here.

Another thing you can do is check the pins for the DCF77, also I2C SDA and SCL, and eventually comment out those lines in esp32-DCF77_Analyzer_Clock/Config.h
:

#define USE_RTC
#define USE_BUTTONS

... to see if you still get the same error

[edit] managed to reproduce the same error message after flashing my TTGO-T1 with the wrong board, the timeout was related to the buttons pins not responding, so keeping USE_BUTTONS commented out to see if the display responds is indeed a valid test

Sorry for the late reply have been away....
Have set
DCF77 Pin to 13
commented out USE_BUTTONS
DS1307_SDA to 21
DS1307_SCL to 22
The code now seems run as I am getting serial out
E][UI.h:1126] InitUI(): Successfully created spritePtr
[E][UI.h:1133] InitUI(): Successfully created logoSpritePtr

Scanning I2C Bus:

 0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
         -- -- -- -- -- -- -- -- -- -- -- -- --

01: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
02: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
03: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
04: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
05: -- -- -- -- -- -- -- 57 -- -- -- -- -- -- -- --
06: -- -- -- -- -- -- -- -- 68 -- -- -- -- -- -- --
07: -- -- -- -- -- -- -- --
Meteo Data Decoding Hour 12 Min 50
Meteo Data Decoding Hour 12 Min 51
Meteo Data Decoding Hour 12 Min 52
Meteo Data Decoding Hour 12 Min 53
Meteo Data Decoding Hour 12 Min 54
City 17 Le Havre (France)
Period/Day Day 3
Day Forecast 0100 02Partly cloudy
Clear
Extreme weather 0000 00 None
Rain probability 100 01 15 %
Weather Anomality 0 00 No
Temp 001101 2c 22°C max
Decoder status 10 OK
City not displayed
Hour 12 Min 54

Meteo Data Decoding Hour 12 Min 55
Meteo Data Decoding Hour 12 Min 56

The display however still does not work.
It does flicker a bit then just goes to a grey screen.

[brettoliver]

re lighter weather cache system
I don't need weather data for those cities so the bug can be left as a tech debt.