December updates

Logging reduction

- Remove MQTT logs
- Only log exceptions for ingestion
- Moved monitoring dashboard to Grafana

One thing that's always bugged me is how much we spend on logging useless info and the cost to have our pretty status monitoring dashboard. Fixing the first is a tricky problem as by default Lambda functions log their start and stop execution details - and the only way to turn this off is to turn off all logging.

To work around this we have to create our own custom logger that handles just exceptions. That's now been implemented. We also turned off much of the SNS to MQTT logging to CloudWatch as it was mostly noise. We can turn them back on if needed. That has reduced the PutLog costs (or USE1-DataProcessing-Bytes).

The CloudWatch dashboard was migrated to Grafana which we already had setup for querying radiosonde and amateur data.

It's not a lot in the grand scheme of our bill but its wastage I'd prefer to avoid.

OpenSearch Improvements

- Disk type change GP3
- Upgrade version
- Disk space reduction
- Less hot data

Mostly just maintenance tasks of ensuring our DB is the latest version and not falling behind. This also requires updating some of our API requests to match new requirements. 

The disk type was switched over to GP3 which provides some better performance and more fined controls. 

To save on disk space, CPU and memory utilisation we've also reduced the amount of hot data we keep in the OpenSearch database. The idea here is that we don't need to increase the size of the DB if the amount of "hot data" is kept low. All the historic data is still retained in our S3 bucket though.

Minor improvements

- Updated sonde types lists
- Backend multi sonde search (once frontend changes are implemented we'll be able to filter on multiple amateur callsigns)
- Various bug fixes and tweaks (some related to Habhub shutdown) 

Future

At the moment we keep every single prediction ever run. These are slowly growing in size. We are part way through developing software to archive just the oldest entry (closest to the ground) and discard the rest to save space.