Improve historical chart data management

[colinl]

Description

Currently, for charts of xAxis type category, old data points are never deleted server side, so this eventually clogs everything up. See #1977.
This corrects that issue, only keeping one data point for each category in each series, for charts of xAxis type category, except for histogram charts, where all points must be kept.
In addition there are some efficiency improvements related to pruning of old data from the chart.
However, client side, the chart itself discards all except the latest values for category charts, so there appears to be nothing to do client side.
However, looking at the equivalent code for time based xAxis I see that it ends with

            // apply data limitations to the vuex store
            this.$store.commit('data/restrict', {
                widgetId: this.id,
                min: cutoff,
                points
            })

I don't fully understand what this is for, so don't know if anything similar is necessary in the categorical case. The PR has an empty function for the client side. If nothing is required then this, and the call, can be removed.

Related Issue(s)

Closes #1977

Checklist

• [ x] I have read the contribution guidelines
• Suitable unit/system level tests have been added and they pass
• Documentation has been updated
  • Upgrade instructions
  • Configuration details
  • Concepts
• Changes flowforge.yml?
  • Issue/PR raised on FlowFuse/helm to update ConfigMap Template
  • Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production
• Link to Changelog Entry PR, or note why one is not needed.

Labels

• Includes a DB migration? -> add the area:migration label

[Steve-Mcl]

Approved - with one comment suggestion

[Steve-Mcl]

Hmmm, didn't spot the conflicts before.

Can you please resolve them Colin so I can merge?

[Steve-Mcl]

look for the button

[colinl]

Oh, that's not right, something is messed up. I will sort it.

[colinl]

clearOldCategoricalPoints() in UIChart.vue could be removed completely, or do you prefer to keep an empty function in for clarity?

[Steve-Mcl]

After some discussion with Colin, this PR will become a general "Improve historical chart data management" aimed at the inefficiencies and slow downs witnessed in charts.

1 quick win - short circuit the logic in canSaveInStore function
1 big win - a new filter function in the data store to avoid repeating the msg cloning and calls to canSaveInStore for every existing data store message when a single new msg arrives.

For this reason, I will retract approval until the above are implemented.

improvements discussed and agreed

[colinl]

I have run some tests and it still seems to work ok.

[Steve-Mcl]

Happy to merge - will follow up with the maxPoint pruning improvements in a new PR