Bias and Error Graphed Against Forecast Wind Speed
Another page, Results and Discussion, shows the bias and error in wind speed plotted against measured wind speed. That is useful in determining where the model is lacking and helps suggest areas that could be improved. This page shows results plotted against forecast wind speed. This approach allows for improvements by making adjustments to the forecast. Such adjustments can be a useful way to improve a modeled forecast because, at the time of the forecast, measured wind speed is not yet available.
To see larger versions of the graphs, click on them or open them in another window.
The next graph down shows the distribution of counts of data points for each
forecast wind speed. That explains some of the wild swinging at the
higher speeds. Other general details about graphs, biases, and data can
be found throughout the page: Results and Discussion.
Whenever I find nearly straight lines in graphed results, it suggests to me that I have found something useful and worth understanding. I have a couple speculations about the graph to the left.
First, notice that the absolute bias in the wind speed forecasted by WRF increases as that forecast speed increases. Please note that a negative bias indicates that WRF forecasts speeds are too fast. So as the forecast speed increases, the more likely it is to be faster than the measured speed, except at the speeds below 5 m/s where little or no power is generated. Farther down, I show my attempts to take advantage of this relationship by removing bias from the overall error.
And notice that the RUC lines are fairly consistent and level. They are not graphed against the RUC forecast speed but against the WRF forecast speed, so the implication is that RUC is generally predicting speeds slower than WRF. This is consistent with the bias graph on this page: Results and Discussion. That page also tells about my shortcuts in getting the RUC forecast speeds.
|
This graph just shows the distribution of data points used in the graphs on this page. Obviously, the data is much more dense for slower speeds, so data at those speeds will be averaged out to smoother lines. The data come from the same runs described in a similar section on the page: Results and Discussion.
|
This graph shows the average of the absolute error for WRF forecasts. I do not have a whole lot to say other than that that much error is too much for forecasts which power grid operators need to use. The above graph shows that more data points exist for the slower wind speeds. That is why slower speeds have more smoothness; the data has been averaged to similar values. But the lack of smoothness at higher speeds with fewer data points indicates a problem that would be shown in a graph of standard deviations: there is a lot of variability in the quality of forecasts, which makes them less easily relied on.
The WRF errors are generally less than the RUC errors.
|
This is the result if the exact biases shown three graphs up is removed from the MAE shown in the above graph. At the higher speeds where data points are few, there is little or no distinction between bias and error, so the error sometimes goes to zero. The change in scale is a little deceptive, but the errors do improve slightly for slower and mid range speeds by removing the bias. Considering the amount of bias, I would have expected better improvement. RUC showed more improvement than WRF. Perhaps that implies that the errors in WRF are a little more complex. But now that the error in WRF is about the same as the error in RUC (and even if it were not), it seems reasonable to ask "How much of the error in WRF is coming from the RUC data?" I have another page that explores that possibility: Attributing Error to RUC and NAM.
|
|
|