Add page on cloud detection performance at low sun elevation

[kandersolar]

Courtesy of @williamhobbs:

The sharp “horns” that are noticeable during sunrise and sunset in the BON data in Fig. 5 are due to the false negative identification of clouds by the Bayesian cloud classification algorithm (Heidinger et al. 2012) at extreme solar zenith angles. This is a common feature in the NSRDB and can be visualized in animated maps where clouds can be seen to “blink” into existence as the sun rises.

Source: https://doi.org/10.1016/j.solener.2022.01.004

Other examples: the GCR SURFRAD station around these dates: 2021-01-09, 2021-02-01, 2021-02-08

[williamhobbs]

Thanks for starting this issue. Here are plots of those date ranges, with code below. They just happen to be the first dates I noticed. I haven't checked to make sure that timestamp conventions are correct, but the concept is clear either way.

import pvlib
import pandas as pd
import matplotlib.pyplot as plt

lat, lon = 34.2547, -89.8729 # Goodwin Creek SURFRAD
nsrdb_data, _ = pvlib.iotools.get_psm3(latitude=lat, longitude=lon,
                                       api_key='DEMO_KEY', email='your.email@gmail.com',
                                       names='2021', interval=30)

# convert to UTC
nsrdb_data.index = nsrdb_data.index.tz_convert('UTC')

surfrad_data1, _ = pvlib.iotools.read_surfrad(filename='https://gml.noaa.gov/aftp/data/radiation/surfrad/Goodwin_Creek_MS/2021/gwn21008.dat')
surfrad_data2, _ = pvlib.iotools.read_surfrad(filename='https://gml.noaa.gov/aftp/data/radiation/surfrad/Goodwin_Creek_MS/2021/gwn21009.dat')
surfrad_data = pd.concat([surfrad_data1,surfrad_data2])
surfrad_30m = surfrad_data.resample('30T').mean()
mask = (nsrdb_data.index > '2021-01-08 10:00') & (nsrdb_data.index < '2021-01-10 02:00')
plt.plot(nsrdb_data.GHI[mask], drawstyle='steps', label='nsrdb ghi')
plt.plot(surfrad_30m.ghi, drawstyle='steps', label='surfrad ghi')
plt.legend()
plt.show()

surfrad_data3, _ = pvlib.iotools.read_surfrad(filename='https://gml.noaa.gov/aftp/data/radiation/surfrad/Goodwin_Creek_MS/2021/gwn21032.dat')
surfrad_30m = surfrad_data3.resample('30T').mean()
mask = (nsrdb_data.index > '2021-02-01 10:00') & (nsrdb_data.index < '2021-02-02 02:00')
plt.plot(nsrdb_data.GHI[mask], drawstyle='steps', label='nsrdb ghi')
plt.plot(surfrad_30m.ghi, drawstyle='steps', label='surfrad ghi')
plt.legend()
plt.show()

surfrad_data4, _ = pvlib.iotools.read_surfrad(filename='https://gml.noaa.gov/aftp/data/radiation/surfrad/Goodwin_Creek_MS/2021/gwn21038.dat')
surfrad_data5, _ = pvlib.iotools.read_surfrad(filename='https://gml.noaa.gov/aftp/data/radiation/surfrad/Goodwin_Creek_MS/2021/gwn21039.dat')
surfrad_data6, _ = pvlib.iotools.read_surfrad(filename='https://gml.noaa.gov/aftp/data/radiation/surfrad/Goodwin_Creek_MS/2021/gwn21040.dat')
surfrad_data = pd.concat([surfrad_data4,surfrad_data5,surfrad_data6,])
surfrad_30m = surfrad_data.resample('30T').mean()
mask = (nsrdb_data.index > '2021-02-07 10:00') & (nsrdb_data.index < '2021-02-10 02:00')
plt.plot(nsrdb_data.GHI[mask], drawstyle='steps', label='nsrdb ghi')
plt.plot(surfrad_30m.ghi, drawstyle='steps', label='surfrad ghi')
plt.legend()
plt.show()

[williamhobbs]

In several NREL reports that include validation, it was only performed on time intervals with solar zenith angles less than 80 degrees:

https://www.nrel.gov/docs/fy17osti/67722.pdf
https://www.nrel.gov/docs/fy20osti/74386.pdf
https://www.nrel.gov/docs/fy22osti/82063.pdf

There's also https://doi.org/10.1063/5.0030992, which is referenced in the paper Kevin linked (emphasis mine):

However, under certain nonoptimal conditions (e.g., extreme solar zenith angles or viewing angles), the DCOMP model can fail to retrieve these cloud properties, which forces the PSM to perform a gap-filling procedure that is described in Section 2.2; this is a long-known issue and is well-described by Yang (2021, Section V.B.)

But I can't seem to find a free version of that paper.

[AdamRJensen]

There's also https://doi.org/10.1063/5.0030992, which is referenced in the paper Kevin linked (emphasis mine):
But I can't seem to find a free version of that paper.

@williamhobbs It should be available on Research Gate.

[williamhobbs]

Thanks! Apparently I didn’t look very hard…