CASSSIDECAR-484: Fix the spurious oldest segment age in CdcRawDirectorySpaceCleaner

[Klose6]

CdcRawDirectorySpaceCleaner re-reads segment metadata via File.lastModified() after the initial listFiles() scan. If Cassandra (or any other actor) reclaims a segment in the interim, the File.lastModified() will return 0, and nowInMillis - 0 collapses to wall-clock milliseconds.

[jyothsnakonisa]

Thanks for the patch @Klose6 left some minor comments

[jyothsnakonisa]

There could be a race condition between two assignments.

You could use Files.readAttributes here to get both of them atomically.

[jyothsnakonisa]

"segment might already be gone" is represented by lastModified == 0. There are two if conditions that you are adding on the lastModified == 0 condition to fix the bug. This works but what do you think about this alternative?

In the listing segment files pipeline, why don't you filter out segment files which are no longer present, so that you don't need to have if conditions later on?

Suggested change

	.map(CdcRawSegmentFile::createSegmentFile)
	.filter(Objects::nonNull)
	.filter(

[jyothsnakonisa]

CdcRawSegmentFile.createSegmentFile could be a static method to create segment files by reading file attributes atomically to avoid race conditions. Could be something like the following.

static CdcRawSegmentFile createSegmentFile(File logFile)
        {
            try
            {
                BasicFileAttributes attrs = Files.readAttributes(logFile.toPath(), BasicFileAttributes.class);
                return new CdcRawSegmentFile(logFile, attrs.size(), attrs.lastModifiedTime().toMillis());
            }
            catch (IOException e)
            {
                // most commonly NoSuchFileException: segment was reclaimed concurrently
                LOGGER.warn("Skipping cdc segment that disappeared before its attributes could be read path={}",
                            logFile, e);
                return null;
            }
        }