Fix Fermi level variation at zero temperature

[niklasschmitz]

This is a problem I encountered from AD, but the underlying problem seems to lie in src/response/chi0.jl:

Currently, for zero temperature the variation of the Fermi level returned by apply_χ0_4P is always exactly zero (see chi0.jl). This is inconsistent with the zero-temperature convention to define the Fermi level as $\varepsilon_F = (HOMO + LUMO)/2$ which we also use in DFTK (cf src/occupation.jl), and disagrees with finite differences.

However, another problem to anticipate is that the zero-temperature definition will be non-differentiable at eigenvalue crossings. I'm not yet sure what the best solution is in such a case. Maybe we can throw a warning if we have near-degenerate eigenvalues at HOMO or LUMO with very different derivatives, indicating an eigenvalue splitting.

TODO list:

• Add (failing) extra tests on apply_χ0_4P for the variation δεF of the Fermi level against finite differences.
• Fix the problem at zero temperature

[antoine-levitt]

It's fair to move epsF as well for consistency , although you shouldn't depend on its result for anything (since it's just a convention)

If there is no gap at zero temperature, the system is not well behaved and we just have to tell the user to add temperature (currently; this may eventually change)

[niklasschmitz]

I think it's even worse, since the problem even appears in a system with a large gap: In the case that either HOMO or LUMO has a degeneracy (e.g. GaAs band structure at gamma point) which splits due to an external perturbation (for example due to strain, breaking lattice symmetry - something I am currently looking at, so that's how I got here :) ).

[antoine-levitt]

Ah, I see what you mean. Why do you care about this anyway? the variations of eps_F are not observable

[niklasschmitz]

I'm thinking about applications of AD to inverse design problems for engineering band structure properties. That led me to correctness questions around our AD ForwardDiff SCF rule in particular for bands & fermi level.

Since at zero temperature εF has a wrong derivative wrt external perturbation but eigenvalues has the correct derivative, then the shifted bands eigenvalues - εF get a (uniformly) wrong derivative, which is what currently happens.