Using a modified Lanczos method instead of full factorization for large-scale problems?

[paulstapor]

Hi @FFroehlich !

Many thanks again for your implementation!
I just had an idea, what one might want to do in the case of using Fides on a really large-scale system, where complete factorization of the Hessian may become expensive (some k variables). A version of this method which can deal well with indefinite matrices is discussed by Stephen Nash.

This is anything but a request, it's rather a question whether this might be a meaningful add-on at some point...

Best!

[FFroehlich]

I think thats a very good idea, i have read multiple paper recommending exactly that, but without an appropriate testcase at hand, its difficult to anticipate other problems that might come up (can we even compute hessian in that case? Is BFGS even viable in that case). There are tons of papers out there that one could apply here.

[paulstapor]

Agreed, zillions of methods for large models in this case.
What Nash suggests is doing some precoditionning using e.g. BFGS or SR1 and then shooting some steps of his truncated Newton algorithm on it, instead of computing an actual Newton-step. Would be a good use for Hessian-vector-products, as this is all his algorithms actually needs...

However, that can only be tested reasonably with (semi-)analytical Hessian vector products. And that has still to wait...