HTTPCLIENT-2381: use system properties as defaults in builder classes

[ok2c]

This turned out to be a big change-set but mostly due to many test classes affected by the API changes.

@arturobernalg @rschmitt @garydgregory Please review. For context please see

https://issues.apache.org/jira/browse/HTTPCLIENT-2381
https://lists.apache.org/thread/n1ckyf4soz0djw0bfo2sy3bhsso30vmj

[arturobernalg]

@ok2c LGTM

[ok2c]

If I hear no loud objections I will merge the change-set in the coming days

[rschmitt]

I'm going to need some time to get caught up on this. I won't be able to do any non-trivial testing until I get back from vacation in about a week.

[rschmitt]

I think "defaults" are a red herring here. The Apache client defaults are never going to work for the original requester, because the client is obviously not going to use his particular corporate proxy as the default value for proxyHost and proxyPort. All he's asking for is a way to override the defaults, without changing code:

Should this project be changed so that by default without code you can use system properties to configure the proxy?

All we need to do to solve his problem is define a new system property for the Apache client that, when set, has the same effect as calling .useSystemProperties() everywhere. This fixes the actual problem (locus of control) in a fully backwards-compatible way and without all the API churn I'm seeing in this PR. We already seem to be in agreement that "the client never queries system properties by default" is a totally obsolete constraint (and the Java platform feature that motivated it is deprecated for removal), so we're well within our rights to add a new opt-in system property that meets this need. The API impact would be zero, and the code change would probably be small enough to backport to previous versions.

Regarding backwards compatibility: flipping the default behavior after twenty years is virtually guaranteed to cause problems. There may be applications with inert copy-pasted system properties that don't actually work. There may be applications that are using those system properties to configure other libraries, e.g. HttpsURLConnection, while supplying the equivalent configuration to Apache programmatically. There may be test suites with fixtures or parallelization configs that don't expect client configuration to be affected by system properties (global mutable state). There's just a huge list of things I don't have to worry about if we simply make this an opt-in behavior (which, as I pointed out, it already is -- it's not a "default" in the sense that a socket timeout value or a retry strategy is a default).