Add support for pluggable keystore and truststore for SSL connections #129972
Description
Description
Many big enterprises use private/proprietary key management solution, especially the ones with on-premise deployments. According to current configurability in ElasticSearch, it already supports PEM and JKS/PKCS12 based key materials for SSL communication for HTTP/Transport/Remote-Cluster-Server reference. However, it could be tedious to manage multiple distributed systems, including ElasticSearch, and comply to such company's internal security guidelines for integrating with in-house key management solution. Also, operationally it is challenging to manage large fleet of servers (of various kinds - like Kafka, Cassandra and ElasticSearch stack) with file based key materials.
ElasticSearch already provides a straightforward way to configure SSL key/truststores with SslKeyConfig and SslTrustConfig interfaces. The only missing piece is to have configuration keys to allow providing custom Key/Trust Config implementations. Also for all use-cases like HTTP, Transport and Remote Cluster Server - the code seems to be common to load/initialize the Ssl Key/Trust Configs via SslConfigurationLoader class's settingsPrefix. Hence enabling one additional config will unlock the same benefit to multiple use-cases easily.
The proposal here is to introduce additional SSL configurations with suffix .custom-key-config and .custom-trust-config (subjected to name review etc), that will take a custom implementation for SskKeyConfig and SslTrustConfig interfaces. Also, the new configuration can take additional parameters (key-value pairs) to allow configuring the runtime implementations for the same.
Logically the code changes will be in the SslConfigurationLoader (here and here) after it checks for PEM and JKS/PKCS12 key materials. If the custom-key/trust-config is specified it will be used to initialize the key/trust config.
There could be another possibility of making the SslConfigurationLoader as pluggable or make SslConfiguration created in a pluggable way with defaults to existing code. However, I will seek community guidance on what makes more sense. In general, providing pluggable mechanism to create Java's SSLContext is most flexible and doing one of these two would come close to that.
Currently, I am not quiet sure how this applies to other Elastic stack (Kibana etc). However, with the community guidance we can take this further.
If there is enough agreement on the value of this, I can make necessary changes (following the contribution guidelines) and raise a PR for review.
References to similar customization available on the open-sourced Apache projects-