[StackOverflow/kubernetes] spark-submit encounter PKIX path building failed

### ROOT CAUSE The PKIX path building failure occurs because the Java Secure Socket Extension (JSSE) in the Spark environment cannot establish a trust chain for the server's TLS certificate. This typically happens when: 1. The server's certificate is self-signed or issued by a Certificate Authority (CA) not trusted by the JVM. 2. Intermediate certificates are missing in the trust chain. 3. The Spark driver is misconfigured to use a specific truststore that excludes the server's CA. ### CODE FIX 1. **Add the Server's CA Certificate to the Truststore**: - Locate the server's CA certificate (e.g., `server-ca.crt`). - Import it into the Java truststore using: ```bash keytool -import -trustcacerts -file server-ca.crt -keystore $SPARK_HOME/conf/jssecacerts -alias server-ca ``` - Ensure the `jssecacerts` file is in your Spark distribution's `conf/` directory. 2. **Explicitly Trust the Certificate in Spark Submit**: Pass JVM options to trust the certificate directly: ```bash spark-submit \ --conf "spark.driver.extraJavaOptions=-Djavax.net.ssl.trustStore=$PWD/custom-truststore.jks -Djavax.net.ssl.trustStorePassword=changeit" \ --conf "spark.executor.extraJavaOptions=-Djavax.net.ssl.trustStore=$PWD/custom-truststore.jks -Djavax.net.ssl.trustStorePassword=changeit" \ --jars /path/to/jsse libraries/... \ your-spark-job.jar ``` - Create `custom-truststore.jks` with the server's CA certificate (use `keytool` to import). 3. **Verify Certificate Chain**: Ensure the server presents all intermediate certificates. Use tools like `openssl` to inspect the certificate chain: ```bash openssl s_client -connect server-host:port -showcerts ``` 4. **Check Spark Configuration**: Ensure no conflicting Spark properties (e.g., `spark.ssl.enabledAlgorithms`) override the trust configuration. **Note**: Replace placeholders (e.g., `$PWD/custom-truststore.jks`) with actual paths and passwords. This solution ensures the JVM trusts the server's certificate, resolving the PKIX error.