1 year ago

#334021

test-img

Aaditya Rathi

Getting PKIX path building failed: sun.security.provider.certpath.. Exception when trying to connect SQL SERVER with Pyspark locally

I am trying to connect to locally installed sql-server-2017 with pyspark. My intend is to create Df from it.

While doing so, I am getting the error as:-

An error occurred while calling o51.load. : com.microsoft.sqlserver.jdbc.SQLServerException: The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: "sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target". ClientConnectionId:fe719377-52ee-412f-9080-e97f5ae480db at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:3680) at com.microsoft.sqlserver.jdbc.TDSChannel.enableSSL(IOBuffer.java:2113) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3204) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2833) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:2671) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1640) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:936) at org.apache.spark.sql.execution.datasources.jdbc.connection.BasicConnectionProvider.getConnection(BasicConnectionProvider.scala:49) at org.apache.spark.sql.execution.datasources.jdbc.connection.ConnectionProvider$.create(ConnectionProvider.scala:77) at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$createConnectionFactory$1(JdbcUtils.scala:64) at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.getQueryOutputSchema(JDBCRDD.scala:62) at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:57) at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:239) at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:36) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:350) at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:274) at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:245) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:245) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:174) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) at py4j.ClientServerConnection.run(ClientServerConnection.java:106) at java.lang.Thread.run(Thread.java:750) Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alert.createSSLException(Alert.java:131) at sun.security.ssl.TransportContext.fatal(TransportContext.java:369) at sun.security.ssl.TransportContext.fatal(TransportContext.java:312) at sun.security.ssl.TransportContext.fatal(TransportContext.java:307) at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:652) at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:471) at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:367) at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:376) at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:479) at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:457) at sun.security.ssl.TransportContext.dispatch(TransportContext.java:199) at sun.security.ssl.SSLTransport.decode(SSLTransport.java:155) at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1315) at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1228) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:412) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:384) at com.microsoft.sqlserver.jdbc.TDSChannel.enableSSL(IOBuffer.java:2021) ... 30 more Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439) at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) at sun.security.validator.Validator.validate(Validator.java:271) at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:312) at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:232) at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:109) at com.microsoft.sqlserver.jdbc.TDSChannel$HostNameOverrideX509TrustManager.checkServerTrusted(IOBuffer.java:1702) at sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:1255) at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:636) ... 42 more Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434) ... 50 more

My code is below pasted, `import os import sys

from pyspark.sql import SparkSession sparkady = SparkSession
.builder
.master('local[*]')
.appName('Connection-Test')
.config('spark.driver.extraClassPath', r'C:\Projects\pyspark\sqljdbc_10_2enu\mssql-jdbc-10.2.0.jre8.jar')
.config('spark.executor.extraClassPath', r'C:\Projects\pyspark\sqljdbc_10_2enu\mssql-jdbc-10.2.0.jre8.jar')
.getOrCreate()

sqlsUrl = 'jdbc:sqlserver://127.0.0.1:1433;database=feederDB'

qryStr = """ ( SELECT * FROM Sparkbuilder ) as table """

sparkady.read.format('jdbc')
.option('url',sqlsUrl)
.option('driver', 'com.microsoft.sqlserver.jdbc.SQLServerDriver')
.option('dbtable', qryStr )
.option("user", "aditya")
.option("password", "Tiger@90")
.load().show()`

I downloaded the jdbc driver for sql server from this page- https://learn.microsoft.com/en-us/sql/connect/jdbc/download-microsoft-jdbc-driver-for-sql-server?view=sql-server-ver15

sql-server

jdbc

pyspark

jupyter-notebook

pkix

0 Answers

Your Answer

Accepted video resources