Page Comparison

Problem

Pipelines created to read from Microsoft SQL Server or writing to Microsoft SQL Server fail with Your pipelines reads or writes to a Microsoft SQL server and you get the following error:

Error: "Socket is closed". ClientConnectionId:<ID>

Symptom

Pipelines that are configured to run on Dataproc, either reading from SQL Server or writing to SQL Server fail while running the pipeline with Your Cloud Data Fusion pipeline reads or writes to a SQL Server. When you run your pipeline, it fails, throwing a Socket is closed exception. The complete You get the following error message is as follows:

avajava.lang.RuntimeException: java.lang.RuntimeException: com.microsoft.sqlserver.jdbc.SQLServerException: The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: "Socket is closed". ClientConnectionId:<ID>
	at org.apache.hadoop.mapreduce.lib.db.DBInputFormat.setConf(DBInputFormat.java:171) ~[hadoop-mapreduce-client-core-2.8.5.jar:na]

This happens because the Dataproc by default uses Conscrypt SSL provider that has a bug when creating SSL Context using Conscrypt SSL ProviderBy default, your Cloud Data Fusion pipeline is configured to use Cloud Dataproc, which uses the Conscrypt SSL provider. There is a known bug, that pertains to Cloud Dataproc’s use of Conscrypt, that causes the Socket is closed exception.

Solution

To fix the issue while running the pipeline disable using conscrypt while creating , disable the use of Conscrypt in your Cloud Dataproc cluster. This can be done by setting the following runtime argument for the pipelineYou can do this while your pipeline is running.

1. Go to the Control Center.

2. In the Data Pipeline card, click on the wrench tool icon.

...

3. In the Preferences window that opens, paste the following argument into the Key field.

system.profile.properties.dataproc:dataproc.conscrypt.provider.enable false

The following screenshot shows how to set this for a pipeline using the UI

...

4. Click Save & Close.