SparkSessionBuilder

Defined in: spark-session.ts:515

Fluent builder for SparkSession. Returned by SparkSession.builder() in @spark-connect-js/core; runtime adapters (e.g. @spark-connect-js/node) usually subclass it to add their own transport defaults.

Example

const spark = SparkSession.builder()
  .remote("sc://localhost:15002")
  .transport(myTransport)
  .arrowDecoder(myDecoder)
  .getOrCreate();

Constructors

Constructor

new SparkSessionBuilder(): SparkSessionBuilder;

Returns

SparkSessionBuilder

Methods

arrowDecoder()

arrowDecoder(decoder): this;

Defined in: spark-session.ts:531

Inject an Arrow IPC decoder. Required for collect() and similar actions.

Parameters

Parameter	Type
decoder	ArrowDecoderFn

Returns

this

---

arrowEncoder()

arrowEncoder(encoder): this;

Defined in: spark-session.ts:537

Inject an Arrow IPC encoder. Required for createDataFrame([...]) with plain rows.

Parameters

Parameter	Type
encoder	ArrowEncoderFn

Returns

this

---

getOrCreate()

getOrCreate(): SparkSession;

Defined in: spark-session.ts:558

Construct the session. In Spark Connect, “getOrCreate” is a server-side concept: the server may return an existing session if the session ID matches. On the client we just instantiate the handle.

Returns

SparkSession

---

remote()

remote(connectionString): this;

Defined in: spark-session.ts:519

Set the Spark Connect endpoint URL (sc://host:port). Required.

Parameters

Parameter	Type
connectionString	string

Returns

this

---

sessionId()

sessionId(id): this;

Defined in: spark-session.ts:547

Reuse an existing server-side session by ID. Must be a canonical UUID string in the form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx. If unset, a fresh UUID is generated on the client.

Parameters

Parameter	Type
id	string

Returns

this

---

transport()

transport(t): this;

Defined in: spark-session.ts:525

Inject a Transport implementation. Required.

Parameters

Parameter	Type
t	Transport

Returns

this