StreamingQuery

Defined in: streaming/streaming-query.ts:48

Handle for a streaming query running on the Spark Connect server. Returned by DataStreamWriter.start and DataStreamWriter.toTable.

Properties (id, runId, name) are cached from the start response. Methods (status, lastProgress, stop, and so on) each issue a StreamingQueryCommand RPC.

isActive is asynchronous on this client even though PySpark and Scala expose it as a synchronous property. Every other inspection method on this class crosses the wire; a synchronous getter would lie about its source of truth.

See

Spark source: StreamingQuery.scala

Properties

id

readonly id: string;

Defined in: streaming/streaming-query.ts:50

---

name

readonly name: string | undefined;

Defined in: streaming/streaming-query.ts:52

---

runId

readonly runId: string;

Defined in: streaming/streaming-query.ts:51

Methods

awaitTermination()

awaitTermination(timeoutMs?): Promise<boolean | undefined>;

Defined in: streaming/streaming-query.ts:155

Block until the query terminates.

• With timeoutMs: returns true if the query terminated within the timeout, false otherwise.
• Without a timeout: blocks forever and returns undefined once the query terminates.

Parameters

Parameter	Type
timeoutMs?	number

Returns

Promise<boolean | undefined>

Remarks

The timeout is in milliseconds, matching the Scala client and the Spark Connect timeout_ms wire field. PySpark’s awaitTermination takes seconds, so porting awaitTermination(10) from PySpark gives 10ms here, not 10s. Pass 10_000 for the PySpark 10 equivalent.

Throws

SparkConnectError if the query terminated with an exception.

---

exception()

exception(): Promise<
  | StreamingQueryException
| null>;

Defined in: streaming/streaming-query.ts:104

The exception that terminated the query, or null if the query is still running or stopped cleanly.

Returns

Promise< | StreamingQueryException | null>

---

explain()

explain(extended?): Promise<string>;

Defined in: streaming/streaming-query.ts:120

Logical and physical plan as a string. extended=true adds the analyzed and optimized plans.

Parameters

Parameter	Type
extended?	boolean

Returns

Promise<string>

---

isActive()

isActive(): Promise<boolean>;

Defined in: streaming/streaming-query.ts:63

Whether the query is still running.

Returns

Promise<boolean>

---

lastProgress()

lastProgress(): Promise<
  | StreamingQueryProgress
| null>;

Defined in: streaming/streaming-query.ts:84

Most recent progress report, or null if no batch has been processed yet.

Returns

Promise< | StreamingQueryProgress | null>

---

processAllAvailable()

processAllAvailable(): Promise<void>;

Defined in: streaming/streaming-query.ts:135

Block until all currently-available source data has been processed and committed to the sink. Useful in tests; do not use in production code with unbounded sources.

Returns

Promise<void>

---

recentProgress()

recentProgress(): Promise<StreamingQueryProgress[]>;

Defined in: streaming/streaming-query.ts:95

Recent progress reports, oldest first. The server keeps a bounded number (default 100); older entries fall off.

Returns

Promise<StreamingQueryProgress[]>

---

status()

status(): Promise<StreamingQueryStatus>;

Defined in: streaming/streaming-query.ts:68

Current runtime status (message, data availability, trigger activity).

Returns

Promise<StreamingQueryStatus>

---

stop()

stop(): Promise<void>;

Defined in: streaming/streaming-query.ts:126

Stop the query. Returns after the server confirms termination.

Returns

Promise<void>