Data Flow

00Overview

Run Oracle Cloud Data Flow — Oracle's managed Apache Spark service — straight from a flow. Define and update Spark applications, launch runs and follow them through to completion, pull a run's log files back as text, and wire private endpoints so Spark can reach resources on a private network. For interactive work, submit statements to a running Spark session and list, check or cancel them as they go.

Every field below is exactly what you see in the Flomation editor. Fields marked ● live picker let you choose from a list pulled live from your account — no IDs to look up.

01Connecting Data Flow

1. Choose how to connect with the Authentication dropdown. Connect Oracle Cloud is the simplest path — Flomation manages the Oracle Cloud connection for you, so there is no private key to enter. API signing key (advanced) lets you supply your own key instead.
2. For Connect Oracle Cloud, add an Oracle Cloud connection in your Flomation environment and follow its prompts to authorise it, then confirm it works before using it in a node.
3. Back in the node, leave Authentication on Connect Oracle Cloud and pick your connection in the Oracle Cloud connection field.
4. To use your own key instead, sign in to the Oracle Cloud Console, open the Profile menu → My profile → API keys → Add API key, let Oracle generate the pair and download the private key. From the Configuration file preview Oracle then shows, copy tenancy → Tenancy OCID, user → User OCID, fingerprint → Key Fingerprint and region → Region (a plain region key such as uk-london-1).
5. Set the Compartment OCID on every action to the compartment your Data Flow applications and runs live in — open Identity & Security → Compartments in the console to copy its OCID.
6. For the advanced key method, store the downloaded private key as a Flomation environment secret (e.g. dataflow_secret) and select it in the node's Private Key (PEM) field; if the key is passphrase-protected, add the passphrase as a secret too and pick it in Private Key Passphrase.

02Application

OCI Data Flow: Change Application Compartment

oracle/dataflow/application_change_compartment · Action

Move a Data Flow application into a different compartment — the application keeps its OCID, only its compartment placement changes.

Returns: tool_result, id, destination_compartment_id, success, error

OCI Data Flow: Create Application

oracle/dataflow/application_create · Action

Create a Data Flow application — a reusable Apache Spark job template fixing the driver/executor shapes, executor count, Spark version, language and the object-storage URI of the Spark program.

Returns: tool_result, application, id, lifecycle_state, success, error

OCI Data Flow: Delete Application

oracle/dataflow/application_delete · Action

Delete a Data Flow application by its OCID — its Spark definition is removed and can no longer be run.

Returns: tool_result, id, success, error

OCI Data Flow: Get Application

oracle/dataflow/application_get · Action

Fetch a single Data Flow application by its OCID — its language, Spark version, executor sizing and lifecycle state.

Returns: tool_result, application, id, lifecycle_state, success, error

OCI Data Flow: List Applications

oracle/dataflow/application_list · Action

List the Data Flow applications in a compartment. Optionally filter by exact display name and cap the page size. Walks pagination up to a safe limit.

Returns: tool_result, applications, count, truncated, success, error

OCI Data Flow: Update Application

oracle/dataflow/application_update · Action

Partially update a Data Flow application — change only the display name or number of executors you supply; blank fields are left unchanged.

Returns: tool_result, application, id, success, error

03Private

OCI Data Flow: Create Private Endpoint

oracle/dataflow/private_endpoint_create · Action

Create a Data Flow private endpoint into a VCN subnet so Spark can reach private resources by their DNS zone names. Returns a work-request id — poll Get Private Endpoint until ACTIVE.

Returns: tool_result, work_request_id, success, error

OCI Data Flow: Delete Private Endpoint

oracle/dataflow/private_endpoint_delete · Action

Delete a Data Flow private endpoint by its OCID — returns a work-request ID that tracks the teardown.

Returns: tool_result, id, work_request_id, success, error

OCI Data Flow: Get Private Endpoint

oracle/dataflow/private_endpoint_get · Action

Fetch a single Data Flow private endpoint by its OCID — its subnet, DNS zones, host count and lifecycle state.

Returns: tool_result, private_endpoint, id, lifecycle_state, success, error

OCI Data Flow: List Private Endpoints

oracle/dataflow/private_endpoint_list · Action

List the Data Flow private endpoints in a compartment. Optionally filter by exact display name or lifecycle state, and cap the page size. Walks pagination up to a safe cap.

Returns: tool_result, private_endpoints, count, truncated, success, error

04Run

OCI Data Flow: Create Run

oracle/dataflow/run_create · Action

Launch a run of a Data Flow application — one execution of its Spark job. Optionally override the display name. Returns the run, which starts in ACCEPTED; poll Get Run until it reaches SUCCEEDED or FAILED.

Returns: tool_result, run, id, lifecycle_state, success, error

OCI Data Flow: Delete Run

oracle/dataflow/run_delete · Action

Cancel and delete a Data Flow run by its OCID — the Spark job stops and its resources are released.

Returns: tool_result, id, success, error

OCI Data Flow: Get Run

oracle/dataflow/run_get · Action

Fetch a single Data Flow run by its OCID — its application, language, lifecycle state and duration.

Returns: tool_result, run, id, lifecycle_state, success, error

OCI Data Flow: List Runs

oracle/dataflow/run_list · Action

List the Spark runs in a compartment. Optionally filter by application, exact display name or lifecycle state, and cap the page size. Walks pagination up to a safe cap.

Returns: tool_result, runs, count, truncated, success, error

OCI Data Flow: Get Run Log

oracle/dataflow/run_log_get · Action

Fetch the content of a single named log file from a Data Flow run, returned as text.

Returns: tool_result, content, content_type, content_length, success, error

OCI Data Flow: Update Run

oracle/dataflow/run_update · Action

Partially update a Data Flow run — change only the maximum duration, SESSION idle timeout or free-form tags you supply; blank fields are left unchanged (a run's display name and other properties are immutable once launched).

Returns: tool_result, run, id, lifecycle_state, success, error

05Statement

OCI Data Flow: Submit Statement

oracle/dataflow/statement_create · Action

Submit an interactive statement (a block of Spark code) to a running Data Flow SESSION run and return the created statement.

Returns: tool_result, statement, id, success, error

OCI Data Flow: Cancel Statement

oracle/dataflow/statement_delete · Action

Cancel an interactive statement running against a Data Flow session run.

Returns: tool_result, id, success, error

OCI Data Flow: Get Statement

oracle/dataflow/statement_get · Action

Fetch a single interactive statement on a Data Flow session run — its lifecycle state and progress.

Returns: tool_result, statement, id, lifecycle_state, success, error

OCI Data Flow: List Statements

oracle/dataflow/statement_list · Action

List the interactive statements submitted against a Data Flow session run. Optionally filter by lifecycle state. Walks pagination up to a safe cap.

Returns: tool_result, statements, count, truncated, success, error

06Notes & Limitations

Behaviours and constraints worth knowing before you build with these nodes.

• Launched runs and newly created private endpoints come up asynchronously — a run begins in ACCEPTED and a private endpoint in CREATING, so poll Get Run until the run finishes or Get Private Endpoint until it is ACTIVE before a later step depends on it.
• Interactive statements can only be submitted to a run of type SESSION; a run launched to execute an application's Spark job runs once and does not accept statements.
• List actions stop after an internal page cap and flag the result as truncated, so a very large compartment can return a partial set — narrow the results with the display-name or lifecycle-state filter rather than expecting a single call to return everything.
• Update actions change only the fields you fill and leave the rest untouched, and a run is largely fixed once launched — only its maximum duration, session idle timeout and free-form tags can still be changed afterwards.
• Each connection reaches only the single region it was set up for, so working with a compartment in another region requires a separate connection pointed at that region.