How to Use the Data Pipeline Tool

This guide walks you through building, configuring, running, and exporting pipelines using the Data Pipeline canvas. You will learn how to start with an input or AI Agent node, chain processing nodes, validate and execute your workflow, and finally download your pipeline as well.

Getting Started

When you open the Data Pipeline canvas, you’ll see a blank grid and a toolbar on the right listing available node types: Converter, Encoder/Decoder, Hasher, Formatter, Cryptography, Input, and AI Agent. Before you can execute, you must place either an Input node or the AI Agent node at the very beginning of your pipeline.

Adding an Input or AI Agent Node

Drag the Input node from the right panel onto the canvas. In its settings, choose “Text” to paste or type your data directly, or “Fetch” to supply a URL and optional JSON headers for remote data retrieval.
Alternatively, drag the AI Agent node to start with a prompt. Enter your prompt in the text area, then select an output type (Text, Hex, Binary, Base64, CSV, or JSON). The AI Agent will supply dynamic content to feed downstream nodes. Additionally, the AI agent can also be used later on as a data processing node where you can attach a input to the left handle so it can use that data along with the prompt.

Chaining Processing Nodes

After your initial node is in place, drag any combination of Converter, Encoder/Decoder, Hasher, Formatter, or Cryptography nodes onto the canvas. Connect the small circle on the right side of one node to the left circle of the next to pass data along.
You can branch outputs to multiple downstream nodes or loop results back into earlier stages for iterative processing. The pipeline will always execute from left to right following your connections.

Configuring Each Node

Click a node to open its settings panel:

  • - Converter: select input type (Text, Hex, Binary, etc.) and output type.
  • - Encoder/Decoder: choose Base64 or URL encoding, then click Encode or Decode.
  • - Hasher: pick MD5, SHA-1, SHA-256, or SHA-512.
  • - Formatter: choose JSON formatting or CSS/JS minification.
  • - Cryptography: select RSA, ECDSA, AES, or DES, enter your keys/parameters, then choose Encrypt/Decrypt or Sign/Verify.

Running & Validating the Pipeline

When your graph is complete, click **Validate/Run** in the top toolbar. The tool will highlight any missing connections or misconfigured nodes. Once validation passes, the pipeline executes automatically, processing data through each connected node in order.

Viewing & Downloading Results

After execution, toggle **Show Output** to inspect the final node’s result in the sidebar. To save your work or output, click the download icon (⬇️) next to **Validate/Run**. You can export the pipeline definition as JSON or download the processed data in its current format.

Tips & Best Practices

  • - Always begin with an Input or AI Agent node—pipelines won’t run without one.
  • - Use named keys and clear labels when configuring cryptography nodes for easier troubleshooting.
  • - For large data sets, consider streaming or chunked processors to avoid UI freezes.
  • - Remove unused nodes by clicking the red “×” on their panels.
  • - Zoom in/out using the controls at the bottom-left to manage complex workflows.