Bring in data

An app might start with a spreadsheet, orders from an ERP, or records from an internal API. Bring that data into Vortexify when you want to query it, prepare it in the background, or give an assistant access to it.

Use an upload for a file you already have. Use a connector and sync when the data lives in another system and needs to be refreshed. If users need to create and update app-owned records, such as approvals or work orders, use a managed database.

Upload an existing fileCopy Anchor Link

Upload a CSV, TSV, or XLSX file and wait for publication:

vortexify upload table ./customers.csv --name customers --wait

Then consume that uploaded table directly in transform/API inputs or agent context:

from vortexify import Table, vxfy_transform

@vxfy_transform(inputs={"customers": Table("_upload.customers")})
def prepare(customers):
    return customers

_upload is reserved and cannot name a connector. An explicit upload name is used exactly or rejected if it already exists; no mock connector or sync is needed. Wait for the uploaded table to exist before committing its consumers.

Connect once, choose what to syncCopy Anchor Link

A connector owns source access: its client, credentials, connection check, and dataset discovery. A sync selects and extracts one dataset through that connector. Several syncs can share a connection without duplicating its credentials.

Your coding agent implements the connector and sync locally using a source guide. If a template does not fit, use Custom API as the starting point for a custom implementation.

Connection and dataset settings each have a config.json. These files describe the fields the platform should collect; actual configuration values are supplied separately. Mark credential fields with x-vortexify-secret: true and supply their values through platform configuration. Keep real credentials out of source and schema defaults.

Save the connector before configuring it. Then save the sync and supply its dataset settings through the sync configuration service. The CLI configuration reference covers configuring a connection from your terminal. The current frontend resource workspace does not expose a dedicated sync configuration form; a required configuration must be supplied before starting that sync.

Discovery and ingestion do different workCopy Anchor Link

A connection check confirms source access. Connector discovery lists the datasets and column metadata your implementation exposes, optionally with bounded samples. It does not create sync implementations or ingest all discovered tables.

Use the connector discovery service to inspect available datasets and choose the required settings. The current frontend does not yet provide dedicated check/discovery controls; these operations are separate from opening an ingested table. Your coding agent then implements that sync's extraction and pagination. The optional Sync.preview() method reads a small sample with disposable state. A CLI or Playground preview run is different: it executes extraction and publishes output. Incremental run checkpoints are scoped to the saved producer version, so a preview run is not a dry-run substitute for the sampling method.

The sync can use full extraction or incremental extraction. In incremental mode, your extraction code reads and advances a cursor/checkpoint according to the source contract. Use a cron schedule when the dataset should refresh automatically after deployment; a manual preview run does not activate production scheduling.

After the code and configuration are saved, your coding agent starts a run:

vortexify run sync shop/orders --wait

The table becomes available to Python through Table("shop.orders"). Use the actual registered connector and sync names. For a CSV/TSV/XLSX uploaded as customers, use Table("_upload.customers") in transform/API inputs or agent context; no connector is needed. _upload is reserved and cannot name a connector.

Wait for the result your next step needsCopy Anchor Link

There are separate stages after extraction:

The sync waits for output schema discovery before completing publication. Primary key and relationship analysis runs as a separate knowledge graph job afterward. A successful sync alone does not mean those findings are complete.

For a query using known columns, inspect the published schema and rows. For a plan that relies on inferred joins, wait for key analysis on the relevant tables and inspect their relationship evaluations first. A completed evaluation can find a relationship, find none, or remain inconclusive; choose joins from the actual findings and column semantics. Inferred keys are findings about analytical data, not automatically enforced database constraints.

Your coding agent can inspect the selected output and its analysis status through the CLI:

vortexify tables
vortexify table shop.orders --json

The table details include the knowledge graph status, phase, and job ID when available. Use vortexify job JOB_ID --wait for that analysis job, then reread table status. Detailed key and relationship payloads are requested explicitly by the platform Relationships view; the current CLI table request does not ask for those detailed analysis payloads. Inspect both sides of a planned join: a table's analysis can finish before a later table becomes eligible for comparison. Findings are tied to output snapshots; later runs may reuse previous findings when the checks find no meaningful change. On a later publication, compare its status rather than treating an older finding as fresh evidence.

In the platform, open the connector's Tables view, choose a table, then use Data, Schema, and Relationships. The Relationships view shows discovery progress and key findings, lets you inspect the discovery job, and can open the exact compared snapshot of another table. Open a sync's Runs & logs to inspect its ingestion history.

Platform screenshot placeholder: A published table with its data-ready status and separate key/relationship analysis progress. Show a completed relationship evaluation alongside the columns it connects.

Start with a file insteadCopy Anchor Link

For CSV, TSV, or XLSX data, your coding agent can upload a table directly:

vortexify upload table ./orders.csv --name orders --wait

No connector is required. Inspect the returned table ID, schema, and rows before using it. As with synced data, inspect the separate knowledge graph status before relying on inferred keys or relationships. Uploaded PDF, TXT, and Markdown documentation is an attachment, not a tabular dataset; see CLI uploads and attachments.

For table inspection, publications, and inferred joins, see the Data section.

Next: Query and prepare data. See connector and sync contracts for extraction, configuration, and checkpoint details.