Compute and schedules
Compute controls the resources allocated to a run. Schedules decide when a deployed sync or transform starts. Keep these declarations alongside the Python behavior so the saved version describes how that code should execute.
Choose an allocation
Start with a preset and adjust after inspecting real run behavior. Memory and temporary storage use MiB; CPU fields use cores. Explicit field values override the preset. Omitted values are filled from the platform's small allocation at execution time.
For an API:
from vortexify import ApiCompute
compute = ApiCompute(size="small")API compute
| Size | Memory | Temporary storage |
|---|---|---|
| xsmall | 256 | 512 |
| small | 512 | 512 |
| medium | 1024 | 1024 |
| large | 2048 | 2048 |
| xlarge | 4096 | 4096 |
ApiCompute accepts size, memory, and storage. The backend accepts API memory from 128 to 10,240 MiB and storage from 512 to 10,240 MiB, subject to plan limits. APIs have request execution limits and no schedule declaration; move long background computation into a transform.
Transform and sync compute
TransformCompute accepts size, memory, cores, and disk:
| Size | Memory | Cores | Temporary disk |
|---|---|---|---|
| xsmall | 1024 | 1 | 2048 |
| small | 2048 | 1 | 4096 |
| medium | 4096 | 2 | 8192 |
| large | 8192 | 4 | 16384 |
| xlarge | 16384 | 8 | 32768 |
Use it in a transform decorator or as a sync class attribute. Temporary disk is useful for extraction and intermediate files; published tables are stored separately.
For PySpark, use TransformSparkCompute. Driver and executor resources are separate, and executor allocations are per executor:
| Size | Driver memory / cores / disk | Executor memory / cores / disk | Executors |
|---|---|---|---|
| xsmall | 1024 / 1 / 4096 | 2048 / 1 / 4096 | 1 |
| small | 2048 / 1 / 8192 | 4096 / 2 / 8192 | 2 |
| medium | 4096 / 2 / 16384 | 8192 / 2 / 16384 | 2 |
| large | 8192 / 4 / 32768 | 16384 / 4 / 32768 | 4 |
| xlarge | 16384 / 4 / 65536 | 32768 / 8 / 65536 | 8 |
Explicit fields are driver_memory, driver_cores, driver_disk, executor_memory, executor_cores, executor_disk, and executor_instances. Plan limits and Spark availability are checked before work is admitted. A supported preset is not necessarily available on every plan.
Refresh on a clock
Use a cron schedule for regular work, such as an hourly source refresh:
from vortexify import CronSchedule
schedule = CronSchedule(expression="0 * * * *")The server validates a five-field expression: minute, hour, day of month, month, day of week. The public declaration has no timezone argument. Confirm the deployment's scheduling timezone when the exact wall-clock time matters.
Syncs support one CronSchedule or None. Assign it as a class attribute beside mode and compute. Transforms accept a cron schedule, an upstream trigger, or a list of schedules in their decorator.
Refresh after upstream data changes
Use an upstream trigger when a transform should react to new table publications:
from vortexify import Table, TriggerSchedule
schedule = TriggerSchedule(
triggers=[Table("shop.orders"), Table("shop.customers")],
mode="all",
)any, the default, reacts when any configured dependency updates. all waits for updates from every configured dependency. Trigger declarations support synced Table, uploaded Table("_upload.table_name"), and TransformOutput references; managed database row writes are not file-backed publication triggers.
Use all when a calculation needs fresh inputs from both sources; use any when each update should refresh the result using the other available inputs. Neither mode implements transactional coordination between independent source systems.
Save, deploy, and observe
Commit source changes so parsed compute and schedule declarations become part of a saved version. A sync-based trigger references the committed sync producer, so the consumer can commit before the sync materializes its first table. Uploaded-table triggers require an existing uploaded table. Triggers are observed after successful output publication, not when source is committed or a pod merely exits. Production scheduling uses deployed resources. A manual preview run executes explicitly and does not activate production schedules or fire downstream production triggers.
In the platform, the resource Overview shows upstream/downstream connections. API and transform Playground views let you prepare a runtime, run saved tests, submit runtime inputs, and inspect history. The Processing result view distinguishes execution from publication and links related preparation, schema discovery, and downstream preview jobs.
Keep schedule and compute edits in source for the CLI workflow. The current frontend's Playground Configure panel prepares/tests the runtime; it is not a complete schedule or compute editor.
See transforms, connectors and syncs, and CLI execution.