# Weaviate connector

Source: /docs/connectors/weaviate.md

Use this as an implementation starting point: adapt the worked example to the requested dataset and the source documentation. See [Custom API](/docs/connectors/custom-api) for the shared SDK pattern and document-upload workflow.

## What this connector is for [#what-this-connector-is-for]

Vector database for objects, semantic search, and retrieval workflows.

Use this guide to choose the resources and row meanings for a Vortexify source integration. Connection ownership belongs to a connector; each dataset gets its own sync, configuration, and extraction behavior.

## Data and tables you can pull [#data-and-tables-you-can-pull]

The following table families are a starting point for source selection. They describe possible datasets to implement using the [vendor reference](https://weaviate.io/developers/weaviate/api), not a guarantee that the example implements every table. Actual availability depends on the product edition, deployment, account permissions, and API/schema version.

| Suggested table | One row represents |
| --- | --- |
| `collections` | a collection |
| `objects` | a object |
| `object_properties` | a object property |
| `vectors` | a vector |

### Relationships and extraction considerations [#relationships-and-extraction-considerations]

These are proposed dataset names, not a universal schema guaranteed by the vendor. Preserve source object IDs and relationship references, and split nested child collections into separate tables when appropriate. Choose the concrete endpoint/report and row meaning from the vendor reference before implementing a sync.

## Connecting to Weaviate [#connecting-to-weaviate]

Use the collection schema and authorized instance credentials. Object iteration/export is different from vector similarity queries; preserve object IDs and requested vector fields.

Supply cluster_url and api_key for the selected Cloud instance and collection for the dataset. Use the Python v4 connection below; self-hosted networking needs its own configuration.

### requirements.in [#requirementsin]

Declare these packages in the resource's `requirements.in`. Use the published `vortexify` package for the SDK and the source-specific driver/client shown below. Choose compatible dependency versions for your runtime; the [connector and sync reference](/docs/reference/python/connectors-and-syncs) explains where each file belongs.

```text
vortexify==0.1.1
pandas>=2,<3
weaviate-client
```

Use the source SDK and its supported runtime. Close source clients and keep credentials in platform secret configuration.

## Worked example: collection objects [#worked-example-collection-objects]

This source-read pattern shows how to shape the selected resource into DataFrames. Use it inside the dataset sync, retaining shared connection ownership in the connector. Its scope is the named resource and filters; adapt the projection, batching, and preview to the requested table.

### Connection factory [#connection-factory]

```python
import weaviate
from weaviate.classes.init import Auth


def make_client(config):
    return weaviate.connect_to_weaviate_cloud(
        cluster_url=config["cluster_url"],
        auth_credentials=Auth.api_key(config["api_key"]),
    )
```

This selects Weaviate Cloud. Self-hosted deployments need their documented connection configuration. Close the client in the owning connector.

### Source read [#source-read]

```python
import pandas as pd


def read_batches(client, config):
    collection = client.collections.use(config["collection"])
    batch = []
    for obj in collection.iterator():
        batch.append({"uuid": str(obj.uuid), "properties": obj.properties})
        if len(batch) == 100:
            yield pd.json_normalize(batch)
            batch = []
    if batch:
        yield pd.json_normalize(batch)
```

### What to adapt [#what-to-adapt]

Use a configured Weaviate Python v4 client with the appropriate HTTP/gRPC connection and auth. The iterator handles object traversal; vectors and multi-tenancy require explicit options. Close the owning client; preview uses an independent bounded collection query.

Use the [custom API SDK starting point](/docs/connectors/custom-api#worked-example-cursor-paginated-json-records) to connect the read operation to `Connector` and `Sync`. For a source with its own driver, retain that client in `Connector.client()` and replace the HTTP request with the driver read. An `extract(*, avg_row_size_bytes=None)` implementation yields bounded DataFrames. An independent `preview()` uses a fresh connector and a small read; it must not call extraction or advance real state.

Choose explicit output columns and stable IDs for the requested table. The examples use `json_normalize` where useful to illustrate row shaping; nested arrays still need deliberate child tables and a stable declared output schema. A first-page sample is useful for development, while a full sync must follow all relevant continuation/parent-resource paths. Design incremental watermarks, update/deletion behavior, and rate-limit handling for the actual source before enabling incremental mode.

The purpose is to give a coding agent a concrete pattern to adapt, including for similar sources. These snippets are examples rather than a claim of complete vendor coverage. Check the [source operation reference](https://docs.weaviate.io/weaviate/manage-objects/read-all-objects) and representative responses before running with a customer's account; live vendor execution has not been verified here.

## Sources and further reading [#sources-and-further-reading]

- [Vendor connection and resource reference](https://docs.weaviate.io/weaviate/manage-objects/read-all-objects) — The interface/access prerequisites underlying this connection path; follow the selected resource's documentation for fields, queries, and pagination.
- [Vortexify connectors and syncs](/docs/reference/python/connectors-and-syncs) — Connector ownership, schemas, preview isolation, extraction, and checkpoint contracts.
- [Vortexify CLI workflow](/docs/reference/cli/overview) — Registering resources, committing source, and providing credential configuration.