Uploading Data

Upload data from pandas DataFrames to TitanRDM. The SDK supports full and incremental upload patterns, and handles multi-table imports in a single batch.

TitanRDM keeps every version of every record. An import never overwrites a row: an update closes the current version and inserts a new one, and a delete writes a tombstone. Rows written by an import are attributed to the import (trdm_actor_type = 'I', trdm_actor_id = the import id) rather than to a user. A key that was deleted and later reappears in a file becomes a new record with a new trdm_record_id.

---

Note: If you are using Databricks or another Spark-based environment, consider using the SparkSync class instead of manually creating uploads. The SparkSync class provides a more convenient way to upload data by automatically discovering tables and creating uploads for them. Note: SparkSync class uses the ConventionSync class internally. You can use ConventionSync directly with pandas DataFrames if you prefer more control over the upload process.

Concepts

Import Patterns

pattern is required on upload.get_table_upload() — it decides whether records absent from the file are deleted, so there is no safe default. The SDK validates it before calling the API.

The values are also available as an enum:

from titan_rdm_sdk import ImportPattern

ImportPattern.FULL                     # 'full'
ImportPattern.INCREMENTAL_UPSERT_ONLY  # 'incremental_upsert_only'
ImportPattern.INCREMENTAL_WITH_DELETE  # 'incremental_with_delete'
ImportPattern.INCREMENTAL              # 'incremental' (legacy)

Rules that apply to every pattern

• Each primary key may appear only once per file. A duplicate fails the import rather than picking a row arbitrarily; the SDK pre-checks the DataFrame and raises ValidationError before uploading.
• Imports write versions, they never overwrite.
• A key that was deleted and reappears becomes a new record with a new trdm_record_id.

---

Full Upload Workflow

import pandas as pd
from titan_rdm_sdk import TitanRDMClient

client = TitanRDMClient(
    url="https://your-tenant.titanrdm.com",
    client_id="your_client_id",
    client_secret="your_client_secret",
)

# Step 1: Create an import batch
upload = client.get_upload(
    branch_id=174,
    description="Customer data update",
    correlation_code="batch-001",
)
print(f"Import batch created: ID={upload.id}, Status={upload.status}")

# Step 2: Create a table upload
table_upload = upload.get_table_upload(
    table_definition_key=456,
    import_mapping_key=123,
    pattern="full",
)
print(f"Table upload created: ID={table_upload.id}")

# Step 3: Send data
df = pd.DataFrame({
    "customer_id": [1, 2, 3],
    "name": ["Alice", "Bob", "Charlie"],
    "email": ["alice@example.com", "bob@example.com", "charlie@example.com"],
})
table_upload.send(df)
print(f"Data uploaded. Status={table_upload.status}")

# Step 4: Complete the import batch
upload.complete(message="Upload completed successfully")
print(f"Import completed. Final status={upload.status}")

---

Incremental Upload

Use pattern="incremental_upsert_only" to add or update rows without touching records that are absent from the file:

upload = client.get_upload(
    branch_id=174,
    description="Incremental customer update",
    correlation_code="incremental-002",
)

table_upload = upload.get_table_upload(
    table_definition_key=456,
    import_mapping_key=123,
    pattern="incremental_upsert_only",
    correlation_code="incr-customers",
)

# Only send the changed rows
df_changes = df.head(5)
table_upload.send(df_changes)
upload.complete()

Incremental with Delete

Use pattern="incremental_with_delete" when the file states its own deletes. Every row must carry a trdm_tran_code:

Codes are case-insensitive and whitespace-trimmed, so a mislabelled i on an existing record is harmless. trdm_tran_code is read straight off the file and is not part of the import mapping — don't map it to a column definition.

df_changes = pd.DataFrame({
    "customer_id": [1, 2, 3],
    "name": ["Alice", "Bob", "Cleo"],
    "trdm_tran_code": ["u", "d", "i"],   # update, delete, insert
})

table_upload = upload.get_table_upload(
    table_definition_key=456,
    import_mapping_key=123,
    pattern="incremental_with_delete",
)
table_upload.send(df_changes)
upload.complete()

If the column is missing the SDK raises before uploading, with the same wording the server uses:

ValidationError: The incremental_with_delete pattern requires a 'trdm_tran_code' column in the file

---

Export → Import Round Trip

An incremental export emits the same trdm_tran_code vocabulary an incremental_with_delete import consumes, so an export from one table can be piped straight into an import of another (for example, promoting a delta from one branch or tenant to another):

from titan_rdm_sdk import prepare_export_for_import

# 1. Export the delta, tombstones included
download = client.get_download(
    branch_id=174,
    table_definition_key=50,
    pattern="incremental",
    high_water_mark="2026-09-09T19:00:00+10:00",
)
download.wait_until_ready()
changes = download.receive()

# 2. Drop the system columns, keep trdm_tran_code
to_import = prepare_export_for_import(changes)

# 3. Import the delta into the target table
upload = client.get_upload(branch_id=200, description="Delta sync")
table_upload = upload.get_table_upload(
    table_definition_key=456,
    import_mapping_key=123,
    pattern="incremental_with_delete",
)
table_upload.send(to_import)
upload.complete()

for result in upload.get_results():
    print(result.table_definition_name, result.insert_row_count,
          result.update_row_count, result.delete_row_count)

prepare_export_for_import(df, keep_tran_code=False) drops trdm_tran_code too, for a full or incremental_upsert_only import.

---

Multi-Table Upload

Upload multiple tables in a single import batch:

upload = client.get_upload(
    branch_id=branch.id,
    description="Multi-table sync",
    correlation_code="multi-upload-001",
)

# Upload customers
customers_upload = upload.get_table_upload(
    table_definition_key=100,
    import_mapping_key=10,
    pattern="full",
)
customers_upload.send(customers_df)

# Upload orders
orders_upload = upload.get_table_upload(
    table_definition_key=101,
    import_mapping_key=11,
    pattern="full",
)
orders_upload.send(orders_df)

# Complete the batch (processes all tables)
upload.complete(message="Multi-table upload done")

---

Using Default Import Mappings

If you don't know the import mapping key, use the SDK to look it up:

# Get deployed table definition
tables = client.get_deployed_table_definitions(branch_id=5, domain_id=1)
table = tables[0]

# Get the default import mapping
import_mapping = client.get_default_import_mapping(table.id)

# Use it in your upload
table_upload = upload.get_table_upload(
    table_definition_key=table.key,
    import_mapping_key=import_mapping.key,
    pattern="full",
)

---

Upload Object Properties

Upload (Import Batch)

TableUpload

Import Results

upload.get_results() reads GET /api/v1/imports/:id and returns one ImportTableResult per table. The row counts are populated for every pattern, incremental ones included.

---

Parameters Reference

client.get_upload()

upload.get_table_upload()

---

Next Steps

• Downloading Data — Export data from TitanRDM
• Convention Sync — Automate uploads for entire domains
• Error Handling — Handle upload errors gracefully