Quickstart Code Example

The Prep API works bucket to bucket: you tell it where a document already lives in your cloud storage and where to write the output, and it reads and writes directly. Nothing is uploaded through the API, so this example needs a little more setup than a plain file post — you supply storage credentials alongside your own.

Create a client

The API accepts either of two credentials, and get_auth_session returns the right kind of session for whichever you pass. Two things hold for both: the API URL is the first positional argument — it is what the issued token is scoped to — and credentials is a dictionary.

Each tab below is complete on its own: the storage credentials are the same for both, your own are not.

$
pip install emtellisplit-sdk-python
$
$
export EMTELLISPLIT_API_KEY='...'
$
$
# read your input bucket and write your output bucket
$
export AWS_ACCESS_KEY_ID='...'
$
export AWS_SECRET_ACCESS_KEY='...'

1
import os
2
3
from emtellisplit_sdk_python import auth, engine
4
5
SERVER = "https://api.us.emtelligent.com:50001"
6
7
session = auth.get_auth_session(
8
    SERVER, credentials={"api_key": os.environ["EMTELLISPLIT_API_KEY"]})
9
client = engine.Emtellisplit(SERVER, session)

Describe the job

InputDocument names the object to process, the buckets to read and write, and the credentials the service should use for them. operation='split-ocr-ep' runs the full set of functions — split, classify, OCR and metadata extraction.

The keys in cloud_credential_set are passed straight through to the cloud client, so they are that provider’s own parameter names — aws_ prefix included. For S3 they are aws_access_key_id and aws_secret_access_key, plus aws_session_token if you are using temporary credentials. For Azure, either connection_string, or account_name with one of account_primary_access_key or sas_token. Anything else comes back as Invalid cred format for s3.

1
from emtellisplit_sdk_python.data import InputDocument
2
3
doc = InputDocument(
4
    cloud_provider="s3",
5
    cloud_credential_set={
6
        "aws_access_key_id": os.environ["AWS_ACCESS_KEY_ID"],
7
        "aws_secret_access_key": os.environ["AWS_SECRET_ACCESS_KEY"],
8
    },
9
    input_object_keys={"document": "subdir/my_document.pdf"},
10
    input_bucket="my-input-bucket",
11
    output_bucket="my-output-bucket",
12
    output_bucket_prefix="my-jobs",
13
    operation="split-ocr-ep",
14
)

Submit and wait

process returns a ResultFuture, and it has to be awaited — it polls the job on an asyncio task, so it only works inside async code. Both spellings below do the same thing.

1
result = await client.process(doc)        # the future is awaitable itself
2
result = await client.process(doc).result()   # or await its result()

Awaiting gives you a CloudResult if the job succeeded and a plain Result if it failed or was cancelled, so isinstance is the success test.

1
from emtellisplit_sdk_python.data import CloudResult
2
3
result = await client.process(doc)
4
if not isinstance(result, CloudResult):
5
    raise SystemExit(f"processing did not succeed: {result.job_status.status}")

Prefer to stay synchronous? submit returns a DeferredResult instead, which is deliberately not awaitable — you poll it yourself. A JobStatus is truthy once the job reaches a terminal state.

1
import time
2
3
deferred = client.submit(doc)
4
while not deferred.done():
5
    time.sleep(10)
6
result = deferred.result()

Fetch the output

The split documents are in your output bucket, and get_result reads them back. These credentials are not sent to the service — this call is your own client listing and reading your own bucket — so they only need list and read on it.

1
output = result.get_result(
2
    "s3",
3
    "my-output-bucket",
4
    {
5
        "aws_access_key_id": os.environ["AWS_ACCESS_KEY_ID"],
6
        "aws_secret_access_key": os.environ["AWS_SECRET_ACCESS_KEY"],
7
    },
8
)
9
if not output.docs:
10
    raise SystemExit(f"job {result.job_id} succeeded but no output was found")
11
for doc_result in output.docs:
12
    print(doc_result.emtellipro_category, doc_result.emtellipro_subcategory,
13
          doc_result.page_span)

One bundle in gives many documents out — that is the split — each with the category and subcategory the classifier assigned and the pages it came from.

The whole thing

Only the session differs between the two, so pick the tab you set up above.

1
import asyncio
2
import os
3
4
from emtellisplit_sdk_python import auth, engine
5
from emtellisplit_sdk_python.data import CloudResult, InputDocument
6
7
SERVER = "https://api.us.emtelligent.com:50001"
8
INPUT_BUCKET = "my-input-bucket"
9
OUTPUT_BUCKET = "my-output-bucket"
10
OUTPUT_PREFIX = "my-jobs"
11
MY_STORAGE = {
12
    "aws_access_key_id": os.environ["AWS_ACCESS_KEY_ID"],
13
    "aws_secret_access_key": os.environ["AWS_SECRET_ACCESS_KEY"],
14
}
15
16
17
async def main():
18
    session = auth.get_auth_session(
19
        SERVER, credentials={"api_key": os.environ["EMTELLISPLIT_API_KEY"]})
20
    client = engine.Emtellisplit(SERVER, session)
21
22
    doc = InputDocument(
23
        cloud_provider="s3",
24
        cloud_credential_set=MY_STORAGE,
25
        input_object_keys={"document": "subdir/my_document.pdf"},
26
        input_bucket=INPUT_BUCKET,
27
        output_bucket=OUTPUT_BUCKET,
28
        output_bucket_prefix=OUTPUT_PREFIX,
29
        operation="split-ocr-ep",
30
    )
31
32
    # polls every 10 seconds until the job reaches a terminal state
33
    result = await client.process(doc)
34
    if not isinstance(result, CloudResult):
35
        raise SystemExit(f"processing did not succeed: {result.job_status.status}")
36
37
    output = result.get_result("s3", OUTPUT_BUCKET, MY_STORAGE)
38
    if not output.docs:
39
        raise SystemExit(f"job {result.job_id} succeeded but no output was found "
40
                         f"in {OUTPUT_BUCKET}")
41
    for doc_result in output.docs:
42
        print(doc_result.emtellipro_category, doc_result.emtellipro_subcategory,
43
              doc_result.page_span)
44
45
46
asyncio.run(main())

If get_result finds nothing, the output is still there — list your bucket under <output_bucket_prefix>/<job_id>/ and you will see it. That gap between where the service writes and where the SDK looks is the version difference described above, not a job that produced nothing.

Next