> ## Documentation Index
> Fetch the complete documentation index at: https://wb-21fd5541-docs-1917.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Trace data loss in worker processes

Weave uploads trace data in background threads to minimize impact on your application's performance. However, when using a multiprocessing or task queue system or a worker processes like Celery, Weave loses trace data if the worker process exits before background threads finish uploading traces.

To prevent data loss in worker processes, you must ensure that background uploads complete by calling `client.flush()` or `client.finish()` before the worker task completes. This helps when working in short‑lived environments like [AWS Lambda](https://aws.amazon.com/lambda/), [Google Cloud Run](https://cloud.google.com/run), or notebooks that exit right after execution.

These methods have different purposes:

* `weave.flush()`: Simple, silent flushing. Recommended when Weave is integrated into worker processes or CI environments.
* `weave.finish()`: Includes progress feedback with a progress bar or status callbacks. Recommended for interactive scripts or notebooks.

Both methods block until all background uploads complete, ensuring no trace data is lost when the worker process exits.

The following example demonstrates `client.finish()`:

```python theme={null}
from celery import Celery
import weave

app = Celery('tasks')

@app.task
def process_task(input_data):
    weave.init("your-team-name/your-project-name")

    try:
        # Your task logic that creates traces
        result = your_processing_function(input_data)

        # Ensure all traces are uploaded before task completes
        weave.finish()

        return result
    finally:
        pass
```

You can also use the `with` context manager to automatically call `weave.finish()` on exit:

```python theme={null}
with weave.init("your-team-name/your-project-name") as client:
    result = your_processing_function(input_data)

    return result
```

You can also improve the performance of your application using `weave.flush()`. See [Flushing](/support/weave/articles/long-eval-clean-up-times) for more information.

For additional resilience against data loss, consider enabling the [write-ahead log](/weave/guides/tracking/write-ahead-log), which writes trace data to disk before sending it to the server.

***

<Badge stroke shape="pill" color="orange" size="md">[Trace Data](/support/weave/tags/trace-data)</Badge><Badge stroke shape="pill" color="orange" size="md">[Performance](/support/weave/tags/performance)</Badge>