Most automatic sources assume the file already exists somewhere: on an FTP server, at a URL, in a mailbox. But sometimes the data doesn't exist as a file until someone asks for it — it lives in your database, behind your own business logic, and producing the export takes time.
The custom data extractor covers exactly that case: WeTransform asks your server for data, your server produces the file at its own pace and hands back a link, and the transformation resumes on its own.
🧭 When to use it — and when not to
The file is already reachable at a stable address → a URL or FTP source is simpler
Your system knows when new data is ready and can push it → the REST API is simpler
WeTransform needs to ask, and your system needs a moment to answer → custom data extractor
🔄 How it works — four steps, two of them yours
PING — when a transformation needs fresh data, WeTransform sends a signed request to your server, telling it which customer needs data and where to answer
You produce the file — at your own pace. Nothing is blocked while your export runs
PONG — once ready, your server posts a download link back to the address provided in the PING
Download — WeTransform fetches the file and the transformation continues exactly as if it had been uploaded by hand
📄 Any format your target format accepts works — CSV, Excel, XML, JSON — and the usual normalization runs on it.
⚙️ Setting it up
Go to Integrations → Custom data extractors and create your extractor: a name, a handle, the ping URL on your server (it must be reachable from the public internet), and a signature secret, generated for you, that signs every request so your server can verify it genuinely comes from WeTransform
Create the sources that use it via the API — each source can carry metadata (a shop ID, a warehouse code, a report type) that is sent back to your server with every request, so it knows exactly which export to produce
🧪 Test before going live. A Test button on the extractor's settings page sends a real signed request to your server and shows you exactly what it answered — status code, body, connection errors. It's the fastest way to validate your endpoint.
🏭 Example — orders produced on demand
A retail platform stores orders in its own database; extracting a customer's orders takes a couple of minutes of processing. With a custom data extractor, the daily scheduled transformation pings the platform's server at 6am with the shop ID in the metadata. The server queues the export, generates the file, and posts the link back a few minutes later. WeTransform downloads it, transforms it, and the exporters deliver the result — all before the team arrives.
🔍 Reliability and debugging
If your server is temporarily unreachable, the source waits rather than failing, and you can simply re-run the import once your endpoint is healthy. Every exchange — request, answer, download — is recorded with its full details for 30 days, so misbehaving integrations can be diagnosed precisely.
👨💻 For your developers: the complete implementation guide — payloads, signature verification, source creation via API, error handling — is at api.wetransform.com/en/documentation/custom-data-extractor.
👉 What to do next