Use a managed connector tool when the source is common and the connector is mature. Write custom ingestion when the source is unusual, the security rules are strict, or the volume makes the tool's pricing painful. The main question is whether your time is better spent on something else.
What Fivetran and Airbyte give you
- Prebuilt connectors for hundreds of SaaS apps (Salesforce, Stripe, Google Ads) and databases.
- Incremental sync, with state tracking handled for you.
- Schema change handling: new columns appear in the destination automatically.
- Retries, rate-limit handling, and monitoring.
- Setup in hours, not weeks, and the vendor maintains the connector when the source's API changes. That last point is the one people underestimate: APIs change, and keeping 30 custom connectors working is a continuous job.
Costs and limits
- Price. Fivetran charges by monthly active rows (MAR), and Airbyte Cloud by volume or credits. A table with many changing rows can become expensive as the company grows. Estimate based on the real change rates.
- Less control. You get the schema and sync behaviour the connector provides. Unusual needs (special filters, odd pagination, custom transformations at the source) can be hard.
- Vendor dependence and data leaving your network, which security teams may question.
- Connector quality varies, especially for less popular sources.
Airbyte options
Airbyte is open source, so you can self-host it, which avoids per-row fees and keeps data inside your network. In exchange you run and upgrade it yourself, and also keep an eye on its connectors. It also has a framework for building your own connectors.
When custom code wins
- Internal systems with no connector.
- Very high volumes where per-row pricing is too much (large event streams are usually better handled by Kafka and your own pipelines).
- Strict control over what data is extracted and how, for compliance reasons.
- Special logic, such as complex API pagination or the need to join before loading.
Practical advice
Use tools for the long tail of common sources, and custom code for the core, high-volume or strategic ones. Compare total cost, including engineer time, and keep an exit plan: land data in open formats in a place you control.