What is a pipeline?
A Pipeline moves your data from a Source to a Destination.
The pipeline has the following components:
- Source: A Source is a database, an API endpoint or a File Storage which has your data that you want to analyze. Hevo integrates with a variety of Sources. Complete documentation on using various Sources can be found here.
- Transformations: Transformations are used when you want to clean, enrich or transform your data before loading it to your destination. Check out Introduction to Transformations to know more.
- Schema Mapper: Schema Mapper lets you map you source schemas to tables in your destination warehouse. Check out Introduction to Schema Mapper to know more.
- Replay Queue: Replay Queue where your data is parked when we encounter an error or failure. Data parked in the Replay Queue can be replayed whenever the errors/failures have been resolved. There can be many reasons for the data to be parked in the Replay Queue. Some of them are listed in Errors in Replay Queue.
- Destination: Destination is a data warehouse or data lake where that data collected from Sources is stored. Various destination supported by Hevo are listed here.