iFLOWXpress

ETL, Data Quality and Data Testing Platform

Not many tools stack in the market has brought ETL, Data Quality and Data Testing in a single platform. iFlowXpress is a unique of its kind which brings all three solutions in a single platform. It facilitates manage, schedule and monitor complex data migration/flow process

iFLOWXpress consists of three components – MigXpress, DQXpress and DTXpress. MigXpress is a platform used for moving data from source to target systems. It supports most of the major RDBMS, cloud databases e.g. AWS Redshift, RDS, Azure databases, Snowflake, Nosql e.g. Mongodb, other variety of data e.g. XML, JSON, Avro, Parquet, ORC tc. DQXpress is a compete platform managing systematically data quality issues. DTXpress is a platform used for testing data movements between sources and targets. iFLOWXpress put all these together and used for creating complex data flow process. Each of the component can be used standalone or together with iFLOWXpress.

Solution Need

  • Manages entire life cycle of data flow process – data extraction, transformation and loading, data quality handling and test of data movement.

  • A diagram to visualize whether all the processes are created for a data flow job e.g. data quality, data testing and data flow.

  • No proprietary software application dependency

  • Must support modern data platform e.g. cloud, nosql

  • CICD support

Features

  • Built of open source technologies

  • Support cloud and nosqls databases e.g. snowflake, Azure SQL, Redshift, Mongodb etc.

  • In-built Scheduler

  • Callable from external programs though REST API

  • CICD support to integrate with leading CICD tools e.g. Jenkins, Git etc.

  • Dependency graph to see whether all processes followed e.g. test , data quality handling

  • Email notification of jobs•

Benefits

  • 20-30% reduction in data flow management time, effort and cost

  • Easy to integrate with external application

  • Easy to manage complex ETL process

  • Easy to do impact analysis

  • Easy to monitor code coverages of ETL processes

  • Unique of its kind in this category of tools by providing a single platform for all three components – data migration, data quality and data testing

Technology Stack

Python, Javascript, Luigi

Request for detailed information on this asset:

Studios