Staff Engineer

May 2022 – Present

Data Integration Developer

July 2021 – May 2022

• Design, build, secure, implement and maintain Data ETL platform with DBT, Airflow, Kafka and Snowflake
• Build automation to provision, maintain, and monitor all aspects of Kafka, Airflow and AWS SageMaker environment via infrastructure as code.
• Successful migration of on-premise data ETL job orchestration system (Airflow) to AWS managed Airflow.
• Administered Data Lake (Snowflake) using Terraform.
• Deployed AWS Python Lambdas to create and deliver usage reports.
• Implemented CI/CD for Machine learning Pipelines using Terraform and AWS SageMaker.
• Used Terraform, SaltStack, GitHub Actions etc. to implement CI/CD pipelines for Kafka and Airflow.
• Conceived and wrote detailed software implementations while ensuring that their code adhere to the security, logging, error handling, and performance standards and non-functional requirements.
• Skills:Application Development(Python, Java, KafkaStreams, KafkaConnect, JUnit),AWS (IAM, S3, Lambda, Managed Airflow, SageMaker, SQS, SNS, CloudWatch, CloudFormation) Other(Terraform, Docker, SaltStack, Airflow, DBT, Snowflake, GitHub Actions, Jira, Confluence, Asana).

Software Developer

January 2020 – July 2021

• Design, build, secure, implement and maintain Multi-data-center Kafka infrastructure.
• Build automation to provision, maintain, and monitor all aspects of the Kafka environment via infrastructure as code.
• To meet stringent Recovery Point Objective (RPO), converted traditional Kafka clusters to multi region clusters.
• Helped build real-time streaming data pipelines to reliably get data between systems and applications.
• Identify bottlenecks and tune to optimize Kafka performance and throughput.
• Built real-time streaming applications Kafka streams that transform & react to streams of data.
• Conceived and wrote detailed software implementations while ensuring that their code adhere to the security, logging, error handling, and performance standards and non-functional requirements.
• Automating on-boarding of clients to Kafka by developing CI/CD pipelines for Kafka Streams, Topics, ACLs, Schemas, Kafka Connectors etc.
• Skills:Application Development(Java, KafkaStreams, KafkaConnect, JUnit), Other(JSON, AVRO, WSDL, HTML, CSS3, JavaScript, Angular, Git, Ansible, Bamboo, Jira, Confluence).

Software Developer

March 2018 – June 2019

• Design & Develop Spring Batch jobs in Java using dependency management and build tool Maven.
• Develop SQL stored procedures to perform data intensive logic and Shell Scripts for file manipulation, Job dependencies etc.
• Develop SQL stored procedures to perform data intensive logic.
• Excellent time-management skills through deadline driven projects and tasks.
• Skills:Application Development(Java, Spring Batch, JDBC, Hibernate, JUnit), Scripting (Bash, Python), Other(Jenkins, Bitbucket, SonarQube, Confluence, Jira, HTML/CSS).