Data Pipeline Engineer

TMP Worldwide - USA - The programmatic team at TMP is seeking a Data Pipeline Engineer to support building new data products and services. About TMP Programmatic works on data services across the programmatic platform within TMP, and supports building customer facing data visualization products. The team has extensive experience in ETL development and works with large scale data in real time Responsibilities Responsibilities Providing technical support for all data pipeline environments. Set up, configure, maintain and enhance proper infrastructure to support a large scale data analytics environment. Evaluate the technical tradeoffs of every decision Build and maintain ETL pipelines utilizing Python Work with Cloud Computing Platforms (AWS), Kafka and other open-source technologies Conduct data modeling, schema design, and SQL development Ingest and aggregate data from both internal and external data sources Collaborate with Product Owner and domain experts to recognize and help adopt best practices in reporting and analysis: data integrity, test design, analysis, validation, and documentation Assist with the development and review of technical and end user documentation including ETL workflows, research, and data analysis Work with Product team to define data collection and engineering frameworks Responsible for daily integrity checks, performing deployments and releases Own meaningful parts of our service, have an impact, grow with the company Qualifications Technical Requirements Excellent and proven knowledge of Python Excellent and proven knowledge of SQL Excellent and proven knowledge of streaming technologies, such as Kafka or Kinesis Excellent and proven knowledge of queue based infrastructures such as AWS SQS, SNS, etc. Good knowledge of the following technologies is a bonus: Postgres on Amazon RDS Amazon Redshift Docker Proficiency in Git, JIRA and Teamcity are a plus 2+ years in a production environment a plus Flexible Location: Remote within the USA... - Permanent - Full-time

source http://jobviewtrack.com/en-in/job-4f1e417d44140a024e1a45432d0f090e0b45111a6b374b5f5c0d680a0807491145116a310717004c1d060c536e4a494c2f34061e4218490d0d632b0902491a0d0c012b1e08/a71431a90817665388a3975aacc63622.html?affid=f584d43114bf1954a48e3ec6be21b6ec