Data Engineering role at Thoughtworks 

Thoughtworks India is looking for talented data engineers passionate about building large-scale data processing systems to help manage the ever-growing information needs of our clients. 

Our developers have been contributing code to major organizations and open source projects for over 25 years now. They’ve also been writing books, speaking at conferences, and helping push software development forward -- changing companies and even industries along the way.  As Consultants, we work with our clients to ensure we’re delivering the best possible solution.  Data engineers play an important role in leading these projects to success. 

You will be responsible for - 

• Creating complex data processing pipelines, as part of diverse, high energy teams
• Designing scalable implementations of the models developed by our Data Scientists
• Hands-on programming based on TDD, usually in a pair programming environment
• Deploying data pipelines in production based on Continuous Delivery practices 

Ideally- 

• You’re familiar with and interested in any one of these languages - Python/Scala/Java. 
• You enjoy working with SQL, both querying for reports and creating projections
• Understand how to build data pipelines and data-centric applications using distributed  storage platforms like HDFS, S3, NoSql databases (Hbase, Cassandra, etc) and  distributed processing platforms like Hadoop, Spark, Hive, Oozie, Airflow, Kafka etc as  part of the curriculum or project 
• You have an understanding of building data pipelines (ETL) and conceptually understand how distributed storage and distributed processing works. It helps if you have played  around with some tools within the space (e.g. Hadoop MapReduce, spark, Hdfs etc) 
• It's great if you have hands-on experience in (one or more) cloud offerings like AWS  EMR, Azure HDInsights etc or with data specific offerings like Databricks or Cloudera
• Knowledge of software best practices like Test-Driven Development (TDD) and  Continuous Integration (CI), Agile development  
• Strong communication skills with the ability to work in a consulting environment are essential 
• You work well with others and have a collaborative attitude 
• You’re keen on making things as efficient as possible and solving challenges.
• Have the passion and analytical skills to play around with data to gain insights  

And here are some of the perks of being part of a unique organization like Thoughtworks: 

• A real commitment to “changing the face of IT” -- our way of thinking about diversity and inclusion. Over the past ten years, we’ve implemented a lot of initiatives to make 

Thoughtworks is a place that reflects the world around us, and to make this a welcoming home to technologists of all stripes. We’re not perfect, but we’re actively working towards true gender balance for our business and our industry, and you’ll see that diversity reflected on our project teams and in offices.  

• Continuous learning. You’ll be constantly exposed to new languages, frameworks and ideas from your peers and as you work on different projects -- challenging you to stay at the top of your game. 
• Support to grow as a technologist outside of your role at Thoughtworks. This is why  Thoughtworkers have written over 100 books and can be found speaking at (and, ahem,  keynoting) tech conferences all over the world. We love to learn and share knowledge,  and you’ll find a community of passionate technologists eager to back your endeavors,  whatever they may be. You’ll also receive financial support to attend conferences every year. 
• An organizational commitment to social responsibility. Thoughtworkers challenge each other to be just a little more thoughtful about the world around us, and we believe in using our profits for good. All around the world, you’ll find ThoughtWorks supporting great causes and organizations in both official and unofficial capacities. 

If you relish the idea of being part of Thoughtworks’ Data & AI Service Line that extends beyond the work we do for our customers, you may find ThoughtWorks is the right place for you.  If you share our passion for technology and want to help change the world with software, we want to hear from you!