Big Data Engineer , DP&I - D&A - Revenue

Description

Sales, Marketing and Global Services (SMGS)

AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success. AWS Support also partners with a global list of customers that are building mission-critical applications on top of AWS services.

The AWS Sales, Marketing, and Global Services Operations (SMGS Ops) Revenue technology team is responsible for publishing Daily Estimated Revenue (DER) and monthly Amortized Sales Revenue (ASR). DER and ASR are used by downstream customers in AWS Finance, Accounting, Sales Operations, Segmentation & Planning (S&P), Data & Analytics and Compensation to set quotas, manage sales goals, forecast, and derive business insights. We collect and process billions of usage and billing transactions every single day and relate it to the largest data feed supported by Salesforce.com. We transform this raw data into actionable information in the Data Lake and make it available to our internal service owners to analyze their business and service our external customers.

We are truly leading the way to disrupt the big data industry. We are accomplishing this vision by bringing to bear Big Data technologies like Elastic Map Reduce (EMR) in addition to data warehouse technologies like Spectrum to build a data platform capable of scaling with the ever-increasing volume of data produced by AWS services.

You should have expertise in the design, creation, management, and business use of large datasets, across a variety of data platforms. You should have excellent business and interpersonal skills to be able to work with business owners to understand data requirements, and to build ETL to ingest the data into the data lake. You should be an authority at crafting, implementing, and operating stable, scalable, low cost solutions to flow data from production systems into the data lake. Above all you should be passionate about working with huge data sets and someone who loves to bring datasets together to answer business questions and drive growth.

Key job responsibilities

• Expertise in the design, creation, management, and business use of large datasets, across a variety of data platforms.

• Excellent business and interpersonal skills to be able to work with business owners to understand data requirements, and to build ETL to ingest the data into the data lake.

• Be an authority at crafting, implementing, and operating stable, scalable, low cost solutions to flow data from production systems into the data lake.

• Above all you should be passionate about working with huge data sets and someone who loves to bring datasets together to answer business questions and drive growth.

About the team

Diverse Experiences

AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.

Why AWS?

Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.

Inclusive Team Culture

Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness.

Mentorship & Career Growth

We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.

Work/Life Balance

We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.

Basic Qualifications

• This position requires a Bachelor's Degree in Computer Science or a related technical field, and 3+ years of related employment experience.

• 3+ years of work experience with ETL, Data Modeling, and Data Architecture.

• Expert-level skills in writing and optimizing SQL.

• Experience with Big Data technologies such as Hadoop, Hive, Spark.

• Proficiency in one of the scripting languages - python, ruby, linux or similar.

Preferred Qualifications

• Experience working in very large data warehouses or data lakes.

• Sound knowledge and hands-on experience in ETL optimization, designing, coding, and tuning big data processes using Apache Spark or similar technologies.

• Experience with building data pipelines and applications to stream and process datasets at low latencies.

• Demonstrate efficiency in handling data tracking, lineage, ensuring data quality, and improving discoverability of data.

• Sound knowledge of distributed systems and data architecture (lambda)- design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning in MPP architecture.

• Knowledge of Engineering and Operational Excellence using standard methodologies.

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $91,200/year in our lowest geographic market up to $185,000/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit This position will remain posted until filled. Applicants should apply via our internal or external career site.