South East, South East, United Kingdom
dependant on experience
Senior Data Engineer
Job ID: 000_103
ABOUT WHITEHAT ANALYTICS
Whitehat Analytics is a boutique Data Science consultancy and our staff works on long term projects with big corporations in Retail, Finance, Energy and Life Sciences as well as Government bodies.
We pride ourselves on the quality of technical and science talent we have. We value our developers as people as well as employees. We also offer a very attractive benefits package including 24 days annual leave, private health care, dental treatment, discounted gym membership & private pension.
We are an equal opportunities employer and value diversity. We do not discriminate on the basis of race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.
We are expanding our team and are looking for a senior data engineer who is driven, passionate and able to develop innovative and effective approaches to solve our clients’ most critical business and data challenges. The ideal candidate will possess a background in computer science, natural sciences, mathematics or similar. We are looking for individuals that thrive in cross-functional teams and are eager to share their knowledge and learn from others. The majority of work will be focused on Java/Scala engineering within Apache Spark.
We engage with our clients to serve as catalyst pushing their abilities to apply Data Engineering and Data Science in production settings. As such we offer a varied working environment with exciting and challenging projects that ensure continuous opportunities to develop and progress.
Most of our work is onsite with one of our clients in London or Southeast England, with travel to other parts of the UK as necessary. The company will cover any travel and accommodation costs.
o Work closely with colleagues in a cross-functional data engineering, dev ops and data science team to develop analytical platform products
o Provide expertise and implementation of ETL pipelines to enable data science functionality.
o Ensure developed pipelines fit within given architectural designs
o Support testing of pipelines to provided quality standards
o Ensure that data is handled in accordance to client governance structures and implement governance structures in accordance to industry standards as necessary
o Initiate, and contribute to, a team mentality of high professional standards and continuous process improvement.
o Continuous Professional Development with new technology.
EXPERIENCE & TECHNICAL SKILLS
o In-depth knowledge of low latency, high throughput programming in Python, Java or Scala
o In-depth knowledge of at least one SQL dialect (preferably Postgres or MySQL)
o In-depth knowledge of Apache Spark, HDFS, YARN and Hive or similar.
o Good knowledge of ETL, Data Cataloguing, Data Lineage and Data Warehousing
o Working with diverse data formats in differing use cases (e.g. xsv, json, Avro, rc, orc, parquet)
o NoSQL and MPP databases (e.g. Teradata, HBase, MongoDB, Neo4j, DynamoDB and others)
o Overlap with DevOps tools and good working knowledge of AWS, Azure or Google.
o Streaming data and related tools (e. g. Flume, Nifi, Kafka, Storm)
o Proven track record of working in computationally intensive environments.
o Exposure to the DevOps toolchain and CD/CD best practices.
o Good working understanding of systems, SCM and virtualisation (e. g. Docker, Git, Linux/bash)
o Data visualisation – creating KPI dashboards and using BI tools (e.g., Tableau, PowerBI, Qlik, Pentaho, D3 and related libraries)
o Front end and API development (e.g. Flask, Django, Tkinter, Swing, AWT)
Project & Non-technical
o Project ownership and team leadership – responsible for key aspects of projects, including identifying user needs, prioritising tasks and determining tools, data and techniques to address them in an Agile way.
o Developing business cases for stakeholders.
o Mentoring junior data engineers via creation of personal development plans, encouraging team-wide diffusion of knowledge and being a source of analytical best practice
o Knowledge of software development processes including object-oriented programming, rapid prototyping and agile development techniques.
o Presenting on technical topics to a wide variety of stakeholders and co-workers
To apply, please submit your CV and cover letter care of Human Resources to firstname.lastname@example.org with the subject line: Lastname, First name - Job ID: 000_103 by the closing date of 31/07/2019. We look forward to hearing from you.
As much as we would like to hire everybody, we do have to ensure that we both fit together.
No recruitment agencies please