Big Data Developer - Java/Kafka

  • location

    London, England

  • sector:


  • job type:


  • salary:

    Up to £110000.00 per annum

  • Contact:

    Jake Daffern

  • Email:

  • job ref:


  • published:

    over 2 years ago

  • expiry date:


  • Startdate:


Key Capabilities

  • A passion for event driven architecture and appreciate for when and when not to apply CQRS.
  • Demonstrable experience in building out or extending streaming frameworks around Kafka to support event discovery and multiple consumers.
  • Deep experience in Java developments. Scala and Python are also beneficial.
  • Design Authority for development work and reviewing the work delivered by other team members.
  • Ability to identify and manage complex technical deliverables to meet program requirements.
  • Experience in the functional decomposition of existing systems into micro services.
  • Strong technical architectural and design skills combined with a confident white board manner.
  • Exposure to open banking api standards and other mainstays of retail banking like core banking systems is beneficial but not essential.
  • Exposure to risk platforms and HPC grids is beneficial but not essential.
  • Experience in AWS is essential and GCP and Azure exposure is beneficial.
  • Experience with Kafka via the confluent or Hortonworks platforms is beneficial.

Technical Skills

  • Programming languages: Python, Scala, Java
  • Big data frameworks: Apache Spark, EMR, Hadoop
  • Apache Ranger or equivalent role based access control to event streams.
  • Streaming frameworks: Spark streaming, Apache Flink, Apache Kafka, AWS Kinesis
  • ETL/ELT: Airflow, NiFi, AWS Glue, GCP Pub Sub
  • Hadoop infrastructure: HDFS, YARN, Hive, Impala, Sqoop and Flume
  • Data warehousing: Redshift, Snowflake or other MPP like Netezza or Exadata
  • Cloud platform knowledge: AWS, , Azure GCP
  • NoSQL and SQL knowledge: DynamoDB, MongoDB, PostgreSQL, MySQL, Oracle, SQL Server, ScyllaDB
  • Documentation and Process: Jira, Confluence, GitHub
  • Swagger, RAML

Personal Skills

  • An ability to apply structured thinking, simplifying complex problems into component parts, to the resolution of project challenges.
  • Working practices; thorough with high standards of work.
  • Effective communication and interpersonal skills to be able to establish rapport with people at all levels.
  • An eye for detail and accuracy, and an affinity with design issues and the ability to help establish an own a vision for our solutions.
  • The ability to work in teams and knowledge of agile processes & ceremonies such as Scrum, SaFe and Kanban.
  • The ability to size and scope stores at an appropriate level to support completion within a sprint. Exposure to BDD would be a plus in this regard.
  • Organizational ability and tenacity; able to think ahead in order to establish an efficient and appropriate course of action for self and others.
  • Take a proactive approach to work, good with handling uncertainty and can manage their work without close supervision.
  • A strong sense of ownership and a desire to deliver quality outcomes for the business.
  • Demonstrated strong oral and written communication skills.
  • Provide industry thought leadership via speaking at conferences and events, wiring blogs posts and white papers.