Financial Services
Tokyo
Posted 1 month ago

Summary of Responsibilities:

The primary responsibilities of Data Engineer are to design and develop robust, reliable, highly scalable applications in Java/Scala/Spark that runs on Hadoop platform for processing extreme large amount of data.

Principal Responsibilities:

  • Collaborate with users and project team on requirements and expectations
  • Analyze business requirements and create proper data solutions that meets functional and non-functional requirements
  • Define the data structure, storage, security model of data in HDFS, Hive, HBase, Solr
  • Design and develop highly scalable applications in Scala/Spark/Java for processing of extreme large amount of data on Big Data platform
  • Implement automated Unit testing and CI/CD pipeline
  • Create scripts for operations, releases, data migrations
  • Maintain data lineage information
  • Provide technical support for releases, issues, and incidents

 
Knowledge/Skills/Competencies Required:

  • Bachelor’s Degree in Computer Science or related field. 
  • Solid knowledge and experience of Hadoop ecosystem: HDFS, Hive, HBase, Phoenix, Solr, Oozie, Impala, Knox, Zookeeper, YARN, Spark, Livy, HUE, Zeppelin
  • Strong capabilities in building Database schemas on top of Hadoop platform.
  • Good knowledge of Oracle, MS-SQL Server, MongoDB, Kerberos, Linux, network
  • Proficient in Java, Scala, Spark, ORM frameworks, SQL, git, unit testing, CI/CD
  • Understanding of design principles, coding standards and best practices
  • Good problem-solving skills.
  • Excellent communication, presentation, and interpersonal skills.
  • Understanding of REST API, micro-services, cloud computing, ETL, and Salesforce is preferred

Personal Attributes:

  • Detailed and precise, able to drive alone the design and development of complex system.
  • Able to articulate technical aspects in an easy-to-understand way for non-developers or tech savvy people.
  • Team player able to adapt to technologies (programming languages, development environments and business requirements).
  • Strong performer with focus on delivery, fast learning and able to apply his knowledge in various domains.
  • Basic Japanese language skills and business level English language

Willing to learn new technologies and pick up challenging work

Apply Online

A valid email address is required.