Posted 2 months ago Financial Services Tokyo
Financial Services
Tokyo
Posted 2 months ago

Responsibilities:

  • Collaborate with users and project team on requirements and expectations
  • Analyze business requirements, create proper data solutions that meets functional and non-functional requirements
  • Define the data architecture, structure, storage, security model of data in HDFS, Hive, HBase, Solr, RestAPI
  • Design and implement highly scalable data pipelines in Scala/Spark/Java for processing of extreme large amount of data
  • Create and maintain reusable frameworks for data ingestion, validation, normalization, and transformation
  • Automate Unit testing and release with CI/CD pipeline
  • Create scripts for operations, releases, data migrations, data validations
  • Maintain data lineage information
  • Provide technical support for releases, issues, and incidents

Requirements

Experience:

  • Minimum 3 years of working experience as developer or data engineer

Skills:

  • Solid knowledge and experience of Hadoop ecosystem: HDFS, Hive, HBase, Phoenix, Solr, Oozie, Impala, Knox, Zookeeper, YARN, Spark, Livy, HUE, Zeppelin
  • Strong programming skills in Scala and Spark
  • Strong capabilities in designing Database schemas on data platform
  • Good knowledge of Oracle database, MS-SQL Server, MongoDB, Kerberos, Linux, networking
  • Proficient in SQL, shell scripting, git, unit testing, CI/CD tools
  • Understanding of design principles, coding standards and best practices
  • Good problem-solving skills
  • Good data analytics skills
  • Excellent communication, presentation, and interpersonal skills

Preferable:

  • Understanding of REST API, micro-services, ETL, and Salesforce is preferred
  • Hand-on experience with Cloud services is preferred
  • Understanding of streaming data processing and change data capture is preferred
  • Experience with BI tools is preferred

Language:

  • Japanese: intermediate
  • English: intermediate

Personality:

  • Detailed and precise, able to drive alone the design and development of complex system.
  • Able to articulate technical aspects in an easy-to-understand way for non-developers
  • Strong performer with focus on delivery, fast learning and able to apply his knowledge in various domains.
  • Willing to learn new technologies and pick up challenging work
  • Team player

Job Level

2. Mid-level

Apply Online
A valid email address is required.