Big Data Engineer


Job description

  • Data engineers are full stack engineers/developers
  • Architecting solutions to collect, stream, store and analyze a variety of data
  • They designing scalable, future proof, systems
  • Working with large sets of structured and unstructured data
  • Design trend detection/pattern recognition algorithms for predictive analytics
  • Design classification models and pipelines
  • Design and embed methods for dynamic parameter optimization
  • Research and develop new technologies as need changes
  • Working with other teams to define/refine requirements and provide solutions


  • BA/BS/MS/or PhD, with strong academic record, ideally in Artificial intelligence, Mathematics, Computer Science, OR, Statistics or other quantitative field
  • Experience working with large data sets (both structured and unstructured) using technologies such as MapReduce, Hadoop, HBase, Hive and NoSQL
  • Experience with visualization technologies
  • Experience with predictive analytics using R or SaS in PoC/prototyping is a plus
  • Experience with algorithm design and machine learning
  • Experience with automated parameter optimization techniques
  • Experience with real-time analysis of sensor/data from Internet of Things is a plus
  • Preferred candidates have several years of software engineering experience in Big Data
  • Preferred skills / tool experience:
    • Scripting / dynamic languages, such as Python / Bash / Ruby
    • Programming languages: Java /Scala/C++
    • Hadoop, Cassandra, MongoDB, Neo4J etc.
    • Linux and Bash scripting
    • Spark, Kafka, Storm
    • GIT, Jenkins, Puppet
    • Search systems (Lucene / ElasticSearch / SOLR)
    • Agile software development
    • Front-end coding (HTML5 / JavaScript / AngularJS

Company profile

Our client is a consultancy company in the area of big data engineering, data science and data architectures. In all these segments of big data, their provide consultancy, training & support. They help customers in their transition from previous generation information architectures to the new generation of big data technologies and analytics.

Our client provides their customers with future proof, sustainable solutions that are able to grow and adapt as their company does. Our client consultants have the scientific background but are pragmatic in the execution and understand it all comes down to business value. Their consultants are industry experts and thought-leaders who have worked extensively with big data technology.

At a glance
Match criteria
  • Data Science & Analytics
  • Medior
Recruitment form
  • Permanent
More information
Jonne Mostert
Mobile: (06) 4210 6327
Vye Professionals uses cookies to remember certain preferences and align jobs interests.