Senior Data Engineer | Canberra, ACT
|Position Type:||Full Time|
|Attachments:||No File Attached|
|Application Close Date:||03-May-2018|
Our Data & Validation Group collects and manages critical validation and algorithm training datasets and applies their statistical analysis expertise to algorithm validation. The team acts as an internal service division and provides critical assistance to the Research & Development team and other engineering groups.
We believe this is a genuinely stimulating and challenging opportunity to work on exciting cutting edge Big Data projects.
The Senior Data Engineer reports directly to the Director, Data and Validation and is responsible for designing and implementing best-practice scalable database solutions for our Image/Video data warehousing.
This will involve:
- Leading cross-division consultations to develop workflows and data tools to support all facets of the data lifecycle, beginning with data collection to storage, processing, analysis and archiving
- Researching and advising on (a) data standards to support continual development of the metadata specification and (b) measures to meet the requirements of data privacy and protection regimes
The successful applicant will have tertiary qualifications in computer science, data analytics or data science or software engineering with strong commercial experience in:
- Advanced database design using various database technologies (e.g. SQL, NoSQL), and database design patterns
- Expertise in logical data modelling
- Expertise in performance optimisation of all aspects of the data warehousing workflow, whilst maintaining data integrity
- Experienced in Cloud base technologies and services (e.g. AWS storage solution and database services)
- Experience with scripted programming languages such as Python
- Experience with file container formats (HDF5, JSON, CSV, Apache Parquet)
- Knowledge of computer networking and file systems (e.g. Windows, Linux, object storage), and distributed storage systems technologies
- Strong analytical and problem-solving skills
- A strong team player and ability to engage with various internal stakeholders and external customers
- Good oral and written English communication skills, and ability to express technical concepts clearly
- Video data streaming technologies and techniques
- Data standards, Data privacy and protection regimes
- User-centered design / user-driven development
- Search engine indexing architecture
- Data analytics skills, and MapReduce paradigm
To apply for this role, please include an updated resume and a cover letter outlining briefly the qualifications, skills and experience that you would bring to the role.