Job title: Data Engineer
Location: UK
Duration: 6-month contract Inside IR35
Working Mode: Fully Remote
Vetting: Active SC Clearance
Job Description:
Data Engineer Skills and Technologies Data Engineering:
– Strong understanding of data concepts
– data types, data structures, schemas (both JSON and Spark), schema management etc
– Strong understanding of complex JSON manipulation – Experience working with Data Pipelines using a custom Python/PySpark frameworks
– Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data.
– Strong understanding of Data Security principles
– data owners, access controls
– row and column level, GDPR etc including experience of handling sensitive datasets
– Strong problem solving and analytical skills, particularly able to demonstrate these intuitively
Requirements
Required Skills:
Languages / Frameworks
– JSON
– YAML
– Python (as a programming language, not just able to write basic scripts. Pydantic experience would be a bonus.)
– SQL
– PySpark
– Delta Lake
– Bash (both CLI usage and scripting)
– Git
– Markdown
– Scala (bonus, not compulsory)
– Azure SQL Server as a HIVE Metastore (bonus) Technologies
– Azure Databricks
– Apache Spark
– Delta Tables
– Data processing with Python
– PowerBI (Integration / Data Ingestion)
– JIRA
...An innovative robotics company is looking to add a Robotics Engineer to its growing team. This role focuses on integrating, configuring, and optimizing robotic systems for deployment in real-world environments. Youll be at the forefront of automation, working hands...
...Recruiting Owner-Operators for Cargo & Sprinter Vans We are looking for experienced Owner-Operators with well-maintained cargo or sprinter vans... ...semi trucks.Our services are much faster than regular delivery, and you can take advantage of that. We will deliver your...
...Department BSD NEU - Clinical Research Staff About the Department The Department of Neurology at The University of Chicago Medical... ...disease. Job Summary The Clinical Research Coordinator II (CRC II) is an onsite, patient-facing, specialized researcher...
...providing high-quality staffing solutions and this role plays a crucial part in our commitment to excellence. As a Locum Anesthesiologist Assistant, you will be responsible for providing anesthesia care under the supervision of an anesthesiologist. You will play a key...
...range for this position is $41,780-$64,340/yearIt is Northwell Health's policy to provide equal employment opportunity and treat all applicants... ..., national origin, immigration status or citizenship status, sexual orientation, military or veteran status, sex/gender, gender...