Job Description
Remote Lead Data Engineer needs 7+ years' experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales)
Remote Lead Data Engineer requires:
Strong background in math, statistics, computer science, data science or related discipline
- Advanced knowledge one of language: Java, Scala, Python, C#
- Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake
- Proficient with:
- Data mining/programming tools (e.g. SAS, SQL, R, Python)
- Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
- Data visualization (e.g. Tableau, Looker, MicroStrategy)
- Comfortable learning about and deploying new technologies and tools.
- Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines.
- Good written and oral communication skills and ability to present results to non-technical audiences
- Knowledge of business intelligence and analytical tools, technologies and techniques.
- Familiarity and experience in the following is a plus:
- AWS certification
- Spark Streaming
- Kafka Streaming / Kafka Connect
- ELK Stack
- Cassandra / MongoDB
- CI/CD: Jenkins, GitLab, Jira, Confluence other related tools
Mandatory Skills Advanced knowledge one of language: Java, Scala, Python, C#? Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake
Proficient with Data mining/programming tools (e.g.?SAS, SQL, R, Python) Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
Remote Lead Data Engineer duties:
- Create and manage cloud resources in AWS
- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies
- Data processing/transformation using various technologies such as Spark and Cloud Services.
- Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations
Employment Type: Full-Time
Salary: $ 110.00 Per Hour
Job Tags
Hourly pay, Full time,
Similar Jobs
The Boeing Company
...be required up to 10% of the time; Domestically depending on business needs.****This position requires an active U.S. Top Secret Security Clearance (U.S. Citizenship Required). (A U.S. Security Clearance that has been active in the past 24 months is considered active)**...
The Maersk Group
...lead time and goals despite unexpected conditions. Our extensive experience, connections, and deep expertise across different sectors,... ...Direct hire-Full benefits Duties: Perform various warehouse duties such as order picking, building pallets, and loading/unloading...
ABC Legal Services
...DC, and more. Seattle is our home and headquarters. We've been successful... ...our technology and business processes to remain years ahead of what... ...by ABC Legal. This role works closely with the e-Fulfillment... ...documents using internal systems and email Participate in ongoing...
Christus Health
DescriptionPOSITION SUMMARY: Promotes the safe and effective use of medication for all patients of the hospital through proper interpretation of physician orders, accurate dispensing of pharmaceuticals, and monitoring of therapy. Provides accurate, adequate and timely drug...
TJG Civil Engineers and Land Planners
...includes engineering plans for fish passage and stream restoration projects as well as stormwater mitigation. Analyzing basin hydrology using WWHM, MGSFlood, USGS Regression and other models. Hydraulic analysis using HEC-RAS and SRH-2D models for streams and culverts...