Data Engineer (Hadoop, SQL Server)

Data Engineer (Hadoop, SQL Server)

Description:

Position Summary

This role is responsible for leading efforts to develop and deliver complex data solutions to accomplish technology and business goals.

Key responsibilities include code design and delivery tasks associated with the integration, cleaning, transformation, and control of data in operational and analytics data systems.

They guide stakeholders, Product Owners, and Software Engineers through the design and execution processes.

Provide technical thought leadership based on the significant experience they have built through implementing multiple complex data solutions, and their knowledge of the interaction across multiple data flows and systems.

Ensure that the design and engineering approach for complex data solutions is consistent across multiple flows and systems.

Collaborate amongst team members to assist with knowledge gaps and or provide instructional training when needed.

Assist other work groups in application development using python or writing advanced SQL queries when deemed necessary.

Maintain, improve, clean and manipulate data for the operational and/or analytics data systems.

Constantly looking for better ways of solving technical problems and designing the solution without being afraid of challenging the status quo.

Collaborate with development teams to understand data requirements and ensure the data architecture is feasible to implement and subsequently implemented accurately.

Assemble large, complex data sets that meet functional/ nonfunctional requirements.

Build processes supporting data transformation, data structures, metadata, data quality controls, dependency and workload management.

Define and build data pipelines that enable faster, better, data informed decision making within the business.

Contribute to existing test suites (integration, regression, performance), analyze test reports, identify any test issues/errors, and triage the underlying cause.

Document and communicate required information for deployment, maintenance, support, and business functionality.

Adhere to team delivery/release process and cadence pertaining to code deployment and release

Identify gaps in data management standards adherence and work with appropriate partners to develop plans to close gaps.

Lead concept and experimentation testing and synthesize the results to validate and improve analytical solution.

Strong experience using a variety of data mining/data analysis methods, using a variety of data tools, building and implementing models, using/creating algorithms and creating/running simulations.

Operational support monitoring and maintaining daily ETL jobs (Autosys/SQL Agent)

Required Skills

Strong problem solving skills with an emphasis on product development.

Experience using statistical computer languages (Python, SQL, hive, etc.) to manipulate data and draw insights from large data sets.

Experience working with and creating data architectures.

Knowledge of a variety of machine learning techniques (clustering, decision tree learning, artificial neural networks, etc.) and their real-world advantages/drawbacks.

Knowledge of advanced statistical techniques and concepts (regression, properties of distributions, statistical tests and proper usage, etc.) and experience with applications.

Experience with distributed data/computing tools: Hadoop, Hive, MySQL, etc.

As an equal opportunity employer, ICONMA provides an employment environment that supports and encourages the abilities of all persons without regard to race, color, religion, gender, sexual orientation, gender identity or express, ethnicity, national origin, age, disability status, political affiliation, genetics, marital status, protected veteran status, or any other characteristic protected by federal, state, or local laws.