System Administrator 3

Company:  First Tek, Inc.
Location: Vancouver
Closing Date: 18/10/2024
Hours: Full Time
Type: Permanent
Job Requirements / Description
POSITION RESPONSIBILITIES

Note: All official drafts, documents and recommendations, as listed below, must be reviewed, finalized and approved / accepted by appropriate BPA manager or other federal personnel with the authority to do so.

With BPA Manager BPA technical lead oversight, use professional level skills/ knowledge to perform system administration of the following systems, including the following:

Configuring applicable software and systems, maintaining local documentation, installing/updating/removing software, applying change control processes and procedures, planning and preparing for future growth, trouble-shooting and resolving technical issues, and performing system-related training for Splunk infrastructure.

With BPA Manager oversight and approval provide the following Splunk support:

Hands on Splunk administration experience of a multi-site distributed Splunk environment including experience with data onboarding, indexer clustering, search head clustering, and index replication.

Monitoring Splunk system updates and planning, reviewing, and executing patch/upgrade deployments.

Perform on-boarding of standard and custom data sources in Splunk and have a thorough knowledge of using regular expressions to create extractions.

Provide production and infrastructure support, root cause analysis, troubleshooting, health monitoring, etc.

Installing and configuring new deployments of Splunk Enterprise and Premium Apps such as Splunk Enterprise Security and Splunk IT Service Intelligence.

Setting up Splunk Forwarding for new application tiers introduced into the environment.

Constantly re-evaluate our product to improve architecture, knowledge models, user experience, performance and stability.

Understand current issues and propose potential Splunk solutions.

Educating new users on the value that Splunk provides and its technical capabilities.

Monitoring the Splunk infrastructure (Identify bad searches, dashboards, overall health of Splunk).

Conduct frequent capacity review of Splunk environments.

Build automation to improve day-to-day operations towards self-service capabilities.

Integration of threat intelligence feeds and with other security tools to facilitate automation.

Perform deployments of Splunk configurations using Splunk Deployment Server, Cluster Master, and Deployer to support data onboarding and other maintenance/upgrade activities.

Develop search queries for support incident investigative efforts to correlate events.

Define, develop, implement, tune and monitor process and procedures for to support and maintain Splunk ES and ITSI, to enable proactive issue identification and resolution.

Configuration management of Splunk ES and ITSI with future growth in mind to enable a balance between scalability, performance, stability, reliability, and agility.

Apply expertise in Splunk administration in UNIX/Linux and troubleshooting OS-level issues. Use Linux commands and Splunk Monitoring Console to analyze and tune both a Splunk environment and a UNIX/Linux server for optimal performance.

Work with the Automation Team to leverage solutions, including Puppet, SCCM, for managing multiple Splunk deployments.

Apply understanding of computer networking at a WAN scale and troubleshooting TCP/IP protocol issues.

Review and contribute to designs and processes to improve existing and new system/process/service using new frameworks.

Deploying TAs and apps from Splunk Base and normalizing data according to Splunk CIM

Knowledge of Splunk Processing Language (SPL) and developing complex search queries, reports, and dashboards and reports using JavaScript, CSS, XML, and HTML to educate and support user base.

Creating and modifying Python, Perl, Bash scripts to parse and transform data as well as automate server maintenance tasks.

Experience with automation tools including Puppet, SCCM, and Docker

Applying understanding of computer networking at a WAN scale and troubleshooting TCP/IP protocol issues.

Install, configure, test and maintain assigned applications, software and system management tools.

Drive performance improvements, maintain patch levels and assist in performing platform upgrades.

Work with application owners to define requirements and build custom dashboards.

Work with infrastructure owners to facilitate system best practices.

Monitor and test application performance for potential bottlenecks, identify possible solutions and develop/implement fixes.

Identify and handle data discrepancy problems within the Splunk monitoring system.

Write and maintain installation, configuration, operation, and other support documentation over and above vendor provided documentation that details system, application, security, and operational procedures specific to the operating environment.

Handle multiple on-going assignments while also working independently on other tasks with a focus on managing continuous improvement of service engineering, delivery, and operational practices.

Participate in the design, logic and flow-charting, data ingestion, data governance, visualization development, testing, debugging, documentation and support of our Splunk infrastructure and provide analysis of problems and recommend solutions and assist in the continuous improvement of the systems.

Create clear documentation for new code and systems used. Document systems designs, presentations, and business requirements for consideration at the manager level as needed.

Write and maintain custom scripts to increase system efficiency and lower the human intervention time on any tasks.

Participate in the design of information and operational support systems.

Implement, design, and maintain least privileged and role-based access control models.

Maintain files / filing system(s) in accordance with compliance requirements. File and disperse documents/letters as appropriate. Validate that all official records are accurately maintained for auditing purposes. Maintain file records in accordance with the Information Governance & Lifecycle Management (IGLM) standards and procedures.

Provide advice and recommendations for process / procedural changes that may become necessary due to system changes, upgrades, etc. to appropriate BPA manager / team lead / BPA stakeholders.

Provide input and recommendations, to the BPA manager, technical lead, and staff, regarding computer infrastructure decisions and create tasks to fulfill management-directed goals.

Support and assist the BPA team lead and North American Electric Reliability Corporation - Critical Infrastructure Protection (NERC-CIP) subject matter experts (SMEs) with implementing, analyzing, and reporting on the operational compliance of all Control Center cyber assets.

Develop, draft, and recommend improvements to the Control Center's procedures and processes for compliance with BPA and NERC-CIP standards and policies.

Develop, draft, and recommend improvements to the Control Center's procedures and processes for BPA's Grid Modernization project, as well as other capital and expense projects.

Assist BPA management with the implementation and promotion of approved operational compliance process improvement efforts, specifically as relates to the administration of the above referenced systems.

REQUIREMENTS

Education & Corresponding Experience (required on matrix)

A degree in Computer Science, Information Technology, or a directly related technical discipline is preferred.

With an Associate's degree in applicable fields, 8 years of experience is required.

With a Bachelor's degree in applicable fields, 6 years of experience is required.

Without an applicable degree, 10 years of progressive hand-on experience in, design, implementation, and support of moderate to complex Splunk Solutions is required.

Experience must include direct work experience in Information Technology performing System Administration.

Experience includes a minimum combination of work-related experience, on-the-job training, and/or vocational training.

Required Technical Skills & Experience (required on matrix)

Experience using specific technologies such as Splunk, Remote Desktop Services, IIS, Wireshark, Trend Micro, Nessus, Ivanti, Puppet, PowerShell.

Solid grasp of best practices as recommended by Splunk.

Install, configure, test, and maintain Splunk application, related software, and system management tools.

Experience setting up Splunk Forwarding for new application tiers introduced into the environment.

Drive performance improvements, maintain patch levels and assist in performing system upgrades.

Work with application owners to define requirements and build custom searches and dashboards.

Experience with administration in both the Linux and Windows operating systems; comfortable on the command line interface, including writing python and basic shell scripts.

Working knowledge or recent experience with scripting languages (bash, python, or Perl), networking, system administration, databases, and analytical tools

Experience with following Change Management systems and ticketing management systems.

Preferred Skills & Experience (optional on matrix)

Splunk Admin, Splunk Certified Architect or Splunk Enterprise Security Certified Admin.

Practical experience with Splunk in Multi-cluster/multi-site environment.

Hands-on experience architecting, building, deploying Splunk instances.

Self-starter, able to gather requirements, plan, execute Splunk architecting and deployment efforts. Able to perform gap analysis and initiate and execute architectural improvements.

Hands-on experience administering, maintaining, and scaling Splunk instances.

Hands-on experience with scripting languages such as Pearl or Python or Bash etc.

Hands-on experience with Regular Expressions (RegEx)

Analyze security monitoring and reporting requirements and define, design, develop, and maintain Splunk dashboards, reports, alerts etc.

Must have working knowledge and understanding of network infrastructure components such as routers, switches, firewalls etc.

Must have working knowledge and understanding of networking and switching protocols and infrastructure services able to troubleshoot and identify DNS, NTP, routing, switching, and firewall issues affecting connectivity of Splunk instances.

Customer-oriented with excellent issue follow-through and resolution abilities.

Excellent written and oral communication, and presentation skills.

Ability to effectively work both autonomously as well as on a team.

Outstanding interpersonal skills, strong work ethic, and self-motivated.

Utilize tools and analytical skills to plan and execute technical changes.

Experience troubleshooting LAN connectivity problems.

Working knowledge of Splunk Validated Architectures.

5 years of IT Security Center Operations (SOC) experience in complex environments with extensive knowledge of computer networking and platforms, including (but not limited to) network operating systems; firewalls; intrusion detection/prevention systems; wireless security including wireless intrusion prevention systems; web proxies; vulnerability scanning technologies; VPN's; Windows and Unix-based platforms; identity management; IT incident response; security configuration automation, logging and network architecture.

Additional Requirements (not required on matrix)

Valid U.S. Driver's License is required.

Apply Now
Share this job
  • Similar Jobs

  • System Administrator 3

    Portland
    View Job
  • System Administrator 3

    Portland
    View Job
  • System Administrator 3

    Vancouver
    View Job
  • System Administrator 3

    Portland
    View Job
  • System Administrator 3

    Portland
    View Job
An error has occurred. This application may no longer respond until reloaded. Reload 🗙