Data Engineer

GA-CCRi maintains and deploys production systems for users across the Intelligence Community, Department of Defense, and commercial industry. We build and develop best-in-class all domain and globally focused situational awareness capabilities, including THRESHER and DRAGONSPELL, that process petabytes of data from numerous streaming data sources in near real time. Our systems apply state-of-the-art algorithms and machine learning techniques to extract features and fuse data from multiple phenomenologies to form a rich live view of objects in the sky, on the sea, and on the ground. These analytics are designed to determine not just where something is, but what it is, where it's been and what it's doing. All of this "data to knowledge" is made available to end users in our own browser-based application for visualization, analysis, and understanding. We always want to do more, and that's where you come in!

CCRi is looking for a Data Engineer with a mix of DevOps, Software, and Systems Engineering experience who is comfortable working in a secure, cloud-based environment.

Positions available in: Charlottesville, VA

Duties and Responsibilities

• Build and maintain data pipelines

You’ll be responsible for bringing in new data sources as well as maintaining existing data sources. “Data to Knowledge” is our mantra. Ensuring that data sources are ready and available for use in our products is priority for both our customers and users.

• Analyze and organize data

We bring in numerous data feeds and additional feeds are always in the works. You’ll be responsible for understanding what’s in the data, how should we bring it in, and how best to maintain it going forward.

• Data ingestion and quality checks

Bringing in streaming data sources can sometimes be a tricky process. You’ll be responsible for reaching out to data providers, utilize Apache NiFi and other tooling for wiring data flows, and then bringing that data into our system.

• Metrics and monitoring

You can’t manage what you can’t measure. We utilize data visualization and alerting tools to help monitor feeds and help ensure availability. As a data engineer you will help scope the appropriate alerting thresholds through evaluation of the feed volume and flow rates.

• Work alongside project management

You'll work closely with the Data Project Manager in preparation for meetings with our customers, such as the monthly Data Council.

Essential Functions

• Under general direction, this position is responsible for the design and implementation of secure automation solutions for development, testing, and production environments.
• Builds and deploys automation, monitoring, and analysis solutions.
• Manages continuous integration and delivery pipeline to maximize efficiency.
• Implements industry best practices for system hardening and configuration management.
• Develops and maintains solutions for operational administration, system/data backup, disaster recovery, and security/performance monitoring.
• Continuously evaluates existing systems with industry standards and make recommendations for improvement.
• Builds and maintains tools, solutions and microservices associated with deployment and our operations platform, ensuring that all meet our customer service standards and reduce errors.
• Tests system integrity, implemented designs, application developments and other processes related to infrastructure, making improvements as needed.
• Manages code deployments, fixes, updates and related processes.
• Works with open-source technologies as needed.
• Works with CI and CD tools, and source control such as GIT and SVN.
• Offers technical support where needed, developing software for our back-end systems.
• Stays current with industry trends and source new ways for our business to improve.
• Work with the Program Manager, Project Technical Lead, and the Customer to create technical plans for new deployments, including requirements, implementation details (e.g. sizing of the resources), verification, and validation.
• Ongoing verification, validation, and documentation of deployments.
• Plan and execute maintenance and upgrade tasks.
• Support the deployments including monitoring and providing answers to customer inquiries.
• Participate in an on-call rotation for supporting the systems during work and weekend hours.
• Work 100% on-site at our Charlottesville facilities.

Requirements

• Typically requires a Bachelor's or Master's degree in Computer Science, Engineering, or related field and five years of related experience.
• Understanding of system administration in Linux.
• Strong knowledge of configuration management tools
• Scripting/coding with Python, Java/Scala, Bash· Strong communication and documentation skills.
• An ability to drive to goals and milestones while valuing and maintaining a strong attention to detail.
• Excellent judgment, analytical thinking, and problem-solving skills.
• Self-motivated individual that possesses excellent time management and organizational skills.
• Strong Customer Advocacy skills / Passion for the Customer
• Strong understanding and experience with at least one cloud environment such as Amazon Web Services (Preferred), Google Cloud and/or Microsoft Azure
• Experience with software development life-cycles and best practices
• Strong troubleshooting skills, and capable to solve problems logically.
• Collaborative and willing to work with remote team members.

Desired Skills and Qualifications

An ideal candidate will have additional specific experience with many of the following:

• Apache NiFi and Apache Kafka, Secure File Transfer Protocol, Web scrapers
• Experience in systems engineering or architecture (balancing hardware and networking capacities with data transactions)
• Containers and container orchestration (docker, kubernetes, helm, istio, rancher)
• CI/CD, GitOps
• Provisioning (CloudFormation, Terraform) and configuration management (Puppet) tools
• Security & networking - cloud security, IAM, ABAC, SSO, Okta, Keycloak, A&A
• Databases (Postgres, MySQL, and Accumulo)
• Support or operations engineering roles
• Preference will be given to candidates with an active TS clearance.

Travel Percentage Required

0-10%

Relocation Assistance Provided

Yes

US Citizenship Required?

Yes

Clearance Required?

Ability to obtain and maintain security clearance required.

Clearance Level

TS with SCI eligibility

Pay Range

$110,000-$150,000

Benefits

• Casual Work Environment
• Intellectually Challenging Work
• Health Insurance including FSA, HSA and Tricare Supplement options
• Short/Long Term Disability Insurance
• Generous Defined Retirement Benefit, including both a 401K match and pension plan.
• Very Flexible Vacation Policy

The job description above is not intended to be comprehensive list. Responsibilities, activities, duties, and/or tasks may change or be assigned at any time.

CCRi is committed to a diverse and inclusive workforce because we know that our differences benefit our employees, our customers, and our community. We are proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, color, religion, age, sex, sexual orientation, gender identity, national origin, status as a an individual with a disability, status as a protected veteran, or any other applicable legally protected characteristics.