Data Engineer (Distributed Simulation), Defence

Job Description

Improbable has created SpatialOS: a networking solution combining low latency, tools for faster iteration, and a ready-to-go architecture capable of supporting innovative games. Now the Improbable Defence division, building on the backbone of SpatialOS, has combined world class scientific modelling, market leading AI, mission specific user interfaces and a uniquely flexible and secure deployment model to create a powerful simulation platform tailored to the needs of the military. Our mission? To enable the most realistic and effective military simulations ever experienced, making defence users more effective on operations and decreasing the cost of military preparedness. Your Mission is to build the data pipeline and analysis frameworks that support the development of scalable, secure and auditable data storage, appreciation, transformation, and analysis solutions for the data that underpin and are produced by simulations. The platform, tools and data framework that we are developing enable modellers and engineers to create and run some of the largest, most complex, and realistic simulations ever built. These simulations enable government and defence organisations to better understand and prepare for action in complex environments so that they can preserve peace and minimise damage and loss of life. Whether it is a power station being destroyed, a road network becoming blocked, or fake news propagating through a social network, our technology enables people to understand these consequences so that more resilient societies can be engineered and our defence and national security is at the cutting edge of capability. Our core engineering teams are focussed on building complete product solutions to tough engineering problems. Overview of our division: bit.ly/2lt8DnI

Your Impact

• Contribute to building a next generation product which will help governments gain a richer understanding of their most critical problems through the power of synthetic environments.
• Lay a foundation for scientific rigour, repeatability and complete auditability of all usages of our platform.
• Understand the requirements of our applied scientists and provide them with the framework and tools that they need to build effective data pipelines for data preparation and the analysis of simulation results.
• Collaborate with other members of our multi-disciplinary team to optimise the end-to-end process of preparing data, running simulations and analysing results; reducing the time it takes for our customers to answer the most critical and time-sensitive questions.
• Design and development of the framework and tools that enable applied scientists to build pipelines to extract, transform, synthesise and integrate data from various sources.
• Provide tools for the automation and scheduling of data extraction from both static and live sources.
• Helping to design and build the services needed for accessing and updating the data and models stored within a synthetic environment.
• Utilising standard data science tooling where possible to provide an intuitive experience for users with minimal onboarding time.
• Engineering for scalability and efficiency so that our customers can process large volumes of data and quickly initialise simulations and perform analysis at critical moments in rapidly evolving environments.
• Build security, auditability and provenance tracking into the core of our platform.

We’d like to hear if you if you identify with ANY of the following:

• You are an experienced Data Engineer.
• You have experience working with big data processing technologies such as Hadoop, Apache Spark, Kafka, Rabbit MQ, etc.
• Knowledge of Go, Python or C++ is a benefit but not essential.
• You are passionate about sharing knowledge, learning and collaborating with others.
• You are pragmatic and able to identify the most impactful work among competing requirements.
• You drive integration efforts across teams and the tech stack.
• You are skilled in Object Oriented or Functional programming.

Nb: While we think the above experience could be important, we can’t predict the future and so we’re keen to hear from applicants that believe they have valuable experience. If you identify with the team & mission, but not all of the suggestions, then please still apply!! Equal Opportunity The best ideas are often the least expected and require new ways of thinking; that’s why our teams at Improbable are made up of an incredible range of talented people. Improbable is proud to be an equal opportunity employer. We do not discriminate based on race, ethnicity, colour, ancestry, national origin, religion, sex, sexual orientation, gender identity, age, disability, veteran status, genetic information, marital status or any other legally protected status. Apply for this job