This is a nontrivial challenge. Let’s go?
Price feeds run 24x7. DeFi applications depend on the accuracy and availability of these feeds; an inaccurate price or offline feed can cause severe financial losses. Each feed in turn depends on many different services, some of which are run by our data providers and some by us. It’s a complex system with many failure modes, but it must always work correctly.
We also run a variety of off-chain services. These services run in a Kubernetes cluster that is managed using Terraform. We also need to ensure these services are running and always healthy.
We’re looking for people to help us operate this system and improve its reliability over time. This job includes providing front-line support for incidents, developing automation to manage our infrastructure, and defining deployment plans for high availability.
About Us:
-
We are a small team. About half the team is technical; the other half manages relationships with data providers, developers, and the broader community (Building a network requires talking to people)
-
We are primarily remote. Team members live worldwide, in the US, Europe, and Asia. We do have an office in Porto, Portugal if you prefer!
-
Our team communicates with each other and external developers in English. Strong spoken and written English skills are required
-
We operate like a startup in the rapidly growing and changing DeFi ecosystem. In order to be successful, we must adapt to meet the market's current needs. Good candidates will help our organization adapt; they are flexible problem solvers who are willing and able to jump on whatever the occasion demands.
-
Most of our software development is open source. You can look at our GitHub repositories to understand what we typically work on.
-
We offer a competitive salary and generous benefits package
About The Job:
Infrastructure
We maintain server infrastructure on the cloud and on bare metal. Our applications move a lot of data over the network, and that is our focus.
Most of our infrastructure is provisioned using infrastructure-as-code (IaaC) tools such Terraform and Ansible, and orchestrated with Kubernetes. We avoid provisioning resources manually using a web console or typing commands on an SSH session because that results in losing track of what and why resources were created.
Another important aspect of how we build infrastructure is where it happens: we avoid doing tasks from our laptops and we avoid exposing secrets and keys outside of the context where they are used.
Finally, some of our software runs on blockchains which means we do not have full control over its infrastructure, so we work with each blockchain community of nodes to coordinate on-chain operations.
Operation & Monitoring
We operationalize the software written by the engineering team. Software tends to break when it is changed, and it changes many times over its lifecycle, so we try to make the integration and deployment process as automated as possible.
Integration means taking the source code and producing a runnable artifact (e.g. a binary or container image) while deployment means taking the produced artifact and running it on a production environment.
Our current model for deploying software is keeping a manifest with the ideal state of all workloads, and running tools that read these manifests and reconcile the current state of the workloads. Part of this model is usually referred to as GitOps.
Once the software is deployed, we need to know how well it is running. This work begins in the development phase where engineers instrument their code with logs and metrics, and we connect the logs and metrics to our monitoring stack. Different workloads have different instrumentation needs.
Development
We build our own tools. We try to use well-established, off-the-shelf tools, but that is not always possible. Our development work covers tooling used by ourselves, by engineering, and by other teams. Here are some examples of use cases for our tooling:
-
Track the balance of blockchain addresses and notify when it goes below a threshold
-
Observe price feeds and send Slack messages when something interesting happens
-
Configure on-chain programs
-
Manage development environment
-
Serve a real-time dashboard of price feeds and publishers
Skills we’d love you to have:
-
4+ years of experience working in Engineering, DevOps, or SysAdmin roles
-
Understanding of Linux fundamentals and comfortable operating Linux servers
-
Experience packaging and deploying workloads in production-grade environments
-
Experience programming in high-level languages such as Go, Python, and/or JavaScript
-
Understanding of blockchains and smart contracts
Think you have what it takes? We’re looking forward to meeting you!
At Douro Labs, we value diversity and inclusion in our workplace. We are an equal-opportunity employer committed to preventing discrimination or harassment. We oppose unfair treatment based on protected characteristics such as race, religion, color, national origin, gender, gender expression, sex, sexual orientation, age, marital status, veteran status, disability status, or any other characteristic protected by law. This policy applies to all aspects of employment, including hiring, promotion, termination, leave, and apprenticeship. Our hiring decisions are based on qualifications, merit, and business needs. We strive to create an inclusive and welcoming environment for everyone.
