Job Information
Confluent Senior Software Engineer-Kafka (remote Canada) in Montreal, Canada
With Confluent, organizations can harness the full power of continuously flowing data to innovate and win in the modern digital world. We have a purpose that drives us to do better every day – we're creating an entirely new category within data infrastructure - data streaming. This technology will allow every organization to create experiences and use the power of data in ways that profoundly impact the way we all live. This impact is our purpose and drives us to do better every day.
One Confluent. One team. One Data Streaming Platform.
Data Connects Us.
About the Role:
Our team is responsible for managing the metadata that makes Kafka a reliable, fault-tolerant, and high-performance distributed streaming platform. This includes coordinating the election of leaders, monitoring broker availability, persisting and distributing metadata, coordinating cluster-wide actions, and more. In this role, you will be working on building highly available, reliable, secure, scalable, and multi-tenant metadata layer. You will be instrumental in driving the technical solutions end-to-end, working closely with the team and key stakeholders to design, architect, and develop top-notch solutions. The team just completed a major initiative to implement its own consensus protocol based on the Raft protocol. This new protocol, named Kraft, has allowed the team to remove the dependency on Zookeeper which allows for more scalable and robust metadata management as well as a more simple deployment model for Kafka.
Here are examples of some of the challenges we've taken on in recent years:
Kora: A Cloud-Native Event Streaming Platform For Kafka (https://www.vldb.org/pvldb/vol16/p3822-povzner.pdf)
KRaft: Apache Kafka Without ZooKeeper
Why ZooKeeper Was Replaced with KRaft – The Log of All Logs (https://www.confluent.io/blog/why-replace-zookeeper-with-kafka-raft-the-log-of-all-logs/)
KIP-866 ZooKeeper to KRaft Migration
What You Will Do:
Driven and excited about challenges of delivering complex projects in production with a focus on quality
Customer focused - making customers more successful by taking on their most challenging problems motivates you
Have a strong sense of teamwork and able to make decisions which benefit the team and company
Identify root causes, and get beyond treating symptoms - motivated to dig deep and solve hard problems
What You Will Bring:
BS, MS, or PhD in computer science or a related field, or equivalent work experience
4+ years of relevant cloud infrastructure/cloud networking experience
Strong fundamentals in distributed systems design and development
Experience building and operating large-scale systems in the Cloud
Solid understanding of basic systems operations (disk, network, operating systems, etc.)
A self starter with the ability to work effectively in teams
Proficiency in Java, Scala, C/C++, Go or other statically typed languages
What Gives You an Edge:
In-depth familiarity with the JVM
A strong background in distributed storage systems or databases
Experience/knowledge with public clouds (AWS, Azure or GCP)
Interest in evangelism (giving talks at tech conferences, writing blog posts evangelizing Kafka)
Experience working on databases or data storage
Come As You Are
At Confluent, equality is a core tenet of our culture. We are committed to building an inclusive global team that represents a variety of backgrounds, perspectives, beliefs, and experiences. The more diverse we are, the richer our community and the broader our impact. Employment decisions are made on the basis of job-related criteria without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other classification protected by applicable law.
Click HERE (https://www.confluent.io/legal/confluent-candidate-privacy-notice/) to review our Candidate Privacy Notice which describes how and when Confluent, Inc., and its group companies, collects, uses, and shares certain personal information of California job applicants and prospective employees.
#LI-Remote