Rethinking Data Streaming With Rust And InfinyOn

Drew: So InfinyOn is in data streaming, but what specifically are you guys doing in that space?

Alan: We provide an integrated platform–all in Rust. It’s sort of a rethink of the big streaming platforms, which have been historically in Kafka and Java and have a certain memory, compute and storage footprint.

Alan: The opportunity that the founders saw was that you could rebuild a streaming stack in Rust with much higher performance, much less memory use, and a lot of the inherent safety guarantees in Rust. We're also adding analytics onto it in an integrated way. Usually you have to build the big data storage and then separately you bolt on an analytics engine. We focus on smaller teams and teams that want to just have one integrated platform.

Drew: Can you tell me a little about the history of the company? How did the company go from zero to where you guys are now?

Alan: The founders of the company, A.J. and Sehyo, the CEO and CTO, had been in a couple of companies together before this, including a startup or two. Probably the most recognizable name is NGINX. They were both there together and built portions of the mesh networking microservice offering. So, they’ve been working together for a long time. I think InfinyOn is their third startup together. Basically they started exploring the streaming space with some seed funding they raised. The company's been around for about five years, and I signed on like two and a half years in.

Drew: You were there pretty early then?

Alan: Yeah, and the main reason I signed on was because I was looking for a company that would make a real, serious investment in putting Rust in production.

Drew: Yeah, that part is very interesting to me. You mentioned that the founders thought that they could bring a lot of value to this space specifically by applying Rust to it. Could you explain what that means in a bit more detail?

Alan: There’s a lot of competing products in this space, and they are very good. They're mature. They deliver a certain level of performance. But, most of them are in Java, and there are certain inefficiencies inherent in running a Java virtual machine for everything. For one, it’s less efficient in terms of memory. And then there's always garbage collection, which can hit your performance at times. So, if you're looking for predictable performance, it's nice to start with a clean slate and start with Rust. There was also an additional opportunity to design it with the cloud in mind. A lot of the other streaming protocols are pre-cloud. They're well adapted to the cloud now, but there's fundamental architectural decisions that are different when you start from scratch with the cloud in mind. You can lay a foundation that plans on being cloud first.

Drew: One of the interesting aspects of Rust’s growth that your company represents is the replacement of Java. It feels like we mostly talk about Rust replacing C and C++, but there’s this whole other group of software that is maybe a layer up and is written in Java. Rust is replacing a lot of that too.

Alan: I think the features that Rust has when you're comparing it to C and C++ are the memory safety that people frequently talk about. In comparison to something like Java, it also has some other features that are nice for scale and for distributed computing. Rust really delivers in the concurrency space. The whole ecosystem and language help you not make a lot of first order concurrency mistakes.

Drew: Right. The classic fearless concurrency. That's a big deal for InfinyOn, isn’t it?

Alan: I think so, because we have a fairly small team and we have delivered a pretty good set of cloud distributed components. We rarely have to worry about the first order errors that you see a lot of with more legacy stacks. That lets us put more energy and focus on higher level features.

Drew: We started to talk about how you got involved in this project. You said part of it was Rust. Maybe you can speak more to that.

Alan: I come from a pretty mixed background. Most recently I did a lot of C and C++ in really low level storage and distributed storage. I also spent time at a startup that was doing bare metal SSD controller programming. We were specifically interfacing with cloud distributed storage stacks. I’ve also done embedded C and C++, C at the kernel level, and then various higher level things in languages like Java.

Alan: So, to speak about joining InfinyOn, I had been playing with Rust for a few years, and this opportunity came up. I basically made a big bet that working in it as a production stack was going to be a valuable skill in the future. I think so far I’ve been proven right with that bet. Rust is still growing. And, nowadays, Rust is really a choice that no one would blink at for lots of projects. I feel like just in the last few years it's kind of crossed that chasm from “Should we really be using this newish language?” to “OK, that's clearly the right choice for this project.” And I'm not someone that says, “Oh, you have to rewrite everything in Rust,” but for the right project, starting at the right time, it doesn't make sense to choose some of the legacy choices.

Drew: How did you find out about the company?

Alan: I don't know, actually. I think I just came across one of their jobs posted somewhere. I don’t think it was a conventional job board, because I wasn't really looking at the time. I actually talked with them six months to a year before, and we didn’t end up working together then because our needs didn’t quite match at the time. But, a year or so later the CEO reached out to me. I happened to be at the right crossroads and joined up.

Drew: So we talked about Rust. WebAssembly also seems to be a big part of the stack. What's the deal with Wasm? How are you guys using it?

Alan: Wasm is nice, because it's well-supported. There are multiple engines to choose from that are implemented in Rust. And it's a bytecode language. You can compile other languages down to it. So it's a nice way to host smaller pieces of logic in a way that's denser even than containers. There’s actually an interesting trend you can look at. First, there were VMs, which took a certain amount of host resources. And then containers came along. And everyone loves containers, because they're smaller and faster and a little bit more efficient.

Alan: Wasm is an even smaller container if you want to think of it that way. It allows you to run pre-compiled pieces of logic, which is useful in the streaming analytics space, because you're often transforming the data with these little pieces of logic. They’re usually simple transforms that you do many many times. For example, you might be trying to extract two fields out of a JSON object for 100,000 objects. You can run that in Wasm and use it as a piece of loadable, compilable logic that you can apply to data streams really cleanly.

Drew: That's really cool. I remember when I was learning about Rust, there was this talk that Steve Klabnik did about Rust and Wasm. Have you seen it?

Alan: It sounds familiar. But at this point, I've seen so much Wasm content that it's all mixed together.

Drew: He was basically just painting this vision of using Wasm on the server in place of a container- similar to what you’re describing. And I remember thinking at the time that it was very compelling. Now I'm talking to someone doing it. So that's really cool!

Alan: I'm glad you brought that up. It's something to keep in mind, because Wasm is also in the browser. Our use is very much server-side. And so there are some slight differences in the environment. It presents standalone versus with all the browser accompaniments hanging onto it. Still the conversation around Wasm is very much browser-oriented, right?

Drew: Most of it, yeah. I agree.

Alan: Figma is a big example. They use Wasm in the browser to do the cool UI stuff that they do. And we actually do some of our website content through Wasm.

Drew: Oh, really?

Alan: Yeah, but that's not a core piece of the platform. It's just giving us some nice UI. And it was easy for us to do because it’s what we're familiar with.

Drew: I’ll admit, I don’t totally understand how server-side Wasm works in practice today. The engine that interprets the Wasm on the server side, what is that?

Alan: It's basically a bytecode interpretation engine. So when Wasm compiles down to its lowest form, it's a set of bytecodes. It's a virtual engine. It’s actually ironic that I was throwing some shade at Java. That's a virtual engine, too. And then now I'm talking about using Wasm. We use Wasm in a very contained way though. It's sort of like having a full server database versus an embedded database. It's just nice that the Wasm engine is very embeddable into other larger processes. So we embed it with some other network processing and network interfaces. And we can transport a Wasm engine from our catalog of Wasm compiled modules and just pull it in and then run it without a lot of separate setup. It's not running on its own server. It's not running in other places. But, we can set up multiple instances of it. For example, if we have multiple customers with two different pieces of Wasm, they're not running in the same space. They're running in two completely separate firewall areas.

Drew: Yeah, that's super cool. So, overall what's the most interesting technical problem you or your team are working on right now?

Alan: I don't think we're going to be showing this for a little while, but like everyone else, we're working on some AI integration with our stack.

Drew: You don't say (laughing).

Alan: So, we're working on that. I also find certain things interesting that others might not. Platform support is one of those. There's a whole set of platform expansion under the hood that we're doing to increase reliability or separate out workloads. This is maybe just routine SRE stuff, but it's being done in a Rust and Kubernetes context, and it's pretty exciting to me that we can pull it off with such a small team.

Drew: Why don't we talk about that for a minute? I think you’d be surprised at what people take interest in.

Alan: One thing we're talking about is basically how do we separate workloads for our paying customers versus some of the trial customers that all go into the same multi-tenant cluster. For example, from time to time, we hold a hackathon. We actually just did one last week. So, we get this surge of people trying, but they're all using the same resources that our regular customers are using. So now we're going to separate that out a little. It's a standard Kubernetes feature, but we have Kubernetes operators written in Rust that we need to apply the right meta information to so that the pods of the streaming units of work for paying customers are well-separated from the trial type customers. To date, it hasn't been a problem at all, but this one hackathon surge was big enough that it affected a few things more than we liked.

Drew: I mentioned before that filtra.io is a hiring platform, so we always like to ask some job-focused questions. How would you describe the culture of the engineering team at InfinyOn?

Alan: We are a remote first team with people across the US and South America. We have a common base of Slack and GitHub. We do like two to three stand-ups a day or a week just to coordinate and see other human faces. It's mostly quiet. We tend to rely on ad hoc meetings and GitHub issues. We’re very adaptable in the workflow.

Drew: Since you're talking about it, I know there's been this big return to office push for a lot of companies. Has there been any temptation for you guys?

Alan: I think it would literally be impossible with our current mix of people.

Drew: Yeah, you have people outside of the country.

Alan: Right. So I don't think it would work. We do try to have an annual get together somewhere just to meet each other in person, do some whiteboarding, and relax and have some fun. That helps break down those barriers too. One thing that I think is interesting about smaller startups like InfinyOn is that they can exploit the efficiency of remote work better than large corporate players. For people looking to be remote, I would say look toward the smaller companies and the companies that were founded remote first.

Drew: Are there any different hiring considerations you've made around that? Is there a different type of person that you think thrives in the remote environment?

Alan: I think so. We try to give as high level work objectives as is possible and then coordinate when we can. So the people that thrive with that are people who can wear a lot of hats and fill in the gaps. You have to have the imagination and drive to say, okay maybe I didn’t get a 10 page spec on this. It's three sentences, but I can ask questions if there are gaps to fill in and try to do some experiments myself. So there's a level of autonomy and ownership that needs to be there. And I think sometimes that does trend better with slightly more senior people, but it's not 100% that way for sure. We've had cases where we started with some very fresh out of school type people and it’s worked out really well.

Drew: Speaking of hiring, what do you tend to look for when you're adding people to your team?

Alan: Basically a good conversation. It sounds strange, but we're not a company that does crazy quizzes or any of the gotcha stuff like “Can you give me a red black tree in 20 minutes? Go!” We've done pretty well by just talking about technology with people, talking about rust, what they like, and exploring topics they are personally curious about. We might do some very general architectural level questions, but I'm not even looking for a specific solution in those cases. I'm just looking for the thought process of how you explore it and how you form pictures of what needs to be done or what problems you are able to anticipate.

Drew: That's interesting. I was just thinking about this today, because someone mentioned to me that it's very hard to hire now because everybody's got these AI cheating apps that they can use to help them solve the problems. I think those things just exposed the flaws that already existed in that interview style.

Alan: Yeah, I think so. In a conversation, you're not gonna stop and go do an AI query of something and then come back to it. We have also used open source to evaluate people. If people are interested in working on some outstanding issue in our GitHub repo, we let them have at it and then we’ll talk about it. But, we're really most interested in talking about it afterwards than in the particular solution. You can definitely get a sense of someone's depth just by having a conversation. You can really get a sense of their ability to learn something too. If you find something that they've spent some time learning about and you go deep on it with them, you can really get a sense of how they learn and problem solve.

Alan: I think we got a little lucky too because Rust was first picked up by people who were curious about programming languages and curious about technology. So, the Rust community has been somewhat self-filtering in terms of who invested in learning the language and becoming proficient enough to have a good conversation about it. Maybe it's a little less so now. Maybe if we were hiring 100 engineers we'd have a tougher time.

Drew: Yeah, that's very true. I do still see that to some extent, but you're right that as Rust becomes more ubiquitous it'll probably trend toward the mean for all programming language communities.

Alan: Yeah.

Drew: What's the culture at InfinyOn in terms of issues of compensation like salary, equity, and benefits?

Alan: We're a mix of things because we hire in and out of the country. We tend to pay so that we can be slightly above average for the role and the country that the person's in. You have to also remember that this is an early stage startup. You do get equity and you learn a lot. So there's sort of that trade-off. I of course think there’s a lot of value for prospective engineers in that this is one of the few companies that does production Rust in almost all of our stack. So, if you want that experience, this is a great opportunity.

Drew: One of the things I've noticed is that we'll see people who are in a position where they basically don't care what the salary is at all. They just want to write Rust. People are very enthusiastic that way.

Drew: Is there anything you wish we covered but didn’t already?

Alan: We have an open source community with a lot of interesting plugins. If you're interested in Rust or in data streaming and trying to do analytics, I just suggest dropping by the website, the Discord, or the GitHub. That’s the Fluvio community.

Drew: Alan, thank you so much.

Alan: Thank you!